Small+Group+Evaluation

FORMATIVE EVALUATION ACTIVITIES 


 * Small Group Evaluation **

1.** Participants: **
 * Teachers – Representative from each school – (1) Senior High School, (1) Middle School, (1) Intermediate School, ((2) Elementary- skills levels range from highly proficient with technology to proficient. High School teachers tend to have a highler level of proficency due to curriculum requirements at the high school level. Middle School through Elementary are proficient with the technology within their classrooms. Troubleshooting skills are also varied. The majority call for IT support for most technology related issues.
 * Administration - (1) Principal - skill levels, like the teachers, vary with the individual. However, the majority will call for IT support if any issues are noted.
 * Teachers Aide - (1) Representative from each school - same as above - skill level is proficient, but usually only in the areas they are required as dictated by the teachers they support. Tend to report technology issues to teacher, who will report them to IT Dept.
 * Secretary - (1) Representative from each school - in most instances, they are all very proficent in technology as the scope of their job requires. However, they tend to call IT support to resolve most technology issues.
 * Head Custodian - (1) - proficiency depends on the individual. The majority are proficient with technology but only as dictated by their job responsibilites. Will call IT support for most technology issues.
 * Total participants = 17

 2. **Procedures:**
 * **Introduction: **
 * The participants will be given a detailed introduction of why they are there, what they will be accessing, and what their role in the overall process will be.
 *  Explain to the learner that a new workshop is being proposed for the District and that they have been chosen to evaluate and perform the instructional materials first hand for their reactions/attitudes, comments and possible revisions.
 * Explain to them how important their comments will be to the overall design. Remind them that most mistakes in the learning process are usually due to poor design. <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;"> Further explain that the materials are still in formative stage of development and their active participation in this process will be instrumental in the success of the proposed workshop.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;"> Review the areas that they will be participating in and evaluating to include: Pre-Assessment Test; PPT presentation; TRP Manual; TRP Workbook; and Post-Assessment Test
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;">The Small Group Evaluation will be conducted just like the actual proposedworkshop.
 * **<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Time Allotted: **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;">Should reflect the same time span as proposed training - not to exceed 2 hrs per session. 15 minute break between sessions. Time may vary dependent how the evaluation is proceeding.
 * **<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Materials: **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;">TRP Manual,
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;"> TRP Workbook,
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;"> PPT Presentation
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;"> PPT Handout with Notes
 * **<span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;">Environment: **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;">The environment will mimic the actual training environment . Learners will be at divided up in smaller groups of 4 and assigned to a table. Each table will contain a network data cable, network data card, a computer tower, and a front cover of a network data box. There will be a white board/projector screen, projector, CPS Data Collection system, and a CPS data remote for each participant.


 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">3. ****<span style="font-family: 'Times New Roman',Times,serif; font-size: 16px;"> Sequence or Instruction: **


 * **<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Pre-Assessment Test **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Before the test begins, Instruct the learners to do the following:
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Fill in name and school.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Circle any words they do not understand and "check mark" or "x" on any vocabulary or instructions that need further clarification.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> Mark any questions that are not clear or concise to them and how they could be revised.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Encourage the participants to add any additional notes or comments while reviewing the materials. Remind learners that they need to work through the materials and save any questions to the end of the instruction.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Observe and take notes of the learners overall reactions, facial expression, body language. Do not interrupt the test at anytime as to not interfere with evaluating the actual time needed for completion.
 * **<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Instructional Materials **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Instruct the learners that you need their help in evaluating the materials. As with the Pre-assessment test encourage them to make any comments or revisions on the margins of the materials to include the Objectives, TRP Manual, TRP Workbook, and the PPT Presentation. As with the Pre-assessment test, remind the learners that they need to work to the end without stopping to ask questions. Encourage them to jot down all question to be addresses at the conclusion. Observe the learners while they review all the materials. Watch for facial expressions, did they look confused, were they able to read through the material in a timely manner. After reviewing all the materials ask the learners to answer the following questions.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">__Objectives__ - Were the Objectives stated clear and concise? Do you know what is expected of you at the conclusion of the workshop?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> __TRP Manual__ - Did the manual clearly state the sequential steps needed to accomplished the instruction goal? Did you find the step-by-step exercises easy to understand and follow? Did you enjoy using the TRP Workbook in conjunction with the manual to complete the exercises? Were the manual's examples, illustrations and descriptions easy to follow? Were you able to complete the exercises with the instruction given in the manual?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">__PPT Presentation__ - was the PPT a good visual portrayal of the information as conveyed in the workshop? Was the sequence of instruct clear in conjunction with the TRP manual and workbook. Did you enjoy the graphics, were they clear and easy to read?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">__Additional Training materials__ - Did you find the hardware components for demonstration purposes helpful to the instructional process.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Do you feel the "hands-on" portion was valuable portion in the instruction?
 * **<span style="font-family: 'Times New Roman',Times,serif;"> Post- Assessment Test **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;">As with all portions of instruction - Instruct learners to to circle words they do not understand and place a check mark beside questions or instructions that are unclear.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Remind learners to answer all the questions on the post-test even if they are unsure of the correct answer (evaluate for clues as inadequate instruction)
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">For questions that they found unclear - have them indicate if it required them to "guess" have them explain their answer.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Record time for post-test completion
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Questionnaire**
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Determine attitude towards training
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">If the Instruction held their attention
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Was the instruction too long or too short?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Was the instruction difficult or too easy
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">What parts of the instruction were unclear, and why?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Was the PowerPoint presentation graphically appealing, informative, work well in conjunction with the TRP Manual
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Did you find the TRP Manual easy to follow?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Were the instructions and the sequential steps as outlined in the TRP Manual easy to follow and mimic in a true life classroom scenario
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">What did you like most from the training?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">What did you dislike from the training and why?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Do you think you will use the problem solving techniques learned today?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Do you think having the materials available to see and use in the instruction made the problem solving strategies easier to comprehend?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Did you enjoy the "hands-on" potion of the training?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Would you like to see more training conducted as the TRP Workshop was conducted today?
 * **Q<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">uestion/Answer session **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Open forum to discuss instruction, listen to learners feedback
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Review comments, ask for suggestions were they indicated a problem with the materials, questions, presentation as a whole.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Ask for comments regarding certain aspects of the instruction - in Media Center opposed to the hands'on training in the classroom - which did they prefer and why?
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Ask about potential issues they might encounter when training is mimicked in the classroom to resolve connectivity issues
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Discuss Questionnaire and their attitudes about the implementing the procedures - if they would willing to actually implement the trouble shooting strategies to meet the Instructional Goal
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Answer any of their questions and thank them for their participation in today's training.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**4. Arrange for a "Field Test" of the instruction** - ask for volunteers to take the training materials back to the classroom and review and perform step in their actual work environments. Make arrangements to meet within a 2 week time period to discuss their finding and if the training received was successful.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 16px; line-height: 23px;">**5**. **Describe the revisions to instruction and assessment**
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Per comments as indicated - retool all difficult questions
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Incorporate all the comments and revisions into the training materials
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Evaluate the test scores to see if there was a significant increase from the pre-test scores to the post-test scores
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Review the questionnaire and make any additional changes as indicated.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> Evaluate all collected evaluations and notes you took during the small group evaluation to determine what areas were most effective and make appropriate revisions to instructions as necessary.