Search
Program Calendar
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Unit
Browse By Session Type
Search Tips
Annual Meeting Housing and Travel
Personal Schedule
Sign In
X (Twitter)
Purpose
This study tested the effectiveness of verbal feedback in the design of online digital instructions and quizzes in i-Ready instruction, which are online instructional modules for students in Grades K-8. Previously, we found passive lecturing was often ineffective at holding children’s attention and engagement. We hypothesized that children would pay more attention to instruction if that instruction was given in response to an incorrect answer. To evaluate the degree to which this feedback was effective we gathered evidence to understand the following questions:
• When a student answers a question wrong are they paying attention and reacting to digital feedback?
• What software features lead children to pay attention and emotionally engage with digital feedback?
Methodology
151 children were included in this study, with the majority of participants coming from Title 1 schools in Hartford and Denver. Children were asked to either use current i-Ready software (baseline) or a prototype where we iteratively altered functionality to improve feedback effectiveness. Wireless eye tracking glasses helped assess whether children were visually paying attention and watching the animations. We used skin conductance sensors to measure when children were emotionally reacting to content, a correlate to engagement (Hedman, 2014). We had a well-trained ethnographer observe and interview the students about their experience with i-Ready.
Results
From our baseline, we observed children often did not pay attention to the passive, lecture-like feedback. During feedback, children would often look off the screen, even read texts on the walls. Most students could be seen clicking and staring at their next guess, waiting for the feedback to end. Children often felt guessing a second time was a valid strategy and feedback was useless or cheating. How much a child cares about getting these answers right, we labeled as “weight.” Typically, there was little weight to answering correctly and correspondingly little attention to feedback. During our prototype iterations we tried multiple approaches including shortening feedback to a few seconds and implementing rudimentary reward systems, neither of which changed outcomes. In our final iteration of the lesson we implemented a strong, working reward system: you have to get every problem right to earn a medal (you get 2 guesses for each). With this reward system, problems typically had considerably more weight. With this increase in weight, we observed children actively asking for feedback/hints from the instructions, something in our six months of research we seldom observed.
Educational significance
This study shows how effective digital feedback can be dependent on children’s personal value of answering a question correctly, the question’s “weight”. Our rule of thumb now is: if a child would not want to ask for feedback, they likely will not pay attention to it. This finding has significance on digital systems that are employing new ways to dynamically teach. When evaluating those systems, researchers should take careful note of how invested children are in arriving to a correct answer.
Hedman, Elliott B. Thick psychophysiology for empathic design. Diss. Massachusetts Institute of Technology, 2014.