And What if They had a Choice? The Impact of Learner Control Over Automated Feedback

• In Event: Advancements in Feedback Research: Beyond Perceptions to Learning Impact

Abstract

Objectives and Framework
The objective of this experiment is to investigate the impact of providing students with the choice not to retrieve a task-related feedback message. It aims to explore the benefits and potential pitfalls of such feedback-related learner control.
Although formative assessment systems with automated feedback can effectively foster learning, they have been criticized for often lacking feedback-related leaner control (Deeva et al., 2021) such as a choice option for feedback nonretrieval. In learning, choices can increase students’ sense of autonomy (Ozogul et al., 2013) and drawing on affective-motivational theories (e.g., Pekrun, 2006), having the mere choice not to retrieve it may be beneficial to the impact of automated feedback. For example, the control-value theory suggests that students appraise feedback most positively when it gives them a sense of agency and control (Pekrun, 2006). Thus, students may evaluate deliberately retrieved feedback more positively than system-administered feedback, which might also encourage students to study the feedback more attentively. However, missed learning opportunities due to the nonretrieval of feedback may potentially pose a threat to the well-documented formative benefit of automated, immediate feedback (Mertens et al., 2022). This experiment compares the impact of system-administered vs. optional task-level feedback in an assessment. We assumed positive effects of the feedback-related choice option on student motivation and emotion but also that the nonretrieval (vs. system-administered feedback) may have selected negative effects (e.g., unpleasant emotional state, lower transfer performance).

Method
In this preregistered experiment, 160 undergraduates worked on a computer-based geometry assessment with 20 tasks while either receiving system-administered, immediate elaborated feedback (n=80) or choosing after each task whether they wanted to retrieve the feedback (n=80; choice group; see Figure 1). Students repeatedly rated their control-value appraisals and emotions during the treatment test and the system logged the time spent on the feedback, the feedback-retrieval behavior in the choice group and the posttest transfer performance.

Results
Mixed-effects analyses showed that compared to the system-administered group, the choice group did not react more positively to the feedback cognitively (no significant differences in transfer learning and time spent on feedback, p≥.175), emotionally (no significant differences in positive/negative emotions, ps≥.122) nor motivationally (no significant differences in control-value beliefs, ps≥.312) but students in the choice group showed very high feedback-retrieval rates of .90 (see Figure 2). Further analyses showed that the nonretrieval of feedback in the choice group was associated with a less pleasant emotional state compared to the system-administered feedback (ps<.010; Table 1). Data indicate that the presence of a choice may not substantially improve students’ reactions to the feedback itself, but that it may be worth further investigating the consequences of feedback nonretrieval.

Significance
This study provides valuable insights indicating that implementing an option not to retrieve task-level feedback may not lead to substantial differences in student outcomes. Nevertheless, for educators, it is crucial to consider the negative emotional states associated with feedback nonretrieval, as this may serve as indicators of potential disengagement among students. Test developers should be mindful of these implications when designing assessment strategies.

Author

• Livia Kuklick, Humboldt University - Berlin