Characterizing and Understanding Contextual Variation in Learning From Providing and Receiving Feedback

• In Event: Advancements in Feedback Research: Beyond Perceptions to Learning Impact

Abstract

Objectives and Framework
Meta-analyses have established that students learn from peer feedback, (e.g., Double et al., 2020), but with large, unexplained heterogeneity of effects. In addition, there are separate learning opportunities in providing and receiving feedback. When experimentally examined, larger benefits were obtained from providing feedback (Lundstrom & Baker, 2009). Here we share two lines of work trying to characterize the varied benefits of providing and receiving and then understand their mechanisms from a cognitive and behavioral engagement perspective.

Methods
For the first part, we use a massive dataset (20,879 students enrolled in 243 courses in many different disciplines at 76 institutions) of student peer feedback using a shared online peer feedback system. With a given assignment, we use multiple regression to characterize the relationship between the amount of or helpfulness of provided or received comments and the amount of growth in performance seen into the next assignment. Across the 505 assignments, we apply meta-regression to capture variation in effect sizes.
For the second part, we use qualitative analysis of extreme cases of particularly high levels of learning from providing or receiving with particularly low levels of learning from providing or receiving. The cases were selected to make sure there were no confounds with type of assignment, discipline, or course (e.g., similar courses in similar institutions). A random sample of longer comments are taken from each selected assignment, and then systematically coded for both contents (i.e., a focus on high vs. low level issues) and features (e.g., inclusion of praise, criticism, suggestions, and explanations). Then Hierarchical Linear Modeling was applied to the patterns of comments across selected assignments to determine predictors of differences in learning effects.

Results
For providing, detailed comments on fewer issues were associated with the most learning, in contrast to shallow comments on many different issues (see Figure 1). In addition, relative focus on lower-level issues predicted lower learning.
For separately cases of high vs. low levels of learning from receiving, the pattern was quite different in terms of what was more common in high-learning cases: 1) form-focused comments were 1.4x more common; 2) general suggestions for high-level issues (but not detailed suggestions) were 1.2x more common; and 3) including questions were 1.6x more common.

Significance
The current study provides insights into learning from providing vs. receiving at the micro-level. These findings also clarify why meta-analyses have been unsuccessful in characterizing variation: it is not about variation in context, but rather about variation in the substance of feedback, which is influenced by micro-structures.

Authors

• Christian D. Schunn, University of Pittsburgh

• Qiuchen Yu, Central China Normal University

• Jing Yang, Dalian University of Foreign Languages