Assessing Middle School Students' Feedback and Revision Behavior in a Computer-Based Task

• In Event: 29.079 - Investigating Performance Tasks as Measures of Social and Emotional Learning Skills

Abstract

Background and Objectives
One of the main goals of education is to prepare students to learn on their own. However, educational assessments rarely focus on the processes and strategies that students use to learn. We describe a new type of assessment of how students behave when given the opportunity to learn from feedback. This measure, Posterlet, collects evidence on students’ self-management competencies, in particular, how they regulate their behavior when given the chance to hear constructive feedback and revise their work. Choosing more constructive feedback and revising in Posterlet have been associated with students’ achievement (Cutumisu, Blair, Chin & Schwartz, 2015). Here, we ask whether students who initially perform better on their posters bring a different set of behaviors to their tasks. If so, results would suggest that instructors, particularly those interested in enhancing SEL competencies related to feedback and revision, should attend to students’ performance early in the learning process.


Assessment Environment: Posterlet
Posterlet is a choice-based digital assessment game in which players design posters and learn about graphical design principles through feedback. The game offers students two main choices. First, students choose between positive and constructive feedback on their posters. Second, students choose to revise their posters based on feedback. The game also scores the quality of the posters.

Data Sources and Method
In a correlational study, nearly five hundred middle-school students from New York City and Chicago played three rounds of the Posterlet game, in which they designed a poster on each round and then completed a post-test that assessed their knowledge of graphical design principles.

The first round of the game was largely exploratory, with students becoming familiar with the choices available in the game. Thus, we use students’ score on their first poster as a pretest score. We tested whether scores on the pretest could predict students’ behaviors on the subsequent two rounds of the game. Specifically, in each of the schools, two groups of students were identified: low performers (those who scored lower than the median on their pretest) and high performers (those whose pretest scores were higher than the median). We then compared behavior of high performers and low performers on the last two rounds of the game.

Results
For each school, results indicated that high performers chose to revise significantly more than low performers on each of the last two rounds of the game, and they performed better on the poster designs and on the posttest of their knowledge of graphic design principles. Additionally, the pretest predicted students’ choices to seek critical feedback on the combined last two rounds of the game.

Significance
These findings have implications for the design of learning technologies used in assessments of students’ academic, social and emotional competencies. Assessments that provide both behavior metrics (e.g., the number of times students’ choose to seek critical feedback and to revise their posters) and performance metrics (e.g., the quality of students’ posters) can be used by instructors to identify potentially struggling students early in their learning process.

Authors

• Maria Cutumisu, University of Alberta

• Daniel L. Schwartz, Stanford University