Behavioral Indicators of Learning and Emotional Regulation in Small Groups: Instrument Development and Diagnostic Information

• In Event: Student Coregulation in Small-Group Learning

Abstract

Introduction
The promotion of cooperative group learning as an instructional method foregrounds questions of student learning and emotional regulation in small groups. Current research suggests the importance of social and motivational dynamics in self- and co-regulation processes involved in classroom learning (Boekaerts & Corno, 2005; Volet, Summers & Thurman, 2009). Observation instruments have been designed to document student learning regulation behavior in the classroom; however, these seldom focus on regulation in small groups. This paper presents development and reliability information for two observation systems for student regulation in small groups.
Instrumentation
The systems presented here were designed within a self- and co-regulated learning theoretical framework. Their purpose is to identify observable behavioral indices present in small groups that are indicative of student regulation processes. The Group Behavior Checklist (GBC) is a low-inference system that includes items to capture student verbal behavior. GBC items are divided into seven categories: Task Structure, Communication Pattern, Tone, Content, Questions, Interpersonal Press and Help Seeking. The presence or absence of each item is coded in 30 second intervals. The Group Environment Summary Sheet (GES) is a moderate-inference system that includes items to capture overall group functioning and affective environment.GES items are divided into five categories: Task-related Behavior (including leadership dynamics), Group Tone, Group Affect, Coping Behaviors, and Health Statements. The presence or absence of each item is recorded at the midpoint and end of each lesson for each group.
Inter-Rater Agreement
As recommended by Herbert and Attridge (1975), inter-rater percent agreement was calculated to determine the clarity of the structure and procedures of each system, as a measure of both observer training and bias and the ambiguity of the observed events. Five staff members individually coded approximately six lessons each; two coders were assigned to each of the 22 lessons resulting in ten coder pairs.
The GBC and GES include primary and subordinate codes, thus, three forms of percent agreement were calculated: overall percent agreement (primary and subordinate codes), primary codes percent agreement, and subordinate codes percent agreement (see Table 2).
Table 2
GBC
Overall GBC Primary GBC Subordinate GES
Overall GES Primary GES Subordinate
90.13 90.93 89.19 82.10 84.30 71.75

All codes except for the GES subordinate codes reached at least 80% inter-rater agreement. For the purpose of system diagnostic information, this indicates adequate clarity of the systems and procedures, sufficient coder training, and suggests the observed events were distinct enough to be captured consistently.
Percent agreement was also calculated by item. Several items were identified as “trouble” items for coders. These items were reviewed and determined to be difficult because of the level of inference required to code them. Items identified as “trouble” items are being considered for removal from future revisions of the GBC and GES. However, the validity of both systems is substantiated by the four other poster session papers. The presentation of the small-group observation systems will include examples of code development and discussion of observation difficulties.

Authors

• Ruby Inez Vega, The University of Arizona

• Erin Elizabeth Anderson, University of Arizona

• Lauren Ballard, The University of Arizona

• Christine Nicole Calderon, The University of Arizona

• Angela M. Labistre, The University of Arizona, Tucson, AZ

• Mary M. McCaslin, The University of Arizona