Accounting for the Multidimensional Nature of Feedback: Latent Measures of Quality of Teachers' Feedback

• In Event: Channels of Feedback: Examining Sources, Types, and Contingencies

Abstract

Objectives and Theoretical Framework
Researchers have proposed models, taxonomies, and feedback coding systems that describe different types of feedback and help us to better understand links between different feedback messages and students’ responses to them. Unlike previous studies, that classified comments according to a specific taxonomy and analyzed the distribution of various categories in terms of frequency of usage, the current study takes into account the multidimensional nature of feedback comments, which can be classified into multiple categories. We attempted to estimate latent variable for in-depth exploration of the quality of feedback constructs. Hence, the purpose of this study is to (1) examine the underlying dimensions of feedback comments derived from classification of feedback written message in a sample of pre-service teachers , and to (2) explore the relationship between task and individual characteristics on type of feedback message.
Method and Data Sources
The sample included 255 Master's degree students in Secondary Education Teacher Training from a Spanish public university (64% female). Participants completed an online protocol and were asked to provide comments and marks on two essays with different level of student performance the same way they would to their own students in a class. Essays were examples from a national standardized test in writing communication. Participants completed the Receptivity to Instructional Feedback scale (Lipnevich et al., 2021), the Big Five Personality Inventory (Goldberg, 1993), and reported their demographics and academic background information.
1835 feedback comments were classified into twelve subcategories by two researchers (See Table 1). The agreement was above 85%, and inter-rater reliability of .66 (Cohen's Kappa). Internal consistency reliability for RIF ranged from 0.77 <α<0.86. And for BFI 0.69 <α<0.86.
Factor Analysis (EFA and CFA) were fitted to examine the factorial structure of the comment classification. Scores for individual factors were estimated using an IRT – 2PL model. Hierarchical Linear Model was used to estimate the variability of participants' latent variables, and a stepwise regression was fitted to predict the effect of participant’s characteristics on each latent variable score.
Results
Analysis suggested two factors that emerged from a classification of feedback comments. Therefore, the taxonomies can also be useful for estimating latent variables about the quality of feedback. These factors can be arranged along a continuum, instead of limiting the analysis to a unidimensional perspective, suggesting both a dynamic interrelation between content and function of comments (Lipnevich & Panadero, 2021) and the presence of patterns in how teachers provide written feedback messages. Additionally, results indicated that individual characteristics could play an essential role in explaining differences in the quality of feedback messages. Specifically, cognitive engagement, conscientiousness, and gender were the stronger predictors of feedback message factor scores (See Tables 2 and 3)
Scientific Significance
The results can help practitioners design optimal feedback and further our understanding of the role individual characteristics play in providing feedback. In sum, initial exploration suggested a promising route for future studies in this field that could also help explore how comments can trigger student cognitive and non-cognitive outcomes when receiving feedback.

Authors

• Carolina Lopera-Oquendo, Graduate Center - CUNY
  Presenting Author

• Anastasiya A. Lipnevich, Queens College - CUNY
  Non-Presenting Author

• Ignacio Mañez, University of Valencia
  Non-Presenting Author