Poster #84 - The TOCA-C: An Efficient Approach to Measuring Children's Social, Emotional, and Behavioral Functioning by Teachers

• In Event: 1-169 - Poster Session 04
  In Poster Session: PS 04 Section - Education, Schooling

Integrative Statement

Introduction: Developmental and intervention researchers alike continue to struggle with efficient yet valid methods for assessing the ways in which social-emotional and behavioral aspects of children’s functioning change and develop over time. In fact, in school-based research, teachers are often asked to rate entire classrooms of students simultaneously on several developmental dimensions over several time points, resulting in a significant burden on these key informants. As such, there is a growing need for valid measures that can be administered efficiently in school settings. The current paper aimed to establish a balance between assessment efficiency, reliability, and the measurement properties of a measure widely used to assess the impact of school-based prevention programs, called the Teacher Observation of Classroom Adaptation (TOCA; Werthamer-Larsson, Kellam, & Wheeler, 1991). Originally used in studies like Fast Track and to evaluate widely-used programs like PATHS and Good Behavior Game, the TOCA has gone through many iterations over the past 25 years (Koth, Bradshaw, & Leaf, 2009). The current study leveraged item response theory (IRT) analyses to create a shortened, more targeted version of TOCA, called the TOCA-C (checklist; Koth et al., 2009), which is both valid and efficient for large-scale use in schools to track development over the course of elementary school.
Methods: We analyzed data from a 39-item version of the TOCA-C (6-point Likert scale) across 7 previously established subscales: 1) Concentration Problems, 2) Aggressive/Disruptive Behavior, 3) Prosocial Behavior, 4) Emotion Regulation Problems, 5) Internalizing Problems, 6) Family Problems, and 7) Family Involvement (Crowder, 2014; Koth et al., 2009). We used IRT analyses to develop a shortened, more targeted version of the scale based on teacher-rating data from 17,456 children in Kindergarten through grade 5 (47.7% female, 54.2% African American, 7.3% Latinx).
Results: The IRT analyses resulted in the retention of 33 of the original 39 items. Andrich’s (1978) rating scale model produced item difficulty estimates reflecting an assessment of multiple locations along the latent trait continuum for all TOCA-C subscales, allowing for a more targeted version. Visual inspection of item maps (Huynh, 1998) revealed similar findings, in that estimated locations of individual items along the latent trait continuum were shown against estimates of person ability scores (see Figure 1). Standardized beta coefficients from confirmatory factor analyses indicated strong, positive relationships for nearly all items (see Table 1). Mean standardized beta estimates for each subscale ranged from .489 for the Emotion Regulation subscale to .781 for the Family Involvement subscale. Lastly, in an effort to identify any potential items exhibiting bias, differential item functioning (DIF) analyses revealed no measurement differences based on gender, race, or grade for 96 of the 99 parameter estimates.
Conclusions: The results suggested that the 33-item TOCA-C is both valid and reliable for use in elementary school settings, and thus could be a helpful tool for developmental and intervention researchers to use in large-scale school-based studies where teachers are asked to rate multiple students (e.g., entire classrooms).

Authors

• Joe Kush, University of Virginia
  Presenting Author

• Catherine Bradshaw, University of Virginia
  Non-Presenting Author