Evaluating Validity Evidence and Measurement Invariance of the Elementary School Climate Assessment Instrument Across Five Caribbean Countries

• In Event: Transforming the Future of Educational Research: RISE Caribbean

Abstract

The Elementary School Climate Assessment Instrument - Student (SCAI-E-S, version 3.1) is an elementary student version of School Climate Assessment Instruments (SCAI) developed by the Alliance for the Study of School Climate (ASSC) (California State University–Los Angeles). SCAI-E-S measures students’ perceptions of their school environment and is utilized by schools to “identify needs and conceptualize and evaluate interventions” (Voight & Hanson, 2012). To use this instrument for educational research in the Caribbean contexts, it is imperative to evaluate its psychometric properties with student samples from the region because a tool to hear voices of students in the Caribbean and collect their perspectives about schools should be “about what exists in local spaces” not “bringing a Western perspective to bear on these spaces” (Warrican, 2020, p. 145).
The purpose of this study is twofold. Given the lack of school climate assessment instruments specifically designed for use in Caribbean educational studies, we examine validity evidence of the SCAI-E-S for the Caribbean students and schools. Furthermore, because cultures and educational systems differ across Caribbean countries, this study tests measurement invariance (MI) to ensure SCAI scores can be constructed and compared across Caribbean countries.
The data in this study include 1222 Grade-5 students from 74 primary schools across five Caribbean countries (Antigua and Barbuda, Barbados, Grenada, St. Kitts and Nevis, and St. Vincent and the Grenadines). The data were originally collected as part of an empirical study that investigates the factors related with academic performance of students in the Caribbean (Warrican et al., 2020), and sampling and data collection procedures are detailed in their report. The SCAI-E-S consists of 30 items with three response categories each.
For the purpose of this study, we conduct multilevel confirmatory factor analysis (ML CFA). In addition to the multilevel structure of the data (students nested within schools), the construct of SCAI-E-S is conceptualized as multilevel: student perceptions of school climate at the student level and aggregated school climate at the school level. Using ML CFA (Kim et al., 2016) we examine the underlying factor structure of SCAI-ES and assess reliability at both student and school levels. To test MI across five Caribbean countries, we use Alignment Optimization (Asparouhov & Muthén, 2014) which allows researchers to examine MI and compare means across multiple countries.
If the results of the present study support the validity of the SCAI-E-S scores for Caribbean student populations, researchers will have a valuable tool to facilitate educational research in the Eastern Caribbean region for evidence-based practices and decision-making. The collaboration of international partners via an international research practice partnership (Weinstein & Worrell, 2016) to validate this tool also presents opportunities for clarifying how ‘truth’ can be more clearly made visible for enhancing educational change in and beyond the Caribbean.

Authors

• Eunsook Kim, University of South Florida
  Presenting Author

• Robert F. Dedrick, University of South Florida
  Presenting Author

• Erin Anne Mahon
  Presenting Author

• Constance V. Hines, University of South Florida
  Presenting Author

• Coreen J. Leacock, University of the West Indies - Cave Hill
  Presenting Author

• S. Joel Warrican, University of the West Indies
  Presenting Author

• Patriann Smith, University of South Florida
  Presenting Author