Content Validation of the Elementary School Climate Assessment Instrument for Use in the Caribbean

• In Event: Deconstructing Inequities and the Marginalization of Learners in Caribbean Educational Systems

Abstract

Researchers interested in making valid interpretations from a questionnaire developed and validated in one cultural context, but now being used in a different cultural context, need to evaluate the appropriateness of the questionnaire in the new context. In this paper, we first describe the evaluation process employed to assess the appropriateness of using The Elementary School Climate Assessment Instrument - Student (SCAI-E-S, version 3.1), which was originally developed in California (United States) by the Alliance for the Study of School Climate, in four countries in the Eastern Caribbean. Secondly, we present the results of the evaluation process that address the content validity of the school climate measure for use in the Eastern Caribbean. Our evaluation process was motivated by the Standards for Educational and Psychological Testing (2014) and consisted of a review of the items by a 12-member expert panel of educators from the Eastern Caribbean, and cognitive interviews conducted with eight children, ages 9-12 years, from the Eastern Caribbean.



Beginning with an expert panel review, a Qualtrics online survey was used to collect twelve experts’ ratings on each of the 28 items and response scales on the climate measure. Experts were asked to rate the relevance of the items for the Eastern Caribbean context (1=not relevant to 4=very relevant) and the clarity of the items (1=not clear to 4=very clear). Experts also provided comments for each item that raised concerns. The content validity index (CVI; Polit & Beck, 2006) was computed for each item for relevance and clarity. The CVI is the proportion of experts providing ratings of 3 or 4 (acceptable ratings). Values for the CVI > .80 are considered acceptable. Results from the expert panel review showed that only one item had an unacceptable CVI for relevance (item 27, CVI=.73); 17 items had CVIs of 1.0, 6 items had CVIs of .92, and 4 items had CVIs of .83. Similar CVIs were found for the clarity of the items. Item 27 was identified as unclear (CVI=.33) with the remaining items having acceptable CVIs: CVI = 1.0 (15 items); CVI = .92 (15 items); CVI = .83 (3 items); and one item had a CVI = .82.



Cognitive interviews were conducted by educators from the Eastern Caribbean and included think-aloud and probing methods (Willis, 2005). The cognitive interviews were designed to determine whether the children: (a) understood the survey directions, (b) understood the language used in the survey, and (c) felt that answering the questions was easy or difficult.



To use a measure like The Elementary School Climate Assessment Instrument for educational research in the Eastern Caribbean context, it is necessary to evaluate its content validity with experts and students from the region. The present paper has illustrated two methods (expert review and cognitive interview) that can be used to evaluate the appropriateness of the questionnaire for the Eastern Caribbean context. Results from these methods can also be used to provide insights into the quantitative analyses conducted on the measure.

Authors

• Crescentiana Alfred, University of the West Indies - Cave Hill

• Robert F. Dedrick, University of South Florida

• Constance V. Hines, University of South Florida

• Eunsook Kim, University of South Florida

• Verna Charleen Knight, University of the West Indies - Cave Hill

• Erin Anne Mahon, University of the West Indies - Cave Hill

• Nitha Mauricette-Philip, University of the West Indies - Cave Hill