Using the Rasch-Guttman Scenario (RGS) Framework to Develop a Sociocultural Competence Measure for Dual Language Students

• In Event: Promoting and Assessing Sociocultural Competence in Dual Language Bilingual Education Programs

Abstract

Objectives
This paper introduces the Dual Language Assessment of Sociocultural Competence (DL_ASC), a measure designed to assess sociocultural competence (SCC) among dual language bilingual education (DLBE) students in grades 3-6. We describe the different phases of measure development using the Rasch-Guttman Scenario Scales (Ludlow et al., 2020).

Conceptual Framework
The DL_ASC is based on the Reference Framework of Competences for Democratic Culture (RFCDC, COE, 2016) and the Learning for Justice Standards (Southern Poverty Law Center, 2017), supplemented by relevant literature in areas where there are gaps, particularly with regard to critical consciousness, culturally sustaining pedagogy, anti-racist pedagogy, and decolonizing pedagogy.


Methods and Data Sources
To develop likert-type items for this measure, we reviewed the bodies of literature noted above to identify various dimensions of SCC. The cyclical process resulted in the research team shifting away from the traditional classical test theory approach (Crocker & Algina, 2006; Furr & Bachrach, 2014; McCoach et al., 2013) to the Rasch-Guttman Scenario (RGS) framework (Ludlow et al.; 2014, 2020) to create a developmentally appropriate measure for the target population. RGS scales require the creation of scenarios that include descriptions of lived experiences with hierarchical progressions (Ludlow, 2020). Thus, the RGS framework is preferable in this context, both because it promotes the use of contextualized scenarios as prompts and because it allows for the possibility of showing changes over time.
After following Ludlow et al.’s (2020) six- steps for developing scenario-based items, we invited 15 scholars and practitioners with expertise in SCC and intercultural competence to review the scenarios and provide their opinions about the facet and level of progression of each item. We also asked DLBE teachers for their feedback, and also carried out cognitive labs with English-dominant and Spanish-dominant DLBE students in grades 3-6. We finalized the scenarios based on all of this feedback, then translated the scenarios and instructions to Spanish. We also audio-recorded the scenarios in English and Spanish to ensure that all DLBE students in Spanish-English programs could access them in either language.

Findings
Key findings and outcomes to date include 1) the four facets of SCC: identity affirmation, appreciation for diversity, interpersonal collaboration, and civic-mindedness; 2) the final scenarios and progression levels that comprise the pilot version of the measure; and 3) lessons learned in shifting measurement development frameworks mid-way through the process in order to create a more robust measure that will yield more accurate results.

Scholarly Significance
SCC is important in understanding how culture contributes to human development and learning (Author, 2010). Within DLBE in particular, it is one of the three major goals, yet there is no existing measure to assess it. This paper describes the complex and non-linear process of creating a measure for such a complex construct.

Authors

• Aarti P. Bellara, Western Carolina University

• Elizabeth R. Howard, University of Connecticut

• Sandra S. Silva-Enos, Providence College

• Elena Sada, Boston College

• Dominique Galvez, University of Connecticut

• Manuela Wagner, University of Connecticut