Adaptation and Psychometric Validation of a Measure of Organizational Conditions for Infant-Toddler Center-Based Settings

• In Event: Nurturing Early Childhood Teacher Well-being and Teaching Quality Through Essential Organizational Conditions and Leadership

Abstract

New measurement capacity and research in early childhood education is accumulating evidence that leadership and organizational conditions may be central to successful teaching and learning in preschool programs. As the field continues to understand effective infant-toddler education practices, attention to these organizational drivers of implementation may be critical for accelerating improvement. The current study sought to scientifically adapt existing preschool teacher/staff and parent surveys of organizational conditions and test psychometric validity for infant-toddler settings.

Setting and Population: All survey adaptation and pilot test data collection took place in four Early Head Start-Child Care Partnership grantees and programs located in California, Colorado, Florida, and Washington, DC.

Research Design and Data Collection:
Adapting the existing preschool surveys for infant-toddler settings required an iterative development and testing process; we used qualitative and quantitative feedback from one cycle to inform the next phase. Survey development includes the creation of measures, with corresponding items that comprise them, followed by analysis using the Rasch model (Wright & Masters, 1982).

Measure and Item creation: The authors and research partners relied on the existing preschool survey measures and items to determine whether existing items fit into one of three categories: (a) items that were appropriate as-is for the infant-toddler context, (b) items that needed language and terminology revisions for infant-and toddler practice, and (c) items that were irrelevant and, should be discarded. Then, we identified constructs and items that are not represented and, therefore, should be created.

Cognitive interviews. To test our adapted and newly generated items, we conducted cognitive interviews. Cognitive interviewing is a method of pre-testing, involving one-on-one interviews with individuals in the target population. Cognitive interviews ensure items are easy to understand, that question stems and answer choices are unambiguous, and that questions asked are being interpreted as intended (Conrad & Blair, 2004). A total of 88 cognitive interviews were conducted with infant-toddler teachers, staff, and parents from EHS-CCP grantees and programs in CO and FL. Cognitive interview data was analyzed and used to inform revisions to measures and items.
Pilot testing and Rasch analysis. Both surveys were translated into Spanish, Arabic, and Amarhic, and then administered online using Qualtrics summer 2020 to 500+ teachers/staff and 1,100+ parents across the EHS-CCP grantees and programs in CA, CO, FL ,& DC. Rasch measurement is a model that creates unidimensional measures, measuring how people feel on a single, unitary concept.

Research Questions: The following research questions were used to guide the psychometric validation study of the adapted surveys, particularly whether the measures were reliable and differentiated among survey-takers:
• Do responses to questions within a measure match the conceptual difficulty of those questions?
• Do questions differentiate among teachers/staff and parents who have different levels of belief or experiences?
• Do questions within a measure tap into the same underlying construct?
• Are the survey measures reliable?

Results:
Rasch analyses will be presented emphasizing findings of item difficulty, person separation, scale reliability, dimensionality, item fit, differential item functioning. Tables and Wright Maps will facilitate interrogation and discussion of psychometric findings.

Group Authors

Debra Pacchiano, PhD and Isabel Farrar, MA, The Ounce of Prevention Fund; and Joshua Stewart, PhD and Marc Brodersen, PhD, Marzano Research.

Authors

• Debra Pacchiano, Start Early
  Presenting Author

• Isabel Farrar, Start Early
  Non-Presenting Author

• Joshua Stewart, Marzano Research
  Non-Presenting Author

• Marc Brodersen, Marzano Research
  Non-Presenting Author