Where the Rubber Meets the Road: Connecting Policy Guidance with Observed Data

• In Event: Oklahoma Pre-K: A State-Level Look at Using Research to Support the Policy-to-Practice Process

Abstract

Despite much discussion of the importance of research-policy-practice partnerships, links between these sectors remain fragmented. Yet, as policymakers continue efforts to formalize expectations and leverage data for continuous program improvement, the possibility for high-impact learning is tremendous if connections can be forged and optimized. The current paper describes one such effort to crosswalk research data with a rubric for Early Childhood Programs of Excellence currently under development at the Oklahoma State Department of Education. Most likely, school leaders will not engage in the detailed and time-intensive level of observation conducted by the research team, but this crosswalk of the rubric with existing observation measures can identify potential ways to operationalize key aspects of expectations, consider where frequency counts or other guidance can help administrators evaluate the extent to which they meet the rubric expectations, and to highlight components for which school leaders may need additional supports to meet expectations.

The data for this presentation are drawn from a longitudinal research study in Tulsa, OK. Children (n = 1,087) were recruited beginning at age 3 and participated in Head Start or school-based Pre-K in the 2017-18 school year. All participating classrooms (n = 128) were observed one morning during winter of 2018 using the Narrative Record (Farran et al., 2017), the Adapted Teaching Strategies Rating Scale (Morris et al., 2014), and the Classroom Assessment Scoring System (Pianta et al., 2008). This abstract focuses on the Narrative Record, and the final presentation will extend this work to other observation measures to compare the types of information available using these methods and to highlight feasibility of these approaches for school leaders’ self-evaluation efforts. The Narrative Record utilizes a mixed-methods coding scheme, in which observers collect data segmented into “episodes,” with a new episode coded when the Activity Type changes (e.g., whole group to small group) and/or the Content of Instruction changes (e.g., Math to Reading Literature). Observers code numerous teacher and student behaviors throughout each episode, and codes are aggregated across episodes (m = 29 episodes per classroom).

Relevant observed items and subscales were selected to assess the extent to which observed classrooms meet the expectations outlined in the Program of Excellence Rubric. Specifically, features of the rubric describing curricular alignment to standards, quality of instruction, and quality of the learning environment were operationalized using items from the research observation tools, and an illustrative set of codes and descriptive data are provided in Table 1. Overall, results highlight that pre-K classrooms vary widely in the experience provided and how time is allocated across mode of instruction and content that is included. Many classrooms are currently meeting the expectations of providing integrated instruction with many episodes of instruction covering mixed content (e.g., literacy and social studies topics concurrently). Some concerning patterns arose, including classrooms in which students were in transition more than half of the time observed and high levels of behavior disapprovals.

Discussion will focus on how researchers and policymakers can synergistically partner to mutually improve one another’s work and maximize impact.

Authors

• Sherri L Castle, University of Oklahoma
  Presenting Author

• Diane M. Horm, University of Oklahoma-Tulsa

• Anna D. Johnson, Georgetown University