Applying a Four-Step Protocol for Capturing Young Children’s Engagement as a Measure of Implementation Fidelity of a Semi-Structured Block Play Intervention

• In Event: Unpacking the Black Box of Early Interventions through Broadened Measurement of Implementation Fidelity

Abstract

Participant engagement is a key component of implementation fidelity yet is rarely measured in child participants (Darrow et al., 2013). Individual differences in children’s active engagement in an intervention represents a potential point of disconnect between the intervention as designed and as implemented (Berkel et al., 2011), as children who are physically present but unengaged during intervention sessions may lead to decreased efficacy of the intervention as measured by estimates of average impacts. As such, we have developed a four-step protocol for capturing the multi-dimensional and varied engagement of young children in the intervention context. We apply the protocol to data from the pilot of a semi-structured block play intervention for supporting the development of social-emotional and math skills in preschoolers.
Children (N= 59; Mage = 55.20 months) were randomly assigned to either the semi-structured play condition or a business-as-usual control. Twice a week for seven weeks, small groups of children in the semi-structured play condition (n = 24) engaged in 15-minute block play sessions. They were given wooden blocks and provided a building goal at the beginning of the session by the interventionist (e.g., “Today your job is to build a tower together!”), and then allowed to build freely. Over the 14 sessions, the prompts gradually became more complex (e.g., Week five: “Today your job is to build a house… It needs to have four walls, a roof, a way to get inside like a door, and at least two rooms”). Results from the pilot study have found positive effects of the intervention on SEL and math skills, especially for children from low-income backgrounds (Schmitt et al., 2018).
Following our four-step protocol for capturing child engagement (see Figure 1), we 1) identified points of opportunity for child engagement with the intervention and specified multiple dimensions of engagement, 2) operationalized and measured the dimensions of child engagement, 3) analyzed the dimensions of child engagement, and 4) linked the dimensions of child engagement to other variables. Specifically, we determined that children could engage with multiple aspects of the intervention including the block play itself, with their peers in the group play setting, or with the building goals. We coded these dimensions from video by observing the children’s behavior in 1 to 5-minute increments across sessions. Descriptive analyses revealed that children differentially engaged with the three dimensions of engagement (see Figure 2 for heat map across dimensions). That is, although some children were uniformly engaged or disengaged across dimensions, some children were highly engaged with one dimension (e.g., block play) but not with another (e.g., peer interaction). Furthermore, children’s engagement was varied across and even within sessions, highlighting the need to consider within-individual variability when capturing engagement as a measure of implementation fidelity. Importantly, our protocol allows for the measurement of a broadened conceptualization of fidelity that includes children’s engagement, accounting for young children’s autonomy in the intervention context. In doing so, we hope to help unpack the black box of early interventions and determine the most efficacious ways to support children’s early learning.

Authors

• Brianna Devlin, University of Oregon
  Presenting Author

• Tanya Maria Paes, St. Jude Children's Research Hospital - Memphis

• Elyssa A Geer, University of Oregon

• Lindsey Bryant, Purdue University, West Lafayette

• Tracy Zehner, University of Oregon, Eugene

• Irem Korucu, Yale University

• Kathleen Morse

• Robert Duncan, Purdue University, West Lafayette

• David James Purpura, Purdue University, West Lafayette

• Sara Anne Schmitt, University of Oregon