Effects of Virtual Tutoring and Group Size in K–2 Early Literacy

• In Event: Equalizing Learning Opportunities Through High-Impact Tutoring: Results From Four Randomized Controlled Trials

Abstract

While research demonstrates that high-impact tutoring (HIT) interventions have substantial positive effects on student learning (Nickow et al., 2020), more evidence is needed on the specific features of HIT programs that are most effective in improving student outcomes. For example, much of the evidence on effective tutoring programs are based on in-person delivery, while virtual tutoring programs are increasingly being taken up by districts and schools (Authors, 2023). Additionally, there are mixed results about the relative benefits of one-on-one tutoring compared to tutoring in pairs or larger groups (Neitzel et al., 2022).With this randomized controlled trial, we add evidence on the impact of both virtual HIT and on tutoring group size, both of which have implications for cost-effectiveness and scalability.

We partnered with a virtual tutoring provider and school district in Texas to conduct a factorial, two-stage, stratified randomization at the school x class period level. Across 10 schools in the district 2,085 students were paired, and within each class period within each school, pairs of students were first randomly selected to receive tutoring (N=1,080). The remaining pairs of students were assigned to a business as usual control group and did not receive tutoring (N=1,005). Among the pairs of students selected for tutoring, 510 were randomly assigned to receive one-on-one (1:1) tutoring and 570 were assigned to receive tutoring in pairs (2:1).

Our primary outcomes are reading achievement on the end-of-year (EOY) DIBELS and NWEA MAP assessments. We also collected administrative data on student gender, race/ethnicity, ELL status, indicators for economically disadvantaged and eligibility for special education services, as well as data on tutoring sessions and school attendance. Finally we collected demographic information on tutors.

To estimate the effect of virtual tutoring on student outcomes we use regression models predicting EOY reading scores (DIBELS, MAP) by study condition, and include controls for the beginning-of-year DIBELS scores, student demographics, as well as a fixed effect for the strata. To estimate the effect of group size on student achievement, we restrict our sample to students assigned to tutoring and predict EOY reading scores by assignment to the tutoring model (1:1 or 2:1), controlling for beginning-of-year DIBELS scores, student and tutor demographics, and strata.

Preliminary analyses indicate that, on average, students assigned to receive tutoring performed .05 SD higher on EOY DIBELS scores than students in the Control group (p<.10). However, we find no effect of tutoring on EOY MAP scores. Examining the effect of group size, we find that students assigned to the 1:1 model scored 0.06 SD higher on EOY DIBELS scores compared to students assigned to the 2:1 model(p<.10). We find no effect of group size on students’ EOY MAP scores.

Future analyses will examine effects by grade and subgroups of students, as well as attrition and compliance across study conditions. Results may inform districts’ and schools' decisions about the designs of programs they adopt to equalize access to accelerated learning opportunities.

Authors

• Cynthia Pollard, Harvard University

• Carly D. Robinson, Stanford University

• Sarah Novicoff, Stanford University

• Sara White, Vanderbilt University

• Susanna Loeb, Stanford University