Virtual Algebra Study: Study Design and Methodology

• In Event: Access to Algebra I: The Effects of Online Mathematics for Grade 8 Students

Abstract

This paper describes the study design and methodology. Schools were eligible if they did not typically offer a full section of Algebra I to grade 8 students as of the 2007/08 school year. In June 2008, a total of 68 mostly rural middle schools in two states were randomly assigned to treatment and control conditions. The average number of “algebra-ready” (AR) students in the participating schools was 6.5. In all cases, the identification of AR students occurred prior to random assignment.

Schools assigned to the treatment condition received the online Algebra I course to offer to their AR students during the 2008/09 school year. Control schools implemented their usual mathematics curriculum.

Outcome data were collected from two distinct student samples: 1) 440 AR students who attended the participating schools, and 2) 1,445 N-AR students who were in grade 8 in the participating schools in 2008/09, and had not been identified as “AR” the spring before random assignment.

The study addresses two primary research questions:

• What is the impact of offering an online Algebra I course to AR students on their algebra achievement at the end of grade 8? Algebra achievement was measured with a computer-adaptive assessment administered in spring 2009.

• How does offering an online Algebra I course to AR students affect their likelihood of participating in an advanced course sequence in high school? Advanced coursetaking was measured by collecting and coding AR students’ actual grade 9 and planned grade 10 course information in spring 2010. Based on prior research on course sequences, the high school course data were coded as advanced if the students successfully completed a full-year course above Algebra I in grade 9 with a grade of C or higher, and enrolled in the next course in the sequence for grade 10.

Secondary questions address potential unintended consequences (or side effects) of offering online Algebra I to AR students. They include:

• What is the effect of providing online Algebra I to AR students on their general mathematics achievement at the end of grade 8? General mathematics achievement was measured with a computer-adaptive assessment that drew 30 items for each student from an item back of over 1,000 items ranging in difficulty from grade 5 to grade 8 level.

• What is the effect of providing online Algebra I (to AR students) N–AR students’ algebra achievement, general mathematics achievement, and likelihood of following an intermediate course sequence in high school? N-AR students took the same algebra and general mathematics assessment as did AR students in spring 2009. Intermediate coursetaking was measured by collecting and coding planned grade 9 course names for N-AR students in spring 2009. Planned grade 9 courses were coded as indicating an intermediate course sequence if the course was at or above Algebra I (vs e.g., prealgebra).

The paper will describe in detail sample recruitment, random assignment, data collection and analysis strategies, and the framework for evaluating the overall “success” of the intervention.

Authors

• Jessica Heppen, American Institutes for Research

• Peggy Clements, Education Development Center, Inc.