An Examination of Implementation Costs: Is Affordability a Barrier to Using ASSISTments to Improve Learning?

• In Event: 54.028 - Efficacy of an Online Mathematics Homework Intervention: Rigorous Research Transforming Knowledge to Action

Abstract

Objective
We examine affordability by comparing ASSISTments implementation costs to the cost of business as usual homework practices.

Perspective
To make good decisions about implementing particular instructional interventions, educators need to understand the likely costs. Reliance on effectiveness alone may encourage adoption of interventions that are too costly to sustain with fidelity (Harris 2008; Hollands et al 2014; Bakia et al 2011). Analytic approaches that examine costs are relevant in context of rising costs in education and decreasing educational budgets (Hollands et al 2014; Bowen 2013).

Methods
Costs for ASSISTments were estimated using an “ingredients approach” to systematically identify required resources associated with the adoption of an intervention, regardless of the source of funding (Levin and McEwan 2001). Development costs of the existing platform were not included. Likewise, the infrastructure that supports individual student use of computers outside school, such as the availability of Internet and computers, is outside scope.

Ingredients were valued using economic data about salaries and prices, to estimate annual implementation costs at a school and per student unit of analysis. Data were collected and analyzed for two conditions: those using ASSISTments and those in the control group. T

Data sources
Descriptive information about a “typical” implementation of ASSISTments was collected through interviews with evaluators and ASSISTments developers. Interview protocols asked about activities likely to influence total and per student costs, such as planning and administration, professional development and technical support. Other program details such as class size and scale of ASSISTments implementation were also collected. State salary data was collected from the Maine Department of Labor’s 2015 Estimate for: Education Administrators, Elementary and Secondary School.

Results
The adoption of ASSISTments by schools for this study required administrative planning and training for teachers. Staff time accounted for most of the cost. Each school invested on average $1,100 worth of staff time to prepare for the use of ASSISTments. This includes ODCs for participation in a 2.5 day training workshop each year.

Available data based on site visits suggest that the amount of preparation time required by teachers does not change with the adoption of ASSISTments. While teachers may save time grading student homework problems when using ASSISTments, they use this time to review data reports generated by ASSISTments.

ASSISTments also provided limited coaching support to teachers during the school year, estimated at about ½ day per teacher over the course of the year.

We found that ASSISTments implementation requires about $15 per student per year, if current professional development practices (2.5 days per year) are maintained for as long as ASSISTments is used by a district.

Significance
Cost is a practical consideration that can dramatically shape how knowledge about effective practices gets translated into action in districts. Teacher time associated with professional development represented about ⅔ of the costs identified for this study. Future studies can examine the intensity of PD required to sustain the intervention over time.

Authors

• Marianne Bakia, SRI International

• Mingyu Feng, WestEd