A Systematic Review of Evaluation Methods in Studies of Educational Technology in Mathematics

• In Event: AERA Roundtable Session Friday 9:50 am Four Seasons Ballroom 2-3
  In Roundtable Session: Design, Evaluation, and Impact of Technology-Enhanced Learning Environments in Education (Table 13)

Abstract

As educational technology programs continue to grow in number, research methodologies that are both rigorous and practical are needed to evaluate such programs in this saturated “ed tech” space. This study presents results from a systematic review of ed tech math evaluation studies to identify and describe current evaluation practices and to ascertain best practices in the evaluation field. This study screened all studies of ed tech and math published since 2000 and analyzes the methods of the 163 which fit the inclusion criteria. Important findings included the small percentage of study RCTs fully meeting WWC Standards Without Reservations/ESSA Tier 1 criteria and QEDs fully meeting WWC Standards With Reservations/ESSA Tier 2 criteria. Program implementation and duration also varied widely.

Authors

• Ashley Anne Grant, Johns Hopkins University

• Jennifer R. Morrison, Johns Hopkins University

• Alan Reid, Johns Hopkins University

• Joseph M. Reilly, Johns Hopkins University

• Michael Cook, Johns Hopkins University