Experimental Design of the AMSTI (Alabama Math, Science, and Technology Initiative) Evaluation

• In Event: Evaluating Alabama’s Math, Science, and Technology Initiative: Results of a Three-Year, Statewide Randomized Experiment

Abstract

This presentation will briefly outline the major characteristics of the experimental design and timeframe. The study is a group randomized control trial, with paired randomization at the school level. The research program consists of two cohorts of schools, a group of 40 that started fall of 2006 and a second cohort of 42 who start fall of 2007.Schools were randomized within each cohort. The schools were randomly selected from pools of eligible schools prior to the two week training sessions that occurred in the summer of 2006 (for cohort 1) and 2007 (for cohort 2). The program schools received a two week summer institute (over two years) and extensive equipment and materials for hands-on math and science instruction. The control schools received the AMSTI training and materials in the second year. Data was collected at the school, teacher and student levels at grades 4 through 8 (grades in which students are tested in math and reading; science tests are given only in grades 5 and 7). Data from the two cohorts were pooled and analyzed together. The full sample included 82 schools with about 780 teachers and 30,000 students.

Key research questions concerning impact are:
• What is the impact of AMSTI on student achievement in mathematics, science and reading after one year?
• What is the impact of AMSTI on classroom instructional practices, teacher reported level of content knowledge, and teacher reported student engagement after one year?
• Do the one-year impacts of AMSTI on student achievement vary for specific subgroups of students?
• What is the impact of AMSTI on student achievement in mathematics and science after two years?

Author

• Denis Newman, Empirical Education Inc.