Teacher Professional Development and Its Effects on Students: Evidence From the [email protected] Program in Italy

• In Event: The Effectiveness of Italian Educational Policies Toward Reaching the ET2020 (Education and Training 2020) European Objectives in Education and Training

Abstract

1. Objectives

The [email protected] teacher professional development program in Mathematics aims at raising student results in the four regions of Sothern Italy characterized by the lowest math achievement.
The question is to understanding whether the [email protected] program makes a measurable difference in promoting student achievement. The study answers this question through random assignment (at school level) evaluation design.


2. Theoretical framework

The study focuses on the relevance of professional development and rules out issues resulting from hidden heterogeneity by designing ex ante a randomized experiment. This approach allows us to objectively compare the results of two statistically-equivalent groups, to test whether teacher professional development plays a role in maths achievement, and to test whether this kind of policy could help filling the achievement gap of the southern regions


3. Methods

The paper presents the design and the results after the first two years of activity of an ongoing 3-years study on the effectiveness of the [email protected] program. . The schools, enrolled on a voluntary basis for the 2009/10 school year, are randomized into two groups: one group receives immediately the specialized training,, the other group is delayed admission into treatment for one year.
The initial sample includes 175 schools and 666 teachers. The control-by-waiting group comprises about 50 schools, 184 classes, and roughly 4,000students. The immediate-treatment group comprises roughly 100 schools, 350 classes, and 7,000 students. The design allows detecting a minimum detectable effect size of 0.20 with a power of 0.8.
Effects on student will be measured over three years by using standardized achievement tests produced by INVALSI (Italian National Institute for Educational Evaluation).


4. Data sources

- Standardized tests in Mathematics and background student information from the INVALSI yearly national assessment;
- school administrative information from the Ministry of Education;
- teachers’ background information collected via CATI on treated and controls;
- pre/post teacher questionnaire investigating the degree of teacher involvement in the program and use of [email protected] experience and their believes, attitudes and practices;
- qualitative information on the program from in-depth interviews to a selected group of teachers and the analyses of their log-books;


5. Results and/or substantiated conclusions or warrants for arguments/point of view

So far, the evidence is mixed. Although the program doesn’t appear to have affected overall students’ math achievement in the first year (which is consistent with the literature), we are finding evidence indicating that [email protected] has been more effective among students from schools with the lowest levels of math achievement. Moreover, we are finding slight evidence that this effect could be due to the increase of math achievement among students with better math's marks before the treatment.
The analyses over the 2-year span shall allow developing these initial conclusions.


6. Scientific or scholarly significance of the study

The study brings new quantitative evidence on the effectiveness of a large scale professional development program, and is the first random assignment evaluation of a large scale program ever conducted in Italy.

Authors

• Gianluca Argentin, Università di Milano-Bicocca/INVALSI

• Aline Pennisi, Ministry of Finance

• Daniele Vidoni, Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione