Are Schools Adding Value? Preliminary Results of a System-Wide Program for Promoting Educational Meritocracy in Italy

• In Event: The Effectiveness of Italian Educational Policies Toward Reaching the ET2020 (Education and Training 2020) European Objectives in Education and Training

Abstract

Objectives or purposes

The paper investigates whether the PQM Teacher Professional Development program in Mathematics and Italian bears measurable effects in terms of student achievement and attitudes towards the discipline after one year of implementation.



Perspective(s) or theoretical framework

Effective teacher professional development is one of the key mechanisms for improving student achievement. PQM is a (quasi)-national program financed by the Ministry of Education and EU federal funds involving 620 classes in 310 schools for the Italian program and an equal number for mathematics. In each participating school, two teachers receive 50h of blended training to improve their teaching methodology and didactic planning skills. They receive incentives for socializing practices and information via peer formation and (in Southern Italy) funding for organizing remedial classes for students in the afternoons.



Methods, techniques, or modes of inquiry

The program did not foresee ex-ante any evaluation design. Thus, we estimate of the effect of the first year of the program at the school level ex post by using quasi-experimental techniques.
We are interested in the average treatment effect on the treated (ATT) of PQM on students’ achievements and students’ attitudes towards the disciplines, where the treatment is defined as being enrolled in a school involved in the PQM program.
A control group is chosen via matching on observable pre program characteristics, between all the schools not enrolled in PQM. The ATT will then be estimated using a difference-in-differences estimator to get rid of the selection time invariant unobservable characteristics at the school level.
As described above, treatment characteristics and teachers’ monetary incentives vary in Northern and Southern Italy. Thus, we will produce different estimates for Northern and Southern Italy.



Data sources

With the support of INVALSI (Italian Institute for Educational Evaluation), the project team is collecting a large amount of primary data on both students and their teachers:

2009-10 and 2010-11
• Standardized tests in Mathematics and Italian and background student information from the INVALSI yearly national assessment of student achievement;
• school administrative information (school budget, resources, other professional development programs) from the Ministry of Education

2010-11
• teachers’ background information collected via CATI on treated and controls
• qualitative information on the program from in-depth interviews to a selected group of teachers;



Results and/or substantiated conclusions or warrants for arguments/point of view

The yearly national assessment of student achievement was just conducted in May, and the data will be available from October.
The analysis will allow us to investigate the following key points:
• are different incentive strategies for teachers leading to a differential effect on students’
achievement?
• is the effect (if any) larger as the peer participation gets stronger?
• provided that most schools are selected for both subjects, is there any addictive effect of the treatment?



Significance

The study brings some new empirical evidence on the effectiveness of teacher training and on the constraints on such impact assessment evaluations in an environment – as Italy – traditionally not accustomed to such practices

Authors

• Giovanni Abbiati, University of Milan

• Elena Meroni, University of Padova