Individual Submission Summary
Share...

Direct link:

Can Teaching Quality Be Identified From a Test? Validity of Mexico’s Teacher Entry Examination After The 2013 Education Reform

Thu, April 18, 1:30 to 3:00pm, Hyatt Regency, Floor: Atrium (Level 2), Boardroom B

Proposal

Mexico introduced in 2013 a historic reform amending the entry, performance assessment, promotion, incentive programs, and retention of teachers, with the aims of advancing teachers’ careers, and eliminating discretional practices by the Teachers Union. Teacher selection and performance evaluation are now guided by teaching standards, following the Latin American practice (Chile, Colombia, Peru, El Salvador), but scant research has been conducted on the subject. Therefore, this research analyzed Mexico’s teacher selection, focusing in the state of Puebla. It offers evidence on whether standards-based teacher evaluations, and specifically written teacher entry examinations, are a valid method to select teachers, and provide with policy recommendations for the newly-elected authorities, who announced they would cancel the reform.

The core component is a validity study of the teacher selection method, assessing whether the teacher entry examination results can predict teacher performance evaluation results after two years. This is supplemented with 31 teacher semistructured interviews and content analysis of administrative documents, contextualizing the quantitative findings.

From the classical perspective on predictive validity, there was some evidence showing that the entry examination had a positive and adequate validity to predict teacher performance, with correlation coefficients ranging from .23 to .28 between the subject-matter test and the global performance evaluation score (the indigenous language test was not correlated, and an the ethics test inadequately correlated). However, there is a potential upward bias in the predictive validity finding, since the subject-matter test and the exam instrument of evaluation were similar in content and method. Correlations between the entry examination and the lesson plan instrument of evaluation offered better evidence, with correlation coefficients ranging from .22 to .29 (the portfolio instrument of evaluation showed no correlation).

OLS regressions, showed that the teacher entry examination was one of the factors that best explained the variability in the global performance evaluation score, with 1% increase associated with a 3.8% increase in the global performance evaluation score (equivalent to 30 points). Grades were also found to be an explanatory factor, but half the size of the teacher entry examination effect. Previous teaching experience in public schools was associated with a negative effect on performance evaluation, of the same size of the teacher entry examination effect, possibly related to the low quality of teacher preparation institutions, or the socialization of teachers with old practices that were counterproductive. Teacher decision of staying in the same school during the first years was associated with an increase of 27 points in teacher performance, highlighting the effect of a healthy school environment. A 1% increase in the marginalization index was associated with a 1.2% increase in performance evaluation, suggesting that an adverse socioeconomic context is not necessarily unfavorable for teachers’ performance. However, being in a lowly dense community was constraining on teachers’ performance (- 42 to -92 points less).

Finally, an innovative strategy estimated the teacher selection error rates, using as validity criteria success and failure measures of predicted teacher performance. The error and severe error rates may be inexact, but the best prediction models showed an underselection error rate of 7% for the global performance evaluation score, 8% for the lesson plan score, and 14% for the portfolio score, reflecting the probability of leaving out of the teaching career promising teachers. They also showed that the overselection error rate was 12% for the global performance evaluation score, 13% for the lesson plan score, and 14% for the portfolio score, describing the probability of selecting underperforming teachers, the worst of outcomes, and the most common error.

Author