Learning about educational system effectiveness from intra-national state differences

Session Submission Type: Group Panel

Description of Session

The theme of the panel is that in large federal systems, where it is possible to compare educational quality differences among administrative entities that are, indeed, in charge of educational policy at the primary and secondary school levels, comparisons among these entities can be more illuminating in understanding the characteristics of “good” and “bad” educational policies than comparing results across countries with even greater variation in educational culture and political context. The panel consists of reports on interstate comparison research in three major countries in the Western Hemisphere, the US, Mexico, and Brazil.

Moderator and Chair: Martin Carnoy, Stanford University

Paper 1: Gains in Student Achievement in U.S. States, on NAEP (1992-2011), and Comparing U.S. States with TIMSS and PISA Results in Other Countries.

Presenter: Emma Garcia, Economic Policy Institute

This paper will report on a massive study done at the Economic Policy Institute using microdata files that compared National Assessment of Educational Progress achievement results in math and reading across US states for the past 10 and 20 years adjusting for socio-economic background, school characteristics, and teacher characteristics. Results are also compared with other countries for US states that participated in TIMSS (1999-2011) and PISA (2012). Inferences are drawn about effective educational policies across states.

Paper 2. Gains in Student Achievement on PISA across Mexican States, 2003-2012.

Presenter: Mariana Barragan, Stanford University, School of Education

This paper will report the results of a study of student achievement of 15 year-olds in mathematics and reading on the PISA tests of 2003, 2006, 2009, and 2012. Since the Mexican PISA sample in those years was randomized in each of the 32 Mexican states, it is possible to analyze state fixed effects controlling for student socio-economic background, school variables, and percent rural schools. The analysis also estimates the effects of state level variables including union strength, an index of violence, % of students in school at 15 years-old, and state poverty levels. Paired comparisons are also made of neighboring states with different gains trajectories.

Paper 3. Gains in Student Achievement on the Brazilian national test (SAEB) across Brazilian States, 1999-2011.

Presenter: Paula Louzano, Universidade de Sao Paulo, School of Education

This paper will report the results of an extensive study of 8th grade student achievement on the Brazilian national assessment test of mathematics and Portuguese, the SAEB, a large randomized sample of Brazilian 8th graders in all 27 Brazilian states. The study compares student achievement state fixed effects controlling for the variation in student and school socio-economic conditions and teacher resources in the schools. The results suggest great variation in state performance over the 12-year period, with some states making large gains and others losing ground. The analysis attempt to explain these differences in terms of political stability and state reform policies.

Sub Unit

• General Pool

Chair

• Martin Carnoy, Stanford University

Presenters

• Paula Louzano, Universidade de Sao Paulo

• Emma Garcia, Economic Policy Institute

• Mariana Barragan, Stanford University

Individual Presentations

• Gains in Student Achievement in U.S. States, on NAEP (1992-2011) - Emma Garcia, Economic Policy Institute

• Gains in Student Achievement on PISA across Mexican States, 2003-2012. - Mariana Barragan, Stanford University

• Gains in Student Achievement on the Brazilian National Test (SAEB) across Brazilian States, 1999-2011. - Paula Louzano, Universidade de Sao Paulo