Defining the Terrain of Educational Accountability: An Experimental Approach

• In Event: Using Experiments to Study Public Policy

Abstract

The 2019 APSA theme of “Populism and Privilege” raises important issues around the framing and definition of policy goals and instruments. Populists, whether of the political right or left, typically present policy problems and solutions in terms supportive of their own interests. This is not unique to populists, of course. Shaping a policy message to fit one’s interests and goals is typical in the policy process.

Accountability is critical in this context. Through elections, public forums, media, and other means, citizens hold public officials ‘to account’ for the policies they propose and implement. This is particularly important in a democratic polity. Yet, as we conclude in our paper, even accountability systems do not escape the contest of contending interests and perspectives that shape the policy process.

We explore this dynamic by focusing on one element in an accountability system – indicators – in the context of a major domestic policy – public education. Indicators used in educational accountability, such as test scores, student attendance, and teacher surveys, include measures or metrics designed to assess student or school performance. There is a robust debate over the best choice of indicators (Education Week 2018; Martin et al. 2016; Hess and Eden 2017). Through a review of the accountability literature, public opinion polls and existing accountability systems, we identify contending perspectives on accountability indicators. Using an experimental design in a general population survey, we offer a more rigorous analysis of indicator preferences, correlated with accompanying political and economic interests.

The range of educational accountability indicators is quite broad, and often the source of disagreement. Student test scores, for example, are the most long-standing indicators used in many accountability systems. They have strong support among many policymakers, particularly at the state and federal levels. Yet in public opinion polls, such as the PDK annual poll, support for test scores as accountability indicators is much weaker. In more recent years, the role of test scores has become more complicated through a debate over the relative importance of achievement scores at a single point in time versus growth measures capturing change in scores over time. State accountability systems pursue different directions on this issue, some give greater weight to achievement while others give preference to growth. In public opinion polls, measures of growth tend to be more popular than achievement scores.

Recent federal policy has encouraged the inclusion of “school quality or student success” indicators. This has contributed to an expanded debate over additional indicators. Some states give considerable weight to measures in this category, such as student surveys and college and career readiness, while others take a minimal approach with much less weight to this indicator.

This debate offers competing perspectives on indicator preferences and their corresponding political and economic interests. As our contribution to that debate, we turn to an experimental approach designed to ascertain the public’s preferences on accountability indicators. We will include questions in a survey experiment imbedded in a nationally-representative sample of approximately 1,000 respondents conducted online by YouGov in spring 2019. The questions will provide a series of options regarding different indicators and their perceived efficacy in an accountability system. We will correlate these preferences with political and demographic characteristics of the survey respondents to identify trends in support for different types of indicators. This analysis will contribute to our understanding of indicator preferences in educational accountability as well as the interests that animate those preferences.

The major research tasks include conducting and analyzing results from the experimental survey as well as a review of the accountability literature, public opinion polls, and accountability systems. Part of this work builds upon previous research by the author, including a current study of state accountability systems as represented in plans submitted to the federal government under the Every Student Succeeds Act. With respect to public opinion polls, we rely upon several existing national surveys, including annual surveys by Phi Delta Kappan (http://pdkpoll.org/) and Education Next (https://www.educationnext.org/edfacts/).

Education Week. 2018. “Approved ESSA Plans: Explainer and Key Takeaways from Each State.”
https://www.edweek.org/ew/section/multimedia/key-takeaways-state-essa-plans.html.

Hess, Frederick and Max Eden. 2017. The Every Student Succeeds Act: What It Means for Schools, Systems, and States. Harvard Education Press.

Martin, Carmel, Scott Sargrad and Samantha Batel. 2016. “Making the Grade: A 50-State Analysis of School Accountability Systems.” Center for American Progress.

Author

• John Portz, Northeastern University