Beyond the Nation’s Report Card: Opportunities to Strengthen and Scale Equity Indicators

• In Event: Appraising Equity in Education With The Nation’s Report Card and Beyond

Abstract

Answering critical questions about how students progress through their education and beyond—and how schools help (or hinder) their chances of success—requires understanding complex forces that shape social and economic mobility. Yet researchers, policymakers, and practitioners are often hampered by the narrow set of measures available to them. For example, while NAEP data provide a clear picture of drops in math and reading test scores during the COVID-19 pandemic (CEPR, 2022; NAEP, 2022a, 2022b), federal sources provide less data on students with different learning needs or in grades that aren’t typically tested, and even less on how students were affected in other ways, such as their physical, mental, and social well-being.

Data can be a powerful tool for promoting equity, but without the right data, communities can struggle to align on the root causes behind disparities, and where to focus data collection and analysis efforts. Many communities want more comprehensive data, but knowing where and how to start can be a significant hurdle. Forty-nine states have received SLDS grants since the program’s inception (ECS, 2021), and 29 states proposed using Elementary and Secondary School Emergency Relief funds to create or improve their education data systems (Karva, 2021). However, only 19 states currently have longitudinal systems that connect student data from pre-K, K-12, postsecondary, and workforce sectors (ECS, 2021).

This presentation is grounded in a publicly available resource that highlights opportunities to improve data collection and use in service of equity goals. The Education-to-Workforce Indicator Framework (Gonzalez et al., 2022) recommends 20 questions every state and locality should answer to understand disparities and how to address them, along with a comprehensive set of indicators and metrics that can provide meaningful answers. To develop these recommendations, Mathematica, Mirror Group, and the Gates Foundation’s education data team worked with researchers, policymakers, practitioners, and community advocates to gather input and review over 40 existing data frameworks.

We will present a new analysis building on the framework that explores where to go next with education equity measures. The analysis draws on two data sources: desk research on data coverage in three leading state longitudinal data systems, and ratings assigned by subject matter experts assessing whether each indicator can be measured feasibly and comparably across contexts. The 99 framework indicators map to three key opportunities to advance equity data:
1. New research and validation of measures. Indicators with a less established, more emerging evidence base that require additional funding, future research, and engagement around efforts to track data. Example: Early grades on track.
2. Increase measurement alignment across data systems. Indicators with an established evidence base, already being collected in leading data systems, but in various ways, preventing meaningful comparisons across contexts. Example: College preparatory coursework completion (AP/IB/dual credit).
3. Expand adoption of core measures in public data systems. Indicators with an established evidence base, already being collected widely and consistently, and feasible to measure at scale in a comparable way across contexts. Example: High school graduation.

Authors

• Megan Shoji, Mathematica Policy Research, Inc

• Sonia Pace, Mathematica Policy Research, Inc

• Naihobe Gonzalez, Mathematica Policy Research, Inc

• Beth Alberty, Mathematica Policy Research, Inc