Poster #80 - Who Gets Flagged, Who Gets Help? Predictive Analytics, Education Governance, and Equity in Intervention Targeting

• In Event: Friday Poster Reception

Abstract

Predictive analytics systems are increasingly used in U.S. public higher education to identify students at risk of course failure, stop-out, or noncompletion. These tools are often justified as a means of improving the efficiency of intervention by directing scarce advising and support resources toward students with the greatest need. Yet recent research suggests that fairness concerns arise not only from prediction error, but from the institutional rules through which risk scores are translated into intervention. The central policy question is therefore not simply whether predictive systems are accurate, but whether they allocate support equitably across student groups. This study examines how differences in the governance of predictive analytics shape intervention targeting in public higher-education systems. The empirical focus is a comparative analysis of three U.S. public higher-education systems that adopted predictive tools under contrasting institutional arrangements: a centralized statewide community college system, a multi-campus public university system with campus-level implementation, and a vendor-supported hybrid model in which institutions retain discretion over intervention thresholds and follow-up procedures. This design makes it possible to compare how similar technologies operate under different administrative rules. The analysis draws on administrative student records, risk classifications, intervention assignment data, academic outcomes, and policy documents governing model use, threshold-setting, override authority, subgroup monitoring, and audit practices. In the statewide system, the data permit direct observation of how risk scores are translated into student support decisions; in the comparative systems, institutional records and implementation guidance are used to reconstruct these decision rules. Together, these materials support evaluation of both predictive performance and the downstream allocation consequences of institutional design. The empirical strategy combines subgroup performance analysis with comparative institutional analysis. On the quantitative side, the study evaluates calibration, classification error, and intervention allocation at decision-relevant thresholds, with particular attention to racial and socioeconomic disparities among students near cutoff points. On the institutional side, the analysis compares systems that differ in whether models are routinely recalibrated, whether human overrides are permitted, whether subgroup equity reviews are required, and whether risk flags are linked to funded support capacity. This design shifts attention from the model alone to the administrative system built around it. Preliminary evidence indicates that governance arrangements strongly condition the equity effects of predictive analytics. Systems relying on fixed thresholds with limited subgroup review appear more likely to misallocate support at the margin, especially when intervention capacity is constrained. By contrast, systems with regular recalibration, transparent override procedures, and explicit equity monitoring appear better able to balance predictive efficiency with more equitable intervention allocation. The findings also suggest that technically strong models do not guarantee fair outcomes where support services are scarce, unevenly distributed, or weakly integrated with risk classifications. The study contributes to science and technology policy by connecting predictive analytics to public-sector resource allocation and evidence-based intervention. It shows that educational AI systems are not simply technical instruments, but institutional arrangements whose distributive consequences depend on governance, implementation, and support capacity.

Author

• Ruoxi Li, The Hong Kong University of Science and Technology (HKUST)
  Presenting Author