ASSESSMENT: Evaluation of a Multiple Measures Placement System Using Data Analytics

• In Event: Developmental Education Reforms on Assessment, Instruction, and Math Pathways

Abstract

Purpose: This paper evaluates the impact of multiple measures placement on student outcomes using an experimental study conducted at seven community colleges within a northeast state university system. The placement system being evaluated uses data on prior students to develop predictive algorithms at each college to weight multiple measures — including placement test scores, high school GPA, and years since high school graduation — that are then used to place incoming students into remedial or college-level courses. Incoming students who arrived at these colleges in the fall 2016, spring 2017, and fall 2017 terms were randomly assigned to be placed by either the status quo system (control group) or the alternative placement system (program group). Students were followed for three to five terms.

Theoretical framework: Most students in developmental math or English courses are referred to these programs based on scores they earn on standardized placement tests. Research shows that some students assigned to remediation would likely pass a college-level course in the same subject area if given that opportunity. It has been hypothesized that using more information, including high school GPA, should lead to more accurate placement and improved student outcomes (Belfield & Crosta, 2012; Scott-Clayton, 2012). This study evaluates a multiple measures placement system that employs an algorithm with thresholds set by college administrators and instructors.
Methods: The study employs an RCT that meets WWC evidence standards without reservations. Impact analyses were conducted using OLS, controlling for college fixed effects and a range of student characteristics. For both math and English, we consider three outcome measures: the rate of college-level course placement, the rate of college-level course enrollment, and the rate of college-level course completion. We also examine impacts on overall credit accumulation, persistence, and degree completion.

Data: Data for this study come from seven participating community colleges that cover over 13,000 eligible students who entered the colleges in fall of 2017 through fall 2018. Data include demographic information, placement test records, and students’ high school and college transcripts.

Results: Early results from the study’s first cohort of students are broadly positive. We find that many program group students were placed differently than they would otherwise have been. In math, 14% of program group students placed higher than they would have under a test-only system, while 7% placed lower. In English, 41.5% placed higher, while 6.5% placed lower. Further, program group students were 3.1 and 12.5 percentage points more likely than control group students to both enroll in and complete a college-level math or English course in the first term. Prior to the conference we will conduct further analyses using tracked data from all three cohorts, including impacts on credit accumulation, persistence, and degree completion.

Significance: As evidence accumulates that traditional placement systems are inadequate, colleges are increasingly turning to the use of multiple measures for assessing and placing students. The results of this study shed light on whether multiple measures placement decisions based on data analytics lead to better student outcomes than a system relying on test scores alone.

Authors

• Elizabeth Kopko, Community College Research Center

• Elisabeth Barnett, Teachers College, Columbia University