District Perspectives on a Correlational Validity Research Study

• In Event: 58.077 - District-Driven Problems of Practice: What Can Practitioner-Scholars Learn and How Do Districts Benefit?

Abstract

Objectives

One school district participating in the research-practice partnership recently began utilizing a new assessment system: the Independent Reading Level Assessment Framework (IRLA). The district had previously been using the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) assessment, and wanted to make a change to an assessment that provides more extensive, diagnostic information and is aligned to the Common Core State Standards (CCSS); IRLA was an assessment system that met these criteria (American Reading Company, n.d.). To better understand the ramifications of this decision, the district requested of the partnership a correlational validity study of IRLA and Smarter Balanced (SBAC) to better understand the predictability of the new IRLA tool within the district.

Perspectives

Little validity evidence exists on the IRLA. A 2014 study conducted by Measurement Incorporated appears to be the only existence of validity evidence of the IRLA; they conducted concurrent validity correlation coefficients between the IRLA and Northwest Evaluation Association’s (NWEA) Measurements of Academic Progress (MAP) tool. Correlation coefficients between the two measures ranged from 0.88 to 0.90.

Methods and Data Sources

The district provided the partnership with concurrently completed IRLA and SBAC data. The analytic population included all Grades 3-5 students with an IRLA score and SBAC reading score (n = 1,953) who attended one of the elementary schools within the district. Descriptive statistics and Pearson r correlational statistics were the primary data analysis methods.

Substantiated Conclusions

Overall, it appears that IRLA was a moderate predictor of SBAC English Language Arts (ELA). Across all students, the correlation between the two measures was 0.748 and 62% of students meeting benchmark on IRLA also met the SBAC benchmark. Correlation for third grade, fourth grade, fifth grade, and English Language Learners (ELL) students waere 0.671 (p < .001); 0.756 (p < .001); 0.676 (p < .001); and 0.701 (p < .001), respectively. Students meeting the benchmark on IRLA but not on SBAC ELA were more likely to be males and in third grade. Students meeting the benchmark on SBAC ELA but not on IRLA were more likely to be White, females, and in fifth grade.

Significance

This school district invested in a new way to teach and measure reading skills, and wished to understand how this tool predicted performance on the state test, SBAC. Engaging in a project this size required tools, expertise, and time the district did not have, and the partnership stepped up to analyze data for the district. Overall, the research revealed that the two tools are fairly predictive. This research has enabled the district to directly support and ensure even more use of the instructional tool, in a targeted way, through professional development and coaching. By understanding that the instructional tool is predictive of the summative state test, the district has been able to place more value in the daily work of teachers and the reading instruction they deliver to students which, in turn, reduces the amount of uncertainty and stress produced by taking the SBAC assessment with no idea how students will place.

Author

• Amy Jackson, Reynolds School District