Estimating Multilevel Reliability for School-Based Behavioral Measures

• In Event: Enhancing Validity and Reliability in Educational Measurement and Assessments

Abstract

Many schools utilize universal behavior screening to quickly evaluate all students to determine who may need additional support. Most commonly, a single teacher will rate all students in their homeroom class or a specified hour of the day for secondary schools. Like many other school-based assessments, screening data has a nested structure, is typically ordinal, and presents with a non-normal distribution. In this study, we simulated data with characteristics similar to behavior screening data with varied levels of inter-item correlation and within-cluster variability. We conducted single-level and multi-level reliability analyses per Lai (2021) to examine the impacts of ignoring nesting when estimating reliability. We provide recommendations for estimating reliability for future studies of behavior screening tools and other similar instruments.

Authors

• Katie Scarlett Lane Pelton, University of Connecticut

• D. Betsy Mccoach, University of Connecticut