Examining the Role of Administrators as Student Learning Objective Approvers

• In Event: 29.046 - Administrative Sense-Making in Assessments of Teaching

Abstract

The Student Learning Objective (SLO) process was originally piloted in the Denver Public Schools system as an instructionally-based approach to evaluating student growth for educator evaluation and merit pay purposes (Slotnik, et al., 2004). This process was conceived on the assumption that the components of an SLO – setting meaningful learning goals, establishing targets for students performance on assessments of these goals, and modifying instruction based on student progress toward attainment of those goals – constitute the work that good teachers already do and thereby represents an authentic evaluation of student growth taking place in the classrooms (Lachlan-Haché et al., 2012; Marion et al., 2012; Slotnik, et al., 2004). As SLOs have gained more traction nation-wide for use in educator evaluations, it is increasingly recognized that administrators play a critical role in this process. Despite this, current research on the SLO process has not focused on how well school administrators charged with approving the quality of the SLOs submitted are rating the SLOs used to inform educator evaluations.
This study presents results from exploratory analyses using SLO data from one state and one large urban school district to provide insights on how administrators are rating the quality of SLOs along key dimensions including: the learning goal specified, assessments used, the articulated instructional strategies, and growth expectations set for students. In the case of the district, SLOs have been implemented for several years to support teacher compensation and administrators are charged with ensuring the quality of SLOs submitted by teachers and approving results. Compared to the district, the state has only recently implemented the SLO process for all teachers during the 2013-2014 school year but has also charged administrators with quality-control and approving SLOs. The guiding research question used in this study was: to what extent would the quality ratings assigned by administrators to the SLOs align with the quality ratings assigned to the same SLOs by content experts?
To investigate this question, a random sample of 30 SLOs in mathematics and in English language arts were drawn from the district and a random sample of 110 SLOs in various content areas were drawn from different districts within the state. The SLOs were first rated by content experts using the same quality rubrics used by administrators, and then compared against the ratings provided by administrators. In both new and old SLO implementation sites, the analysis yielded similar results where the SLO quality ratings assigned by content specialists to the SLOs largely diverged from administrators, and were lower than the ratings assigned by administrators to the SLOs on each dimension of the rubric. Focus groups were also conducted with 15 administrators in the district to learn about continuing challenges and perceived barriers associated with their role as SLO approvers. Although generalizations cannot be made about the overall quality of SLOs being submitted by all teachers in both settings, these preliminary findings point to potentially problematic aspects of using the SLO results to support inferences about student growth achieved for high-stakes personnel evaluation purposes.

Author

• Elena Kitaoka Diaz-Bilello, University of Colorado - Boulder