A Case Study Comparison of Leadership Practice Against Formative Assessment Survey Results

• In Event: Roundtable Session 18.
  In Roundtable Session: The Testing, Development, and Implementation of an Online Instructional Leadership Formative Assessment Tool

Abstract

The Comprehensive Assessment of Leadership for Learning (CALL) is an on-line formative assessment and feedback system for middle and high schools which is designed to capture levels of leadership for learning by measuring existing leadership practices from the perspective of school leaders and staff. A critical task in the development and implementation of the CALL involves validation work to both refine the survey tool and inform the research team’s theory of action. As part of the validation process of CALL, the research team conducted six pilot administrations of the survey, the results of which were captured in the following case study report. This case study examines the results of these validation efforts by triangulating three sources of data: the five core domains of school leadership rubrics that serve as the foundation to the CALL instrument, the quantitative scores by domain and sub-domain for each participating school, and the qualitative results from the three rounds of interviews conducted with school leaders and other survey users.
For the first two phases of this research project, the research team developed the CALL survey from a set of school leadership rubrics developed by a Principal Investigator for this project in conjunction with research conducted by the Institute for Learning. The School Leadership Rubrics identify specific observable practices that can be found in more and less effective schools. The practices and tasks are organized within five domains (in italics).
In Focus on Learning, school leaders maintain a school-wide focus on learning, formal leaders are recognized as instructional leaders, the school incorporates a collaborative design of an integrated learning plan, and the school provides appropriate services for students who traditionally struggle. Monitoring Teaching and Learning emphasizes formative and summative evaluation of student learning as well as formative and summative evaluation of teaching. Building Nested Learning Communities involves a collaborative school-wide focus on problems of teaching and learning, an emphasis on professional learning, socially distributed leadership, and coaching and mentoring. In Acquiring and Allocating Resources, school leaders take into consideration personnel practices that contribute to school improvement, effectively structure and maintain time, focus school resources on student learning, appropriately integrate external expertise into the school instructional program, and coordinate relations with families and external communities. Lastly, Maintaining a Safe and Effective Learning Environment involves clear, consistent and enforced expectations for student behavior; a clean and safe learning environment; student support services that provide safe haven for students who traditionally struggle; and buffering the teaching environment.
In the first round of interviews for this pilot study, the researchers inquired about leadership practice focusing on the five domains. The subsequent interviews occurred after survey administration and focused on user feedback. The researchers triangulated data from these interviews, the dimensions of CALL, and the survey scores by domain and subdomain. Preliminary results reveal variation among how school leaders conceptualize their practice in comparison with the five core domains the survey scores as well.

Author

• Mark Blitz, University of Wisconsin - Madison