The Development and Testing of the CALL (Comprehensive Assessment of Leadership for Learning) Survey Instrument

• In Event: Innovations in Measuring and Assessing Instructional Leadership in Schools

Abstract

Developed by researchers from the University of Wisconsin-Madison and The Wisconsin Center for Educational Research, the Comprehensive Assessment of Leadership for Learning (CALL) is an on-line formative assessment and feedback system for middle and high schools. CALL is designed to capture levels of leadership for learning by measuring existing leadership practices from the perspective of school leaders and staff. The survey captures leadership practices and school culture across five core domains:

1. Focus on Learning
2. Monitoring Teaching and Learning
3. Building Nested Learning Communities
4. Acquiring and Allocating Resources
5. Maintaining a Safe and Effective Learning Environment

CALL provides formative feedback for schools to assess and improve leadership for learning. Feedback includes a quantitative report on survey scores, a qualitative narrative of a school’s leadership practices and school culture, and detailed prescriptive suggestions on how to make improvements based on the school’s needs as indicated by CALL.

The foundation of the CALL assessment tool was constructed from the work of this project’s two Principal Investigators. For the first two phases of this research project, the research team developed the CALL survey from a set of school leadership rubrics developed by one Principal Investigator in conjunction with research conducted by the Institute for Learning. The School Leadership Rubrics identify specific observable practices that can be found in more and less effective schools.

In validating CALL survey content, the research team invited a panel of middle and high school leaders and teachers to examine each survey item and provide feedback based on their own experience of practice in schools. The survey was then administered in a large urban area in a mid-western state, the results of which helped inform construct validity within the domains. In the second year of the project, the research team administered the survey in six middle and high schools in a mid-western state, as well as nine school districts in a southern state, and conducted three rounds of interviews around the survey administration with the principals and other survey users (leadership team members, teachers, and other staff).

Validation of the CALL instrument involved examining the dimensionality of items using confirmatory factor analyses, checking internal consistency through reliability analyses for all subdomain scales, and checking correlations of subdomains within a domain and across domains. An important piece of diagnostic information is the degree to which teachers in a school agree about leadership practice.

The tool is not intended to provide a stand-alone summative evaluation, but rather a window into leadership practice for continuous improvement. This focus on formative feedback encourages school leaders to focus on specific areas of to make improvements. Furthermore, the CALL survey represents an innovative approach in using technology to inform practice: the survey is administered online with automatic results returning to the school leaders and also incorporates embedded definitions of specific terms to ensure user understanding of survey intent. The challenges that lay ahead for the development and implementation of CALL concern capturing leadership practice and providing meaningful feedback to practitioners.

Authors

• Carolyn Kelley, University of Wisconsin

• Richard R. Halverson, University of Wisconsin - Madison

• Mark Blitz, University of Wisconsin - Madison

• Seann Mason Dikkers, Ohio University - Athens

• Eric M. Camburn, University of Wisconsin

• Matthew A. Clifford, American Institutes for Research