The Vanderbilt Assessment of Leadership in Education: Conceptualization, Psychometric Development, and Implementation

• In Event: Innovations in Measuring and Assessing Instructional Leadership in Schools

Abstract

The Vanderbilt Assessment of Leadership in Education (VALED) is a 360 degree, evidence-based multi-rater rating scale that assesses principals’ behaviors known to influence teachers’ performance and in turn, students’ learning. The VAL-ED measures critical leadership behaviors for the purposes of diagnostic analyses, progress monitoring, and summative evaluation. This paper presents the conceptual scaffolding of the instrument, its psychometric development, the feedback reports, and its implementation and use for formative development as well as summative decision making such as in pay for performance.
The VALED assessment instrument is rooted in the ISSLC standards and the research literature, and measures six core components and six key processes that define learning centered leadership: high standards for student learning, rigorous curriculum, quality instruction, culture of learning and professional behavior, connections to external communities, and performance accountability. The processes are: planning, implementing, supporting, communicating, monitoring, and advocating (see Murphy et al, 2007; Goldring et al,2009). The principal, all of his/her teachers, and his/her supervisor make judgments regarding the leadership effectiveness of the principal on the 12 dimensions.
The development of the VAL-ED included: (1) specifying the purposes of the assessment, (2) defining content assessed, (3) writing items, (4) developing test specifications for validity evidence plans, (5) piloting test forms, (6) designing scoring and interpretation frameworks for scores, (7) conducting studies that yield evidence for the reliability and validity of the scores, (8) field-testing forms with a representative sample, and (9) writing a technical manual that summarizes technical characteristics and uses of the assessment.
The psychometric development process of the instrument included a sorting study, cognitive interviews with supervisors, principals, and teachers and two pilot tests ( Porter et al, 2010a). A 300-school field trial established the internal consistency reliabilities and factor structure of the instrument, the degree to which the two forms are parallel, and the degree to which the ratings the principal gives him or herself correlate with the ratings from the teachers in the school and from the principal’s supervisor. In addition, a Bookmark proficiency standard setting was conducted. (Cizek, 2001; Porter, et al, 2010b; Cravens et al, 2011). A national panel of 22 experts participated. Ultimately, the standards were set yielding 17% of the national field trial principals below basic, 50% below proficient, and 86% below distinguished.
Feedback reports of the VALED, include graphs, charts, and narrative indicating the level of effectiveness ratings by the various respondents (teachers, principal, and supervisors), principal's performance relative to a national sample of peers (norm scores), and the principal's performance on proficiency standards (below basic, basic, proficient, distinguished). Principal targets of professional growth are identified and specific action and behaviors for improvement are provided.
The VALED assessment instrument can also be used as part of a comprehensive principal evaluation system and should be used in conjunction with other information, most importantly, value-added student achievement. A critical decision is how to weight the various components of a principal evaluation system.

Authors

• Ellen B. Goldring, Vanderbilt University

• Andrew C. Porter, University of Pennsylvania

• Joseph F. Murphy, Vanderbilt University

• Xiu Cravens, Vanderbilt University

• Stephen N. Elliott, Arizona State University