A Tale of Two Measures: Defining Pedagogical Proficiency in the Assessment of Preservice Teachers

• In Event: Multiple Perspectives on Preparing Teachers for Diverse Populations: National, Institutional, and Curricular Responses

Abstract

As a research center affiliated with a large private university in New York City, we have been engaged in the development and implementation of a method and measure for evaluating the pedagogical proficiency of teacher candidates in the university’s school of education. This work must not only take into consideration the program’s mission of preparing candidates for work in urban schools serving students of diverse backgrounds, but also the varied purposes for which such assessment data are used, including annual reporting for national accreditation, providing evidence of program efficacy for faculty and administrative review, and evaluation of pre-service candidates. Since 2004, the center has used the Domain Referenced Student Teacher Observation Scale (DRSTOS-R), an instrument adapted from Danielson’s (2007) Framework for Teaching. However, recent revisions in the New York State teacher certification requirements, set to take effect in 2014, mandate the evaluation of teacher candidates’ pedagogical proficiency via a portfolio-based measure called the edTPA. Given this policy’s impact on a large number of our program graduates, discussion has turned to whether, and to what degree, the DRSTOS-R might be modified or replaced to accommodate new expectations.

This paper presents part of a broader content analysis examining the DRSTSOS-R and edTPA conducted to aid in this transition and to identify functional similarities and differences between the instruments, particularly noting which constructs are included and how these constructs are operationalized. Rubrics and handbooks for each instrument were analyzed to systematically characterize their respective measurement objectives, assessment criteria, structure and organization, as well as their administration and scoring procedures. In addition to descriptive comparison, this analysis aims to glean insights on the theoretical and practical orientations reflected in instruments’ operational definitions of “good pedagogy.” With an eye towards how these measures depict pedagogical proficiency with respect to teachers of diverse learners, this analysis comments upon the extent to which instruments include consideration of the needs of diverse learners at large, as well as how each measure is used with candidates seeking certification in English as a second language and special education instruction. For the DRSTOS-R, additional evidence speaking to the interpretation and functioning of items is drawn from training notes and informal feedback provided as part of the data collection process.

This work affords valuable contributions to theory and practice in both teacher education and teacher evaluation. By deconstructing how pedagogical proficiency is defined and assessed by two measures, one currently utilized by a large school of education and another set to impact the preparation of pre-service teachers statewide, findings offer insight on the dynamic relationship between institutional practice and policy in teacher education. Furthermore, this study points to key tensions and issues regarding the assessment of preservice teachers’ instructional skills. As a growing number of institutions adopt or develop measures for evaluating pre-service candidates for increasingly diverse schools, we must consider the ways in which those measures communicate and instantiate particular values in their definitions of effectiveness, and by extension, shape the goals and practices of teacher preparation programs.

Author

• Kimberly Woo, New York University