Designing and Implementing Nationally Representative Panels of Teachers and School Leaders

• In Event: 72.047 - New Tools for Monitoring Teacher and School Leader Responses to Policy Reforms

Abstract

The purpose of this paper is to describe the process and methodology for the design of the American Teacher Panel (ATP) and the American School Leader Panel (ASLP). The ATP and the ASLP were created to address the need for a longitudinal monitoring tool to track the implementation of policy. The panels are thus designed to understand how teachers and principals experience implementation for specific policies and survey item development is framed within the implementation literature (Pressman & Wildavsky, 1973; Mazmanian & Sabatier, 1983; Havelock & Havelock, 1973; Rogers, 1983, 1995). Four research questions conceptually guide the surveys for the panels:
1. How do teachers and school leaders interpret a given reform?
2. How well are teachers and school leaders prepared to successfully implement a given reform?
3. What do teachers and school leaders do to implement a given reform?
4. What barriers do teachers and school leaders encounter in their efforts to implement a given reform?
Survey items for the panels, including those being reported in this session, are developed in consultation with an advisory committee made up of representatives from National Association of Elementary School Principals, National Association of Secondary School Principals, American Federation of Teachers, National Education Association, University Council for Educational Administration and researchers. In future administrations, researchers may develop their own questions to submit to the panels within broad guidelines developed by the advisory committee.

The goal for the project is to develop and maintain nationally representative panels of public school teachers and leaders that will yield precise estimates of national responses and also permit reasonable sub-analyses based on pre-selected stratification variables at the school and teacher levels (e.g., school level, teaching subject matter). There is no existing sampling frame containing information on all teachers or all school leaders in the U.S., so we used a school-based approach to sampling, relying on the enumeration of schools available in the Common Core of Data (CCD). To permit analyses that associate responses from teachers to responses from school leaders, we selected teachers and leaders from the same schools. To achieve a high level of precision we are sampling 2,300 public schools from the CCD. The principals and two teachers are selected within each sampled school to be part of the panels. This is the approach used by SASS in its recent cycles (2003-4, 2007-8, 2011-12) because it provides relatively good school-based and teacher-based estimates. (Schools and teachers will both be ‘refreshed’ in later years to reflect changes in the populations over time.) At present we have 1/3 of the sample in place. When the full panels are achieved, this school-level sampling plan will give, as an example, 80% power to detect a difference of 5.5 percentage points between elementary and secondary teachers. Response rates for the first survey to the panels (which focused on implementation of state standards), May 2014, were 73% (teachers) and 64% (school leaders). A second survey will be administered in October 2014. Results from both will be reported in this session.

Authors

• Brian Stecher, The RAND Corporation

• V. Darleen Opfer, RAND Education

• Susannah Faxon-Mills, RAND Corporation

• Michael Robbins, RAND Corporation