Measuring Teacher Participation Across National Science Foundation CS10K-Funded Professional Development Providers

• In Event: Measuring the Efficacy of Large-Scale Computer Science Efforts in K–12 Settings: Strategies and Challenges

Abstract

Computer Science is a rapidly evolving field, with many teachers participating in professional development (PD) to offer CS courses in their schools as the entry point to a CS education pipeline. The NSF CS10K program “aims to have rigorous, academic computing courses taught in 10,000 high schools by 10,000 well-prepared teachers.” However, efforts to study these programs is challenged by the lack of a coordinated set of definitions, measures, and system for data collection (Cooper et al., 2016). This research seeks to establish common definitions and a system for data collection by measuring teacher participation across CS10K programs. This study’s goals are twofold: to test the feasibility of a diverse group of projects collecting and reporting on common data and; to generate an accurate count of the number of teachers trained through CS10K projects and location.

Our team of researchers developed a common data collection tool and process for projects to report aggregate-level participation data. We used a peer-driven approach to engage this community, identifying common data elements across projects and soliciting community feedback. We also provided a list of recommended survey questions and instructions for collecting school and student data. In addition, each project was paired with a liaison from our research team to support data collection. Project leads used a spreadsheet template to submit aggregated participant information, while liaisons worked one-on-one with project leads to improve data quality. Data validation interviews were conducted to ensure interpretation of the aggregated data was accurate as well as in an effort to promote transparency, build good will, and increase future participation in other data collection efforts.

To date, 100% of projects submitted at least partial data. The results show that 1,538 new teachers have participated in PD through these 29 CS10K programs; taught in at least 909 public and private schools in 43 states, the District of Columbia, and Puerto Rico. This effort has revealed the following: 1) The CS10K projects were cooperative with respect to the data collection effort. All projects provided at least some data and several revised historical data or provided historical data for the first time. 2) Interpreting the data was more challenging than expected. We learned that both the timing of data collection and methods of data collection matter. For example, projects do not all collect data at the same time (i.e., spring after implementation) nor do they use the same methods (i.e., teacher surveys). 4) Data was provided at the aggregate level, which precluded the research team from doing any outcome-based analysis. 5) Scaling up data collection effort will be challenged by current liaison model, which requires significant individualized attention.

By highlighting challenges and strategies for collecting common data across disparate projects, this research can contribute to the broader CS research community as data collection is more widely supported. Lessons learned from this effort may also inform the CS education professional development community as the field matures. The paper will also contribute to our understanding the landscape of CS education nationally.

Author

• Rebecca Zarch, SageFox Consulting Group