Lessons Learned From Developing a Framework for Evaluating the Impact of Computer Science (CS) Teacher Professional Development on CS for All Outcomes

• In Event: Teacher Preparation for Computing: A State of Affairs

Abstract

Objective
The national CS for All movement is predicated on the assertion that very few students in K-12, and particularly few women and students of color, have access to computer science in K-12 (Margolis, 2008). In response to this challenge, the NSF has made a large financial commitment to support the development of engaging CS curriculum and CS teacher professional development (National Science Foundation, 2016). However, longitudinal measures of the impact of CS teacher professional development on the final outcome of interest--student access to and participation in CS coursework in K-12 schools--is limited primarily to AP enrollment. This paper discusses efforts to define and measure longitudinal objectives in relation to The University of Texas at Austin’s WeTeach_CS project, which focuses on CS teacher training, certification, and capacity building. Since 2015, WeTeach_CS researchers and practitioners have been developing a framework for analyzing statewide impact in CS through an equity lens. Researchers will discuss
the process of defining statewide strategic objectives for K-12 CS Education access and participation,
decision points related to data access and analysis, and
recommendations for other states as they engage in similar efforts.

Theoretical framework
Our effort to build CS teacher capacity is as much a social innovation as it is technical challenge. The process by which outcomes/metrics and data analysis techniques were identified, prototyped, provided feedback, and iterated for improvement reflects a design thinking model for change (Razzouk & Shute, 2012). Five key outcomes frame this effort in Texas.
1. Increasing the number of teachers certified to teach high school CS.
2. Increasing the number of high schools that offer CS courses.
3. Increasing the number of students that complete a CS course.
4. Broadening and diversifying the types of students that complete a CS course.
5. Expanding coding, programming, computational thinking, and CS opportunities in K-8.

Methods
These outcomes were developed through engagement with stakeholders from the Texas CS Task Force (Fletcher, 2014) and specific metrics were further refined in collaboration with CS4TX. The primary data source for this study is the Texas’ Education Resource Center (ERC), a consolidated data center providing access to Texas student data for research.

Results
Researchers will discuss key decision making points in developing strategic outcomes and metrics for any state and the team addressed these issue. This involved achieving consensus on which secondary courses should “count” as computer science courses, what counts as a “high school” in determining the percentage of high schools that offer CS, how to measure underrepresented populations in CS while dealing with FERPA, and establishing consistent K-8 metrics across a state.

Significance of the study
Given the recent increased investment in building CS teacher capacity, it is vital that states establish consistent long range goals and metrics for measuring access to and participation in K-12 computer science. Although the design process appears simple on the surface, it is actually incredibly complex, requiring consensus among researchers, practitioners, policymakers, and business partners. Individuals engaged in similar efforts across the nation can find value in the lessons learned in WeTeach_CS.

Authors

• Carol Fletcher, The University of Texas - Austin

• Jayce R. Warner, The University of Texas - Austin

• Lisa Garbrecht, The University of Texas - Austin

• Carol Ramsey, The University of Texas - Austin