Exploration of the Alignment of State Data and Infrastructure to Mathematics and Science Success Indicators

• In Event: 34.051 - Improving Indicators of K–12 STEM Education Policy and Practice: What Have We Learned?

Abstract

Purpose

Monitoring Progress Toward Successful K-12 STEM Education (NRC, 2013) described 14 indicators for monitoring progress in science, technology, engineering, and mathematics (STEM) education. The driving question for our work is whether current State Level Data Systems (SLDSs) collect data sufficiently robust, comprehensive, and aligned to the indicators to allow their monitoring. And, if not, what are next steps to leveraging SLDS structures to evaluate national progress?

The focal STEM Indicators for the project (as numbered in Monitoring Progress):
1. Number of, and enrollment in, different types of STEM schools and programs, by district;
2. Time allocated to teach science in grades K-5;
3. Science-related learning opportunities in elementary schools;
6. Teachers’ science and mathematics content knowledge for teaching;
7. Teachers’ participation in STEM-specific professional development activities;
8. Instructional leaders’ participation in PD on creating conditions that support STEM learning.

Perspective

Each SLDS has hundreds of data elements (e.g., hours/week spent by each middle school student in math; number of hours of professional development (PD) and title of each PD course completed by each teacher or leader/principal). Some of these are required by the federal government (EDFacts system). Others are unique to specific states. Informed by the latest research on educational structures (e.g., what makes something a “STEM program”), the study sought information on SLDS content and processes.

Methods and Data Sources

We combined qualitative (document analysis and interview) with quantitative (descriptive statistics) methods. Project STEM experts identified classes of data needed to address focal indicators then examined the national Common Education Data Standards (CEDS) for such data elements. We obtained and analyzed 15 states’ data dictionaries and categorized state elements by indicator. Additionally, we talked with SLDS and STEM program staff in 10 states (individual and group interviews) about current use and future viability of SLDSs as an indicator-supportive repository.

Results

The presence of indicator-aligned data elements in CEDS and sample states varied widely (particularly for Indicators 2 and 3). Combining data elements might lead to acceptable composites. SLDS leaders and STEM directors demonstrated eagerness to talk with colleagues, across professional subcultures, to see a big picture that “supports children in learning – that’s what we are all concerned with, really.” Under development now is a set of recommendations and action steps for modifications to state (and local) data communication and collection.

Scholarly (and Practical) Significance

Two national policy agendas intersect here: CEDS and STEM indicators. For CEDS, responses ranged from ignorance to dismissal to passionate support. “STEM indicators” was news to all but one interviewee. SLDS and state STEM program staff can contribute complex, thoughtful, and rich ideas. Carefully orchestrated conversations that clearly value those contributions can be fruitful (they were for us!) in identifying how requisite data elements may be collected. We also saw how SLDSs can function both as an essential tool for ensuring equity and excellence through compliance and as a means for insuring educational equity and excellence through the quality of the STEM indicator data they curate.

Authors

• Ellen B. Mandinach, WestEd

• Shandy Hauk, WestEd