Classroom Artifacts as Indicators of Quality in STEM Education

• In Event: Developing National Indicators for K–12 STEM Curriculum and Instruction

Abstract

Background

This study explores the potential for developing a scalable indicator system that can provide insight into the range and quality of classroom artifacts aligned with emerging college and career readiness standards (CCRS). We develop and evaluate a prototype system that uses classroom assignments and assessments as indicators of the quality of STEM instruction and use the Next Generation Science Standards and Common Core Mathematics Standards to guide the indicator system design.

Classroom artifacts are an important source of evidence of the quality of classroom interactions providing evidence about the nature of expectations held for students in terms of new standards, content, reasoning, and communication.

While most studies of artifacts have drawn inferences at the teacher level, an indicator system makes inference about practice at broader levels (e.g., district, state, nation). Different levels of inference also have design and sampling consequences.

Purpose

We undertake the design of an artifact-based indicator measure to be used as evidence of implementation of instructional practices embodied in CCRS. We then pilot key components of a prototype system to be used as either part of large-scale surveys and/or as a tool for districts to engage in self-study of instructional practice.

Primary research questions include:

1. To what extent can valid and reliable scoring protocols for classroom assignments in math and science be designed?
2. To what extent can artifact quality serve an indicator of progress toward the goals of STEM education as expressed in the CCRS? What is the evidence about the nature of STEM content and practices addressed through classroom artifacts?
3. To what extent can this approach be helpful to districts in evaluating local STEM instructional quality?

Methods/Data

The study has four distinct components:
Phase 1 involved relevant literature review and interviews with experts in CCRS, artifact study, and large-scale educational surveys;
Phase 2 involves designing a protocol to collect and assess the quality of classroom assignments. The protocol will capture both the content covered and the quality of instructional practice with respect to the CCRS. Data collection methods employing simple, readily available technologies will also be developed;
Phase 3 is a pilot study that will include approximately 500 math and science classrooms from multiple districts across the country. We will collect and score 2 assignments from each classroom, together with a random sample of student work. Assignment collection will occur across the school year to ensure that the indicator system captures a representative view of instructional practice and;
Phase 4 will undertake a sharing of findings and a refined instrument for adoption by interested school districts.

Results and Discussion

We will present the protocol design, sampling and analytic approaches and preliminary findings from the initial stages of data collection. We will also discuss progress and challenges in data collection. We will also share findings of initial discussions with districts concerning the potential for use of the protocol to examine instructional practice locally.

Authors

• Drew H. Gitomer, Rutgers University - New Brunswick/Piscataway

• Jeanette Joyce, Rutgers University