Response to the Critiques of Learning Progression Research

• In Event: Modern Measurement for Science Learning Progressions (Book by Routledge)

Abstract

Although significant progress has been made in LP research (Duschl et al., 2011; Jin, Mikeska et al., 2019), several researchers raised critiques of the value of LPs (Shavelson & Kurpius, 2012; Sikorski & Hammer, 2010). According to the critiques, in most LPs, student learning is presented as a sequence of achievement levels. However, students’ knowledge is fragmented, containing many loosely connected facets of knowledge. The process of knowledge development is complex and dynamic. Individual students may learn the same scientific idea and practice in very different ways. Then, how can science learning be presented as a sequence of achievement levels? How can a single progression present the learning of many different students? Although researchers use both qualitative and quantitative methods to validate LPs (e.g., Gotwals and Songer, 2013; Wilson, 2009), there is still a danger that researchers may use empirical data to fit their arbitrary progressions, or in other word, their “procrustean beds” (Shavelson & Kurpius, 2012).
These critiques of the LP research are rooted in a well-known debate between two perspectives on conceptual change: the knowledge-in-pieces perspective and the naïve-theory perspective. The presentation will discuss these two perspectives, which will help us understand the critiques. From the knowledge-in-pieces perspective, student knowledge is fragmented, containing many loosely connected pieces. When answering questions about a scientific idea or practice, depending on contextual factors, certain knowledge pieces are activated to generate explanations. As such, it is inappropriate to represent student understanding of a scientific idea/practice in terms of a few LP levels. If this is true, the validation techniques used in LP research (e.g., interviews and IRT analysis of assessment scores) lack a theoretical basis. This is Critique 1, which is about the nature of student knowledge. The knowledge-in-pieces perspective asserts that learning is contextual and dynamic. Therefore, a single progression cannot accurately represent the learning of a diverse group of students. Due to variations in thinking, development, and learning styles, each student may have a unique trajectory of learning. Thus, it is unlikely that one standardized progression can be applied to all students. This is Critique 2, which is about the process of learning scientific knowledge.
Researchers have used both naïve theories and knowledge-in-pieces to develop LPs (e.g., Smith et al., 2006; Stevens, et al., 2010). If substantial evidence suggests that students do not reason consistently across contexts, or they do not use relatively coherent theories, at least using findings and ideas from naïve theories raises concerns. Therefore, in response to the critiques, we first consider the empirical studies that examined whether students’ knowledge is fragmented or theory-like. Next, we argue that science LPs are instruction-oriented, they should be evaluated in terms of its effectiveness in promoting student learning.

Author

• Hui Jin, Georgia Southern University