A Multiparameter Framework for the Design of Assessments of Epistemic Cognition

• In Event: Conceptual and Methodological Issues and Advances in Research on Epistemic Beliefs

Abstract

Objectives and Theoretical Framework

In this paper we propose and defend a multidimensional framework for designing assessments of epistemic cognition. The framework specifies six different parameters on which the designers of assessments must make decisions. The framework can be used to guide the design of both formal assessments (e.g., questionnaires or interviews) and schemes for analyzing epistemic discourse (e.g., the discourse in inquiry-based science classes).

In previous work we have argued that philosophical scholarship suggests many avenues of research that psychologists have not yet investigated (Chinn, Buckland, & Samarapungavan, in press). In this paper, we continue to apply ideas from philosophy to suggest new ways of thinking about assessments of epistemic cognition.

Data Sources and Methods

Our framework is grounded in data from two year-long studies involving two cohorts of seventh graders (over 500 students) in 26 science classes taught by seven teachers who were implementing a model-based inquiry curriculum. Each class was video recorded on a majority of days during the academic year. Over 250 class and group discussions were analyzed for evidence of epistemic talk, using constant comparative methods.

Results

We conclude that students’ epistemic talk can be analyzed along six distinct parameters. In presenting the framework, we also show how the same parameters apply to the design of formal assessments. The parameters are:

(1) Topic. Following the analysis of Chinn et al. (in press), we identified six topics of epistemic talk, including talk about epistemic aims, views about the nature and structure of epistemic achievements, justifications, stances, epistemic virtues, and reliable processes for generating knowledge. We present evidence that students’ talk spans all these topics. (2) Degree of abstractness. This dimension refers to the extent to which talk is about epistemic categories (e.g., knowledge) in general or about specific instances of knowledge (e.g., explanations of photosynthesis). (3) Productivity. Some student talk was productive, in the sense that students spontaneously produced their own epistemic ideas. Other talk was receptive, in that students responded to others’ ideas (e.g., expressing agreement or disagreement with another student’s epistemic statement). (4) Grain size. Student discourse differed in the grain size of focus, from coarse-grained (e.g., talk about personal experience) to fine-grained (e.g., detailed talk about fit with all high-quality evidence). (5) Conditions. Epistemic talk varied in terms of whether students discussed the specific conditions under which different epistemic concepts were considered applicable to particular situations. (6) Metacognitive level. Finally, students’ comments varied according to metacognitive level (unreflective, reflective, meta-reflective); the latter category involved sophisticated epistemic evaluations of the epistemic categories.

Significance

We believe that this framework provides new insights into how to conceptualize and analyze assessments. We will show that one source of confusion in interpreting results from research on epistemic cognition is that different assessment instruments have different values on the six parameters we identify. We also show that one and the same framework is fully applicable both to analyzing and evaluating formal assessments of epistemic cognition and classroom talk in inquiry-based science classes.

Authors

• Clark A. Chinn, Rutgers University

• Luke Andrew Buckland, Rutgers University