Extending Content Knowledge for Teaching Assessment Design Theory From Elementary-Level Mathematics to Secondary-Level Mathematics

• In Event: Developing and Validating Indicators for Teachers' Science and Mathematics Content Knowledge for Teaching

Abstract

There has been great interest in assessments designed to measure types of mathematical knowledge used uniquely in teaching practice, commonly referred to as Mathematical Knowledge for Teaching (MKT) (National Research Council, 2013). Studies have provided compelling evidence that MKT differs profoundly from conventional mathematical knowledge (Author, 2014a) and can be linked to student performance (Hill, Rowan, & Ball, 2005). These findings support the argument that teaching requires a form of professional mathematical knowledge that is specialized to the work of teaching (Hill, Schilling, & Ball, 2004).

To date, however, nearly all assessments of MKT have been developed at the elementary or middle school level. Many mathematical challenges that teachers face at these school levels differ from how other mathematically skilled adults typically work with and understand mathematics. In part, this is because teachers focus on student-level mathematics that is quite distant from adult mathematics. Such differences may not exist or may be less dramatic at the secondary level, where students work with mathematics that approaches the mathematics non-teaching math majors encounter (Rowland, 2012). And other differences may come into play – for example, differences in the breadth and depth of conventional knowledge teachers hold may be of greater significance at the secondary level. In this study we address the basic validity question, “Do secondary MKT items modeled on design theories established at the elementary level function as intended, where our design theory specifies that the test-taker will apply their mathematical knowledge to a particular task or problem that is encountered in teaching practice?”

In this study, 24 retrospective cognitive interviews (Ericsson & Simon, 1985) were conducted using a set of secondary-level MKT assessment items focused on three content areas (Quadratics, Linear Functions, and Exponents). (The items and authorship are not described in detail in order to provide blind review, but follow design characteristics of MKT measurement that readers may be familiar with from similar measures such as those produced by the Learning Mathematics for Teaching and Measures of Effective Teaching studies.) Participants were selected on the basis of strong conventional mathematical knowledge as indicated by test scores on a test of higher level mathematics. The interviews produced 186 item-responses, which were analyzed, following a previously established methodology, to compare the given answer to the reasoning demonstrated (Author, 2013b). Of the responses, 87% reflected the desired alignment between reasoning and answer selected, providing strong overall evidence of construct validity. However, particular items showed less alignment, suggesting item-level design issues that may be a result of translating to a secondary context.

Results suggest that assessment of secondary-level MKT can build on the foundation laid by early work at the elementary level by capitalizing on item design theory established in that context, which provides validity evidence for such assessments. However, item-level results also provide examples of a number of ways in which similarly designed items function differently than expected due to secondary teachers’ relatively stronger conventional mathematical background, suggesting that caution is in order.

Authors

• Heather Howell, ETS

• Yvonne Lai, University of Nebraska - Lincoln

• Erica Miller, University of Nebraska - Lincoln

• Geoffrey C. Phelps, Educational Testing Service