Using Scenarios to Assess Teachers' Justification of Actions in Mathematics Teaching

• In Event: 66.072 - Multimedia Online Experiences as Research Tools to Study Mathematics Teaching Knowledge at Scale

Abstract

We describe a series of multimedia scenario-based instruments intended to assess practicing mathematics teachers’ recognition of professional obligations as a source of justification for instructional actions. The paper presents early results from our analysis of these instruments using Item Response Theory (Bond & Fox, 2006).

The theory of practical rationality (Herbst & Chazan, 2012) posits both that mathematics teaching is guided by situation-specific instructional norms and that deviations from the norm are justifiable on account of professional obligations to the individual, the class as a whole, the discipline of mathematics, or the institution. To better understand the role of these obligations as a source of justification, we present professionals with scenarios in which a norm of action is breached by instead doing something that might be justifiable on account of a specific obligation and ask participants to rate its justifiability.

We developed storyboards in which a teacher deviates from a norm of instruction. Each set of storyboards was introduced with a statement that indicated the teacher had deviated from her lesson on account of an obligation (stated in teacher friendly language; e.g., "… in order to address an issue of mathematical importance" for the disciplinary instrument). These storyboards are all accompanied by statements of the form: “The teacher should have [done the normative action], rather than [the depicted breaching action]”. The participant is then asked to agree or disagree with the statement using a 6-point Likert scale. After developing 15 to 18 storyboards for each of the four obligations, we screened them with focus groups consisting of experienced mathematics teachers and other people experienced in mathematics in order to assess the face validity of the normative and breach actions.

We then piloted the items with practicing K-12 teachers over the course of a year. Participant responses on each of the instruments (disciplinary = 15 items, n = 228; individual = 16 items, n = 221; institutional = 18 items, n = 153; interpersonal = 18 items, n = 140) were scored dichotomously: 0 if the participant agreed with the depicted teacher’s action, 1 if otherwise. Agreement with the statement indicated that the depicted teacher should have done what was normative in the situation. The reliability of the items was analyzed using a 1-parameter Rasch model (Bond & Fox, 2006). Each group of items had high item reliability (> 0.9582)) which means that each set of items can be well rank-ordered in terms of how justifiable it is to deviate from each normative action. They had lower person-reliabilities (> 0.4740) which means that the items only somewhat distinguish different levels of recognition of the obligation by participants.

By giving mathematics teachers the opportunity to pick a breaching action justified by a professional obligation in the context of an instructional situation, we have developed an instrument that allows us to examine the theory of practical rationality in a truly situated way. This will help in the development of a language for talking about the role of justification in the mathematics teaching profession.

Authors

• Ander Willard Erickson, University of Michigan

• Justin Kelly Dimmel, University of Maine

• Kristi Hanby, University of Michigan - Ann Arbor

• Inah Ko, University of Michigan - Ann Arbor