Novice Teachers' Mathematical Knowledge for Teaching: Conceptualizing, Measuring, and Reporting on Pilot Results

• In Event: 73.023 - Toward a Fair Evaluation of Teachers: Methodological Challenges in a Cross-National Study of Mathematics Teachers

Abstract

Objectives

This paper examines the challenges involved in developing knowledge assessments for primary and secondary novice teachers in this study, the psychometric properties of these assessments, and the procedures used to improve these instruments to measure well across countries.

Theoretical framework

Developing assessments of knowledge for international and comparative studies is a challenging task (see Hambleton, Merenda & Spielberger, 2005; National Research Council, 2002). There are differences across countries regarding when and with what emphasis content topics are covered; even a subject matter area as well defined as mathematics shows substantial variation in timing, coverage, and emphasis of mathematical content (Author, 2011) Thus, deciding what should be included on an assessment is not straightforward Additionally, the breadth of material that potentially needs to be covered is not possible in the period of time that can be obtained from the participants in a country.

Methods

This study’s assessments of the Mathematics and Mathematics Pedagogical Content Knowledge of novice teachers used items developed by experts and also unreleased items from TEDS-M. Item response theory (IRT) was used to analyze the results to calibrate items and score the knowledge assessments (see DeAyala, 2009 for a description of IRT methodology).


Data sources

The data come from the study’s tests of knowledge administered to novice primary teachers and secondary teachers in the 15 countries participating in this proof-of-concept study. The assessments’ frameworks are consistent with those of TEDS-M which defined knowledge for teaching mathematics as consisting of at least three components: disciplinary knowledge, pedagogical knowledge and knowledge of the primary and secondary mathematics curriculum. Operationalization of disciplinary (mathematics) knowledge and (mathematics) pedagogical knowledge has been part of the work on developing instruments for this study and is discussed in this paper, along with the data that were collected.

Results

The results of the piloting of the assessments show that the instruments are successful in discriminating varied levels of mathematical and pedagogical expertise in novice teachers across countries.

Scholarly significance

A central motivation behind current policy is to improve the teaching and learning of mathematics by improving the preparation of mathematics teachers. Mathematics knowledge and the mathematics pedagogy knowledge are considered as key to high quality teaching. Thus, developing assessments to measure teachers’ knowledge has been a priority area of this study. But teaching is complex, involving multiple dimensions, and quality can be completely assessed only on the basis of knowledge and performance, and ultimately on how well pupils learn. The assessments constructed in this study represent one of a number of measures that will allow us to construct such system.

Authors

• Kiril Bankov, University of Sofia

• Michael C. Rodriguez, University of Minnesota