Tests of Teacher Math Knowledge: Are We Using Them?

• In Event: A Synthesis of Mathematics and Statistics Assessments: 2000–2020

Abstract

In this summary we present findings from an analysis of the teacher mathematical knowledge tests (hereinafter referred to as TMK tests) identified from the last 20 years of the mathematics education literature. This analysis is part of a larger project designed to develop a repository of instruments and their associated validity evidence for use in mathematics education. Here we report on the frequency with which TMK tests appear in the mathematics education literature, as well as determining the extent to which TMK tests are constructed, reported, or taken up in ways that represent intent for use by the community of mathematics education researchers.
Methods
We used R script to web scrape 21 mathematics education journal websites (identified as the union of the lists in Nivens & Otten, 2017, p. 350) this resulted in 12,065 articles. The title, abstract, and keywords of these articles were electronically screened for the potential to include a TMK test; this resulted in 2,249 articles. The title and abstract of these articles were manually screened by the author team; this resulted in 267 articles. The full article was manually screened by the author team; this resulted in the identification of 173 articles that contained reference to a TMK test. Within these 173 articles, we documented whether the test included was a study-specific TMK test or a community use TMK test. Community use measures (broadly defined) had to fit at least one of these criteria: stated validity argument; careful investment in instruments (e.g., article written just about the instrument development); intent for use by others stated by authors/developers; or used by multiple researchers in other publications. Anything not meeting any of these criteria was deemed to be study-specific—intended only for use in that study.
Results
We provide a brief summary of results here that will be further elaborated at the AERA 2023 conference presentation. The frequency of journal articles containing a reference to a TMK test in the mathematics education literature was 1.9%. The presentation of TMK tests in the mathematics education research literature is relatively evenly split between community use and study-specific designations. However, because a particular community use TMK test may have been referenced across multiple articles, the number of community use tests (37) identified was less than half of the number of study-specific TMK tests (98). The suite of Learning Mathematics for Teaching (LMT) tests (e.g., Hill, Ball, and Schilling, 2008) and the two TEDS-M scales (e.g., Tatto, 2013) were the most commonly published tests. Six other community use TMK tests were cited twice and the remaining 29 were referenced once.
Questions for Discussion
Our work raised several important questions for the field, we provide a few here: (1) What are the relative values and strengths of study-specific versus community use measures? (2) Are the relatively few community use measures sufficient for the field to measure TMK? (3) How do our findings compare to use of TMK tests in other research fields?

Authors

• Michele Carney, Boise State University
  Presenting Author

• Pavneet Kaur Bharaj, California State University - Bakersfield
  Non-Presenting Author

• Heather Howell, Educational Testing Service
  Non-Presenting Author

• James E. Smith, University of Arizona
  Non-Presenting Author

• Wendy M. Smith, University of Nebraska - Lincoln
  Non-Presenting Author