Exploration of theoretical and empirical congruence in a measure of early math

• In Event: Poster Session 12
  In Poster Session: PS 12 Topic - Methods, History, Theory

Abstract

Educators and researchers need a measure that matches the breadth and depth of children’s mathematical thinking. This study uses data collected from an evaluation project to explore psychometric qualities of a short form of the Research-based Early Math Assessment (REMA). The REMA – Short Form includes 80 items. Data include 2,112 observations of 1,194 children. Data were collected from the Spring of 2016 to the Fall of 2019 from two school districts on the west coast of the United States.

Fifty items were developed to capture children’s understanding of quantity and operations. Thirty items were developed to capture children’s understanding of measurement, geometry, and patterning. These two subsets are typically administered in two separate sessions. While some items are scored dichotomously, others receive partial credit based on timing of the response. Many items are scored with strategy codes in addition to correctness. Strategy codes collect information about the sophistication level of children’s attempt to answer. For example, a child may answer an arithmetic question with small numbers correctly or incorrectly. In an attempt to answer the question, the child may count all numbers involved in the problem, count up from one of the numbers, or have knowledge of the math fact memorized. Assessors are extensively trained to gather this type of information in addition to children’s correctness on items. Strategy codes have been ordered by sophistication level based on research and theory on children’s mathematical learning and development (Clements & Sarama, 2014). Items have been ordered based on research and theory, as well as on Rasch analysis of the longer version of the measure (Clements, Sarama, & Liu, 2008).

Data analysis plans include factor analysis and item analysis to address the hypothesis that the assessment items and their sophistication codes follow an order with empirical congruence to the research and theory behind its test construction (Clements & Sarama, 2014). First, items will be recoded to combine strategy codes and correctness codes to generate a continuum of answers from incorrect answers achieved with increasing levels of sophistication to correct answers achieved with increasing levels of sophistication. Second, factor analysis will be conducted on the dichotomous and polytomous answers using principal axis factoring and a quartimax rotation, as a general factor is expected. Following confirmation of unidimensionality of the measure, or subsets of the measure, the generalized partial credit model (gpcm) will be applied to examine the estimated difficulty (i.e. association of the nominal code to the construct) of the combined correctness-strategy codes (de Ayala, 2009). Specifically, the difficulty order of sophistication codes based on early math research will be compared with the order of estimated difficulty in this large sample of young children.

Implications of the research are twofold. Psychometric evaluation of the factor structure and item parameters (difficulty and discrimination) will inform refinement of the test. Data addressing our hypothesis will add evidence to the field of early math regarding the development of children’s mathematical thinking. Data have been prepared and preliminary analysis has been conducted.

Authors

• Shannon Stark Guss, University of Denver
  Presenting Author

• Douglas H Clements, University of Denver
  Non-Presenting Author

• Julie Sarama, University of Denver
  Non-Presenting Author

• Crystal A Day-Hess, University of Denver
  Non-Presenting Author