Poster #9 - The effect of question type on sex differences in mathematics for 11th and 12th graders: a meta-analysis replication and extension

• In Event: 3-001 - Poster Session 2 during Breakfast

Abstract

The potential existence of fundamental differences in men and women’s mathematical abilities has long been a matter of interest in psychology. Although some evidence points to a small but consistent sex difference in mathematical ability, others have argued that this effect is confounded by test design. To address this topic, we both replicated an existing meta-analysis (Lindberg et al., 2010) and extended its scope to determine whether the magnitude of sex differences in performance in mathematics examinations in the final years of secondary education are moderated by test design.

Our study was a replication of Lindberg et al. (2010) meta-analysis of research on sex differences for 11th and 12th graders' mathematics performance published between 1990 and 2007. We also extend those findings by conducting a separate retrieval of papers with similar search terms covering work published from 2007 and 2015. An a priori power analysis revealed a power coefficient of 0.989. Studies needed to include 11th to 12th graders, or of an equivalent age, and contain original data. We retrieved 3593 unique records covering the years 2007 to 2015, of which 314 met our inclusion criteria and were then downloaded and assessed for eligibility. In total, 46 pieces of research published or submitted between 1990 and 2015 passed this eligibility screening; 29 of which were not previously included in Lindberg et al. (2010).

From these 46 studies, 99 effect sizes (expressed as Cohen’s d) were produced and the total N was 281,193 (152,620 female). A random-effects model was used and conservative adjustments were made for sample size and publication bias. Overall, the weighted effect size was d=0.25, p<.001 (CI: 0.214 to 0.293, SE=0.02). Effect sizes that came from studies that used only multiple-choice assessments (d=0.326, p<.001) were compared with studies that used either a mixed design or solely non-multiple choice questions (d=0.184, p<.001). These two subgroups were significantly different, Q(2)=15.36, p<.001.

This suggests that mathematics tests for 11th and 12th graders show reduced sex differences when tests included non-multiple choice measures. These effects sizes are larger than those in Lindberg et al. (2010); however, their closest equivalent effect sizes included all age groups and not just 11th and 12th graders. These results underscore the importance of assessment type for mathematics. Institutions might consider utilizing non-multiple choice assessment assessments to supplement or replace multiple-choice testing to ensure that the individuals with the greatest potential are being recognized, regardless of sex.

Authors

• Dominic Kelly, University of Michigan
  Presenting Author

• Melissa Hines, University of Cambridge