Are Assessments Involving Picture Stimuli Valid in Global Contexts? Evidence From Preschoolers in Mombasa, Kenya

• In Event: Improving understanding of child development globally through advancements in measurement

Abstract

Many childhood cognitive assessment tools (e.g., Peabody Picture Vocabulary Task) rely on picture stimuli. As these assessments spread globally, researchers make an implicit assumption: that children across early environmental contexts understand pictures in the same way. What if this assumption does not hold? Differences in early picture comprehension may drastically change the validity of assessment scores.

Our research investigates whether children in some global contexts might perform more accurately on a vocabulary task involving object stimuli than on the same task involving picture stimuli. We worked with preschoolers in 12 schools in 5 sub-counties of Mombasa, Kenya. In a preregistered between-subjects design, 96 children (M = 4.54 years, SD = 1.00 years, range = 2.65 – 7.48 years, 43% female) participated in the Object Condition (i.e., a vocabulary task involving objects), and another 96 children (M = 4.50 years, SD = .91 years, range = 2.38 – 6.95 years, 46% female) participated in the Picture Condition (i.e., a vocabulary task involving pictures). Local researchers presented children with a Swahili vocabulary task involving three kinds of words: nouns (i.e., “toothbrush”, “hat”, “crayon”, “fork”), number words (i.e., “one”, “two”, “three”, “four”), and color words (i.e., “red”, “blue”, “yellow”, “green”). These three kinds of words were chosen to create variability in trial difficulty: preschoolers in Mombasa should be quite familiar with nouns, somewhat familiar with number words, and not at all familiar with color words.

In a preregistered analysis, we found that children had a higher proportion of correct responses in the Object condition compared to the Picture condition (49% vs. 41%, t-test p < .001; chance performance = 25%). Moreover, in exploratory analyses, we found that this overall difference was driven by the trials involving words that children had some familiarity with. Children had a higher proportion of correct responses in the Object condition than the Picture condition for trials involving nouns (69% vs. 58%, t-test p < .001) and number words (50% vs. 41%, t-test p = .02), but children's performance on trials involving color words did not differ by condition (27% vs. 26% accuracy, t-test p = .66).

In additional exploratory analyses, we found that children were less likely to demonstrate side-biasing behavior in the Object condition than the Picture condition (bias on 21% vs. 28% of blocks, p = .04). In line with other results, this overall side-biasing difference between conditions was driven by the trials involving words that children had some familiarity with. Specifically, children were less likely to side-bias in the Object condition than the Picture condition on trials involving nouns (bias on 1% vs. 10% of blocks, p = .02) and number words (bias on 15% vs. 31% of blocks, p = .007), but children side-biased equally frequently in both conditions on trials involving color words (48% vs. 44% of blocks, p = .56).

Overall, our findings suggest that assessments involving pictures may underestimate children’s cognitive capacities in some contexts. Ultimately, this research contributes to the development of more appropriate assessments to improve early childhood educational trajectories.

Authors

• Rebecca Zhu, University of California - Berkeley
  Presenting Author

• Helen Osborne Pitchik, University of California - Berkeley

• Tabitha Nduku

• Jan Engelmann, University of California - Berkeley

• Lia C.H. Fernald, University of California - Berkeley

• Alison Gopnik, University of California - Berkeley