Sensitivity to Environmental Volatility Across Development: Evidence from Reinforcement Learning and Pupillometry

• In Event: Novel Approaches to Examining Learning and Self-Regulation Development Across Context and Levels of Analysis

Abstract

Introduction
Across development, individuals must learn to make good choices across varied environments. Some environments have highly stable reward contingencies, such that actions that were frequently rewarded in the distant past continue to bring about beneficial outcomes. Other environments are more volatile, such that reward contingencies change over time, and only the outcomes of recent actions are useful for guiding choice. Prior research has found that during learning, adults’ arousal systems — as indexed by pupil dilation — track the underlying volatility of environmental reward contingencies. Adults also dynamically adjust learning accordingly, by weighting recent outcomes more strongly when making choices in volatile environments. Here, we asked when and how the ability to dynamically adjust learning to the statistics of the environment emerges, by examining whether adolescents track the volatility of the environment during reward learning and flexibly adjust their weighting of recent outcomes accordingly.

Hypotheses
We hypothesized that more implicit sensitivity to environmental statistics, as indexed by pupil dilation, may emerge earlier in development than corresponding changes in the flexible adaptation of reinforcement learning to different environmental demands.

Methods.
To test this hypothesis, a large sample (current N = 101, planned N = 112) of adolescents (N = 43, age range = 12.1 - 17.8 years, mean age = 14.8 years, SD age = 1.7 years), and young adults (N = 58, age range = 18.5 - 25.9 years, mean age = 21.9 years, SD age = 2.3 years), completed two blocks of a reinforcement learning task while their pupil size was measured with an eye tracker. Throughout the task, participants had to repeatedly select between two cats and avoid getting scratched. In the stable task block, one cat scratched on 75% of trials and the other scratched on 25% of trials; these probabilities remained constant for 90 trials. In the volatile block, one cat scratched on 80% of trials and the other scratched on 20% of trials, but these probabilities reversed every 20 trials. The order of the blocks was counterbalanced across participants, and though participants were told the scratching probabilities may change throughout the task, the transition between the blocks was not signaled.

Results
Preliminary results (N = 101 behavioral, 88 pupillometry) revealed that, in contrast to our initial hypothesis, sensitivity to environmental volatility was greater at younger ages. We observed an age x block condition interaction effect on pupil size, F(1, 101571) = 108.76, p < .001, such that younger participants demonstrated a greater increase in pupil dilation in the volatile versus stable block of the task. In addition, reinforcement learning models fit to participant choice data revealed that while neither adolescents nor adults demonstrated significant differences in their weighting of recent outcomes across task blocks, F(1, 106) = 0.72, p = .397, younger participants demonstrated heightened exploratory tendencies in the volatile relative to the stable block of the task, F(1, 294) = 5.19, p = .023. These findings suggest that adolescents may be particularly sensitive to environmental volatility, and may adapt their choice processes to explore more when the environment is changing.

Authors

• Kate Nussenbaum, New York University
  Presenting Author

• Hannah Hamling

• Haoze Zhu, New York University

• Linda Kerbl, New York University

• Bradli Washington, New York University

• Catherine Hartley, Weill Cornell Medicine