Using Randomized Natural Experiment to Estimate the Effects of Evaluee's Gender and Attractiveness on Evaluator's Accuracy

• In Event: 41011 - Experimental Examinations of Social Psychological Processes

Abstract

I study factors that influence people’s memory and ability to evaluate how well others performed at an objectively-measurable task. I use a unique data set that I hand-collected from a randomized natural experiment, TV show The Weakest Link. In each episode, several strangers take turns, individually answering randomly-assigned trivia questions. After each round, players individually vote out the worst player. Players cannot take notes; they must rely on memory to decide who the worst player is.

This creates a randomized natural experiment for studying memory and cognition. We have: (1) each player’s objective measures of performance (ratio of correct to incorrect answers), (2) on randomly assigned tasks (questions), and (3) each player’s objective measures of memory or perception of how others performed (voting record).

I hand-collected data for 44 episodes of the game, 302 players, hundreds of rounds, thousands of questions. For each player, I took a screen photo and fed it to an AI tool that generated an objective, cardinal, consistent measure of each player’s attractiveness. I collected data on player’s performance, voting record, and demographics, such as gender, race, education, age, etc.

I test multiple hypotheses, using multiple statistical models. Main findings: in each statistical model, controlling for everyone’s objective performance: (1) women with higher attractiveness scores are significantly less likely to be voted out; (2) players are more likely to accurately evaluate/remember performance of an unattractive person than an attractive one; (3) the order of “treatment” (order in answering questions) strongly predicts the likelihood of being voted out – the player who was randomly made last to speak was significantly more likely to be voted out; (4) the order effect exists only for men.

This contributes to the literatures on workplace discrimination, gendered evaluation, implicit biases, the beauty premium, serial-position effect.

Author

• Jacob Black, Oakton College