Individual Submission Summary
Share...

Direct link:

One, many, and millions: Is citizen science beyond the reproducibility crisis?

Sat, September 2, 4:00 to 5:30pm, Sheraton Boston, Floor: 3, Exeter

Abstract

As social science, especially psychology, is battered by controversies over unreproducible studies, two alternative "citizen science" methodologies are advanced by the Quantified Self movement. "N=1" studies are undertaken by assiduous self-trackers so as to improve their wellness, longevity, and performance with personalized analytics and interventions. Also, millions of self-trackers can aggregate their data into massive studies, with the benefits of increased subject diversity and analytic statistical power. I will consider "one" and "million" subject studies in light of a recent controversy over a prominent "many" subject study (N=42).

In 2016, Dana Carney repudiated her work on "power poses," undertaken with Amy Cuddy and Andy Yap. Cuddy popularized their work with a TED talk and book wherein she claimed that "power posing," taking a confident posture for two minutes, increases confidence, risk taking, and testosterone, decreases cortisol, and improves how one is perceived in "stressful evaluative situations," like a job interview. Subsequently, Carney, who with Yap did most of the data collection and analysis, disassociated herself from the power pose. In hindsight, Carney feared they may have been measuring an artifact of the experiment rather than "an expansive posture effect."And in the analysis, they ignored data, selectively removed outliers, and reported only the statistical tests that showed significance.

Although citizen science has merits, I will describe how the problems associated with traditional "many" subject studies (e.g., N=42) are equally applicable to N=1 and big data studies, touching on erroneous tracking, flawed methods, placebo effects, and p-hacking.

Author