Big Data vs. Survey Methodology

• In Event: Social Media and the Public Sphere: An Agenda for Future Research

Abstract

For many years, surveys that use rigorous random samples and high-quality operations have been used as leading indicators to inform policy, as well as to answer questions in the social sciences. For decades, scientists and survey practitioners have been refining the different aspects of surveys to consistently provide high quality data. However, surveys are an expensive proposition, and often operate on relatively slow time scales. Recently, some researchers have been using data mined from social media channels to generate similar insights as those as have been traditionally the purview of these “gold-standard” surveys. While there are legitimate and serious critiques of the errors introduced by these Big Data techniques, there are also reasons why so many of these studies have been conducted recently. For example the cost of Big Data techniques can be low compared to survey methods. In addition, it’s easier to collect longitudinal data, and to collect data on emergent events close to the event. While many have reflected on the weaknesses of Big Data techniques, this is still a young field that has not gone through the decades of refinement of survey methods, though it’s an open question of whether such process improvements can overcome limitations of coverage inherent in the systems from which data originates. In addition, we know little about whether topic dependencies affect the tension between surveys and Big Data as methods.

Author

• Cliff Lampe, University of Michigan