Spikes and Variance: Using Google Trends to Forecast Protests

• In Event: Geography and Spatial Analysis

Abstract

The big data revolution has brought the social sciences enormous opportunities. One rich source of big data that political scientists have under-appreciated is Google Trends (GT). GT is a service provided by Google that analyzes the popularity of a search query over a given time period and geographical area. It has proved useful in detecting upsurges in everything from retail sales to international travel. Despite its capacity to provide high-frequency data on everything from protests to political identities, GT has been deployed sparingly by political scientists.

In this article we set out to (1) provide clarity around generating, accessing and processing GT data and (2) show how GT can be used to detect protest events ex-post and predict event occurrence ex-ante. On point one, we emphasize the importance of understanding GT's indexing of searches for its application to any particular analytical question. Given GT's approach, any single search on the volume of a term such as 'protest' or 'police violence' can yield deeply misleading results. On point two, we propose a `variance-in-time' method that smooths GT scores over time and incorporates variance in interest into the final forecast, increasing accuracy. The intuition is that more consistent interest in an event may help us predict it as well or better than the mean GT score. The method collects repeated and overlapping samples over a period of three months for every search term and collect both the mean and variance over time. We compute the coefficient of the variation, which we use in our models as our main predictor variable.

We illustrate our recommendations using the search term `protest' during the lead up to the Baltimore protests of April 2015 surrounding the death of Freddie Gray at the hands of police. We show that the coefficient of variation in hourly GT searches for `protest' in the Baltimore metro area over a period of two months does an excellent job of detecting and predicting the events of the 25th of April, the day of the riots. We also show that our approach outperforms GDELT, probably the most commonly used event dataset in political science, and a simple GT search on the volume of `protest' searches in Baltimore.

We then test the efficacy of our approach by relying on known protest events in the United States from Crowd Counting Consortium (CCC) data. We collect GT data for 130 known protest events and for a set of 130 synthetic controls where no protest occurred. For every event in the dataset, weekly collections for a period of 3 months yield a total of 3.6 million data points. We then use machine learning to produce out-of-sample forecasts. Our model aggregates the data at the protest level and builds a logistic regression classifier to forecast which cities had a protest event and which ones did not. Using the lag of the coefficient of the variation of up to two weeks, our model correctly classifies protests 88 percent of the time out of sample, with an AUC of .91. The results show that our novel approach to GT data can forecast protest with higher levels of accuracy and with more anticipation than previous models or models using other social media data such as Twitter. While this specific analysis contributes to broader efforts to detect and forecast bouts of protest, instability and crisis events, our goal is to describe an approach to event detection using GT that is applicable across a broad range of search queries and, thus, research topics.

This article makes several contributions. First, it provides a better understanding of how GT works, the nature of the data it produces, as well as its strengths and limitations. We hope to make it easier for other researchers to deploy GT for their own purposes. Second, we show that GT can usefully detect protests -- even modest sized ones -- at higher levels of temporal and spatial resolution than other, more traditional, data sources. Our `variance-in-time' method is hourly and leverages regional and city-level data. The method we propose smooths GT scores over time and incorporates variance in interest into the final forecast, increasing accuracy. Ultimately, these benefits are not simply empirical as such data forces researchers to think theoretically about causal processes at finer grain than country-years. Third, and despite the limitations inherent in GT, we show that it has considerable potential for predicting rare events. Thus, GT can provide a means for evaluating out-of-sample model performance even as it offers promise as a tool for policymakers and researchers interested in promoting, preventing, or responding to events in the world.

Authors

• Erik M. Wibbels, Duke University

• Joan C. Timoneda, Purdue University