Comparing Methods of Creating a Nationally-Random Sample of Twitter Users

• In Event: Methodological Developments for Social Media

Abstract

One of the key challenges in working with Twitter (and similar platforms) data is to obtain a random sample of users from a country. The goal is usually to get a platform or population representative sample of users. However, there are at least two major challenges in obtaining random samples of users from a country: 1) several methods have been proposed in the literature and it is not clear which one is the best, and 2) the extent to which these samples are representative of the population or even platform is questionable. In this paper, we compare these methods with respect to their results’ size and representativeness of topics, events, platform, and population.

There are four popular methods to devise a random sample of Twitter users for a specific country: 1) Twitter 1% stream filtered for language and country; 2) Querying Twitter’s search-tweet API for a specific language and country; 3) Randomly generate potential user IDs and filter for language or country; and 4) Querying Twitter API based on a country name or its bounding box coordinates. The extent to which these four methods produce similar results and which one is more representative of population and platform is mostly unexplored. To create a nationally-representative sample, we follow a recently proposed approach based on multilevel regression methods to estimate the probability of an individual with given demographics to be on a given platform. We use a multimodal deep learning model to infer age, gender, and location of Twitter users. We compare the results of each sampling method according to six criteria: 1) number of tweets, 2) number of unique users, 3) geographical distribution of users, 4) distribution of tweets over time and their correlation with major national events, 5) distribution of topics covered by users’ tweets, and 6) being suitable to create a nationally-representative sample.

Authors

• Meysam Alizadeh, University of Zurich

• Darya Zare, Amirkabir University of Tecnology

• Maël Dominique Kubli, University of Zurich

• Fabrizio Gilardi, University of Zurich