Exploring List Experiments in Different Contexts to Measure Drug Consumption

• In Event: Methods for Survey Research

Abstract

Having reliable measures of drug use is a key element to understand the relationship between citizens and illegal substances, and to define public policies aimed to reduce the negative consequences of drug use. Traditionally, researchers have used surveys to capture the prevalence of drug consumption (Smart and Ogborne, 2000); nonetheless there are reasons to think that such measures could be biased since, in many societies, there is a negative image associated with drugs and drugs users. Thus, when answering questions about this topic, citizens may falsify their preferences and experiences as a consequence of what is known as the “social desirability bias” (Fisher et. al., 1993).

Evidence exists that prevalence estimates of drug consumption in countries with stricter drug laws are less valid than in countries with more liberal norms (Steppan et al., 2013). To test the validity of self-reports, a long list of measures was developed: urinalysis techniques, hair analysis, waste water analysis, search engine query data, sales figures of cigarette paper, treatment data (Steppan et. al., 2013; see Harrison 1997 for an extended overview). Looking at those external validation criteria, we know there is enough evidence that the validity of self-reported drug use varies by population subgroup and more stigmatized drugs generate more underreporting (Harrison, 1997).

In an attempt to overcome such bias, it has been common to use methods designed to reduce pressure on respondents and avoid preferences’ falsification because of fearing social sanctioning (Holbrook and Krosnick 2010 review the findings). One of such methods is the list experiment. Despite of the popularization of experimental methodologies in the social sciences, there has been little discussion about the conditions or situations in which the use of the list experiment is needed. The general assumption is that list experiments are a proper tool for measuring preferences and behaviors when dealing with sensitive issues and behaviors. That is the case of racism (Kuklinsli, Cobb & Gilens, 1997), abortion, clientelism (González-Ocantos et. al., 2012), support for militant groups (Matanock and García-Sánchez, 2017), and of course drugs use (Biemer and Brown, 2005; Biemer et al 2005; Couts and Jann 2011).

We think that the pertinence and functioning of this methodology may be conditioned by social contexts. Thus, issues that are deem sensitive in some countries may be less delicate in others (DeMaio 1984). That is the case of drugs use, a behavior that may be openly reported in more liberal societies. We argue that social desirability bias may affect reports of drug use, in particular of marijuana, in more conservative societies and not necessarily in more socially progressive environments. Thus, using indirect or alternative measures to capture drug consumption should be a must in the first type of societies but not necessary in the second type.

To test these arguments, we propose to compare two Latin American societies that differ in terms of social progressiveness: Colombia and Uruguay. Colombia, a majoritarian Catholic country, is one of the most conservative societies in the region both politically and socially, and on the other hand, Uruguay is probably the most liberal country in South America. Thus, reports of drug consumption should be lower when measured directly than indirectly in a conservative society such Colombia; on the other hand, no significant difference, between direct and indirect measures, should exist in Uruguay, since it is a more liberal context.

This paper makes two main contributions. The first one is to validate the list experiment as a proper technique to measure hidden habits or behaviors. Despite the item count technique is proposed as a solution to preference falsification, it is not exempted of problems. In that sense, this paper contributes to the recent discussion about the limits of item count technique to discover hidden preferences (Bauch, Pechenquina and Skinner 2016) by discussing how context impacts the technique's expected functioning. Secondly, it does so by contributing to validate the technique as a proper tool to overcome the social desirability bias registered on direct questions on drug’s use. Knowing in a precise way the prevalence of specific drugs is extremely important to define and evaluate public policies, regardless if the policy is pro regulation or prohibition.

Results from list experiments conducted in Colombia and Uruguay offer evidence that contexts may have an impact on the functioning of list experiments. We find no differences between the direct and indirect measures of marijuana use in Uruguay, the liberal context. On the other hand, the item count technique failed to produce an estimate of marijuana use prevalence in a more conservative context such as Colombia.

Authors

• Miguel Garcia-Sanchez

• Rosario Queirolo, Universidad Católica del Uruguay