Evaluating Special Techniques for Surveying Sensitive Topics: An Approach that Detects False Positives

• In Event: 321 - Regular Session. Methodology, Quantitative

Abstract

The valid measurement of sensitive issues such as norm-violations or stigmatizing traits through self-reports in surveys is often problematic. Special sensitive question techniques such as the Randomized Response Technique (RRT, Warner 1965) and among its variants the recent Crosswise Model RRT (Yu, Tian and Tang 2008) should generate more honest answers by providing full response privacy. However, the RRT sometimes does not work as expected and evaluating whether particular implementations improve data validity is essential before their application in substantive surveys. To this end, studies mostly compare prevalence estimates of sensitive traits and behaviors of different techniques. Assuming that respondents only falsely deny but never falsely admit a sensitive trait or behavior, higher prevalence estimates were interpreted as more valid estimates. However, if false positives occur, i.e., if respondents are misclassified as bearing a sensitive trait although they actually do not, conclusions drawn under this assumption are likely wrong. False positives occurred in a variant of the Crosswise Model. Other RRT variants might be affected too, but studies so far have largely neglected this possibility. We show an evaluation approach that detects systematic false positives without the need of a validation criterion – which is often unavailable. Results from an application in a survey on “Organ donation and health” (N = 1’686) show, that the Crosswise Model RRT produced false positives to a non-ignorable extent. This finding is in line with a previous validation study, but has not been revealed by several comparative evaluation studies that did not consider false positives.

Authors

• Marc Höglinger, University of Bern

• Andreas Diekmann, Swiss Federal Institute of Technology ETH Zurich