Management Accounting Section Midyear Meeting

Theory-hacking in Management Accounting Research: A Questionable Research Practice using Survey Data

• In Event: Ethics

Abstract

The social sciences are increasingly concerned with the integrity of data analyses in empirical studies, and management accounting is not excepted. Retraction scandals and replication failures suggest that researchers substantively engage, probably unintentionally, in questionable or unethical research practices, or even data manipulation and fabrication. In this paper, we complement extant analyses of the problem through an analysis of the potential incidence of theory manipulation and fabrication within the realm of survey-based research in managerial accounting. We explore what theoretical degrees of freedom exist that researchers, consciously or unconsciously, may use to retrofit theory to data rather than seek support for the theory in data. Using a previously collected and published survey dataset measuring various standard management accounting constructs, we apply a systematic empirical model to explore theory-fitting opportunities under contemporary scientific data analytic practices. We name the opportunistic behaviour of retro-fitting theory to data ‘theory-hacking’, a symbolic reference to the infamous, yet widely researched, p-hacking. Additionally, we complement our analysis with an iterative simulation based on 1,000 randomly generated datasets showing the generic scope and broader relevance of the underlying issue. We conclude that the current methodological discretion in survey-based research offers almost infinite degrees of freedom to find empirical support for competing theories. We end this paper by discussing the methodological and ethical significance of our findings to the wider social sciences.

Authors

• Frank Hartmann, Radboud University

• Thomas Niederkofler, Radboud University