When the Experimental Hand Errs How Data Politics can Derail Experimentation in Platform Organizations

• In Event: 40860 - Section on Economic Sociology Roundtables
  In Refereed Roundtable (60 minute): Table 10. Innovation and entrepreneurship

Abstract

Experimentation is central to organizational decision-making, with digital platforms leveraging data analytics and machine learning for rapid iteration. While ‘scientific management’ promises agility and innovation, internal organizational dynamics can disrupt experimental regimes. Drawing on 12 months of ethnographic fieldwork at the digital labor platform “iLabour,” we examine the organizational challenges of implementing a new algorithmic matching system. iLabour sought to replace its loosely structured job-matching system with a taxonomy-driven approach to improve platform efficiency and market legibility. While initially supported, the project became entangled in internal conflicts and technical constraints, ultimately stalling. We show how platform experimentation created a form of “data politics” that facilitated ambitious initiatives but hindered their completion. Two key mechanisms drove this derailment. First, violations of experimental assumptions disrupted clean implementation. Treatment and control groups could not be effectively separated, and the intervention itself altered user behavior, making it impossible to generate clear results. Those advocating fundamental change initially held political influence due to their use of recognized analytical technologies, but ambiguous findings weakened their position relative to groups defending specific user interests. Second, regimes of experimentation led to rapid shifts in organizational beliefs. Early support for change united diverse occupational groups, but once a prototype launched, market feedback triggered divergent positions on whether to refine or abandon it. Struggling to maintain support, the company halted further iterations. Instead of transitioning to a new system, the experimental process produced a hybrid architecture where older features persisted alongside new ones, creating inefficiencies rather than optimization. Our findings contribute to research on technology and expertise by showing how experimentation, when embedded in complex platform architectures, can lead to technological entrenchment rather than iterative improvement. This study underscores the need to reassess how data-driven management interacts with organizational expertise, internal politics, and platform design constraints.

Authors

• Georg Rilinger, MIT Sloan School of Management

• Greetje Corporaal, Erasmus University Rotterdam