Epistemic Authority at the Crossroads: Who Should Be the Arbiters of Knowledge— Humans or Machines?

• In Event: Science, Knowledge & Technology Section Roundtables
  In Refereed Roundtable (60 minute): Table 5: Knowledge Production in the Age of AI

Abstract

This research examines how the scientific community perceives the growing use of artificial intelligence (AI) in peer review. Framed around moral, pragmatic, and epistemic legitimacy, it analyzes how researchers evaluate AI versus human decision-making with respect to bias, efficiency, and the epistemic character of the work under review. Focusing on sociology, we conduct regression analyses using data from a vignette-based factorial survey built from abstracts in the American Sociological Review Digital Archive. The survey assesses whether AI involvement is seen as enhancing or undermining peer review and whether these perceptions vary systematically across methodological and epistemic orientations. By integrating survey responses with text-analytic and bibliometric measures of epistemic structure, we examine whether AI is perceived to advantage particular forms of sociological knowledge, especially research that is more codifiable (quantitative), closely aligned with journal norms, or highly specialized. This approach allows us to address broader concerns about how AI may reshape power and inequality in knowledge production. Findings indicate that researchers who perceive peer review as inefficient are more receptive to AI as a corrective and more willing to accept its risks when they view the current system as not being efficient. In contrast, those who see existing processes as relatively legitimate—particularly regarding bias—are more likely to view AI as introducing new forms of unfairness. Moreover, less codifiable (more qualitative) research is perceived as more vulnerable to bias when evaluated by AI. The findings of our research suggest that integrating AI into peer review raises a deeper question: how is epistemic authority being reconfigured, and who should arbitrate scholarly knowledge—humans or machines?

Authors

• Laurie A Schintler, George Mason University

• Thomas Hinz, University of Konstanz

• James C. Witte, George Mason University

• Connie L. McNeely, George Mason University

• Emma Vetter, George Mason University

• Chad Smith, George Mason University