Integration, Explainability, and Secondary Data in the Age of Machine Intelligence: Toward Retrofitted Research Design

• In Event: AERA Roundtable Session Saturday 3:20 pm Four Seasons Ballroom 2-3
  In Roundtable Session: Advancements in Mixed Methods Research Powered by Artificial Intelligence (Table 8)

Abstract

As machine intelligent systems move toward ubiquity, the secondary data used to train those systems is increasing in importance. Whether or not our research is directly connected to machine intelligence, international pushes for open science will make the data we produce as researchers available as secondary data. This paper aims to make a methodological intervention to “retrofit” design to secondary data. In outline, the procedure is: 1) determine the provenance of the data and perform an evaluation of its suitability; 2) identify the vectors of generalizability represented in the data; 3) determine which sets of data are quantity-dependent and which are quantity-agnostic; 4) identify potential forms of drift; 5) produce a relational map of how various data subsets intersect and tessellate.

Authors

• Jasmine Ulmer, University of Florida

• James Salvo, University of Florida

• Johanna Creswell Báez, University of Colorado - Colorado Springs