Combining Machine Learning and Qualitative Analysis to Study Public Discourse Around Opting Out of Testing

• In Event: 34.094 - Fairmont Royal York Roundtable Session Six
  In Roundtable Session: 34.094-6 - Analytical Approaches in Multimethod and Mixed-Methods Research

Abstract

We introduce an analytic approach that supports the exploration of public discourse around educational policies, using data sources that might otherwise be intractable (e.g., Twitter). Statistical classifiers serve as a first pass at sorting tweets into analytic samples. Drawing on our feedback loop between qualitative research and machine learning, we focus on the impact a systematic review of training data has on both the emerging qualitative coding scheme as well as model performance. We demonstrate this method by using tweets related to the opt-out movement, and iteratively building models on refined training sets aligned to sub- codes. Ultimately, the precision of classified text improves through this approach, and a discriminative model outperforms a generative model.

Authors

• Amy Burkhardt, University of Colorado - Boulder

• Terri S. Wilson, University of Colorado Boulder

• Wagma S. Mommandi, University of Colorado - Boulder

• Michele S. Moses, University of Colorado - Boulder