Sifting Meaning from Data: Integrating Computational Text Analysis into Qualitative Inquiry

• In Event: Dreaming Beyond Numbers: Storytelling as a Methodology and Praxis in Mathematics Education

Abstract

Objectives
This paper introduces a sequential mixed-methods design that integrates computational text analysis between cycles of qualitative inquiry during the analysis of large-scale text corpora. Text mining in education is an emerging practice, and scholars use a variety of methods (Ferreira‐Mello et al., 2019). However, traditional methods are said to suffer from an inability to mine for specific insights given the large-scale nature of the data. Our aim was not to reduce narrative to numbers, but to trace the contours of storytelling at scale—mapping how phrases, metaphors, and rhetorical patterns encoded resistance, hope, and possibility. To this end, we ask: What are some of the measurable benefits and potential pitfalls of a sequential mixed-methods qualitative-computational design in the analysis of semi-structured interview textual data?

Perspectives
We build on research on computational text analysis in the humanities (see, e.g., Kuhn, 2019). In education, there has yet to be a comprehensive analysis on the use of computational tools to advance qualitative studies despite the generally large-scale textual data used in qualitative analysis. While the use of computation can support qualitative research in education, there are important considerations in qualitative inquiry that should be considered. Kuhn (2019), for example, explores differences in a scheduling dilemma, a concept focused on fixed vs. open tasks in the analysis of data, and the subjectivity problem in intersubjective stability.

Methods
This analysis is part of a broader study on mathematics educators’ perspectives on the future of the field. In this study, we explore how two open-source software programs—one qualitative and one quantitative—support a sequential mixed-methodological inquiry across multiple stages of analysis and researcher reflections. We build on Evans’ (2021) framing of computational techniques to “measure the impact of particular cases on broader discourse, [examine] how to validate substantive inferences from [a] small [sample] of textual data, and [explore] how to determine if identified cases are part of a consistent temporal pattern” (p. 1). This analysis was situated as a second step in an iterative qualitative inquiry. This second step was informed by a set of open codes developed by our research team during the initial analysis of data.

Results
Exploratory findings indicate three initial themes: first, as noted by Evans (2021), the need for a reflexive method to make sense of topic themes; second, a computational process within the larger sequential mixed methods analysis that supports thematic and topic model selection in response to the scheduling dilemma; and, third, a post computational reduction analysis to ensure a response to the subjectivity problem in the second, post-computational qualitative analysis.

Significance
This study supports future inquiries into how textual data can be analyzed through computational tools without losing sight of both meaning and context. While computational methods are often viewed as antithetical to qualitative richness, we argue the opposite: when embedded in a critical qualitative framework, these tools can amplify rather than flatten participant voices. In this study, computational storytelling became a way to listen—at scale—to the dreams, doubts, and designs that participants shared.

Authors

• Nathan Alexander, Howard University

• Darnell Leatherwood, University of Michigan