An Investigation Into the Complexity and Robustness of MMR (Mixed-Methods Research) Data Collection and Analysis

• In Event: The Current State of Mixed-Methods Research: A Systematic Review and Panel Discussion

Abstract

Objectives
A focus of this systematic review is centered on data collection and analysis. The review research questions included: What type of data are mixed methods researchers collecting? Is there a level of robustness in data collection across both strands? Are researchers explicitly stating how they emphasized the qualitative and quantitative data and discussing issues of validity? And to what level of complexity are they analyzing the data?

Perspectives
Tashakkori and Teddlie (2010) call for “methodological eclecticism,” a necessary dexterity in mixed methods research to protect against banal methods of data collection (Teddlie & Tashakkori, 2012). They state that mixed methods calls for sophisticated analyses, yet do not specify what these are.

Quantifying qualitative data is one method of collecting both types of data for a mixed methods study. This data reduction technique creates opportunities for statistical analysis, but at the cost of losing the richness of the qualitative data. A potentially even more severe consequence of this method is the possible collinearity the researcher generates in the coding process, which could then create issues for the quantitative analysis (Driscoll et al., 2007).

In 1991, Morse proposed four notations for either simultaneous or sequential designs that used symbols (+, →) and capitalization to communicate the design. Leech and Onwuegbuzie (2009) expanded those four to 21. Additional symbols emerged to represent embedded methodology, including parentheses, brackets, and double arrows. Cameron et al. (2013) proposed the Extended Mixed Methods Notation System for their study, a system that includes additional notations to depict sample size, data samples, and type of data collection and analysis.

Like other methodological decisions, identifying data analysis techniques must be affixed to the research questions and purpose of the study. An interesting development is the call for a new typology of validity centered on the concept of legitimacy (Onwuegbuzie & Johnson, 2006). They outline nine types of legitimacy and propose that these mixed methods validation strategies are incorporated into the established quantitative and qualitative methods of establishing validity.

Methodology and Data Sources
Within the Mixed Methods Educational Research Review, the analysts applied several categories to capture how authors conveyed the conceptualization of their mixed methods studies within the publications. These categories included: (1) whether the emphasis of the qualitative and quantitative data is explicated (in a figure, embedded in the text, or as a notation), (2) the qualitative and quantitative data collected, (3) the type of analyses conducted and (4) discussions of validity.

Initial Results
Initial review results indicate that statistical analyses are more descriptive than inferential and the discussion of in-depth qualitative analysis strategies are atypical. Validity is rarely addressed. Most authors describe how they emphasized the qualitative and quantitative data.

Scholarly Significance
Teddlie and Tashakkori (2012) argue that a mixed methods scholar must be a “connoisseur of methods,” holding all skills necessary to fully address their research questions (p. 777). They push for a kind of “methodological bilingualism” while noting the difficulty of attaining this level of expertise in both qualitative and quantitative methods (p. 777).

Author

• Jessica T. Decuir-Gunby, University of Southern California