What Makes Mixed-Methods Research "Scientific"?

• In Event: Mixed-Methods Education Research: Is It Science?

Abstract

Addressing the question of whether mixed method research is “science” requires a consideration of what science is and what makes an approach “scientific”. The premise of this paper is that being scientific is not a matter of using prescribed methods or following a defined protocol, a point on which I believe most philosophers of science would agree. The website for the American Association for the Advancement of Science states that “Scientific inquiry is not easily described apart from the context of particular investigations. There simply is no fixed set of steps that scientists always follow, no one path that leads them unerringly to scientific knowledge.”
What makes a particular study scientific is fundamentally a matter of its ability to credibly argue for the validity of its conclusions by addressing possible alternative explanations or interpretations of the results, making the latter relatively implausible. This is inherently dependent on evidence and argument, not method per se. Validity is a property of conclusions, not methods (Shadish, Cook, & Campbell, 2002, p. 34; Brinberg & McGrath, 1985, p. 13). Thus, there can be no strict methodological criteria for determining the scientific status of a study. This point is even more important for assessing the “scientific” status of a methodological approach, such as mixed method research. The issue is how the approach is used, in a particular context, to provide evidence for the specific conclusions of a study, and to make implausible alternative conclusions or interpretations of that evidence.
From this perspective, mixed method research has particular strengths. First, it has the fewest restrictions on the sorts of methods and evidence that you can use to gain an understanding of the things you study and to address alternative explanations and interpretations of these. Historically, rather than being a recent development, it is the original form of research, in the natural as well as social sciences (e.g., Galileo’s observations of the phases of Venus; Darwin’s The origin of species), from which all more specialized forms evolved. It thus provides the greatest opportunity for researchers to triangulate observations and data to test or support their initial conclusions.
Second, as argued by Greene (2007), it allows different approaches to develop different understandings of the phenomena studied, and to put these in dialog with one another, allowing researchers to deepen and complexify, rather than simply to triangulate and test, their conclusions.
For these reasons, mixed method research has great potential for being used scientifically to provide valid conclusions about educational and other phenomena. However, realizing this potential will require researchers to use these strengths to generate credible evidence and arguments, systematically assessing plausible alternative interpretations for their results. This is an area where mixed method research could use more development.

Author

• Joseph A. Maxwell, George Mason University