What Are We Mixing Here? Some Questions About Questions?

• In Event: Mixed-Methods Education Research: Is It Science?

Abstract

The question about mixed methods routinely draws fire unless it is a crowd of believers. The question of whether mixed methods constitute legitimate social science, or any kind of science, seems to be the wrong question. The real questions revolve about other issues, such as How shall we define science? Who gets to define it? What kind of findings does mixed methods provide, that is, can such findings be used for policy formulation, and if so, what are the results of evaluating such enabling? Likely no one is theoretically opposed to the idea of mixed methods. I argued for their use long before this debate started (Guba & Lincoln, 1981, 1989, 1995; Lincoln & Guba, 1985, 2000; Lincoln, Lynham & Guba, 2012). The major questions, however, still remain stubbornly poised for answering. What kind of theory of reality or knowing undergirds such methods? Do we retreat to the standards for judging each method of data collection and analysis as a means of fulfilling criteria of rigor and validity? Or is there some additional and overarching “paradigm” which, when deconstructed, yields logical criteria rooted in the theories of reality and epistemologies themselves? So the questions for this author revolve about whether the results of mixed methods research adhere to some well integrated model or theory of knowledge, or whether, as some proponents claim, they can “mix and match” along some set of beliefs about the nature of the social world; about whether the knowledge produced is useful knowledge for guiding public debate about proposed policy; about whether the findings are transparent, whether they are consonant with some inherent logic which is visible to users, whether they add to our understanding of a complex and pluralistic social life; and, whether or not they exhibit high-quality standards of human protection, including attention to diverse stakeholders and conflicting belief systems of respondents, careful data collection, systematic and disciplined analyses, and thoughtful and insightful (rather than hurried, haphazard) discussion of what the findings might mean.
Moreso than the criteria for validity, there is a larger concern surrounding mixed methods research, and that is the ethical dimension. Experimental social science operates with one set of principles, set out by the Belmont statement, and qualitative/ethnographic/interpretive inquiry operates with a far finer-grained set of standards which guide the interactions and relationships between respondents and researchers. Which set of standards will prevail in mixed-methods research—the looser standards of the quantitative practitioners, or the more nuanced and vulnerable ethics of the ethnographic researcher? These are the questions which should be guiding the mixed methods debate, not the National Research Council’s (2002) narrow, rigid, 19th century view of what constitutes science.

Author

• Yvonna S. Lincoln, Texas A&M University - College Station