Utilizing Mixed Data Analysis Techniques to Interpret Science and Evidence

• In Event: Mixed-Methods Education Research: Is It Science?

Abstract

The discourse surrounding the process of implementing science-based research and acquiring credible evidence has been led by high profile organizations, such as the National Research Council (NRC) and the American Educational Research Association (AERA). NRC has defined specific criteria and standards underpinning the conduct of research termed scientifically based research, and it has defined quality indicators of evidence (NRC, 2001, 2002, 2005; The National Academies, Division of Behavioral and Social Sciences and Education, 2006). AERA guidelines have addressed the importance of reporting research that is warranted relative to authenticating the results and conclusions, and transparent in terms of expounding the logic underpinning the study or research program (AERA, 2006). These guidelines have been complemented by a second set of recommendations pertinent toward conducting humanities-oriented research (AERA, 2009). This top down approach to defining the nature of science, indicators of evidence, and the attributes of scientifically based research has generated animated discussions among academics (cf: Eisenhart, 2005; Eisenhart & Towne, 2003; Feuer, Towne, & Shavelson, 2002; Freeman, deMarrais, Preissle, Roulston, & St. Pierre, 2007; Maxwell, 2004), and has initiated calls for including in the discussion the perspectives and values embraced by various intellectual communities who engage in social science research (Collins, Onwuegbuzie, & Johnson, 2011).

Aligned toward broadening the conversation on these topics, the intent of this symposium is to illuminate a variety of perspectives centered on reflecting how the community of mixed methods researchers situates mixed methods research in this evolving discourse. With this in mind, the goals of this presentation are to discuss the distinct characteristics that distinguish mixed methods research hereafter termed mixed research, and to juxtapose these characteristics to the attributes defining the nature of science and the quality of evidence as portrayed in the published literature. Employing a critical stance, the published works on the topics of the nature of science, scientifically based research and indicators of evidence were read to identify the various definitions of science and evidence, and to identify the author(s) positioning (i.e., representing a particular perspective or viewpoint) on these topics as a means of identifying the assumptions surrounding these interpretations. From this body of published work, specific publications were selected via a maximum variation sampling scheme (Miles & Huberman, 1994). A mixed data analysis was implemented. In the qualitative analysis phase, the selected published works representing various perspectives (e.g., quantitative-orientation, qualitative-orientation, mixed research orientation) were coded utilizing constant comparison analysis (Glaser & Strauss, 1967), and categorized into themes to interpret the varying ways the nature of science and the quality of evidence are portrayed to the reader. In the quantitative analysis, the qualitative coded data were analyzed using correspondence analysis to ascertain the relationship among the themes (i.e., meta-themes). The results of these analyses were interpreted to assess the positioning of mixed research, as a third methodological or research paradigm (Johnson, Onwuegbuzie, & Turner, 2007, p. 129), in the evolving dialogue concerning the nature of science and the quality of evidence.

Author

• Kathleen M.T Collins, University of Arkansas