The Math for All Education RCT: Contrasting an Enacted Design With the Potential of Mixed Randomized Controlled Trials

• In Event: 32.066 - Presentation of a Mixed-Methods Randomized Controlled Trial Design and Lessons Learned (So Far)

Abstract

Mixed RCTs may become more common given advances in philosophical work that promotes mixing (e.g., Johnson, 2015). There are potentially several advantages to adopting multi-paradigmatic thinking at philosophical, methodological, and analytic levels in the context of Mixed RCTs because doing so can help researchers better understand different levels of causal inference, including nomothetic (classic group differences), idiographic (intervention impacts for an individual within a study), and causal mechanisms (Hitchcock, Johnson, & Schoonenboom, 2016). If following a full integration style in mixed methods (Teddlie & Tashakkori, 2009) then qualitative work need not be thought of as an add on to an experiment that helps explain findings, one can also use such inquiry to inform causal inference via developing a deep understanding of treatment and control groups (and what about treatment exposure makes them different), helping researchers fully consider sample particulars, explore how treatment effects bear out across different research entities, examine different participants’ perspectives of the intervention, and so on.
Despite the potential advantages of a Mixed RCT, their use is arguably nonexistent in education because advanced mixed analyses needed to carry them out has only started to be described in the mixed methods literature (e.g., Onwuegbuzie & Hitchcock, 2015). Fully working out the details and potential of Mixed RCTs in education is therefore aspirational. To promote the development of a Mixed RCTs, this paper will review details of a multilevel RCT in education that utilizes robust qualitative components. The RCT focuses on a teacher professional development package called Math for All (MFA; Moeller et al., 2013) and it is being carried out in an urban setting that contends with a number of labor disputes and resource limitations. The paper will describe plans for working toward a set of fully integrated analyses and meta-inferences that will inform causal inference at varying levels (see above) and understand broad aspects of implementation details needed to inform the intervention. The contribution of this paper is that detailed design plans of an actual RCT rather than general design advice will be offered, moving the field closer towards the realization of fully Mixed RCTs.

References
Hitchcock, J. H., Johnson, R. B., & Schoonenboom, J. (2016, April). Ideographic and nomo¬thetic causal inference in special education research and practice: Mixed methods perspectives. Paper presented at the American Educational Research Association Annual Meeting [Mixed Methods Research SIG], Washington, DC.
Johnson, R.B. (2015). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 1-18. doi: 10.1177/1558689815607692
Moeller, B., Dubitsky, B., Cohen, M., Marschke-Tobier, K., Melnick, H., & Metnetsky, L. (2013). Mathematics for All: Facilitator guide for grades K–2. Thousand Oaks, CA: Corwin Press.
Onwuegbuzie, A.J., & Hitchcock, J. H. (2015). Advanced mixed analyses. In S. N. Hess- Biber & R. B. Johnson (Eds.), Oxford Handbook of Mixed and Multimethod Research (pp. 275-295). Oxford, England: Oxford University Press.
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative techniques in the social and behavioral sciences. Thousand Oaks, CA: Sage.

Authors

• John H. Hitchcock, Indiana University - Bloomington

• Ellen B. Meier, Teachers College, Columbia University