Designing a Longitudinal and Multilevel Mixed-Methods Study

• In Event: Design-Strengthening Strategies for Longitudinal and Multilevel Mixed-Methods Studies of Education Systems

Abstract

The objective of this introduction to the methodological presentations is to highlight the gap in the methodological literature and initiate discussion about design-strengthening strategies for complex mixed methods (MM) research designs with longitudinal and multilevel features.
Developing research linking data to policy for the purpose of improving education presents formidable methodological challenges. It is, for example, difficult to design research accounting for the complexity and diversity of settings and students. In the absence of information about how schools change over time or differ from one another, policymakers may be hesitant about applying research findings in particular schools or conditions. Recently, three research approaches have gained attention as approaches for making research more relevant to policymakers. Longitudinal designs have been used to reveal trends and characterize effectiveness (e.g., to explore the impact of a policy change over time). Multilevel designs have been used to study the relative contributions of environment and individuals in educational processes (e.g., to explore whether a process is more influenced by school climate or student characteristics). Finally, MM research designs have been used to integrate quantitative and qualitative research strategies and generate findings that transcend what might be discerned from either strategy alone (e.g., to explore how students’ academic performance relates to their perceptions of academic potential). Given the advantages of each approach, it is reasonable to hypothesize that combining the strategies in a single study would increase chances of connecting educational research and policy in a way that promotes enduring equity in education and limits unintended injustices. To our knowledge there are few research studies intentionally designed to maximize the combined benefits of these three research approaches. This may be due to limited discussion in the methodological literature about how to navigate the inherent difficulties of integrating advanced methodological strategies in a single study.
This presentation will be delivered in two parts. First, we review three methodological approaches – longitudinal, multilevel, and MM –and exemplify their combination. The exemplar is a statewide exploratory study of college-readiness study aimed at improving post-secondary outcomes. Exemplar artifacts, including research questions and joint displays (Plano Clark & Sanders, 2015), will be examined to compare the original MM design to the refinements. These overarching research questions, and the data (see Table 1) used to answer them, will be discussed in terms of their longitudinal, multilevel, and MM features:
RQ1. What are the prevalent [English/Mathematics] course trajectories among students, and how do course trajectories relate to students’ college enrollment?
RQ2. What is the impact of proficiency labels on students’ course selection and course trajectories?
RQ3. How do schools influence students’ course trajectories, and what data/processes do schools use to promote college readiness?
RQ4. What data or processes have the potential to inform college-readiness policy and practice refinements within or across schools?
Tensions between the three approaches (e.g., paradigmatic perspectives, feasibility constraints) will be explored. Second, we review Greene’s MM rationale typology (Caracelli & Greene, 1993; Greene, 2007) to introduce the MM concepts that ground the design-strengthening strategies discussed in the following presentations.

Authors

• Marcia Gail Headley, University of Delaware
  Presenting Author

• Henry May, University of Delaware
  Non-Presenting Author

• Jeffrey Robert Klein, University of Delaware
  Non-Presenting Author

• Katrina K. Morrison, University of Delaware
  Non-Presenting Author

• Roderick L. Carey, University of Delaware
  Non-Presenting Author