Comparing Qualitatively and Quantitatively Defined Trajectories: Available Versus Traveled Trajectories

• In Event: Design-Strengthening Strategies for Longitudinal and Multilevel Mixed-Methods Studies of Education Systems

Abstract

The objective of this presentation is to warrant and illustrate the use of a triangulation strategy for strengthening a research design in the context of the college-readiness study.
In the MM literature, triangulation is recognized as a plausible rationale for using mixed methods concurrently. Triangulation is the rationale for using MM when the express purpose of using quantitative and qualitative approaches is to reveal the ways in which the findings generated from two different perspectives point to convergent or divergent conclusion(s).
The initial research design specified a sequential MM (QUAN  qual) approach to answering RQ2: What is the impact of proficiency labels on students’ course selection and course trajectories? The centerpiece of the planned quantitative analysis was regression discontinuity (RD) analysis using secondary data from the SLDS and the NSC (see Table 1). RD analysis is a quasi-experimental method that compares similar (but not randomly assigned) groups. In this study, RD analysis was planned to compare academically similar groups of students (receiving assessment scores within the standard error of cut-scores) to estimate the effect of being classified as proficient (above the cut-score) or not-proficient (below the cut-score) on future high school and college course enrollments. The planned follow-up qualitative strand of inquiry involved interviewing a subset of the students in the RD analysis (from a subset of recruitment schools across the state) to get students’ perspective on the impact of being informed about proficiency level.
To improve the downstream explanatory power of qualitative themes, we refined the plan for selecting recruitment schools for interview students. RQ1 generated qualitative information about schools’ course-offerings (Figure 1) and quantitative information about students’ course-taking (Figure 2). In isolation, these offer little insight into how schools differ in terms of academic climate (i.e., communal college-readiness expectations). We planned a triangulation strategy to increase the likelihood of selecting recruitment schools that differ in this way. We reasoned that we could compare available trajectories (i.e., sequences of courses described in school catalogs) to traveled trajectories (i.e., sequences of courses completed by students). The Sankey charts (see Figure 1 and Figure 2) allowed us to visually compare available and traveled course trajectories within school. Between-school comparisons of visualization contrasts were expected to produce insights into whether there are schools that offer similar academic trajectories but have different proportions of students completing them. The results enhanced the planned recruitment school selection.
This example illustrates important points about triangulation in the context of mixed methods. Triangulation in mixed methods does not always mirror its historical meaning. In antiquity, triangulation referred to pinpointing the location of one offshore point given two known onshore locations (using theorems of geometry). In commonplace contemporary parlance, the term triangulation is often used to emphasize the goal of reaching a single conclusion. However, in mixed methods triangulation may be better understood as the use of two perspectives (Moran-Ellis et al., 2006). In this instance of triangulation, we are mixing methods with an eye toward making rough distinctions, if they exist, in academic climate.

Authors

• Marcia Gail Headley, University of Delaware
  Presenting Author

• Chu Yi Lu, University of Delaware
  Non-Presenting Author

• Henry May, University of Delaware
  Non-Presenting Author