Using Qualitative Analysis of High School Course Catalog to Develop Categories for Quantitative Analysis

• In Event: Design-Strengthening Strategies for Longitudinal and Multilevel Mixed-Methods Studies of Education Systems

Abstract

The objective of this presentation is to warrant and illustrate the use of a development strategy for strengthening a research design in the context of the college-readiness study.
In the MM literature, development is recognized as a plausible rationale for using mixed methods sequentially. Development is the rationale for using MM when the express purpose of using quantitative and qualitative approaches, one after the other, is to further develop or inform the subsequent strand of inquiry.
The exemplar’s initial research design specified a quantitative approach to answering RQ1 (What are the prevalent [English/Mathematics] course trajectories among students and, how do course trajectories relate to students’ college enrollment?) based on secondary data (see Table 1) from the statewide longitudinal data system (SLDS) and National Student Clearinghouse (NSC). A critical first step for analyzing trajectories is to develop policy-relevant category scheme(s) for categorizing courses. The centerpiece of the planned analysis for this RQ was two-level (students within schools) hierarchical linear modeling to determine the relationship between students’ course trajectories and college-readiness.
To improve the interpretability and usefulness of the results, course and trajectory classification systems require context meaningful to educational stakeholders such as state- and district-level partners. Drawing on the development rationale, we reasoned that a qualitative analysis of course catalogs (i.e, primary, publicly available documents developed by high schools to describe academic course offerings) would improve the justification of course categories as relevant statewide and within district. As a result of this development strategy, we refined the plan for answering RQ1 to include a qualitative strand of inquiry designed to answer this question: What courses are offered at high schools across the state? Ultimately, we addressed RQ1 using an exploratory sequential (QUAL QUAN) MM approach.
The analysis of course catalogs (see Table 2) yielded important insight into how course offerings at high schools are similar and different. We used the results to determine options for characterizing courses in terms of content (e.g., Algebra I vs. Geometry) and rigor (e.g., Advanced Placement vs. Dual Enrollment) that could be applied reliably across all schools. We discuss products (e.g., see Figure 1) of the qualitative analysis and explain how they shaped the preparation the subsequent quantitative inquiry.
This development strategy example makes a significant methodological contribution. First, it illustrates the importance of viewing research designs as dynamic documents that can benefit from additions (e.g., research questions, data, and analyses). Second, it highlights that hazard of conceptualizing a component of a complex research design as strictly quantitative or qualitative. Excluding the possibility of using qualitative analysis to inform quantitative analysis (or vice versa) limits the possibilities for strengthening research designs. This example suggests the development strategy can strengthen without increasing scope; however, it does add to person-hour requirements for implementation.

Authors

• Chu Yi Lu, University of Delaware
  Non-Presenting Author

• Henry May, University of Delaware
  Presenting Author

• Marcia Gail Headley, University of Delaware
  Non-Presenting Author

• Jeffrey Robert Klein, University of Delaware
  Non-Presenting Author