Strengthening the role of qualitative research in education

Group Submission Type: Pre-conference Workshop

Description of Session

Qualitative research methods can play an important role in program evaluation, especially with a focus on the Global South and research contextualization, but often they are considered “second class” methods, when compared to quantitative evaluations, and especially to experimental methods. When researchers want to know ‘what works’ quantitative methods are commonly selected instead of qualitative methods, however, without good qualitative data to contextualize these findings, ‘how or why things work’ can often remain obscured. Thus, the Building Evidence in Education (BE2) working group, a group of over 30 funders of education programs, is developing a guidance note to help commissioners of research and researchers design and implement qualitative research that uses a high level of rigor.

The guidance note outlines:

• When to use qualitative data collection for which questions;
• How to manage primary data collection, including sampling;
• Methods in qualitative analysis;
• Ethics;
• Challenges, including researcher bias;
• Available tools.

Proposal

Title: Strengthening the Role of Qualitative Research in Education

Workshop Leaders:
Dominic Richardson, Senior Education Specialist at UNICEF, Office of Research – Innocenti; BE2 Steering Committee Member.
Rachel Hinton, Senior Education Advisor, Research & Evidence Division, DFID; BE2 Steering Committee Member.
Sarah Jones, Senior M&E Advisor, Evidence Team, Office of Education, Bureau for Economic Growth, Education and Environment (E3) United States Agency for International Development (USAID).


Workshop Rationale

Qualitative research methods play an important role in program evaluation, especially with a focus on the Global South and research contextualization, but often they are considered “second class” when compared to quantitative evaluations. When researchers want to know ‘what works’ quantitative methods are commonly selected instead of qualitative methods, however, without good qualitative data to contextualize these findings, ‘how or why things work’ can often remain obscured’. Thus, the Building Evidence in Education (BE2) working group, a group of about 30 funders of education programs, is developing a guidance note to help commissioners of research and researchers design and implement qualitative research that use an equivalent level of rigor and command an equivalent level of respect as quantitative research.

Most qualitative studies are used to explore and understand the mechanisms of social phenomena, rather than assess the effect or attribution of action. Within each of the research streams - interviewing, observation and documentation analysis - various different approaches are used.
The purpose of qualitative analysis is “sense making”, i.e., to understand how in the program understand, think about, make sense of, and manage situations/environments and/or to describe the contexts in which a program is being implemented. This often answers “how” and “why” questions. Qualitative data in program evaluations is most often used to describe or document aspects of a program (i.e., in formative evaluations), rather than to demonstrate causality (i.e., program effectiveness or outcomes).

The key objectives of the workshop are:
• To provide peer evaluation from academics and researchers for the draft guidance note;
• To help evaluators and funders of research in making methodological choices and commission or deliver a usable, credible qualitative research product.
Workshop participants will also learn how assess the quality of a qualitative research and gain an understanding of when qualitative methods are used appropriately.

For this purpose, the workshop leaders will:
• Share proposed guidance about how to conduct excellent qualitative research with workshop participants;
• Familiarize participants with specific components of the guidance note, including when and the extent to which qualitative methods are appropriate, e.g., based on evaluation’s purposes and intended uses;
• Have in-depth discussion on approaches and methods; gauge level of agreement with recommended guidance;
• Get input from potential research commissioners and researchers on how to strengthen the guidance note, e.g., regarding potential gaps in the guidance note, methodological issues, and implementation challenges;

Discussions will include when to use qualitative approaches, sampling strategies, sample size considerations, data analysis and ethical questions.

Timing:
1. During the workshop, the workshop facilitators will present and share different components from the draft guidance note with participants (1 hour).
2. The group will then break into small working groups and use tools and have guided discussions to provide input, feedback and suggestions on the draft document, and contribute suggestions on how to strengthen the content of the note (1 hour).
3. The break-out session will be followed by a feed-back session in plenary (45 min), and finish with a
4. Discussion about dissemination of the final guidance note (15 min).

Rigorous implementation of a sound research design, careful sampling, well thought-out data collection and capture, and methodical data analysis are essential components to a high quality, rigorous qualitative research and will lead to valid, useful findings.

Duration: 3 hours

Size: Maximum 60 people

AV and Room Set-up Needs:
• Projection of slides from computer (presenters will bring presentations on a flash drive; NOTE: please let us know if we need to bring our own computer)?
• Round table set-up so people can have discussions in smaller groups.
• 3 Flip charts, if possible

Sub Unit

• Pre-Conference Workshops

Workshop Organizers

• Rachel Hinton, DFID

• Dominic Richardson, UNICEF

• Elena Walls (Vinogradova), US Agency for International Development (USAID)

• Maria Brindlmayer, Making Cents International

• James O'Donoghue, DFID