Comparing AI and researcher analyses: Analyzing learning issues boards to inform program development of teacher professional learning

• In Event: Exploring AI’s Role in Educational Leadership, Teacher Preparation and Program Development

Proposal

Problem-based learning (PBL) is an inquiry-based model of curriculum and instruction that uses an ill-structured problem to engage students’ natural curiosity. Teachers act as guides, but students drive the learning by asking questions, working together, conducting research, and reflecting on their learning process (Collins & Kapur, 2015). The ill-structured problems are engaging, real-world scenarios that help students develop questioning skills, the ability to collaborate, and problem-solving skills. Key to PBL instruction is the Learning Issues Board, a collaborative graphic organizer that is used at the beginning and end of every class session. The Learning Issues Board is designed to help students sort and process information, note questions, and plan for the prescribed task of the problem-based learning unit.

For teachers, problem-based learning requires a shift in practice away from traditional, teacher-led instruction. To teach PBL with fidelity, teachers need professional learning to understand not only how PBL works, but how to enter and stay in a cognitive apprenticeship with the students. Active engagement with the curriculum and opportunities to learn in the same way they will be teaching students (Hill & Papay, 2022; Opfer & Pedder, 2011) helps teachers learn the skills needed to be good facilitators of PBL. Immersive professional learning using the Learning Issues Board shows teachers the skills they will need to model later in the classroom.

But, even with a well-developed professional learning program, teachers come to understand PBL differently. When teachers return to their classrooms and start to implement PBL on their own, they often struggle the first couple of times. Ongoing support, district or state contexts, and teachers’ tolerance for ambiguity have all been considered as contributing factors to differences in teacher implementation and fidelity.

This applied research study aims to understand how teachers understand problem-based learning through the use of the Learning Issues Board in professional learning sessions while analyzing the data for potential program development. The following research questions are currently being explored:

1. In what ways do teachers understand problem-based learning (PBL) through the use of the Learning Issues Boards?

2. In what ways do teachers consider their individual district or state contexts?

3. How might the Learning Issues Board data used to inform program development and improvement?

Three Learning Issues Boards from Project ExCEL-Ignite (Horak & Shaklee, 2019) professional learning sessions were selected. Project ExCEL-Ignite is a federally-funded (U.S. Department of Education) Jacob K. Javits grant that works with three school districts in the United States and is focused on problem-based learning. Learning Issues Boards were selected based on later participation in PBL curriculum unit implementation. Each Learning Issues Board was completed during a one-day professional learning session that ran approximately six hours excluding breaks. Teacher participants in the professional learning sessions varied in age, years of teaching experience, race, content area taught, and gender. All teachers worked with middle school students.

Two experienced facilitators were responsible for the professional learning sessions. Both are veteran educators and have extensive experience with problem-based learning. In two cases, the facilitators worked together during the professional learning session.

Because the Learning Issues Boards are artifacts rather than interviews, the researcher initially took some time to complete the initial coding of each board to understand the flow of professional learning as represented on the board. After this phase was complete, descriptive and process coding was completed by hand on each board. Codes were refined and another coding run was completed. At this point, the researcher looked at all three boards together and from that, created a codebook complete with examples. Initial themes included various elements of problem-based learning, references to the stakeholder identity, issues and approaches to gifted identification and advanced coursework. From this, the researcher then looked to two different AI tools - ChatGPT 4 Plus and Google Gemini Advanced to complete additional descriptive and process coding with the intent of comparing the hand coding to the findings in the artificial intelligence tools to assist in answering the research questions and refining themes. Using these digital tools also allowed the researcher to complete analyses unaffected by researcher perceptions or biases.

The AI tools employed found ideas similar to the researcher’s hand coding. The themes generated by ChatGPT 4 Plus and Google Gemini Advanced captured the focal points of problem-based learning in addition to the professional learning outcomes. Compared to the researcher, the AI tools analyzed the data more widely, capturing broad strokes of professional learning rather than individual skills. This could help plan future professional learning as different components may be able to be grouped under specific topics. Although analysis is ongoing, this presentation will focus on the potential use of AI tools to inform program development and improvement. The significance of this research to a CIES audience and those attending the conference is the focus on the use of integrating digital technologies, specifically AI tools, into research and professional learning program development. The preliminary findings indicate that these tools may be beneficial for improving teacher professional learning programs as they may suggest additional topics that teachers focus on in professional learning sessions. This will help facilitators and those who design professional learning to better serve teachers’ needs.

Author

• Kimberley Daly, George Mason University