Lessons Learned About Using Interviews to Assess Classroom Practices

• In Event: Math for All: Lessons Learned From Piloting a Randomized Controlled Trial in a Large Urban District

Abstract

Our spring 2015 data collection included the development and pilot of semi‐structured interviews of teachers and principals. Moving forward, in the implementation year (2015-2016), 25 teachers in the treatment and 25 in the control groups will be interviewed, as well as all the principals in the 32 participating schools, and again in the 2016-17 impact year. The information from these interviews will (a) provide qualitative information about the instructional practices of teachers in the study, (b) help us understand the factors that may affect the fidelity of the implementation, and (c) provide insights into the similarities and differences in treatment and control group classroom practices. The interview data will also help us to refine the logic model for the intervention (see Figure 3).
Lessons learned from the pilot
The pilot interviews included a total of 16 teachers (out of a total of 18 teachers), and four principals (out of a total of four). The interviews were conducted over a two-week period in June 2015 and averaged approximately thirty minutes each. The teachers were asked about the program, changes they were making in their practice (if any), the impact of those changes, and school context factors that might be a barrier to implementation.
Important findings emerged from this qualitative phase of the data collection, findings that were used to help guide the project as it moved into the implementation phase. For instance, the teachers explained the novelty of working with special education teachers in the classroom – a new experience for these Chicago teachers – and a factor MFA was able to address more explicitly going forward.
The teachers were also able to identify particular aspects of the Math for All program that had a significant impact on their mathematics instructional perspectives. Teachers were able to explain what aspects of the model they were using most and what was novel for them, such as working with their peers to plan and review lessons. All of this information was significant for MFA developers as they prepared for the 2015-2016 Implementation year.
Finally, teachers and administrators were asked to talk about the barriers that could impact their implementation of the model going forward. One of the findings, for instance, was a concern about limited planning time for identifying and tracking progress of the “focal” student. This context variable was important for the program developers to address to insure overall implementation of the model.
Qualitative data collected directly from participants through interviews provides critical explanatory data. The interviews give “voice” to those in the school even as they provide critical cultural context data to help guide the development of the program as it enters the implementation and impact years (Creswell & Plano Clark, 2011). The use of qualitative data in the pilot year thus helps project developers understand the response of participants to the professional development – central to any PD effort (Guskey, 2000) – and helps shape the way forward for the developers and future school participants.

Authors

• Ellen B. Meier, Teachers College, Columbia University

• Dawn M. Horton, Teachers College, Columbia University

• Caron M. Mineo, Teachers College, Columbia University

• Joanne Beckwith, Teachers College, Columbia University