The Interactional Nature of the Interview and Its Impact on Students' Complex System Performance

• In Event: Supporting Systems Learning 2.0: The Next Phase of Research in Complex Systems in Science Education

Abstract

Recent studies have demonstrated that elementary-aged children are able to learn complex systems concepts (Assaraf & Orion, 2009; Danish, 2009). However, assessing elementary students’ understanding of complex systems can be challenging due to their developing language skills and unfamiliarity with the content matter. To address these challenges, researchers typically use verbal interviews in pre-posttests rather than written measures to assess student performance. Interview questions are designed to elicit performances that demonstrate mastery of complex systems concepts. Students’ talk during interviews is typically coded for knowledge-oriented utterances that are assumed to reflect their understanding.

While prior coding-schemes have been successfully employed to provide a summary of students’ knowledge, they do not explicitly recognize the interactional nature of the interview and its possible impact upon students’ responses (Potter & Hepburn, 2005). In particular, the sequential nature of the interview can lead students to expound upon or transform their responses. Using interaction analysis (Jordan & Henderson, 1995) and principles of conversation analysis (Schegloff, 2007), this paper aims to examine sequence organization in interviews to better understand students’ performance of complex systems understanding in these settings.

We draw on secondary data from post-interviews of an intervention with elementary students (N=30, 6-7 years old) learning complex systems concepts after engaging in a simulation, BeeSign, which focuses on bees collecting nectar (Danish, Saleh, Andrade, & Bryan, 2014). Out of the 30 students who participated, our secondary analyses focuses on fourteen students who were grouped as performing above their zone of proximal development (ZPD) and thus demonstrated some mastery of the content based on a revised coding scheme of components and mechanisms or CMP (Sinha et al., 2013).

Analyses reveal that students' performances shift between fully comprehending complex systems mechanisms to having a non-normative view (e.g., honeybees continue dancing even after they do not find nectar). For instance, when asked what bees would do upon returning to the hive after visiting a flower with no nectar, students’ CMP scores indicated that 8 out the 14 above-ZPD students correctly answered that the bee would fly around and find another source of nectar (chaining forward mechanism) or the bee does not need to dance (implicit negative feedback). However, when the question was asked differently “would other bees go to this flower too?” as a follow-up, the only two students whose response was coded as negative feedback were able to answer the question correctly. This suggests that while coding responses per interview question can reasonably demonstrate students’ learning, it is also necessary to understand how subsequent turns reveal nuances about what students do not comprehend regarding the content matter. Additionally, interviewer’s choice of words and responsiveness can sometimes create problems in communication, wherein students misunderstand words that were used or mistake neutral comments as positive assessments of their talk. Examination of these repair-breakdown sequences reveal that it may be necessary to include finer-grained questions in interview prompts, particularly to address student confusion and to ensure that the desired student performance is obtained.

Authors

• Asmalina Saleh, Indiana University - Bloomington

• Joshua Adam Danish, Indiana University