Examining the Nature and Validity of Epistemic Beliefs Using Cognitive Interviewing

• In Event: Conceptual and Methodological Issues and Advances in Research on Epistemic Beliefs

Abstract

Introduction
Researchers frequently rely on the use of self-report questionnaires to assess students’ beliefs about knowledge and knowing, yet little research has explored whether students’ interpretations of questionnaire items align with researchers’ theoretical assumptions and construct definitions (Muis, Bendixen, & Haerle, 2006). In the present study, we employed cognitive interviewing techniques to address the following questions: (1) How do students interpret items and select responses to epistemic beliefs questionnaires? (2) What is the nature of students’ beliefs across different domains and levels of education?
Methodology
One hundred eighty-five students from four educational levels completed three versions of the Discipline-Focused Epistemic Belief Questionnaire (DFEBQ; Hofer, 2000) adapted for three domains: everyday life, psychology, and mathematics. From this larger sample, 34 students then participated in a semi-structured cognitive interview to answer questions about their response choices and interpretations of five items (representing the five dimensions assessed in the DFEBQ) across the three domains. Participant responses were coded based on multidimensional and developmental frameworks, and word interpretation, confusion, point of reference, and cognitive validity (i.e., congruence between item interpretation, explanations/examples, and response choice). After coding independently, two coders met to discuss themes and quotations using cross-case thematic analyses across educational levels and disciplines.
Results
Thematic analyses of interviews with high school and college students revealed that mathematics was viewed from an absolutist stance, whereas students frequently held a multiplist stance for psychology and everyday life. At higher levels of education, these patterns shifted: responses from undergraduates suggested evaluativist beliefs for psychology and everyday life, and graduate students articulated scenarios where they could imagine more advanced epistemic positions in mathematics, although this was not reflected in their personal beliefs.
Item response analyses revealed that students’ personal interpretation of an item was aligned with examples they provided when describing their corresponding belief and response choice for that item. However, interview responses also revealed that several students interpreted key words from questionnaire items in unexpected ways. For example, ‘truth’ was interpreted in terms of veracity or deceit rather than an ontological reality; ‘expert’ had no clear and consistent interpretation within domains; and ‘complex’ was interpreted as being synonymous with ‘difficult to understand’ rather than representing structure. Furthermore, when interpreting items for a domain with previous educational experiences (e.g., mathematics), students referred to classroom learning and assessment experiences. In contrast, when lacking formal educational experience (e.g., psychology), students interpreted items by drawing on examples from everyday life experiences. Additionally, students often reported selecting ‘3’ on a Likert scale because they held multiple conflicting examples in mind; students could envision two or more scenarios when they could both ‘agree’ and ‘disagree’ with the item.
Discussion
Findings from this research provide insight into the nature of epistemic beliefs across education levels and domains. Moreover, item interpretation and response choice patterns provide support for Muis et al.’s (2006) TIDE framework. These findings also highlight important discrepancies between students’ interpretations of words and the intended meaning, suggesting a need to re-examine phrasing and terminology used in epistemic belief questionnaires.

Authors

• Krista R. Muis, McGill University

• Melissa Duffy, McGill University

• Gregory Trevors, McGill University

• Michael Joseph Foy, John Abbott College

• John Ranellucci, McGill University

• Bogusia Gierus, McGill University

• Xihui Wang, McGill University