Formulating Significant Research Questions

• In Event: Communicating Empirical Research in Mathematics Education: Insights From Flagship Journal Peer Review

Abstract

In 2002, the National Research Council proposed that the first principle of scientific inquiry in education was to “pose significant questions that can be investigated empirically” (p. 3). The report argued that the significance of a question could be established on a foundation of existing theoretical, methodological, and empirical work. However, it is not always clear what counts as a significant question in educational research. One answer is that significant research in mathematics education should advance the field’s knowledge and understanding of the teaching and learning of mathematics. (An analogous statement holds for education research in other areas.) The characteristics that make a research question significant, then, are dependent on context and specifically on assumptions about what kind of knowledge is useful. Further clues as to the field’s view of what makes a research question useful can be found in reviewers’ concerns regarding research questions.

To examine reviewers’ concerns about research questions, we applied analogous methods to those described in Paper 1. Table 2 provides a summary of results from the analysis of reviews regarding research questions. Key concerns included a need for greater clarity and precision in stating the research questions and mismatches between the questions and other parts of the manuscript. In addition, many reviewers highlighted the need to motivate the research question—to make a case for why it was worth answering (e.g., by making connections to prior research). Indeed, for those manuscripts that were ultimately rejected, not a single reviewer stated that the research questions were particularly relevant or insightful. In contrast, for manuscripts that ultimately received a Revise and Resubmit decision or were Accepted, only one reviewer raised the concern that answering the research questions would not make a contribution to the field.

Given these concerns about contributions to the field, it is clear that reviewers attend to whether and how the case is made in a manuscript for the significance of the research questions. Alignment with the theoretical framework provides one avenue by which this case can be made, both by connecting the research questions to prior work in the field and by making clear how answering the research questions explicitly adds important knowledge to the field. Thus, when reviewers highlight disconnects or mismatches between the theoretical framework and the research questions (or the methods used to answer them), they are also critiquing the case for the significance of the research questions. We believe that one way to make the case that a research question is likely to be significant is to show how it addresses practitioners’ shared instructional problems and one whose answer helps the field (students, teachers, policy makers, researchers) understand why and how the answer is a solution to the problem (Maxwell, 2004).

Authors

• Stephen Hwang, University of Delaware

• Anne K. Morris, University of Delaware