Research Synthesis for Pedagogical Improvement in School Mathematics and Science

• In Event: Informing Policy and Influencing Practice: Messages From Cross-Disciplinary Research on Science and Mathematics Education

Abstract

This paper examines the evolving state of research synthesis to inform pedagogical improvement in school mathematics and science. It offers a critical appreciation and comparison of three distinctive approaches that have been employed in recent syntheses and then triangulates these to identify key agreements, anomalies and gaps.
In the “systematic review” developed in Great Britain (Bennett et al., 2005; Mathematics Education Review Group, 2009; Science Education Review Group, 2009; instantiated by Bennett et al., 2005; Kyriacou & Goulding, 2006), a team representing researcher, practitioner and policy communities formulates a tightly-focused research question in response to particular professional or policy concerns; a corpus of relevant studies is then systematically assembled and evaluated according to an explicit protocol; finally, through a process of argued judgement, the team draws on these studies to formulate a response to the research question.
The “best evidence synthesis” developed in New Zealand (Alton-Lee, 2004; instantiated by Anthony & Walshaw, 2007; and prefigured by Hipkins et al., 2002) is more holistic in its focus, and more explicit about how it seeks to negotiate, across the multiple constituencies of practitioners, policy-makers and researchers that form a professional community, a collective understanding of effective practice grounded in the research literature; the emphasis here is on formulating a coherent system of overarching principles extracted from what are identified as “landmark studies”.
Finally, predominantly in the United States, there is a strong tradition of (various forms of) meta-analysis, including a very different notion of “best evidence synthesis” (Green & Skukauskaité, 2008; Slavin, 2008; instantiated by Schroeder et al., 2007; Seidel & Shavelson, 2007; Slavin & Lake, 2007, 2008; Slavin, Lake & Groff, 2008, 2009) in which systematic and transparent procedures are employed to identify studies that measure the effectiveness of teaching methods or programs, to sieve or weigh them by quality, and to identify broad trends in their results, notably in the form of summative indicators of the effectiveness of particular methods or programs; nevertheless, researcher judgement continues to play an important part in core aspects of the formulation of such syntheses.
The paper shows how the results of such syntheses depend to an important degree on crucial features of the analytic process – notably the conceptual framework employed, the strategies used to identify relevant studies, the criteria adopted for screening studies, the types of finding that receive attention, and the processes through which these findings are organised. Nevertheless, by triangulating the major recent syntheses named as instantiations above, the paper develops a critical summary of the now substantial body of research bearing on effective pedagogy in school mathematics and science, identifying consistent trends, anomalous patterns, and underinvestigated areas.
Finally, the paper outlines how this overview has informed the epiSTEMe project, part of the current ESRC Targeted Initiative in Science and Mathematics Education; this project is developing and evaluating a research-informed pedagogical apparatus intended to support improvement at scale of mathematics and science education in the lower-secondary school, and to contribute to filling important gaps in the evidence base on effective pedagogy.

Author

• Kenneth Ruthven, University of Cambridge