Paper Summary
Share...

Direct link:

Science Speaks to Society? Arguments for Research Relevance and Validity in International Large-Scale Assessment Publications

Mon, April 8, 10:25 to 11:55am, Metro Toronto Convention Centre, Floor: 200 Level, Room 206D

Abstract

1. Theoretical perspective
Bush (1945) coined the slogan “Science speaks to society” putting forwards a conception of scientific autonomy and research communication. Nowotny et al (2001) stated “Society speaks back”, arguing instead for socially engaged research. These positions present different perspectives on science-society interaction and socially relevant research. We ask how is this interaction functioning and what does it imply for science and society? Conceptually we situate science-society interaction at an “education agora”, embracing different agents, interests and positions (Henkel, 2007). Agora is referring to contextualization of science and coproduction of science and society in tandem processes (Nowotny et al, 2001). Few studies have used an agora perspective in education (for an exception analyzing education policy, see Brown, 2012).

2. Objectives and purposes
This paper deals with the problematic of science-society interaction in education with an empirical focus on International Large Scale Assessments (ILSA), having vital impact in education policy and practice (Meyer & Benavot, 2013, and Ramirez et al, 2018), though there is often little of correspondence of ILSA research and policy discourses. The purpose is to analyze how educational research is speaking to an education agora by analyzing research publications. We ask how educational research is formulated as socially relevant and valid in research publications; with what arguments and to whom? Based on our studies we portray how ILSA speaks to society and implications of such speech.

3. Methods
We were analyzing research publications published 2002-2016. Thus, we pinpoint how researchers speak in fora they control. The publications were identified by systematic research reviews (e.g. Gough et al, 2013) using Web of Science, Discovery and Scopus. Claims of relevance were identified and statements concerning knowledge contributions were analyzed in terms of statistical truth (Hacking, 1992) and practical reason (von Wright, 1983).

4. Data sources:
We identified more than 10 000 articles, which were filtered and classified. We focused on articles published in peer reviewed scientific journals. Here, we selected 208 articles, all of two kinds, (A) presenting primary research based on international comparisons, and (B) secondary research articles commenting ILSA research.

5. Results and conclusions
In primary research publications, policy-makers were frequent targets, followed by education professionals. Sometimes there was an interest to give directives for political or professional action, but mostly the ambition was to present insights into different kinds of outcomes, such as efficiency, or gender gaps, or social segregation. Secondary research publications were often addressing promises or criticisms of ILSA in terms of educational knowledge or transnational governance. Targets were often very diffuse and abstract. A task for educational research is to concretize maps of agora and to enter critical and constructive conversations on validity problems and conclusions in research contextualization.

6. Significance
This study showed how educational research in form of ILSA is presenting itself at the education agora – arguments for its relevance. This is of importance to capture in order to understand the use and abuse of educational research and to improve its intellectual quality!

Authors