Effects of the Text Feature “Scientificness” on Laypeople’s Evaluation and Resolution of Intertextual Conflicts

• In Event: Learning From Others: Advancing Theory and Research on Learning From Sources

Abstract

Scientific texts represent a genre for which the adherence to specific genre-characteristics allows an indirect, depersonalized evaluation of the source. As the blind peer review procedure demonstrates, further source information (beyond the text itself) are not necessary to evaluate the text and thereby indirectly the credibility of the source. Understanding such features of scientific texts is an element of Scientific Literacy taught in schools, but it is also relevant in informal learning settings: Laypersons seeking for scientific information (for example w/r/t health issues) have to assess the credibility of scientific texts. This is especially the case when they look up science based information in the Internet, where they easily come across contradicting knowledge claims. However, it is an open empirical question whether laypeople are also aware of these specific genre-characteristics and even when so, if they so would use such features to judge about credibility and decide about information validity.

We conducted an experiment that investigated the impact of scientific genre features (e.g. citations, elaborations of methods for justifying claims) on lay recipients’ dealing with textual controversies. Sixty undergraduate students with low topic-knowledge read two conflicting online texts on a medical issue, which were either both written in a Scientific Discourse Style (SDS - SDS), both written in a Factual Style (FS -FS), or which differed in style (FS-SDS). This was embedded in a scenario of looking up information which was critical for further health related behavior. The opposing knowledge claims remain constant in all pairs of texts. Perceived scientificness and credibility assessments were dependent variables, furthermore we asked for lay recipients’ readiness to decide between the conflicting positions or to relay on further expert advice instead.
The results showed that SDS texts were not only perceived as more scientific than texts that contained no features typical of science texts (FS), T(59) = 10.135, p < .001, but were also perceived as more credible, T(59) = 5.807, p < .001. This was furthermore confirmed by content analyses of participants’ justifications of their evaluation of both conflicting positions. However, whether or not two conflicting texts differed in their perceived scientificness had no effect on participants’ readiness to decide between the conflicting positions, F(3.1, 181.2) = .107, ns.

Independently from the experimental conditions, participants were relatively unwilling to decide about the conflict based on the given information, but they were generally more inclined to decide after obtaining further content information or to completely outsource the decision to an expert, F(1.5, 90.9) = 114.819, p < .001, part. η²= .661.
Overall the results demonstrate that laypeople are aware of text features that indicate information scientificness and use them to evaluate the credibility of science-based information. Nevertheless, they seem to be reluctant to completely rely on their own evaluation capabilities and to use them as a base for an actual decision about conflicting information. Finally we will discuss implications for modeling laypersons’ capabilities to deal with scientific knowledge even if they have only a coarse understanding of it.

Authors

• Rainer F. Bromme, University of Münster

• Lisa Scharrer, University of Münster

• Marc Stadtler, University of Münster