An Expert-Penalty Effect in the Use of Social Q&A Forums for Learning

• In Event: Sourcing: Identifying and Evaluating Information in the Digital Age

Abstract

Theoretical Frame

Social question and answer forums are a popular way to search for answers on any topic, from personal issues to technology. But since any user can say virtually anything about any topic in SQA, there is great variability in authors’ expertise and message quality. One of the most salient cues to assess information quality and credibility is authors’ credentials.

Previous studies have found that adolescent and adult students judge messages from experts higher than from novices (Jeon & Rieh, 2013; Salmerón, Macedo-Rouet, & Rouet, 2015; Winter & Krämer, 2012). However, just relying on authors’ credentials is a rather superficial strategy, as it may be easily faked (as in the case of hoax pages). Recent evidence suggests that expert bias is not applied as a quick heuristic in which only a single decision rule is used (i.e. if the message is authored by an expert or not). Salmerón et al. (2015) found that university students recommended experts’ messages over anonymous users’ messages only when experts included an external source (such a hospital webpage) in their messages to support their claims, but not when they simply included their personal experience.

Previous studies used scenarios that included mostly messages that were useful in solving the questions posed in the SQA forum. An unexplored issue is how students judge messages that are less useful. While a heuristic view on expert bias predicts that messages will be judge better if they come from experts, a systematic view of the effect predicts an interaction between authors’ expertise and message usefulness. We explored this issue in two studies.

Methods

In the first pilot study undergraduates (N=56) rated the utility of a list of responses to six SQA forum questions. Later they rated the extent to which a list of professions were competent enough to provide a recommendation for each forum question and the perceive prestige of each profession. From these data, we identified expert/less expert professionals as well as useful/less useful messages on different topics.

In the second study, we constructed four forum scenarios that offered two recommendations, coming from expert or less expert professionals, with a similar level of perceived prestige, on the topic of the forum discussion.
Recommendations were either highly useful or less useful. A different sample of undergraduates (N= 60) read the forums, rated the goodness of each recommendation (not really useful 0-10 really useful), and provided explanations for their ratings.

Results

Results showed an interaction between authors’ expertise and message usefulness, F(1,59)=8.31, p=.005, µ2p=.12: useful messages were rated higher if they were written by expert professionals (M=7.9, SD=1.3) than if the author was non-expert (M=7.4, SD=1.6); the opposite pattern was found for less useful messages (expert: M=4.8, SD=1.9; non-expert: M=5.3, SD=1.7).

Scholarly Significance

In sum, students judge recommendations in SQA forums through a systematic interaction of author expertise and message usefulness. Students give extra credit to expert messages if they are useful (cf. Salmerón et al., 2015), but they penalize them if they are not useful at all.

Authors

• Lalo Salmeron, University of Valencia

• Ana Cristina Llorens, University of Valencia

• Mônica Macedo-Rouet, University of Paris 8

• Jean-Francois Rouet, CNRS and University of Poitiers