BackHome
Refresh: OffView Personal Schedule
Paper Summary
Share...

Direct link:

Performance Indicators of Online Information Processing Behaviors of Young Professionals While Solving Critical Reasoning Tasks

Mon, April 25, 2:30 to 4:00pm PDT (2:30 to 4:00pm PDT), San Diego Convention Center, Floor: Upper Level, Room 9

Abstract

Objectives/Purposes
Internet users develop different search and information processing behaviors [7,9,25]. For instance, professional fact checkers read "laterally," leaving a web page after a quick scan to first verify its credibility, whereas (undergraduate) students often read "vertically" and only stay on a single website [24]. A similar concept is described as "sampling," where the focus is on selecting the optimal source of information by quickly scanning different sources [11]. Generally, high-performing students switch more frequently between search results and websites in the process of seeking information before remaining on one website, which could be interpreted as greater metacognitive engagement [25].

Framework/Hypothesis
Based on these findings, process-related metrics such as dwell time on web pages, number of pages visited, and number of searches should have an impact on the critical use of online information, where, in terms of lateral reading, visiting a higher number of web pages, starting a higher number of search processes, and spending less time on each website could indicate a better outcome. We investigate whether these assumptions can be supported on the basis of process data from young professionals in three domains (medicine, law, and teaching).

Methods/Data
We adapted an assessment tool by McGrew et al. [13] to validly assess the “Critical Online Reasoning” (COR) skills [3] in a realistic online environment, capturing both process data (i.e., log files with timestamps) and performance data (i.e., written open responses). Participants were given a computer-based task to complete in 20 minutes, consisting of a search question and two task prompts. Participants were asked to conduct an open-ended web search and write a short text response, which was evaluated by two independent human raters based on a three-step rating scheme (for details, [3], [4]; for sample description, Table 1 in Appendix 1).

Results
With a mean score of 3.26 (SD=0.80) out of a maximum of 4 achievable points, the overall level of COR skill is relatively high in the sample. Including process and performance data, two clusters were identified: the larger group of "high performers" (HPs, n=46) and a smaller group of "low performers" (LPs, n=10). On average, the HPs scored 1.04 points higher (p=.021). Among HPs, online information behavior is characterized by more frequent information searches with a mean of 3.35 compared to 2.1 among LPs (p=.009). HPs visited a mean of 11.58 websites compared to 7.8 websites among LPs (p=.026). HPs’ mean time spent on the websites is 39.68 seconds and lower compared to the LPs’ 74.79 seconds (p=.019) (Table 2, Appendix1). Remarkably, significant differences between groups were identified with respect to cross-domain contextual factors, such as self-reported cognitive overload and need for cognition, intelligence, and socioeconomic characteristics (such as gender and age). No significant differences were found between domains (Table 3, Appendix1).

Conclusion/Significance
The results from these young professionals are in line with the findings from other studies with graduates and emphasize the need to examine further process data more closely (e.g., which websites or areas within are visited longer/shorter) to identify and promote conducive behavioral patterns.

Authors