Online Book Clubs and Natural Language Processing: Analysis of Fourth-Grade Small-Group Online Discussions in E-Books

• In Event: Cyberschooling and Social Isolation: New Literacy Opportunities for Vulnerable Populations

Abstract

Objectives

The Covid-19 pandemic made online K-12 schooling the norm. Online discussion, a mainstay of distance learning at universities, affords meaningful learner-learner and instructor-learner interaction, vital for distance learning (Moore, 1989). Therefore, we suggest online discussions for remote K-12 schooling. Although, moderating online discussions - key for productive, nurturing learning environments - is labor-intensive, computer algorithms could support teachers moderating online discussions. We collected small-group, book club, discussion postings from fourth graders, reading web-based eBooks, and qualitatively coded the postings. Computer science students developed Natural Language Processing (NLP) algorithms to automatically code the discussion postings. We discuss possibilities of NLP assisting teachers with moderating online discussions.

Perspectives

Online discussions afford three vital types of interaction: learner-instructor, learner-learner and learner-content (Moore, 1989). Most studies of online discussion use university students (Bertera & Littlefield, 2003; Bowen, Farmer, & Arsenault, 2012; Brock, Ballweg, Wick, & Byorth, 2005; Miyazoe & Anderson, 2011; Roberts & Rajah-Kanagasabai, 2013). However, with the pandemic-caused increase of K12 online schooling, we should consider online discussions in elementary school. We investigated how NLP can categorize fourth grade discussion postings and help teachers moderate online discussions.
In using NLP to categorize online discussion postings, we deal with short informal texts, i.e., short-text classification, using semantic analysis and the correlation of texts to obtain the overall logical structure (Song et al., 2014).

Method

Twenty-one Slovenian fourth grade classes went, with their teachers, to computer labs for 1.5 hour sessions. Some 342 students: (1) sat, one at each computer, with a web-based interactive eReader on a browser, (2) logged into the eReader and (3) read through short eBook stories, interacting in online discussions.

Preservice teachers qualitatively coded discussion 3,500 Slovenian postings, with a Kappa (McHugh, 2012) interrater-reliability of 0.95. The coded postings were passed to six groups of computer science students, who developed NLP to predict qualitative codings of discussion codings, including whether postings were book-relevant (yes or no), class (six possible codes) and category (19 codes). Students developed algorithms with 80% of the coded discussions, while 20% were held back to evaluate the algorithms.

Results

The computer science group achieving best results used a BERT model (using word content of postings and discussion questions) successfully predicted book-relevance at 85%, in Slovenian postings and 95% in the English translations.
Another group used ELMO word embeddings and achieved 85% on book-relevance. However, they also used context information (i.e., which discussion question and book-club and time stamps) for path analysis to predict the category of the following posting, e.g., when students posted a chat unrelated to the discussion question, the probability that the next posting attempts to answer the discussion question is 12.9%.

Scientific Significance

NLP dataset sizes typically run in the millions. The small size of our dataset, 3,500, demonstrates that NLP can effectively address smaller datasets. Educationally, our results suggest that NLP could assist teachers moderating online discussions, providing real-time notification of discussion problems.

Author

• Glenn Gordon Smith, University of South Florida