Combining Human and Automated Analyses for Collaboration Dynamics: How Do Small Groups Participate Equitably?

• In Event: 68.049 - Data, Big and Small

Abstract

Collaborative learning occurs at the intersection between the individual and the group (Puntambekar, 2013). One way to understand learning at each level, along with factors such as temporality and context, is to use mixed-methods approaches (Puntambekar, 2013). Mixed-methods approaches allow us to study processes and products of learning at multiple grain sizes, using both qualitative description and quantitative comparisons (Strijbos & Fischer, 2007).

One mixed-methods approach is quantitative discourse analysis. This approach involves transcription, development of coding schemes and reliability measures, and statistical testing (Strijbos & Fischer, 2007). While this approach permits fine-grained analysis, it involves high labor cost and possible researcher bias. Using automated analyses may reduce labor and bias in findings (Martin & Sherin, 2013; Puntambekar, 2013). Natural language processing (NLP) algorithms find patterns in word usage that can describe and predict discourse and written responses (Witten, Frank, & Hall, 2011). However, as both human and automated analyses have limitations, it may be beneficial to consider integration of approaches (Sherin, 2013). For example, Sherin (2013) found that statistical NLP converged with human analyses and identified theory-relevant data features. Integrating automated analyses may benefit researchers by triangulating results, supporting theory, and revealing unanticipated patterns in data (Puntambekar, 2013; Martin & Sherin, 2013).

Small-group collaboration provides an opportunity for integrated analyses. Participation in shared meaning-making discourse impacts individuals’ conceptual outcomes (Barron, 2003; Wegerif, Mercer, & Dawes, 1999; De Wever, Van Keer, Schellens, & Valcke, 2010). In particular, talk that is equitably distributed among group members improves quality of collaboration and co-construction of knowledge (Rafal, 1996; Bennett, Hogarth, Lubben, Campbell, & Robinson, 2009; Esmonde, 2009). Inequitable patterns in participation may actually be detrimental to conceptual outcomes (Barron, 2003; Esmonde, 2009). For example, when students adopt expert roles and dominate group discourse, other group members have fewer opportunities to engage in conceptual talk (Esmonde, 2009; Strijbos & Weinberger, 2010). To remedy inequitable talk patterns associated with poorer conceptual outcomes, we must understand how such patterns form.

Investigating how individuals participate in discourse allows us to explore how collaborative learning dynamics are linked to individual conceptual outcomes. In this study, we use a mixed-methods approach to study how students in small groups learn biology concepts by participating in collaborative activities. We use a diversity of data sources at multiple levels to identify participation patterns, including: written responses to prompts and assessment scores (for the individual level); log data of e-textbook actions (for the group level); and audiovisual recordings of group work (for the intersection of individual and group levels). Our research question is: How do the data sources (individually and collectively) reveal patterns of participation in collaboration? Based on prior research, we expect convergence between individual assessments and group discourse (Dornfeld & Puntambekar, 2015; Dornfeld & Puntambekar, 2016) and between discourse and log data. By understanding alignment between data sources, we hope to further uncover how collaboration unfolds, along with opportunities to improve collaboration through visualizing participation (e.g., Gweon, Jun, Lee, Finger, & Rosé, 2011; Kay, Maisonneuve, Yacef, & Reimann, 2006).

Authors

• Catherine Louise Dornfeld, University of Wisconsin - Madison

• Matthew W. Berland, University of Wisconsin - Madison

• Sadhana Puntambekar, University of Wisconsin