Engaging MOOC Learners in the Creation of Data Science Problems

• In Event: 81.025 - Innovating MOOC Pedagogies

Abstract

Objectives
MOOC learners have an insatiable desire for more content, more examples, and more problems to try, especially in contexts where learners are developing new skills, such as creating data visualizations using Python. Yet, instructors and course staff may be hard-pressed to find the time to create the volume of examples and problems that would satisfy the appetite of MOOC learners (Mitros & Sun, 2014).

To address this challenge, we developed The Mentor Academy, a two-week instructional program that engages expert MOOC learners as Mentors to engage in the creation of data science problems use in future MOOC. However, a remaining challenge was that Mentors were likely novices in the educational task of creating problems that will allow learners to meet learning goals.

Perspectives
Our work draws on the concept of learnersourcing as a form of crowdsourcing, which creates opportunities for participants to collectively contribute new content for future learners, while engaging in a personally meaningful experience (Kim, 2015; Williams et al., 2016). Our instructional program was also grounded in cognitive apprenticeship models (Brown, Collins, & Newman, 1989) and communities of practice (Lave & Wenger, 1998).

We based our instruction on problem creation on the GRASPS (goals, roles, audience, situation, product, performance, purpose, and success) model, an instructional design framework that guides educators in the creation of authentic problems (Wiggins & McTighe, 2005).

Methods and Data Sources
We recruited 120 participants from six countries. We ran four sessions of a two-week online program that included instructional videos that focused on pedagogical concepts of constructivism (Hein, 1991), scaffolding (Quintana et al., 2004), and authentic problem construction (Wiggins & McTighe, 2005). Mentors were tasked with finding a dataset and using it to create a data science problem, following the GRASPS guidelines.
We created an evaluation rubric that followed the GRASPS components to evaluate the quality of Mentors’ problems (see Table 2). Two educational researchers independently coded the problems. Following a process of inter-observer agreement, all disagreements were resolved through discussion (Patten & Newhart, 2017). We analyzed the 39 problems that were submitted across four sessions of the Mentor Academy.

Results
Of 39 problems we analyzed, 2 displayed a high level of authenticity, 16 a medium level of authenticity, and 21 a low level of authenticity. Problems typically had a well-defined goal as well as clear criteria for the product/performance that was required. Problems often lacked the why of the problem — additional context that would motivate a learner to solve the problem. Additionally, problems tended to lack “standards of criteria of success,” that is, a description of how the solution would be evaluated.  

Scholarly Significance
The Mentor Academy represents an advance towards addressing the challenge that many MOOC instructors face, that of creating high quality content, by inviting expert learners to participate in the problem creation process, through close interaction with peers and domain experts. The GRASPS rubric that we developed quite rigorously evaluates the authenticity of problems. Indeed, an interesting extension of our research could involve using the GRASPS rubric to evaluate problems created by MOOC instructors.

Authors

• Rebecca M. Quintana, University of Michigan

• Christopher Brooks, University of Michigan

• Cinzia Smothers, University of Michigan

• Yuanru Tan, University of Michigan - Ann Arbor