Visualizing how novice programmers share code

• In Event: Roundtable Session 98
  In Roundtable Session: Using Learning Analytics and Educational Data Mining to Understand Scripted and Exploratory Learning Environments: Toward a Common Theoretical and Methodological Framework to Investigate the Trajectory to Expertise

Abstract

This study leverages data mining as a tool for describing the programming approaches of 53 high school senior girls learning to program in IPRO, an application for the iPod Touch that allows users to design soccer-playing virtual robots using a simplified functional language (Berland, Martin, Benton, & Petrick, 2011).
While learning science has turned its attention towards programming, design, and collaboration around these areas, these efforts have largely been based on interview and observation of designers, programmers, and groups in action (Hancock, 2003; Resnick et al., 2009). This research has provided useful insights and findings. We hope to capitalize on affordances these methods do not have, primarily, the identification of overarching trends, hidden patterns, and “micro-actions” of interest that are possible because we are working in an environment in which all actions are logged and analyzing this data using tools from scientific data mining and visualization.
To explore student patterns of programming, we built a programming environment designed to collect those fine-grained data. The IPRO programming language was designed to foster quick programming on the fly ("tinkering"); it utilizes a restricted library of sensors, logical operators, mathematical operators, and actions. IPRO programs are immediately executable, and no syntax errors are possible in the environment. We designed IPRO with these features so that: student programs can be easily categorized and analyzed; it is easy to start programming with no background or prior knowledge; and students can share code easily. Berland (2008) and Hancock (2003) both describe how features such as reusable code and easy testability engage novices by allowing them to learn as they make rewarding progress on their programs.
We employ novel data collection and analysis methods to examine relationships between styles of programming and sharing in the corpus. We deployed the system to 53 students across 16 teams (of 3-4 virtual robots each) on the first day of a girls’ programming summer camp. Each student made between 200-1000 edits to her programs over the course of 90 minutes. The new programs produced by each edit were automatically evaluated by the system in terms of how successful that program was at scoring goals. Metrics were created to evaluate similarity of any two edits as well as uniqueness of a program at a particular time. To investigate these relationships, we built two novel analysis and visualization tools: AMOEBA and TRACER. AMOEBA explores the log data by visualizing patterns of sharing across programs. TRACER visualizes process patterns and models of students learning to program.

Through AMOEBA and TRACER, we found that the patterns in which students collaborated seemed to affect their success at programming. Students who collaborated by sharing complex chunks of code with people outside their team were likely to create much better programs immediately afterwards. This suggests that sharing is beneficial, but this effect did not appear at the aggregate level: over the implementation, the average sharing metric was uncorrelated with the average quality metric. In this session, we explore these relationships and present relevant visualizations with AMOEBA and TRACER.

Authors

• Matthew W. Berland, University of Wisconsin - Madison

• Taylor Martin, Utah State University

• Tom Benton, The University of Texas - Austin

• Carmen Petrick Smith, University of Vermont