Using Tools From Machine Learning to Capture Micro-Changes in Knowledge

• In Event: 52.069 - Just a Minute! New Methods for Observing Conceptual Change in STEM

Abstract

The papers in this session are taking on a big challenge—the challenge of capturing learning on the timescale of seconds to minutes. From the point of view of a cognitive perspective, learning is understood as changes to mental representations, the stuff of human knowledge. Thus, from this perspective, if we wish to be able to capture learning on the timescale of seconds to minutes, we need a way of seeing mental representations; we need methods that allow us to see through what people do and say, to the mental representations underlying that behavior.

This is not a new endeavor. For example, in the earliest days of cognitive science, researchers such as Ericsson and Simon (1984) employed a method called protocol analysis. In this method, the task of seeing mental representations is made tractable by looking only at highly constrained tasks, and by making very specific a priori assumptions about the mental representations that might be seen. Other, more recent efforts, have tended to use human coders, working inductively from datasets, and constrained by broader theoretical assumptions (e.g., Chi, 1997).

The contention of this talk is that methods from data mining and learning analytics provide powerful new ways of seeing through behavior to mental representations. Although we tend to think of these new methods as appropriate only for the analysis of “big data;” in reality they encompass a new space of computational methods, with a wide range of uses, including for the analysis of more traditional “small data.” In particular, there are methods that make it possible to discover “latent topics” in text data, which I have argued can be taken to reflect the tacit knowledge of a speaker (Author 8, 2013)

In this talk, I will take describe and demonstrate a new web-based text-mining tool called Tactic, designed to support the application of these methods. Unlike other text mining tools, Tactic is not designed for data miners; rather, it is designed for researchers who work with relatively small amounts of rich text data. In Tactic, users work with algorithms that take the form of visual tiles, which can be combined, interactively, to produce multiple types of analysis, and to explore hidden patterns in text. Users can also reprogram these tiles, or program entirely new tiles.

For illustration, I will apply Tactic to clinical interview data in which middle school students are interviewed about a variety of science topics – spanning the seasons, climate, and parts of human biology. I will show how one algorithm for latent topic discover, latent Dirichlet allocation (Blei, Ng, & Jordan, 2003), can be used to see changes in knowledge on the timescale of seconds to minutes.

Author

• Bruce Sherin, Northwestern University