Measuring Flexibility in a Computer-Based Exploratory Problem-Solving Environment

• In Event: Transfer for Learning in the Classroom: Developing Contexts to Examine STEM Learning Skills and Strategies

Abstract

In recent decades, flexibility has been recognized as an important problem-solving behavior for learners in STEM fields (Common Core State Standards Initiative, 2010; National Research Council, 2013). In traditional, well-structured mathematical problem-solving, flexibility is often defined as the ability to use multiple solution methods to a problem (Star, Newton, Pollack, Kokka, Rittle-Johnson, Durkin, 2015). However, flexibility may look different in more complex, exploratory problem-solving tasks in other STEM domains. While some research has been done to evaluate whether various measures of flexibility in exploratory problem-solving tasks are predictive of learning and transfer (Kapur, 2012; Zhang, Biswas, Chiu, McElhaney, 2019), little research has specifically aimed to define and measure flexibility in exploratory problem-solving. In this work, we seek to identify potential measures of flexibility during exploratory problem-solving in the Invention Coach, a computer-based learning environment for teaching physical science equations. Additionally, we consider how different types of guidance in the learning environment may impact these flexible behaviors. Lastly, we examine whether these measures relate to problem-solving success and transfer.

This research is based on data from a classroom study conducted with 195 7th and 8th graders from a low-to-average performing public middle school in New Jersey (87% free or reduced-price lunch). After completing a pretest and survey, students worked on two exploratory problem-solving tasks in the Invention Coach for ~60 minutes total. In each task, students used a set of examples to invent a numerical index for describing a physical phenomenon.

There were three conditions of the Invention Coach, which varied in the guidance provided:
1) PTM – problematizing activities + task reminders + motivational messages
2) TM – task reminders + motivational messages
3) M – only motivational messages
Problematizing activities engage students with disciplinary ideas that challenge their understanding, while task reminders restate constraints of the task.

Student log data from the Invention Coach was used to obtain three measures of flexibility:

Quantity of solutions (total number of solutions submitted)
Breadth of solutions (number of unique solution types submitted)
Index changes (number of times an index was changed)

Pretest survey data was also used to assess tolerance of uncertainty in problem solving situations (adapted from the IUS-12, Carleton, Norton, and Asmundson, 2007). This trait-level measure has been shown to affect students’ ability to perform in uncertain situations.

Preliminary analyses show that some flexible behaviors differ by condition, suggesting that the type of feedback provided in a problem-solving task can affect a student’s ability to act flexibly. Specifically, breadth of strategies and index changes were lower in the PTM condition than in others. Therefore, although the PTM condition produced greatest transfer (Authors, 2019a), it may also limit opportunity for flexible problem-solving. Regression analyses show that regardless of condition, success on the task has a positive relationship with tolerance of uncertainty, breadth of strategies, and index changes, while transfer from the task is positively associated with tolerance of uncertainty. Thus, we have identified promising measures of flexibility in an exploratory problem-solving task which may predict problem-solving performance and transfer from the task.

Authors

• Helena Connolly, Teachers College, Columbia University

• Catherine Chase, Teachers College of Columbia University