Assessing Problem Difficulty: A Problem Difficulty Scale

• In Event: 53.081 - Roundtable Session 21
  In Roundtable Session: 53.081-12 - All About Problems: Considerations of Problem-Based Learning Problems for Learning Processes and Outcomes

Abstract

Purpose of study
The purpose of this paper is to provide a theoretical framework for creating a scale for evaluating difficulty level of problems by exploring the constructs of problems as well as their constituent elements that contribute to the difficulty level of solving them.

Theoretical Framework & Discussion
Fostering students’ general and domain specific problem solving skills is an unquestionably important educational goal as it is deemed as a critical skill in school learning (CTGV, 1990; Vye, Goldman, Voss, Hmelo, Williams, 1997) and the most demanded skill in work places (Schaafstal, Johnston & Oser, 2001; Middleton, 2002). Therefore, it is not surprising that incorporating problem solving activities into the instruction is the most common instructional strategy in instruction today. The uses of problems in instruction vary greatly. Problem-based learning (PBL) is an example of such a pedagogy that structures students’ entire learning process with problem solving activities (Barrows, 1986; Wilkerson, & Gijselaers, 1996). There is also instruction that uses problems as practice opportunities for students to become familiar with the concept. Thus, regardless of the instructional method employed, use of problems plays an indispensable role in instruction.

Among the issues of using problems in instruction, problem difficulty is one under-discussed yet critical issue. Oftentimes, problem difficulty is thought of as a single dimension construct determined by the percentage of students who solve it correctly (e.g. difficulty index, Nikota, 2004). However, difficulty index provides an ad hoc quantitative aspect to the difficulty because it can only be obtained after the test is administered to a group of students, which does not help designers make informed instructional design decisions during the design stage. Secondly, as Jonassen and Hung (2008) argued, all problems are not equal, not only on the levels but also the nature of difficulty. Especially important in designing PBL instruction, instructional designers must analyze the profile of the problem difficulty. This information is essential for designing effective problems as different disciplines focus on different types of problem difficulty. As Lloyd-Jones, Margetson, and Bligh (1998) stated, “problems should be functionally appropriate” (p. 492).

Thus, this paper proposes a multi-dimensional problem difficulty scale and index in hopes to create a universal scale for evaluating and profiling problems used in instruction. This paper will discuss research on problem difficulty, the nature and constituents of problem difficulty, an assessment scale designed to evaluate the difficulty level and nature of a problem or a given type of problem, and its potential applications with examples.

Scholarly Significance
The purpose of this problem difficulty assessment scale is to improve the effectiveness of problems of all types and purposes of instructional needs. The problem difficulty assessment method serves as a tool to enable designers and teachers to map out the degree and the nature of difficulty of a given problem in order to match the nature of the subject area, supporting the intended instructional purpose and function of the problem, and ensuring appropriate difficulty level for the intended level of the course.

Author

• Woei Hung, University of North Dakota