PISA 2012 Creative Problem-Solving: An Assessment of 21st Century Skills

• In Event: Poster Session: Cross-national research using large-scale international data

Abstract

PISA 2012 Creative Problem-Solving: An Assessment of 21st Century Skills
Objectives
The Program for International Student Assessment (PISA) is an international assessment that measures 15-year-old students' reading, mathematics, and science literacy. For the first time in 2012, PISA included a computer-based assessment of creative problem solving, which was administered in 44 education systems including 28 member countries of the Organization for Economic Cooperation and Development (OECD). A country’s performance on the PISA problem-solving assessment is an indication of how well-equipped its 15-year-olds are to meet the demands for solving problems in the workplace and in other life contexts.
This analysis takes a deeper look at the performance of participating education systems in problem solving by focusing on the following: (a) Overall and relative performance in problem-solving (b) Strengths and weaknesses in problem-solving of participating education systems. There will be some particular emphasis on comparing the performance of U.S. 15-year-old students in problem solving with the performance of their peers in other countries.
Perspectives and Theoretical Framework
Problem-solving competence is defined by PISA 2012 as “…an individual’s capacity to engage in cognitive processing to understand and resolve problem situations where a method of solution is not immediately obvious.”
The shift to non-routine, interactive, and analytic tasks in the workplace (Autor and Price, 2013) is leading to an increasing emphasis in education on empowering students to confront and solve non-routine cognitive challenges. In addition to the core PISA subjects of mathematics, science, and reading literacy, problem solving is essential to evaluate students’ abilities to solve real-life problems as they near the end of compulsory schooling (OECD, 2013). Through the use of computers, PISA was able to present interactive problems to the students similar to those confronted in real-life contexts.
Methods and Data Sources
PISA, which is designed and coordinated by OECD, assesses what students have learned both within and outside of school as they near the end of compulsory education. The PISA problem-solving assessment describes student performance by presenting average scale scores and six established proficiency levels. PISA also collects data from a student background questionnaire and a school questionnaire completed by the school principal.
PISA data can be used to analyze the performance of education systems and gauge where they stand compared to the rest of the world.
Results and Conclusions
Example results include the following:
Overall and relative performance:
• Average scores in problem solving ranged from 562 in Singapore to 399 in Colombia.
• Percentages of top-performing students (those scoring at level 5 or above) in problem solving ranged from 29 percent in Singapore to 1 percent in Colombia, Malaysia, Montenegro, and Uruguay. In the United States, 12 percent of 15-year-olds scored at proficiency level 5 or above.
• The percentage of students scoring below level 2, which is considered a baseline of proficiency by the OECD, ranged from 7 percent in Korea and Japan to 61 percent in Colombia. In the United States, 18 percent of 15-year-olds scored below level 2.
• In Italy, the United States, Japan, and Serbia, students scored about 10 points higher in problem solving than expected (given their scores in mathematics, reading, and science). On the other hand, in Bulgaria, Shanghai-China, Poland, and the United Arab Emirates, students scored about 40-50 points lower in problem solving than expected.
Countries’ relative strengths and weaknesses in problem-solving:
• Students in Hong Kong-China, Korea, Macao-China, Singapore, and Chinese Taipei performed better on problems that required understanding, formulating or representing new knowledge, compared to other types of problems.
• Students in Brazil, Ireland, Korea, and the United States performed better on interactive problems compared to static ones.
More in-depth analyses, using data from the PISA 2012 contextual questionnaires and from external sources, will be carried out to better understand these relationships both within and across countries.
Significance
Creative problem-solving is an essential 21st century skill that education systems need to foster to make students reach their full potential and engage with the world in a constructive manner. To compete in the knowledge economy, countries around the world need to encourage students’ creativity and innovation through policy and practice. In this endeavor, PISA has emerged as an important yardstick for measuring the quality, equity and efficiency of an education system. Furthermore, PISA continues to help countries look beyond their borders for evidence of best practices.

Author

• Anisha Singh, George Washington University