Ninth-Grade Mathematics Item Feature Analysis in South Korea's National Assessment of Educational Achievement Test

• In Event: 64.036 - Feature Analysis Approaches in International Assessments

Abstract

Objectives. This study aims to analyze items used in the National Assessment of Educational Achievement (NAEA) based on a common framework of item feature analysis. Use of feature analysis as a common metric at an international level would provide meaningful information to improve the test and develop new items adapted to the revised national curriculum.

Perspective. The National Assessment of Educational Achievement is a nationally representative assessment of measuring what students know and can do in academic subjects in Korea. The National Assessment of Educational Achievement has been administered annually in the subjects of Korean Language Arts, Mathematics, English, Science, and Social Studies for 9th graders in middle school, and in Korean Language Arts, Mathematics, and English for 11th graders in high school. The National Assessment of Educational Achievement results serve as scientific evidence to monitor students’ educational outcomes and improve teaching and learning by providing student academic progress over time on the four levels of proficiency. As a curriculum-based assessment, the National Assessment of Educational Achievement is currently in the middle of transition since it needs to reflect the new standards and emphasized core-competences in the revised 2015 national curriculum.

Methods. As the other countries participating in this symposium, the common framework of the item feature analysis was employed, however, some contents and concepts of mathematics were adapted to Korean contexts and the 9th grade curriculum.

Data. The analysis in this study is based on data from 2017 9th grade math test. The total of 13,311 nationally representative sample of students from 238 middle schools participated in 2017. The test consists of 33 mathematics items, comprised of 29 multiple-choice items and 4 constructed-response items. The content areas include numbers and operations, variables and expressions, functions, statistics and probability, and geometry.

Results. As to content domain, two domains, i.e., variables and expressions, geometry, are most dominant (9 items each domain, 55%), and two domains, i.e., functions, statistics and probability least dominant (4 items each domain, 24%). There was no statistical difference in terms of item difficulty between 29 multiple choice items and 4 short answer items. As to problem types, there are 5 items for simple calculation, 16 items for understanding, 6 items for mathematical inferences, and 6 items for problem solving. With respect to the relationship between item difficulty and item features, regression analysis indicated that problem-solving items were more significantly difficult than item requiring mathematical reasoning.

Scientific Significance. Within Korea, the descriptive and quantitative findings will be used to influence the revisions of National examinations. Korea is also interested in comparisons to other country examinations as a supplement to its participation in current international comparisons. The use and consistency of findings in feature analysis provides empirical support for this qualitative technique.

Authors

• Jimin Cho, Korea Institute for Curriculum and Evaluation

• Hyekyung Jung, Korea Institute for Curriculum and Evaluation

• Myung-Wha Kim, Korea Institute for Curriculum and Evaluation

• Seong-Kyeong Kim, Korea Institute for Curriculum and Evaluation

• Seunghyun Yang, Korea Institute for Curriculum and Evaluation