A Comprehensive Analysis of Science and Math Interactive Computer Tasks for Assessment

• In Event: 46.027 - Issues and Developments in NAEP Technology-Rich Assessments: Representative Work of NAEP–Educational Testing Service Summer Undergraduate Research Experience/Summer Predoctoral Research Experience Fellows

Abstract

The National Assessment of Educational Progress (NAEP) is currently working on the development of science and math interactive computer tasks (ICTs) for future K-12 assessment. These tasks are being developed as part of the Technology-Based Assessment (TBA) initiative to improve the quality of educational testing at ETS. ICTs are important in test development because the data generated gives insight into student’s actions within an item, which provides information that are difficult to ascertain from paper and pencil assessments (Zoanetti, 2010). Additionally, they have instructional characteristics like environmental simplicity to limit distraction while delivering fast feedback about a student’s performance on any given task. Finally, they increase student motivation and incorporate apparatus that scientists are using in the field (Scalise & Gifford, 2006). All in all, with the rapid growth of technology in the 21st century it is important that educational assessment begins to incorporate the developing technologies in their design of new item banks. This study serves as overview of various technology-rich assessments currently under development. A literature search and analysis was conducted on the current science and math interactive applications that are available online. This research dealt with applications used for learning, which are less structured than standardized ICTs used for assessment. Using the NAEP Science and Mathematics framework, a matrix was created to sort the various ICTs into content specific and grade appropriate levels. Additionally, web research was conducted to analyze the developing interactive technologies that could be utilized in educational assessment.
In a comparison of various modalities of assessment for science and math, science ICTs used more virtual labs, interactive simulations, and interactive games, while math ICTs included interactive simulations and games as well. Both types of ICTs used Adobe Flash player, which is concerning as Flash is dying out in place of other plugins like Unity and Javascript. Moreover, the ICTs were limited to one specific content area which fails to address new educational standards that emphasize crosscutting concepts. Finally, math specific tasks focused on low complexity content areas and were limited in content range and interactivity.
Beyond web-based computer tasks, new platforms are available to enhance interactivity even further. For example, Xbox One will open up their source code to allow people to develop their own games while utilizing Kinect technology. Additionally, there are several different motion capturing technologies being developed that will allow test developers to design tasks that capture optical, hand, and full body motion that will provide a wealth of data regarding student action in assessment. Automatic feedback will be able to generate data about the various ways in which students maneuver the assessment. Additionally, the virtual world creates a platform in which there is more authentic interaction with stimuli than one would observe in the real world. These new modalities of assessment can open up new opportunities to develop items beyond the simple transfer of content from paper and pencil tests to the computer.

Authors

• Peta-Gaye Nicole Bullock, Howard University

• Gregory Vafis, Educational Testing Service

• Joanna Sandra Gorin, Educational Testing Service