NAEP Technology and Engineering Literacy Assessment

• In Event: 58.074 - Discussion and Demonstration of New Digitally Based Assessments for NAEP

Abstract

Technology and engineering have become critical components of 21st-century life. For generations students have been taught about technology and instructed on how to use each new generation’s emerging technologies. However, until the development of National Assessment of Educational Progress (NAEP) technology and engineering literacy (TEL) assessment, there had been no standardized, nationally representative assessments to provide evidence of what students know about technology and engineering; the roles they play in our lives; and the extent to which students can use technologies and understand how they are designed and developed.

The 2014 NAEP Technology and Engineering Framework broadly defines technological and engineering literacy as the capacity to use, understand, and evaluate technology as well as to understand technological principles and strategies needed to develop solutions and achieve goals. In 2014 a national sample of eighth-grade students were selected to participate in the first-ever national TEL assessment. The assessment is designed to assess three interconnected areas of technological and engineering literacy: Technology and Society; Design and Systems; and Information and Communication Technology. Within the content areas, students were expected to demonstrate thinking and reasoning skills specified in three practices: Understanding Technological Principles, Developing Solutions and Achieving Goals, and Communicating and Collaborating.

The results from the TEL assessment revealed that 43 percent of eighth-grade students scored at or above Proficient. Female students scored higher than males students in TEL overall as well as in one content area and two practices. Recurring achievement gaps between race/ethnicity and school locations were similar to other NAEP assessments across different subjects.

For the 2014 TEL administration, NAEP provided the equipment for the assessment, selecting a laptop with specifications that would work well with the items and provide a consistent student experience. Custom software was developed for both students and administrators, and universal design features were incorporated to improve accessibility.

To allow students to demonstrate the wide range of knowledge and skills detailed in the three TEL assessment areas, students were asked to perform a variety of problem-solving tasks based on interactive scenarios reflecting realistic solutions. These tasks are known as scenario-based tasks. Results from these tasks provide valuable insights on students’ ability to understand and apply technology and engineering knowledge and skills. During this portion of the symposium, participants will have the opportunity to examine a sample TEL task and learn more about the student results from these tasks on the 2014 TEL assessment.

In addition to the traditional analysis of “response data”—students’ answers to questions—TEL also tracks students’ interactions with the assessment, or “observable data.” Observable data can be used to inform test design, create research opportunities, and create new reporting opportunities.

Because students’ experiences with technology are not confined to the classroom, the TEL assessment is accompanied by a questionnaire focusing on students’ opportunities to learn about technology and engineering both inside and outside the classroom. We will also share findings from students’ responses to the questionnaire with participants during the symposium.

Authors

• Marc Berger, Educational Testing Service

• Michael Friesenhahn, ETS