New, Generalizable Strategies for Developing Assessments for the Naval Education and Training Command

Session Type: Symposium

Abstract

How do stakeholders and researchers successfully collaborate on new assessments? The NETC’s Training Assessment Framework (TAF) articulates stakeholder’s needs and purpose(s) of assessments, e.g., certification, program evaluation. TAF also provides rules for cognitive and task requirements. Technical challenges are to create efficient methods to acquire and represent disparate content for 60+ course, establish appropriate assessment comparability, and adapt to ever-changing requirements and schedules. From a scientific approach, we introduce just-right performance assessments to trainees, use automatic on-the-fly scoring, and analyze multiple data collections of cognitive, affective, and performance trials. The session highlights new approaches to technical quality and validity for small scale samples. It also demonstrates our collaboration. TAF will likely contribute to the Navy’s new Training Effectiveness Model.

Sub Unit

• Division D - Measurement and Research Methodologies / Division D - Section 1: Educational Measurement, Psychometrics, and Assessment

Chair

• Michele C Harrison, Naval Education and Training Command

Papers

• Training Assessment Framework System - Eva L. Baker, University of California - Los Angeles; Jenny C. Kao, University of California - Los Angeles; Stephen C Court, UCLA

• Assessment Quality and Validity for the Training Assessment Framework - Kilchan Choi, University of California - Los Angeles

• The Naval Education and Training Command: Stakeholder in the Training Assessment Framework - Benjamin Ervin, Naval Education and Training Command

• Use of Affective Measures in the Training Assessment System - Harold F. O'Neil, University of Southern California

Discussant

• Peter Walker, Office of Naval Research