Using the Standards to Improve the Validity of Research Measures

• In Event: 47.015 - Leveraging the Test Standards to Improve Research Validity

Abstract

This presentation shares the measurement perspective on validity, the most fundamental consideration in developing and evaluating any assessment or measure. Validity refers to the degree to which evidence and theory support the interpretations of scores for proposed uses of tests, or the extent to which scores provide accurate information for an intended purpose, providing the foundation for the overarching validity standard:
Clear articulation of each intended test score interpretation for a specified purpose should be set forth, and appropriate validity evidence in support of each intended interpretation should be provided
The validity standards are organized into three thematic clusters:
• Establishing intended uses and interpretation
• Issues regarding samples and settings used in validation
• Specific forms of validity evidence
Specific standards within each cluster that are particularly relevant to research measures are presented, focusing on the first and third clusters. Based on these standards, the presentation concludes by outlining a general validity argument and specific forms of validity evidence that could be appropriate for supporting the validity research measures. These include consideration of the following:

ARGUMENT EVIDENCE FORM
Construct definition Content (literature, expert)
Construct representation Content (expert review), response processes
Psychometric structure Internal structure
Reliability/SEM Reliability/SEM
Criterion-related relationships Relationships with criteria, response processes
Convergent-divergent relationships Relationships with other variables
Sensitivity to research effects SEM, expert

Author

• Joan Herman, University of California Los Angeles/CRESST