Validity Evidence Supporting Science Achievement Assessments in Published Intervention Studies

• In Event: 67.063 - Advancements and Applications in Test Validity Research and Evaluation

Abstract

The following study reports on an analysis of validity evidence reported in science education intervention studies published in the Journal of Research in Science Teaching from 2005 to 2015. Based on a coding rubric derived from the Standards for Educational and Psychological Testing (Standards) (AERA/APA/NCME, 2014), we report on metrics describing the different types of validity evidence present, interrater reliabilities, and a principal components analysis. The latter analysis resulted in three components representing 1) content/construct validity, relationship to external measures, and overall soundness of validity; 2) consequences of testing; and 3) internal structure of the assessment. Content/construct validity is the most prevalent type of validity evidence, and consequences of testing was the least prevalent. Implications of these findings are discussed.

Authors

• Zachary Robert Mayne, University of Utah

• Michael K. Gardner, University of Utah

• Gabriel M. Della-Piana, University of Utah