Using the Standards to Support the Fairness of Research Measures

• In Event: 47.015 - Leveraging the Test Standards to Improve Research Validity

Abstract

This presentation shares core perspectives that motivate the fairness standards, i.e., that fairness is
• an integral concern in the validity of score Interpretations
• a fundamental principle underlying all steps in the assessment process
• not something you think about “after the fact.” You must build it in
• fundamental right of all Individuals and subgroups in the intended test population.
Based on this perspective, the overarching fairness standard is as follows:
All steps in the testing process, including test design, validation, development, administration, and scoring procedures, should be designed in such a manner as to minimize construct-irrelevant variance and to promote valid score interpretations for the intended uses for all examinees in the intended population.
This overall standard is organized into four clusters of specific standards:
• Test design, development, administration and scoring that minimize barriers to valid score Interpretations for the widest possible range of Individuals and relevant subgroups
• Validity of test score interpretations for intended uses for the intended examinee populations
• Accommodations to remove construct-irrelevant barriers
• Safeguards against inappropriate use
The presentation of the most relevant standards for research ends with implications for researchers’ development, selection and/or use of measures that can provide valid results for all subjects in the intended research population, including consideration of these questions:
• Has the range of intended research subjects been fully specified?
• Are the items and tasks appropriate for all intended subjects?
• Were the items reviewed and revised for potential sources of bias?
• Were the items and measure piloted with the full range of indivdiuals for which it is intended?
• Will accommodations be needed for some students? Are some scores accommodated?
• Do psychometric or other analyses support the fairness of the measure? The validity of the accommodations?

Author

• Linda L. Cook, Educational Testing Service