Using Rasch Measurement Theory for Responsive Program Evaluation

• In Event: Research on Evaluation Session

Abstract

This study demonstrates the importance of Rasch Measurement Theory (RMT) in program evaluation when outcome measures need to be constructed from scratch. The paper introduces typical measure validation methods presented in program evaluation texts and discusses room for improvement. The study then illustrates how the seamless transitions from program theory, to ethnographic observation, to item generation, to measure analysis, to effect-size estimation are all possible under one framework using Rasch modelling. The program evaluated in the present study focuses on increasing retention of students in the biology major at a public university. The results presented balance the pragmatics and technical requirements of program evaluation. We show how this approach can align with Robert Stake’s call for valuing in program evaluation.

Authors

• Albert Anthony Clairmont, University of California- Santa Barbara

• Daniel Katz, University of California - Santa Barbara

• Mike Wilton, University of California - Santa Barbara