A Bayesian Method for Multi-Unidimensional Instrument Development With Small Sample Sizes

• In Event: Instrument Development

Abstract

As multi-unidimensional tests are becoming increasingly popular in educational and psychological fields, it is important to gain content and construct validity evidence efficiently to develop high-quality multi-unidimensional instruments. However, traditional methods were challenged by small sample sizes. Analyzing response data without considering the ordinal nature is another challenge. To overcome these two challenges, we proposed a Bayesian method to integrate experts’ ratings and participants’ data to establish a unified model for validity evidence. Simulation studies were done to compare the performance of the Bayesian method to the traditional CFA under multiple situations. The results show that the Bayesian method outperforms the traditional CFA the most when the sample sizes of participants are small and the instruments are dichotomously scored.

Authors

• Jihang Chen, Boston College
  Presenting Author

• Zhushan Mandy Li, Boston College
  Non-Presenting Author