Estimating Classification Accuracy and Decision Consistency With classdecision

• In Event: Measurement and Statistical Modeling

Abstract

Much of the software for calculating true scores for classification accuracy and decision consistency is typically outdated. Despite a proliferation of programming languages and techniques for data science, methods for the estimation of accuracy and consistency of classifications based on test scores have not received the same attention as other methods. The classdecision function was built in R to address the lack of Classical Test Theory methods for estimating classification accuracy and decision consistency. This function was compared with three other software packages that use the same methodology to estimate classification accuracy and decision consistency. We will address the importance of statistical theory and rigorous programming by comparing the performance of different software packages.

Authors

• Alberto Alexander Diaz Tobias, University of Alberta

• Okan Bulut, University of Alberta

• Maria Cutumisu, University of Alberta