Using Generalizability Theory to Explore the Trustworthiness of Survey Data From an International Development Evaluation

• In Event: 32.080 - Measurement and Validity Issues in Survey Research

Abstract

The trustworthiness, or reliability, of survey data may be assessed in many ways, depending on the type of the assessment, the context of the data collection techniques, and whether we want to compare individuals to one another (norm-referenced) or an external criterion (cutscore) (criterion-referenced). Data from an ongoing evaluation in the Democratic Republic of the Congo is analyzed using Generalizability theory to provide some insight into the effects of an informal adaptation method of a survey administered by enumerators. Due to space constraints here, initial analyses were completed using one year of data with enumerator, item/subtest, and language as random facets in a fully-crossed design. These results indicate that there were some sources of variability that require further investigation.

Author

• Louise M. Bahry, University of Massachusetts - Amherst