Method Effects: Concepts and Models

• In Event: Roundtable Session 117
  In Roundtable Session: Perspectives on Validity

Abstract

Although the idea of a “method effect” seems intuitively straightforward—that is, it occurs when any characteristic of an instrument contributes variance to scores beyond what is attributable to the construct of interest—much of the conceptual vocabulary surrounding the concept remains confused, as does the relationship between the concept and many common psychometric models. This paper explores the connections between models from diverse research traditions including the multitrait-multimethod CFA model, IRT models for testlet and facet effects and other forms of local dependence, and generalizability theory, at both a technical and a conceptual level. Three real-data examples are used to illustrate the theoretical and empirical considerations that can lead to optimal model selection relative to a particular assessment situation.

Author

• Andrew Maul, University of Colorado - Boulder