Analyzing Response Styles and Trait-Based Rating Responses With Multiprocess Item Response Theory Models

• In Event: 41.027 - Recent Advances in Modeling Response Style Bias in Rating Scale Data

Abstract

Multiprocess IRT models combine processing-tree models for sequences of decision nodes with IRT models for the node probabilities (Böckenholt, 2012; De Boeck & Partchev, 2012; Jeon & De Boeck, 2015). When applied to rating responses, multiprocess IRT models allow one to disentangle response processes that are related to the traits of interest and response processes that concern the use of rating categories irrespective of item content (Khorramdel & von Davier, 2014). In particular, multiprocess IRT models can be specified to accommodate response styles like the preference for midscale responses (midscale response style, MRS) and the tendency to choose extreme responses (extreme response style, ERS) in addition to trait-based response processes. The decomposition of response styles and trait-based processes through multiprocess IRT models has recently been validated with extraneous measures of MRS, ERS and trait covariates (Plieninger & Meiser, 2014) and in terms of convergent versus discriminant correlations between substantive traits and response style dimensions (Böckenholt & Meiser, 2016). In addition, selective associations of response style dimensions and traits with education, age and personality variables (Böckenholt, 2012; Zettler et al., 2016) support the validity of response processes as specified in multiprocess IRT models.

Our ongoing research aims at the analysis of acquiescent response style (ARS) with multiprocess IRT models and at the integration of multiprocess models and polytomous IRT models. First, we extend multiprocess IRT models to account for ARS in item sets with both regular and reversed items. The multiprocess IRT model for regular and reversed items offers an alternative to bifactor or multitrait-multimethod IRT models for the analysis of item wording effects (Wang, Chen, & Jin, 2015) and allows the simultaneous analysis of additional response processes like MRS and ERS. Second, whereas multiprocess IRT models usually include binary or categorical decision nodes for content-based agreement, polytomous IRT models like the Partial Credit Model (PCM), generalized PCM or Graded Response Model specify an ordinal selection process with gradual degrees of agreement. Although empirical results suggest that the specification of binary agreement nodes does not attenuate estimated correlations between constructs (Böckenholt & Meiser, 2016; Plieninger & Meiser, 2014), the neglect of ordinal trait-related response processes is a critical assumption. At the same time, polytomous IRT approaches that accommodate response styles, like mixed PCMs (e.g., Meiser & Machunsky, 2008) or multidimensional IRT models (e.g., Bolt & Johnson, 2009), contain less specific or unrealistic assumptions concerning the nature of response styles or the correlations between style dimensions and substantive traits. In our presentation, we therefore analyze how content-based and style-based responses generated under one model class can be captured by another model class, and we demonstrate how a-priori specified response styles can be integrated in a model with an ordinal trait-based response process. Related open questions that will be addressed concern parameter estimation and parameter recovery in multiprocess IRT models with traditional and Bayesian methods.

Authors

• Thorsten Meiser, University of Mannheim

• Hansjoerg Plieninger, University of Mannheim