Separating Content-Related Effects From the Tendency Toward Middle or Extreme Categories in Rating Scale Data

• In Event: 41.027 - Recent Advances in Modeling Response Style Bias in Rating Scale Data

Abstract

Rating scales frequently have symmetric response categories of the form strongly disagree, moderately disagree, ..., moderately agree, strongly agree. The choice of one of the response categories is often determined by a response style that is characterized by a disproportionate tendency to middle categories or to extreme categories, that is, the highest and lowest response categories. The preference to extreme categories is often called extreme response style and has been a topic of research for some time. The problem with heterogeneity in response styles is that it can affect the conclusions drawn from rating scale data. In particular, biased estimates can be expected if one ignores a tendency to middle categories or to extreme categories. Approaches to simultaneous modelling of content related effects and response styles are scarce. Most approaches rely on the calculation of specific indices that can be corrected by regression techniques. Exceptions are latent class approaches, which are strong tools, but the existence of latent classes is always a strong assumption and the interpretation has to rely on their existence.

An alternative model is proposed that explicitly and simultaneously models the content-related effects and the heterogeneity in response styles. The model allows us:
• to investigate content-related effects that are undisturbed by the response style for a single item,
• to investigate the response style undisturbed by content-related effects,
• to use covariates to disentangle content and style, and
• to avoid biased estimates of the content-related effects, which are the parameters of interest in most studies.

A visualization tool is developed that makes the interpretation of effects easily accessible. The proposed model is embedded into the framework of multivariate generalized linear model, which entails that common estimation and inference tools that have been developed for this class of models can be used. Existing software can be employed to fit the model, which makes it easy to apply. The crucial difference between latent variable approaches and the proposed adjacent categories model is that the response style is not perceived as an individual trait, but exists in relation to the covariates. The model does not need the additional assumptions that accompany latent variable modelling.

Authors

• Moritz Berger, University of Bonn

• Gerhard Tutz, Ludwig-Maximilians-Universität München

• Matthias Schmid, University of Bonn