Investigating Tendency in Responding to a Self-Report, Likert-Type Rating Scale: The Rosenberg's Self-Esteem Example

• In Event: Using Mixed Models to Investigate Tendency, Process, and Typology in Item Responding

Abstract

1. Purpose
In a rating scale, response categories are written to vary the degree of endorsement. This response format has been a subject of research interest because respondents may have a tendency to prefer or avoid particular categories, which can contaminate the measurement of the true level of attribute. The presence of construct-irrelevant response tendency can seriously distort the meaning of the scale scores. This study investigates the presence of different types of tendency in responding to the Likert-type rating scales (e.g., Strongly Agree, Agree, Disagree, and Strongly Disagree), the most widely used response format for self-report measures.
2. Perspective(s) or theoretical framework
Response tendency has been discussed with different terms such as response style (Cronbach, 1946), response set (Jackson & Messick, 1958), and response bias (Rundquist, 1996; Paulhus, 1991). Reflecting the definition in previous research, this study defines response tendency as a response pattern of choosing particular categories of the scale characterized by degree and direction of endorsement after controlling for item difficulties (thresholds) and respondents’ levels of the measured attribute.
Various response tendencies were suggested. This study focuses on the most frequently investigated: (1) acquiescence response style (ARS), referring to the tendency to agree with or accept the statements by selecting positively-endorsing categories, (2) disacquiescence response style (DARS), referring to the tendency to disagree with the statements by using negatively-endorsing categories, (3) extreme response style (ERS), referring to the tendency to endorse the most extreme categories, and (4) mild response style (MLRS), referring to the tendency to endorse non-extreme categories.
3. Methods and data sources
We used the Rosenberg self-esteem scale (RSES) as an example of Likert-type rating scale to investigate response tendency. The scale consists of 10 items with four response categories (Strongly Agree, Agree, Disagree, and Strongly Disagree). A sample of 1,596 participants were included in the analysis (male=43.2%, M_age = 49.37, SD_age=18.89). Generalized linear mixed model was used to test the presence of the four response tendencies while controlling for item difficulty and respondents’ levels of attribute, as well as the item-keying direction (i.e., reverse scoring).
4. Results
Results showed that there was a statistically significant tendency to prefer mild categories (Agree and Disagree) to extreme categories. On average, respondents had a slight tendency to not choose the extreme categories (odds ratio=0.89). Also, there was a statistically significant tendency to prefer positive categories (Agree and Strongly Agree) to negative categories (odds ratio=2.20). Respondents tended to acquiesce the statements irrespective of the contents in the stem.
5. Significance of the study
Unlike most previous work, our analytical framework revealed “pure” response tendency by controlling for response tendency associated with item difficulty and person attribute. Moreover, compared to previous work that investigated response tendency as one global method effect, this study provides a comprehensive and in-depth understanding of tendency in responding to Likert-type rating scale. The present analytical method can be adopted to investigate tendency in responding to other Likert-type self-report scales.

Authors

• Minjeong Park, The University of British Columbia

• Amery D. Wu, University of British Columbia