Differential item functioning and the Child Behavior Checklist: Using IRT to assess externalizing behavior in Belize youth

• In Event: 3-005 - R-E-S-P-E-C-T Individual Differences: A Methods Approach for Examining Culture and Gender

Integrative Statement

Introduction
The Child Behavior Checklist (CBCL) is designed to measure behavioral and emotional problems in children and adolescents. The CBCL assesses these problems from a multicultural perspective (Achenbach et al., 2008). There are eight areas of assessment with this instrument: anxious/depressed, withdrawn/depressed, somatic complaints, social problems, thought problems, attention problems, rule-breaking behavior, and aggressive behavior (Achenbach, 1991). Studies have been conducted in the Caribbean using the CBCL (e.g., Lambert, Knight, Taylor, & Achenbach, 1994; Lambert, Lyubansky, & Achenbach, 1998). However, little validation evidence of been provided for the CBCL from that region, and none employing item response theory (IRT).
Methods
The current study we use IRT to assess the psychometric properties of an externalizing behavior -- aggressive behavior (17 items) -- from the CBCL in a sample of Belize adolescents. IRT is a method used to analyze items and scales, create and administer psychological measures, and measure individuals on psychological constructs (Reise, Ainsworth, & Haviland, 2005). In IRT an individual’s response to an item is influenced by the qualities of the individual, and qualities of the specific item (Furr & Bacharach, 2008). Specifically, IRT examines item measures based on the abilities or traits of the examinee (Hambleton & Jones, 1994). Specifically we used differential item functioning (DIF) that allows for scoring and comparing individuals of diverse groups onto a common scale (Embretson & Reise, 2000). For the current study we assessed sex and age DIF.
The sample consisted of 2,795 Belize adolescents, ages 12-18 years old with average age 14.1 (SD = 1.20). There are 1,326 males (47.4%), and 1,469 (52.6%) females. There are 1,874 adolescents (67%) ages 12-14 years old, and 921 adolescents (33%) ages 15-18 years old. The data is from the Youth Self-Report form, and students self-rated their responses. DIF analyses was conducted using the flexMIRT software (version 3.51; Cai 2017). Item purification through determining anchor items following the suggestions of by Woods, Cai, and Wang (2013).
Results
Results from the sex-DIF revealed that three items showed DIF: item-10 ‘Scream a lot’, item-12 ‘mood changes’, and item-17 ‘loud’. Male and female adolescents of the same level of aggressive behavior respondents differently on these three items. Figure 1 provides a visual of item-10 (Screams a lot). Item-10 is more reliably assessing males at low levels of aggressive behavior than females of similar aggressive behavior. Similar patterns are shown for item-12 and item-17. We may argue that the 17 item scale is not performing as well as a potentially 14-item measure of aggressive behavior. All 17 items were free of DIF for the age groups.

Author

• Marvin G Powell, George Mason University
  Presenting Author