Are School Psychologists' Decisions Reliable and Unbiased? Results From Experimental Analogue Studies

• In Event: Roundtable Session 9
  In Roundtable Session: Are We Identifying the Right Students for Special Education? Emerging Directions in Disproportionality Research

Abstract

1. Objectives

The objective of this line of research was to examine reliability and racial bias in school psychologists’ eligibility determinations, as few researchers—and none in the past 30 years—have examined the effect of student characteristics on eligibility determinations.

2. Perspectives

Although landmark 1975 federal special education legislation was hailed as a civil rights victory for individuals with disabilities, special education has been criticized as a means of racial discrimination. More than 40 years of large-scale systemic analyses show persistent racial disparities, but the underlying dynamics remain largely unexplored. One important target of study is the evaluation of educational needs to determine the presence of educational disabilities. School psychologists have long been recognized as highly influential in this process and schools’ resultant decisions (Knoff, 1983; Mehan, 1991).
We focused on emotional disturbance, intellectual disability, and autism because racial disparities are currently most pronounced in these categories. For emotional and intellectual disabilities, minority students may be inappropriately labeled and provided poor quality services that restrict their opportunities to learn (Donovan & Cross, 2002). For autism, students of color with legitimate educational needs may be denied much needed services (Sullivan, 2013). Thus, special education is considered a double-edged sword: the same system designed to benefit marginalized students operates to harm others (Sullivan, Kozleski, & Smith, 2008).

3. Method

This project used analogue vignette methodology common in cognitive and social psychology and law to explore bias. We used a 3 x 2 mixed factorial experimental design in each study consisting of one between-subjects factor (race: Black or White) and one within-subjects factor (ambiguity: unambiguous/qualified for disability, unambiguous/no disability, and ambiguous). The low-ambiguity conditions assessed participants’ evaluation competency when presented with uncomplicated evaluation results. The ambiguous case served as the test for racial bias, evaluating the presumption that under ambiguous conditions, evaluators’ implicit biases may affect educational decisions.

4. Data Source

Materials. Materials included: (a) 3 sets of hypothetical vignettes describing assessment results from special education evaluations, (b) a rating scale for participants to indicate their confidence in their decisions, and (c) a questionnaire about participants’ demographic information, educational history, work history, and experiences interacting with racially/ethnically diverse populations.
Participants. Participants in all studies included approximately 300 school psychologists in 3 states where special education eligibility criteria matched federal criteria. Participants were identified via simple random sampling of each state and were randomly assigned to each condition.

5. Results

In each study, referral and eligibility decisions were shown to be unreliable but unrelated to race when participants were making differential identifications for ID and ED, but not ASD. When presented with case studies of students who met criteria for ASD, participants were significantly more likely to identify Black students than White students. Competency and attitudes towards the effectiveness of special education were correlated with false positive eligibility decisions.

6. Scholarly Significance

These results suggest that biased decisions do not contribute to racial disproportionality. The unreliable and inaccurate pattern of decisions underscores the need for improved professional development regarding special education requirements.

Authors

• Amanda L. Sullivan, University of Minnesota

• Alaa Houri, University of Minnesota - Twin Cities

• Shanna Sadeh, University of Minnesota - Twin Cities