Poster #53 - Context Matters: The Importance of Testing for Random Effects in Hierarchical Data

• In Event: Poster Session 15

Abstract

INTRODUCTION: As children navigate educational contexts such as childcare centers or schools, they are impacted by a complex ecological system (Bronfenbrenner, 2009) where children are nested within classrooms (or teachers), which are nested within schools or centers, which are then nested within school districts and communities. Each of these levels has a unique culture that can impact children’s growth, development, and learning (Burstein, 1980; Raudenbush & Wilms, 1995). Multilevel modeling (MLM), also known as hierarchical linear modeling or mixed-effect modeling, is one statistical method to account for these hierarchical settings. However, MLM can do more than just account for the nesting of data, as it can also be utilized to investigate how higher-level factors moderate associations between lower-level factors, called random effects (See Figure 1). This analysis can be conducted even if the higher-level factor was not measured (e.g., a school-level variable was not measured in a student-teacher-school model). To demonstrate the need to test for random effects, two statistical models will be used as exemplars.
EXEMPLARS:
There is currently a lack of clear understanding of the teacher’s role in peer relationships within the classroom (Copple & Bredekamp, 2009). There is also an inconsistency in the literature about the associations between teacher beliefs and student outcomes (See Buehl & Beck, 2015, for review), with one explanation potentially being that the larger school context is not being accounted for (Fives & Buehl, 2012).
RESEARCH QUESTIONS:
1. Do teacher beliefs about reasons for social exclusion directly impact student social outcomes?
2. Does the school context moderate the relation between teacher beliefs and student social outcomes?
METHOD: Data were collected from 932 first-grade students within 101 classrooms, nested within 25 schools. Two models are used as exemplars to demonstrate random effects using MLM. Teacher Reason for Exclusion (TBS-Reason) is tested in both models as the predictor variable, with outcome variables of peer reported Mutual Dislike (Model A; Coie et al., 1982) and teacher reported Bullying (Model B; BASC-II).
RESULTS AND DISCUSSION: For both Models, the fixed effect between TBS-Reason and the student-level variable was not significant (Mutual Dislike, p = 0.400; Bullying, p = 0.371). However, the random effect of TBS-Reason was significant (Mutual Dislike, p = 0.027; Bullying, p = 0.023). This significant random effect, but non-significant fixed effect, demonstrates that in some schools the student-teacher relation is positive and in others, the relation is negative, thus, the overall relation across schools averages to zero. This is visualized in Figure 2 where the yellow line demonstrates the fixed effect of TBS-Reason predicting Bullying and is not statistically different from zero. However, the distribution around the fixed effect (i.e., the random effect) shows that 95% of the schools’ relations fall between the two orange lines. This means that something at the school level is affecting when and in what direction TBS-Reason impacts Bullying (i.e., there is a moderator at the school level). These exemplars emphasize the importance of testing for random effects even if the fixed effect between two variables is not significant.

Authors

• Clarissa M Corkins, Oklahoma State University - Stillwater
  Presenting Author

• Amanda W. Harrist, Oklahoma State University - Stillwater

• Isaac Washburn, Oklahoma State University

• Laura Hubbs-Tait, Oklahoma State University - Stillwater

• Glade Topham, Kansas State University

• Lenka Shriver, University of North Carolina at Greensboro

• Taren Swindle, University of Arkansas for Medical Sciences