It's Not on the Test: Capturing Teachers' Full Effects

• In Event: AERA Graduate Student Research in Progress Roundtable Session One
  In Roundtable Session: Teacher Beliefs, Identity, Cognitive Equity, and Assessment Practices (Table 15)

Abstract

Teachers' test-based value-added (VA) on Math and ELA have been studied with clear consensus that teachers matter for students' short- and long-run outcomes. Recent scholarship expands beyond teachers' test-based lower-order thinking skill (LOTS) VA by estimating teacher VA on higher-order thinking skills (HOTS) and 'noncognitive' skills (NCS), capturing additional dimensions of teacher effectiveness. Still, what is known about the coverage of new constructs studied (RQ1), the degree teachers shape students' growth on them (RQ2), the trade-offs in teacher effectiveness across these constructs (RQ3), and the long-run impact of assignment to teachers who are effective at promoting these new constructs (RQ4)? We systematically review 41 studies on teacher VA to HOTS (n=16) and NCS (n=25). We find the following: 1) Existing coverage of HOTS/NCS studies and constructs limit confidence in some findings (RQ 3&4) and impose critical knowledge gaps (e.g., no teacher VA on Creativity); 2) Teachers, on average, shape HOTS and NCS (except Academic and General Survey Measures) similarly to LOTS with variation across individual constructs--findings are complicated by potential bias; 3) Teachers typically skilled at promoting HOTS are somewhat skilled at promoting LOTS, but teachers are rarely equally skilled at promoting various NCS-- even purportedly similar constructs, raising measurement concerns-- and almost never equally skilled at promoting LOTS/HOTS and NCS; and 4) Assignment to teachers effective at promoting overall tests (limited HOTS evidence) and some NCS (i.e., Behavioral Index) is complementary with each improving distinct long-term academic and non-academic outcomes, but findings are sensitive to construct choice.

Author

• John Wang, University of Virginia
  Presenting Author