Beyond the Trifold: Accurately and Reliably Assessing Quality in Youth Policy Presentations

• In Event: Marriott Roundtable Session 25
  In Roundtable Session: Engaging Students in Systems Change: Embracing Multiple Research-Practice Partnership Stakeholders in Transformative Student Voice Work

Abstract

Objectives
This paper introduces the Measure of Youth Policy Arguments (MYPA), a protocol designed to assess policy presentations associated with action civics education. Using evidence obtained from 20 youth presentations and a focus group with civics educators, we argue MYPA is a valid and reliable tool.

Theoretical Framework
Traditionally civic learning takes the form of stagnant classroom-based projects that rarely result in students engaging in actual civic action (Kahne & Middaugh, 2009). Traditional civics learning culminates in students presenting their work in the form of trifolds, power points, or written reports to inauthentic, or non-decision making, audiences. A new approach, called action civics, facilitates civic learning by authentically engaging youth in research, policy development (Levinson, 2012), and public presentations, where students share their policy solutions, with authentic decision making audiences. Alas, few resources exist to support youth in constructing and delivering high quality, evidence-based policy arguments (Pope, Stolte, & Cohen, 2012). MYPA was created to fill that void and consists of 25 items in six domains: presentation and delivery, problem identification, research methods, policy proposal, collaboration, and responsiveness to questions.

Methods
We draw upon construct validity and inter-rater reliability to establish that MYPA is psychometrically sound. Construct validity was established using a panel of civics educators and policy makers. These experts examined the MYPA protocol and used MYPA to score video examples of youth policy arguments. Inter-rater reliability was established using two raters who attended two action civics events. At each of these events they, simultaneously and independently, scored youth policy presentations. A total of 20 presentations, in which youth presented on an issue that impacted their lives, were observed and scored: 16 in the midwest and 4 in the west. Percent agreement was used to established inter-rater reliability.

Results
A panel of civics education experts agreed that elements of a quality presentation were captured by MYPA and that MYPA scores aligned with their assessment of presentation quality. Raters then observed youth policy presentations and achieved an 81% agreement on items of the MYPA. Given that a percent agreement of over 80% is deemed acceptable (McHugh, 2012) and experts agreed with the content of MYPA, we argue that MYPA is valid and reliable.

Scientific or Scholarly Significance
There are limited resources available for youth on how to leverage their voice in policy context; MYPA fills this void by providing a valid and reliable assessment of the quality of student presentations. Students, and their adult partners, can use MYPA to receive both formative and summative feedback, enabling students to hone their presentation abilities to leverage their voices and advance equitable, systems-level solutions. Further, MYPA is designed to facilitate student power--as opposed to assessing student knowledge or rudimentary presentation skills. By facilitating the empowerment of students’ voice in policy arguments, we hope youth can claim space for student voice in educational and policy spheres. Finally, MYPA provides educators and researchers with a tool to assess the quality of youth action civics work, which might be leveraged to promote transformative youth voice programs.

Authors

• Carlos Porfirio Hipolito-Delgado, University of Colorado - Denver

• Ginnie Logan, University of Colorado - Boulder

• Dane Stickney, University of Colorado Denver