Evaluation of the Education for Sustainability in Galapagos Program

• In Event: 25.082 - Fairmont Royal York Roundtable Session Three
  In Roundtable Session: 25.082-10 - Education for Sustainability: A Public-Private Partnership for Transforming Education in the Galapagos

Abstract

Objective and Purpose
Any program that seeks to improve its program as well as prove its outcomes benefits from a holistic evaluation approach. As the primary research and evaluation arm of the Education for Sustainability in Galapagos project, we ask and answer questions regarding how the program is going and trying to understand what changes, if any, occur in educators' instructional practice and attitudes in specific constructs (e.g., grit, growth mindset, peer collaboration). In the process of doing so, we also examine fidelity of implementation and dosage of intervention to identify the key factors of the implementation that might lead to creating these potential outcome changes.
In order to address needs of myriad stakeholders and implementers, the evaluation of this program utilizes a blend of developmental, empowerment, formative, process, and outcome evaluation approaches for a comprehensive evaluation that caters to different needs and for different purposes. For the purposes of this presentation, the focus will be on formative, process, and outcome data.

Evaluation Questions and Framework

Formative and Process Evaluation Questions:
1. Are evidence-based best practices evident in Institute workshops?
2. Are evidence-based best practices evident in coaching?
3. Are evidence-based best practices evident in learning circles (LCs)?

Outcome Evaluation Questions:
4. Are educators' attitudes changing over time?
5. What demonstrable changes can be seen in teachers' instructional practices over time?

Data Sources and Methods

Formative and Process Data
· Teacher and Administrator Focus groups
· Satisfaction surveys
· Observations of workshops
· Observations of one-on-one coaching
· Observations of professional learning circles

Outcome Data
· Attitude Construct surveys
· Coach feedback on classroom instruction
· Video observation of classroom instruction

Findings
Data on formative and process evaluation has been triangulated over three years. Latest findings from 2017 indicated that workshops were generally strong, with 14 of 18 evidence-based best practices strongly evident in workshops. Coaching and learning circles (LCs) were implemented with less alignment to evidence-based best practices (9 out of 17 and 13 out of 21 respectively). In terms of outcomes, 12 out of 18 constructs were rated higher than 4 on a 5-point scale, indicating generally high attitudes. Additionally, outcome data analysis is currently being planned to see the effects of dosage data (attendance in workshops, coaching, LCs), moderated by attitudes, on outcomes such as coaching observations and video observations of instructional practice.

Scholarly Significance
Evaluation, if designed well, can contribute to a deeper understanding of program implementation for continuous improvement. During this presentation, a Tableau dashboard as a means to effectively display this data results will be briefly demonstrated. This is done in order to empower implementers of the program to take ownership of their own data for continuous program improvement. The triangulation of data in a brief executive summary will also be explained during the discussion of findings in order to make this data accessible to different stakeholders.

Authors

• Hiba Rahim

• Harvey Luna

• Akihito Kamata, Southern Methodist University