Using Empowerment Evaluation to Build a Community Consensus, Self-Assess, and Plan for the Future

• In Event: Marriott Roundtable Session 32
  In Roundtable Session: Empowerment Evaluation and Technology: Building a Virtual Computer Science/STEM+C Evaluation Learning Community

Abstract

The Google organized “Evaluation Wrecking Crew” decided to use an empowerment evaluation approach to focus its efforts, assess the current state-of-affairs, and establish an agreed upon plan for the future. A three-step process was used, including: establishing a mission; assessing current state-of-affairs; and planning for the future.

Mission

Members of the group began by brainstorming a list of mission-related ideals and aspirations. They summarized the list as follows:

“The ultimate purpose of the Wrecking Crew is to increase the quality of K-12 computer science and teacher professional development. The Wrecking Crew’s primary contribution to this goal involves improving the evaluation of computer science education programs.”

This exercise helped the diverse group build a consensus around their purpose or mission, building a solid foundation for the next step - taking stock.

Taking Stock

The group assessed their status, during the second online meeting. The generated a list of 37 activities they thought critical to accomplishing the group’s mission. The list included: building an evaluation repository, mentoring evaluators, and seeking additional support for the group’s activities.

Each member was given 5 dots to vote for the most important activities to assess as a group from this point onward. This was essentially a way to prioritize the list since it was not feasible to assess everything at that time.

The group selected 10 activities (from most important to least important within the context of activities meriting inclusion in the rating exercise). A few included:

creating a centralized hub (for sharing measures, resources and communication)
collecting measures
developing a mentoring system
educating CS community about evaluation
bringing in other corporate stakeholders

The group rated the performance of each activity on a 1 (low) to 10 (high) scale. The facilitator engaged the group in a dialogue about the ratings, calling on individuals to provide the evidence for their ratings.

The ratings and dialogue provided their collective view of their assessment of CS/STEM+ education evaluation. For example,

1. creating a centralized hub received moderate ratings (4.85) because it had been discussed but not implemented.
2. collecting measure received moderate ratings (4.3) because the measures that did exist were not organized, vetted, or easily accessible
3. mentoring received low ratings (2.54) because there was virtually no mentoring f
4. attracting other corporate stakeholders received low ratings (2.77) because it had received little

The ratings and the evidence for a rating were used to guide the next step of the process plans for the future.

Plans for the Future

The third step of an empowerment evaluation approach, planning for the future, is designed to facilitate action, detailing specific and concrete steps for the group. The Evaluation Wrecking Crew’s strategic plans for the future provide a road map for future action and included:

Creating a Centralized Hub
Collecting Measures
Bringing in Other Corporate Partners
Educating CS Community About Evaluation
Teaching About Effective Measures

The second part of this demonstration highlights the technological tools used to facilitate the empowerment evaluation.

Authors

• David M. Fetterman, Fetterman and Associates

• Jason Ravitz, Google Inc