Using Logic Modeling as a Tool to Infuse Evaluative Thinking and Guide Developmental Evaluation

• In Event: All Aboard! Open to Learning Through Applied Research Practices and Evaluative Thinking in the Ontario Public Education System

Abstract

We will share how embedding evaluative thinking in our practices support building the sector’s capacity to learn and use evidence to inform decisions and implementation through integrating research, monitoring, evaluation and capacity building. Learning through evaluative thinking contributes to deepening and broadening the system’s capabilities in meeting changing conditions; adding and using new skills, knowledge and behaviors; and becoming an increasingly sophisticated system through reflection of its own actions and consequences. (Rowe & Kuji-Shikatani 2013; London & Sessa 2007).

DE helps build a culture of evaluative thinking in systems that is adaptive, coherent, precise and personalized through collaborative inquiry-action and reflective practice:
…a tool for organizational learning, and at the heart of organizational learning is the ability to enter into a professional community, to develop modes of inquiry, and to take risks. …communities for practice examine their own practice; They analyse data related to their work to inform their planning and decisions; the dialogue results in learning that builds coherence and capacity for change. Use of the cycle, which begins and ends with questions of evaluation, is crucial to improving practice. (Militello, Rallis and Goldring, 2009)

Making evaluative thinking an inclusive activity is a challenge for leaders. Logic Models offer a way to describe and share an understanding of relationships among elements necessary to operationalize any change effort. (Knowlton & Phillips , 2012) Most of the value in a logic model is in the process of creating, validating, and modifying the model. The clarity of thinking that occurs from building the model is critical to the overall success of the program.

In 2010 the Logic Modeling Initiative began as a Student Achievement Division project that created a basis for ongoing evolution and improvement of our work. Now logic modeling is used as a way we work to: build a culture of evaluative thinking through collaborative inquiry, action and reflective practice that is adaptive, precise and personalized; serves as an important tool for making explicit the logic involved in reaching every student through our commitment to a learning as we go stance; models professional accountability for schools and districts in a process that is analogous to improvement planning; and integrate research, monitoring and evaluation across all initiatives. One such initiative, Board Improvement Plan for Student Achievement (BIPSA) builds evaluative thinking through a cyclical process of collaborative inquiry-action, addresses the board’s own most urgent student learning needs based on needs assessment, planning, implementation monitoring and evaluation. Ministry supports the process by providing regional professional learning opportunities, resources, and teams of educational experts who visits the board to engage in conversations as a professional community. In turn, the ministry learns through engaging in DE - collaboratively analyzing and learning from what the boards have learned individually and collectively – to gain learning that informs the ministry’s next steps. Logic modeling is used throughout the year to infuse evaluative thinking and guide DE for planning, communication, implementation and share learning as a system to inform the next steps.

Authors

• Mary Jean Gallagher, Ontario Ministry of Education

• Keiko Kuji-Shikatani, Ontario Ministry of Education