Accepting the Challenge of Evaluation as a Leadership Development Best Practice

• In Event: The Evaluations Are In: Are You Curious?

Short Description

This presentation will examine two cases to illustrate the use of practical approaches to address the challenges of evaluating leadership development. Key design elements of the leadership development initiatives and the evaluations will be highlighted. Best practices to ensure continuous program improvement and to understand the program’s impact will be discussed.

Detailed Abstract

Organizations need leaders with the ability to succeed in the face of today’s consistent uncertainty, complexity and rapid change (Conference Board and Right Management, 2013). Innovations in strategy or services can move a fledgling or experienced organization to the front of the pack, and effective leadership will help that same organization stay in the lead regardless of the changes that occur or obstacles that arise. Look at organizations that are succeeding and you will likely find many are investing in leadership development (Center for Creative Leadership, 2016). Leadership development (LD) is one of the tools organizations continue to use to help ensure leaders and organizations are prepared to respond effectively to the challenges of turbulent times.

Although much has been written questioning the impact of LD investments, the literature has consistently supported the effectiveness of LD as long as it is designed and implemented according to best practices that have evolved over time (Ortrun Zuber-Skerritt Ina Louw, 2014; Harvard Business Publishing, 2016; The Conference Board, 2015; Center for Creative Leadership, 2015). Based on these reports, which share findings from numerous global surveys of business leaders, LD is more likely to be effective if it is:
1) Designed as part of broader organizational change effort and strategy execution;
2) Supported by senior leadership;
3) Geared to create a strong internal leadership pipeline;
4) Constructed around a continuous learning philosophy and process in which participants apply what they learn over time;
5) Based on the adult experiential learning theory and practice;
6) Targeted to specific and relevant capabilities (or competencies);
7) Tailored to current job and business challenges making it highly relevant to participants; and
8) Evaluated to ensure continuous improvement and to understand overall impact.

Evaluation is included consistently as an LD best practice to “aid in the improvement and effectiveness of the programs themselves” (ASTD Report, 2010) and to measure the effectiveness of the LD initiative (Harvard Business Publishing, 2016). Effective evaluation keeps LD initiatives on track and contributes to organizational learning so that leaders can remain responsive and resilient. Effective evaluation tools can help the organizations determine the quality of impact and implementation of their leadership development initiatives. Evaluation is used as a way to guide us in refining LD work and understanding its impact on various stakeholders. Yet, most companies, even those implementing LD according to the best practices listed above, encounter major challenges with measuring the impact of LD (Harvard Business Publishing, 2016). According to the Harvard report on the State of Leadership Development, only 40% of companies surveyed who have “best in class” LD programs believe they capture their program’s effectiveness (Harvard Business Publishing, 2016).

This presentation will use two cases to illustrate the use of practical approaches to address the challenges of evaluating LD and highlights of the findings of these evaluations. For both cases, the programs being evaluated are designed to incorporate the best practices listed above. Descriptions of each case will include key elements of the LD initiative design, the evaluation design, and key evaluation findings.

Case One. The first case is the evaluation of a comprehensive physician leadership development program designed as part of a broader culture change initiative in a large health system in the United States. The 11-month developmental experience includes:
• Team analysis of individual, group and organizational challenges;
• Face-to-face individual / team dynamics feedback session and goal setting;
• Intensive individual executive coaching;
• Multiple tailored 1 ½ day workshops to support achievement of team goals; and
• Small group action learning projects and data collection and reporting to support application.

The evaluation captured post-program data using surveys and focus groups from participants in five cohorts as well as from participants’ direct reports, peers, and bosses. In addition, participants were asked to link the impact of their action learning projects overall business success metrics. Findings included perceived behavior shifts, leadership knowledge gains, improved leadership competencies year over year, and Return on Investment estimates from the action learning projects.

Case Two. The second evaluation includes 1,895 leaders who attended a five-day open enrollment leadership development program from 2011-2015. The global program aims to strengthen the ability of leaders to manage complexity, balance competing priorities and collaborate up, down, and across the organization to drive tangible results. Each participant applies what they learn to a key leadership challenge and receives coaching post-program to support goal application. The evaluation draws on feedback collected three months post program from 1,895 participants and their bosses. This study had two goals: (1) understand the impact of this leadership development program on organizational outcomes and (2) uncover the key supports that will help leaders and organization experience even greater improvements in these outcomes from the program. It provides evidence of the importance of boss support as one of the most important drivers of impact on effectiveness, management capabilities, and empowerment of participants’ teams.

As organizations look at best practices in designing and implementing LD they should make sure they include evaluation designs that will address the needs and interests of key stakeholders. By ensuring multiple perspectives and interests are captured, findings provide a more accurate and useful understanding than when just single rater survey ratings are used. This presentation, through two specific cases, shares information useful to organizations tackling the challenge of evaluating leadership development initiatives to ensure continuous program improvement and to understand business impact.

Participants

• Holly Downs, Center for Creative Leadership

• Tracy Patterson, Center for Creative Leadership