Utilizing Mixed Methods to Evaluate Professional Development Programs

• In Event: The Parker Hotel Roundtable 2
  In Roundtable Session: Approaches and Issues in Assessing Outcomes of Training/Education Programs: A Healthcare Education Perspective

Abstract

Evaluation can be thought of as a systematic process utilized to determine the worth or value of whatever is being evaluated (e.g., program, product, policy, etc.). Evaluators use systematic inquiry and methodology to arrive at a determination about the entity being evaluated. This determination can be a number of things. For example, an evaluation could be conducted at the end of a program to determine whether the program was a success or successfully achieved its goals, an example of a summative evaluation, which is typically conducted after a program’s conclusion. Another example of a summative evaluation would be one conducted to determine whether or not a program should continue. This would involve conducting an evaluation after a program has been implemented and then using the evaluation results to make a judgement about whether the program should continue or be terminated. However, an evaluation could also be conducted solely for program improvement purposes and not to inform a decision regarding program adoption. For example, an organization might conduct an evaluation to see how program delivery to participants can be improved or while a program is still ongoing to identify ways in which the program can be made more cost-effective. These would both be examples of formative evaluation or evaluation aimed at improving the program in some way.

What separates program evaluation from research, assessment, and the like, is that typically in the land of evaluation, we are dealing with value of an entity. Additionally, more often than not, evaluation relies heavily on the stakeholders of a program to not only inform the evaluator about program history, function, and direction, but also to buy in to the entire evaluation process to make it as meaningful and helpful as possible. Evaluation can serve a unique function in the professions in that not only can evaluation methods tell us if program goals and objectives are being met, they can also tell us reasons why goals and objectives might not be being met (i.e., they can highlight areas in need of improvement). Moreover, evaluation can be an ongoing process that informs program function and delivery along the way, as a program unfolds.

Given the growing demand for clinicians and faculty to be able to provide interprofessional educational experiences for students in the health professions coupled with the critical shortage of health professionals trained in geriatrics, one mid-Atlantic university established a Faculty Development Program (FDP), aimed at providing clinical faculty interprofessional training in geriatrics. To evaluate the effectiveness of this program, a mixed methods evaluation was conducted. Prior to the beginning of the program, participants completed a geriatric knowledge test and a geriatric efficacy questionnaire. After the year long program concluded, participants once again completed the knowledge test and efficacy questionnaire as well as participated in a phone interview. Findings indicate that the FDP program greatly enhanced participants knowledge of interprofessional care for geriatric patients, their ability to teach these topics to students, and, ultimately, the number of faculty and students exposed to interprofessional educational experiences.

Author

• Sarah Anne Marrs, Virginia Commonwealth University