Advancing Innovations in STEM Education Evaluation: Program, Portfolio, and Project Evaluation Initiatives in the National Science Foundation’s Education and Human Resources Directorate

• In Event: Innovations in STEM Education Evaluation: Enhancing the Quality of Evidence Available to Assess the Impacts of Research Portfolios

Abstract

The National Science Foundation’s Directorate for Education and Human Resources has intensified its focus on program, portfolio and project evaluation across the Directorate. An EHR-wide Project and Program Evaluation (PPE) program has been established within the Division of Research on Learning in Formal and Informal Settings to fund evaluation, synthesis, and comparison studies designed to increase knowledge of effective practices in STEM learning and education, and program evaluation. While consistent with, these activities go beyond the existing project-level evaluations required in all EHR programs, and the aggregation of these findings to influence program-, division- and directorate-level strategic planning. In this presentation, the chair of a cross-directorate team working in consultation with experts both inside and outside of the agency will describe a range of activities underway to strengthen evaluation and improvement of STEM education programs, and provide initial results from these initiatives. Activities aimed at strategic STEM education investments to be discussed include:
1. Aligning EHR priorities with new requirements from the Office of Management and Budget. Activities include developing metrics for each program and exploring new areas such as how to develop measures for R & D activities.
2. Reshaping contractual activities so that new approaches such as evaluating themes that cut across programs are explored.
3. Creating an R & D program to advance innovations in STEM education evaluation. In September 2010 NSF issued the first solicitation for a new program to support research on evaluation. The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program places special emphases on: exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; building on and expanding the theoretical foundations for evaluation STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and growing the capacity and infrastructure of the evaluation field. To date, 17 awards have been made through the PRIME program, including projects to develop a unifying logic model and set of common metrics to guide and inform the portfolio review process, to convene representatives from key stakeholder groups involved in conceptualizing, translating, and implementing traditional and innovative evaluation approaches, strategies, and methods to assess efforts which hold the promise of transforming undergraduate STEM education. While several of the funded projects will remain ongoing at the time of the Annual Meeting, select highlights of preliminary findings and work in progress will be provided.

Author

• Janice M. Earle, National Science Foundation