Capturing Problem-Based Learning Critical Competencies Over Time Through Portfolio Assessment

• In Event: Educational Responsibilities in Problem-Based Education (PBE): Principles, Practices, and Networks Creating and Supporting PBE

Abstract

Purpose
In this presentation we explore the development of a digital portfolio tool to capture critical workforce and college-ready competencies as K-12 students engage in problem-based innovation and entrepreneurship challenges. We also illustrate a component of the portfolio tool to highlight how and why student competencies may change during the problem-solving process.

Perspectives
An increased emphasis has been placed on preparing students with Future Ready competencies (creativity, critical thinking, computational thinking, communication, and collaboration) to position them for future success (employability, entrepreneurship, economic development). To provide opportunities to develop and demonstrate these competencies, problem-based learning (PBL) is being increasingly employed. By 2030, our team imagines classrooms transformed into incubator spaces where student teams work alongside community collaborators to create prototype solutions to challenging problems.

The creation of assessment and evaluation tools to capture student and team performance indicators during this process is limited, but essential. Portfolios have been shown to be an effective mode for students to demonstrate problem solving, technical mastery and metacognitive reflection (Keune & Peppler, 2017). Constructivist principles of knowledge building, active and self-directed learning, and collaborative learning can be captured in portfolios (Ligorio & Sansone, 2009) through activities that focus on metacognitive control, formative feedback, interconnected knowledge, and opportunities for social learning (Biggs & Tang, 2011).

Methods
We draw on authentic assessment (Cole, Ryan & Kick, 1995), growth mindset (Dweck, 2008), and longitudinal data analysis methods (Long, 2012) in our approach to portfolio development. Collectively, these methods allow students to demonstrate changes in competencies over time through engagement in innovation and entrepreneurial challenges. The analysis of portfolio artifacts uses discourse and statistical analyses.

Data Sources
Data sources for the development of the digital portfolio include expert reviews by industry
experts, psychometric and factorial analyses, content analysis, and beta testing. The
process was iterative and occurred multiple times across a four-year time span. Student digital portfolio analysis data is drawn from the artifacts of 1,289 high school students from 4 states.
Portfolios include data from students, educators and community members in the form of videos, self-assessments, formative feedback, monitoring, self and peer reflections, and work samples.

Results
Beta testing of the portfolio shows high functionality and usability. Longitudinal growth curve modelling indicates significant growth in critical Future Ready competencies over time (p<0.01, N=1,289 HS students), and growth remains significant when disaggregated by gender and race. These findings are significant as females and minorities are currently limited in fully participating in the innovation economy (Bell, et al., 2018). Preliminary discourse analysis of student reflections indicate they can identify how a critical competency is acquired (or isn’t). Students’ reflections are analyzed to provide insights on how they display critical competencies as they progress through the innovation process.

Scholarly Significance
This work sheds light on the development of a research-driven portfolio tool to capture critical competencies as students engage in problem solving. We advance a vision for the design and implementation of a seamless, authentic mode of assessment for K-12 learners in problem-based learning environments.

Authors

• Leslie Flynn, University of Iowa

• Rachel Crady

• Grace E Trane