Techniques To Develop Prior Knowledge Assessments To Support Students And Instructors In College Courses

• In Event: Scenario-Based Assessment (SBAs) in the Age of Gen-AI: Developing Tools for Postsecondary Instructors

Abstract

It is well recognized that prior knowledge is important for successful reading (McCarthy & McNamara, 2021), and in particular for learning and apply the knowledge learned towards solving problems (O’Reilly et al., 2019). This presents a conundrum for college students and instructors. Students may vary widely in their prior topic and domain knowledge when engaging in course assigned texts, which can be a barrier to comprehension (O’Reilly et al., 2019). Instructors need to choose texts that are accessible to most students who vary in specific course topic and subject/domain knowledge (e.g., biology or psychology); and support students who have significant prior knowledge gaps. However, instructors are often blind to the prior knowledge of students. Prior knowledge assessments administered at the onset of a course could thus valuably inform instructor’s practice. However, these assessments need to fit within the temporal and technological constraints of the course.

We are particularly interested in this problem as it applies to introductory to interdisciplinary thinking courses. There has been a sustained rise in interdisciplinary programs in higher education since the 1970s (Repko, 2019). These courses present challenges because the texts students must read stem from a variety of disciplines but must be interpreted in terms of solving a practical problem. They have to learn ways of thinking and problem solving that are associated potentially disparate disciplines with respect to epistemic perspectives. Thus, the goal of these courses extends beyond students learning topic specific knowledge and, as such, assessing prior knowledge in these courses have unique challenges.

We will discuss this problem in the context of developing prior knowledge tasks for scenario-based assessments for college courses. We will discuss approaches that have been developed and implemented in an Introduction to Interdisciplinary Thinking course (Ari, 2021). Ari has been exploring the use of cloze tasks which involve giving students sentence stems which require general subject area or specific topical course content to complete. Thus, the cloze technique can be applied both to the broad and specific semantic spaces associated with the course, as well as be used to monitor students’ growth of knowledge across the duration of the course.

In the session, we will demonstrate techniques used to efficiently develop and validate such prior knowledge assessments across varied disciplines, as well as practical logistics for feasibly administering and scoring within the constraints of a course. We will then further discuss approaches that we are now exploring in the SBA project to enhance and generalize such techniques in introductory interdisciplinary courses.

References
Ari (2021). Undergraduates as interdisciplinary researchers: Gains in competencies and conceptual knowledge. College Teaching, 69(4), 218-230.

McCarthy, K. & McNamara, D. (2021). The multidimensional knowledge in text comprehension framework. Educational Psychologist, 56(3), 196-214.

O'Reilly, T., Wang, Z., & Sabatini, J. (2019). How much knowledge is too little? When a lack of knowledge becomes a barrier to comprehension. Psychological Science, 30(9), 1344-1351.

Repko, A. Szostak, R., & Buchberger, M. (2019). Introduction to interdisciplinary studies (3rd ed.). SAGE Publications.

Authors

• Omer Ari, Georgia State University

• Joseph Magliano, Georgia State University

• Jonathan Cohen, Georgia State University