Commentary: AI and Formative Assessment—The Train Has Left the Station (Poster 10)

• In Event: AI and Formative Assessment: The Train Has Left the Station

Abstract

Researchers have been questioning AI --whether we can and should use AI for formative assessment (Li et al., 2023). AI is already being employed, for better or worse, to facilitate formative assessment in various educational contexts. We should focus on how we can help educators manage an AI revolution that has outpaced a research community caught unaware and recognize the remarkable progress that AI has contributed to formative assessment.
AI is a Tool That is Neither Good nor Bad
Technology is morally neutral; only its uses relate to moral or societal values, not the technology itself. Questioning AI is different from questioning uses of AI. The challenge is engaging in thoughtful and nuanced consideration of the potential positive outcomes while restricting negative consequences. Scores, regardless of whether derived from AI, merely represent one component in formative assessment. Skilled teachers leverage these data to probe students’ thinking. Against AI-generated scores could apply to all assessments sharing the shortcoming, akin to ‘throwing out the baby with the bathwater.’
AI Can Help Diversify Representational Modalities in Science Assessment Practices
Opening assessment spaces to encompass a greater diversity of communication modalities is needed in science education. Science educators are working hard to ensure that students with lower language proficiencies are included in meaningful science learning, and pictorial modeling might be one solution (Zhai, 2022). AI-based automatic scoring has great potential for facilitating the use of multimodalities in classrooms, providing more equitable educational experiences for all students.
AI Bias in Education Can Be Tested Empirically
Bias composes three essential components (Zhai & Krajcik, 2022): (a) deviation –the deviation between observations and ground truth (i.e., error); (b) systematic error-- instead of random error; and (c) tendency –favor or against some ideas or entity over others. “Pseudo AI bias” relates to misunderstanding of bias, mis-operation of AI, or over-expectation of AI outcomes. Claims of AI bias must specify the form of bias and the evidence that supports it. Vague and unsubstantiated claims of bias do little to help our field move forward.
AI Scoring Bias ≠ Injustice in Education
Equity or inequity is the consequence of formative assessment practices as a whole rather than AI-generated scores alone. Admittedly, AI cannot provide accurate scores for all students, but neither can teachers. Experienced teachers can skillfully practice formative assessment with multiple-choice and increase learning for students from historically excluded communities. Poor assessment practices can be associated with any type of formative assessment tool (e.g., multiple-choice). Equity could be promoted if robust and timely assessment information generated by AI empowers teachers to help struggling students in a timely fashion.
AI Cannot Solve All Problems or Replace Teachers’ Understanding of Students
Researchers (Li et al., 2023) suggest that AI may undermine culturally relevant pedagogy because AI cannot recognize science’s role in upholding racist institutions, or students’ histories, group dynamics, classroom environment, and community context. No existing assessments can do the job and fix all of the factors highlighted. AI-based assessment is still assessment, not a solution to all educational problems.

Authors

• Xiaoming Zhai, University of Georgia

• Ross H. Nehm, Stony Brook University - SUNY