Automated Extraction of Computational Thinking Features in Visual Programming Assessment

• In Event: AERA Roundtable Session Wednesday 3:45 pm - Gold 3
  In Roundtable Session: Computational Assessment in Education: Validity Evidence from Automation to Adaptivity

Abstract

Computational thinking (CT) is an essential skill in the digital age. Based on the performance data of 707 second-year junior high school students who simultaneously participated in a visual programming activity and a standardized test of CT, we constructed a model that uses code features to predict the level of CT. Then, we evaluated the performance of multiple pre-trained code language models (LMs) in automatically extracting these code features. We found that the codes’ correctness, robustness, and technicality all positively impact the CT level. Besides, code LMs have higher F1 performance than baseline models in extracting most code features, which shows the possibility of constructing automated CT assessment systems based on code LMs.

Author

• Yishen Song, Beijing Normal University