Search
Program Calendar
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Unit
Browse By Session Type
Search Tips
What to do in Chicago
Personal Schedule
Sign In
Automatic coding of short text responses opens new doors in assessment. We propose and implement a collection of baseline methods for automatic coding and software components that are available under open licenses. The methods' accuracy is demonstrated by using data collected by the Programme for International Student Assessment (PISA) 2012 in Germany (for 10 items, with n = 41,990 responses in total). This paper reports preliminary results of four items for which high percentages of agreement (84--90 %) between automatic and human coding were reached. Nevertheless, performance varied by item, as agreement beyond chance differed strongly (.23 <= Kappa <= .70). Overall, the methods appear accurate and feasible, especially for items that are solved by expressing specific semantic concepts.