Teaching and Assessing Scientific Thinking: Online Inquiry Units With Automated Scoring

• In Event: Assessment for Educational Accountability in History and Science: Beyond Recall

Abstract

Most science teachers today postpone inquiry activities until after the last state test has been completed. Online inquiry activities and automatically-scored assessments could change this situation by efficiently measuring scientific thinking.

Current situation. Although science standards call for scientific thinking, most standardized tests emphasize recall of details. Consider this item on a California standardized eighth grade science assessment:

Which of the following best describes an atom?
A. protons and electrons grouped together in a random pattern
B. protons and electrons grouped together in an alternating pattern
C. a core of protons and neutrons surrounded by electrons
D. a core of electrons and neutrons surrounded by protons

These detail-oriented questions motivate teachers to stick to the textbook where students can access this information. As a result students receive a very distorted picture of what scientists do and little opportunity to develop the scientific thinking skills they need.
In reality, scientists spend little time memorizing details. They know the details relevant to their own work because they use the information regularly to reason about dilemmas. When scientists need to solve a problem in another field they tend to look up the details and use powerful reasoning skills to make sense of the information as do citizens who need to resolve conflicting information about contemporary problems such as energy conservation or healthy eating.

To be sure, both scientists and citizens need to know what information they need before they can look it up. Thus, rather than details, scientists and citizens need the ability to search for information relevant to their needs, conduct their own investigations, and make sense of the information they find.

Online instruction and automated scoring. To teach and assess these skills we have developed online units on contemporary problems such as global climate change, evolution, and energy conservation that improve scientific thinking about complex topics such as photosynthesis (Linn & Eylon, 2011). A challenge to implementing these units is the time required to score constructed responses.

Recently we have tested automated scoring of constructed responses using the ETS c-rater engine (Sukkarieh & Blackmore, 2009) and the knowledge integration rubric (Linn & Eylon, 2011). We added automated scoring to a tested unit on the importance of plants in maintaining energy balance on the planet. We compared human and automated scores for an item that asked students to, “Explain how the sun helps animals survive.”

Authors

• Marcia Linn, University of California - Berkeley

• Ou Lydia Liu, ETS

• Kihyun (Kelly) Ryoo, University of North Carolina - Chapel Hill

• Jacqueline J. Madhok, University of California - Berkeley