Profiles of U.S. performance in advanced mathematics and physics from the 2015 TIMSS Advanced assessment

• In Event: Diagnostic Studies Using TIMSS and TIMSS Advanced Assessment Data

Proposal

The United States first reported on TIMSS Advanced 2015 in Highlights from TIMSS and TIMSS Advanced 2015 (NCES 2017), which focused on how U.S. students compare to students in the other participating countries in average scores and in the percentages reaching the TIMSS Advanced international benchmarks (Advanced, High, and Intermediate). In terms of average scores, the United States performed relatively well in advanced mathematics (higher than five countries and lower than two) and less well in physics (higher than three countries and lower than four). To better understand these results, the National Center for Education Statistics (NCES) produced a “closer look” report that further mined the TIMSS Advanced data to examine the performance of U.S. students in greater depth and explored how performance relates to students’ opportunity to learn the content covered in the TIMSS Advanced assessment (NCES 2020).

This presentation provides results from secondary analyses of TIMSS Advanced data that describe performance levels (e.g., average item scores and range of item difficulty) for each of the topics covered in the advanced mathematics and physics content domains to identify areas of relative strength and weakness. Topic-level performance data are compared to data from the TIMSS Advanced teacher questionnaires on the extent to which the TIMSS Advanced topics are covered in their mathematics and physics courses. Data from diagnostic scoring are used to identify common approaches, misconceptions, and errors on TIMSS Advanced mathematics and physics items. Example released items from the assessment are used to illustrate these findings. Results are reported for U.S. students overall, by gender, and by course type, including Advanced Placement (AP) calculus and physics courses and International Baccalaureate (IB) courses in mathematics and physics.

In both advanced mathematics and physics, U.S. performance on TIMSS Advanced varied considerably depending on the specific courses taken, with AP students generally outperforming non-AP students, particularly those taking the highest-level AP courses in calculus and physics. In general, across subjects, U.S. males outperformed females. The gender differences may be related to course taking patterns, as higher percentages of males than females took the highest-level AP courses. U.S. students demonstrated some common misconceptions and errors on TIMSS Advanced items, including those assessing topics that had a high level of coverage in U.S. courses and where students performed relatively well on average. In physics, many U.S. students had difficulty applying Newton’s laws of motion in problem-solving situations, demonstrating an understanding of electric and magnetic fields, and correctly solving and showing their work on quantitative problems. In advanced mathematics, many U.S. students had difficulty solving problems in real-life contexts and demonstrating understanding of some concepts and procedures needed to solve problems (e.g., derivatives, trigonometric functions, and simultaneous equations). The prevalence of specific types of misconceptions and errors often varied by course type. For example, misunderstanding concepts and problem-solving situations was less frequent among students taking the highest-level AP courses. In contrast, there were no differences by course type in the frequency of making computational errors or showing incomplete work.

Author

• Maria Stephens, American Institutes for Research