Comparing Rater Experiences With Two Different Assessments

• In Event: Assessment Innovation in Professions Education

Abstract

This study compares rater experiences scoring two bar examination formats: the current and NextGen exams. Thirty-one raters were interviewed about their grading processes, rubrics, and supplemental materials. Analysis revealed that the NextGen rubric, being more detailed, enhanced consistency but required more time, while the February exam’s flexible rubric allowed for discretion but risked inconsistency. NextGen's comprehensive supplemental materials clarified expectations but were potentially overwhelming, whereas February's materials were less helpful. Examinee responses to the NextGen exam were more structured, aligning better with grading criteria. These findings suggest that while NextGen’s structured approach supports consistency and fairness, it increases grader workload, highlighting the need for balanced grading practices in future exams.

Author

• Hilary A. Symes