Assessment Challenges in Creating the Uniform Bar Examination: The Three P’s—Politics, Practicality, and Psychometrics

• In Event: Clarifying Assessment and Measurement Issues Across Disciplines and Professions

Abstract

Objective
This presentation discusses the assessment challenges that were identified and resolved in the process of creating the Uniform Bar Examination (UBE) which was launched in February 2011 for use in American legal licensure.

Perspectives
The legal community formally recognized in 1921 that legal licensure candidates “ . . . should be subject to an examination by public authority to determine his fitness.” Historically, each jurisdiction (i.e., state) has had its own Board of Bar Examiners with independent responsibility to define the appropriate assessment procedures and performance standards for formal determination of applicants’ competency to practice law in that jurisdiction. Over time an independent organization, The National Conference of Bar Examiners (NCBE) created in 1931 to support general legal education and licensure, developed several types of instruments to measure distinct aspects of competencies needed by new lawyers and offered these materials to jurisdictions for their discretionary use. Recognizing the significant commonality in assessment methods and the practical realities of current multi-jurisdictional legal practices, various constituencies (e.g., bar examination candidates and individual licensure authorities) expressed interest in a common assessment designed to produce a single score useful for admission consideration across jurisdictions.

Methods
Guided by psychometric considerations identified during a decade-long research program (spanning more than 50 studies conducted across 45 individual jurisdictions) and with the encouragement, input, and support of key outside constituencies for addressing political and practical issues related to content and process, NCBE developed and implemented a testing model that had the appropriate construct representation while mitigating validity threats that were shown to be potentially introduced throughout the examination process beginning with test development and ending with the release of scores.

Outcomes
The UBE is a uniformly administered, two-day examination comprised of a one-day multiple-choice section Multistate Bar Examination (MBE) one-day written section comprised of six Multistate Essay Examination (MEE) questions and two Multistate Performance Test (MPT) tasks; all developed by NCBE and its committees of content experts. The MBE is scored by NCBE while the MEE and MPT are graded by user jurisdictions following standard guidelines. NCBE calculates the total UBE score for each jurisdiction using a standard algorithm in which the MEE and MPT raw scores are first combined and scaled (within jurisdiction) to the MBE and then the equated MBE score and the scaled written score (MEE and MPT) are combined with equal weight. Each jurisdiction establishes its own performance standard; the UBE score is portable to other jurisdictions that accept it, but the pass/fail decision is determined by each jurisdiction individually. In some jurisdictions, performance on a jurisdiction-specific examination component may be combined with the total UBE score to determine final bar admission status.


Significance
The presentation provides a direct illustration of the 2012 AERA theme; the successful “use of a research” to inform the development of a national-level assessment designed “to serve the public good” by ensuring decision accuracy within the context of professional licensure.

Author

• Douglas R. Ripkey, National Conference of Bar Examiners