Standard Setting

Setting the Pass/Fail Standard

Consistent with other national high-stakes examinations for physician licensure worldwide, the NBOME follows industry standard best practices in determining minimum pass/fail standards for the COMLEX-USA examinations and in periodic resetting of the standards. One of the most important components of a high-quality, defensible examination program is a fair and reasonable approach to identifying a “cut score” – the score at or above which a candidate is deemed to have passed an examination.

For COMLEX-USA examinations, a passing score means the candidate has demonstrated at least a minimal level of competency by scoring at or above the level judged to be required for the safe and effective practice of osteopathic medicine at the supervised level for COMLEX-USA Level 1 and Level 2-CE, or at the unsupervised, independent practice of osteopathic medicine practice level for COMLEX-USA Level 3.

Every three to five years, the NBOME reviews the standards for minimum competence required to pass COMLEX-USA examinations. The standard setting process may result in a change in the cut score needed to pass an examination.

Approach to Standard Setting

The NBOME applies criterion-referenced methods to establish cut scores. Sometimes referred to as “absolute standards,” these are not norm-based or relative to the performance of other candidates.

A process of triangulation is used to establish cut scores. The process is widely used for high-stakes examinations for physician licensure around the world and follows industry standard best practices. Triangulation includes standard-setting surveys, standard-setting panel meetings, and a comprehensive final review.

Surveys

The NBOME surveys osteopathic medical students and residents, osteopathic medical school deans, and residency program directors from programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) learn about their perceptions of the abilities of the candidate pool, and whether current standards are appropriate.

The Angoff method begins with a discussion of the candidate who is minimally qualified to pass the examination. Panelists then review the test questions and estimate the probability that this minimally qualified candidate will answer the item correctly. A passing score is derived for each panelist by summing these item probabilities. The final passing score is calculated by taking the average passing score across panelists.

The Hofstee method does not require individual question judgments. Instead, panelists are asked to give their impressions of what the minimum and maximum failure rates should be for the examination as well as what the minimum and maximum percent correct scores should be. These rates and percent correct scores are averaged across panelists and projected onto the actual score distribution to derive a passing score. The Hofstee method is often used to evaluate or adjust the passing score derived using Angoff methodology. When the two methods produce similar passing scores, they validate each other.

This approach is consistent with the approach used around the world by most testing organizations related to physician licensure.