To help ensure NBOME assessment content is written in an inclusive manner and free from bias, the NBOME has unveiled an additional tool for item writers and editors: the Mindful Language Item Writing Guide, which includes tips for incorporating diversity, equity, and inclusion (DEI) into NBOME test questions. The guide is available to NBOME National Faculty members for NBOME examination development purposes.
“This was a top priority for us and a goal we wanted to accomplish at the end of the first year of our Strategic Plan, Assessment Matters,” said Marie Fleury, DO, MBA, associate vice president for assessment services, who served as executive sponsor on the project. “The new guide is important for assuring incorporation of DEI concepts into our examination content and processes.”
Among guidance on topics such as person-first language; addressing race, ethnicity, gender, and religion; and identifying stereotypes, the guide also provides NBOME item writers and reviewers with an evidence-based checklist to ensure item content is free of bias.
“We have always provided item writers and reviewers with supplemental training resources for writing questions, but with the expansion of our Core Values to strengthen our focus on DEI, we took the opportunity to expand upon that guidance,” said Lisa Mysker, NBOME managing editor.
A core team of test developers, editors, and in-house DOs reviewed external references and NBOME test content to identify the topics to address. The group then met weekly to review specific topics and design a series of recommendations, which became the new guide.
“In recent years we’ve seen an increased focus from our subject matter experts and candidates alike on DEI,” said Evelyn Ronkowski, MFA, MAT, associate director for test development. “The feedback we’ve received from candidates on each question has been particularly helpful in creating this guide.”
In addition to using the guide to create new test items, NBOME editors and test developers have been using the guide to review existing questions and flag any test content that needs to be updated.
“The guide is an evolving document, and it’s going to be continuously updated based on the evolution of language and of health topics,” said Mysker. “As our item writers and reviewers create content, we are asking them to use this guide to keep these topics in mind.”
In addition to the guide, the NBOME also shares the following resources during Item Writing Workshops:
- AMA Health Equity Language Guide
- CDC Health Equity Principles for Inclusive Communication
- Conscious Style Guide
- APA Bias-Free Language Guide
The new guide has been used at recent item-writing workshops, and overall, the Test Development team says the response has been positive.
“The guide helps to recognize bias, avoid using stereotypes, and encourages the use of language that is mindful to the particular population presented,” said National Faculty Chair for DEI Bernadette Riley, DO.
“The guide recognizes healthcare disparities seen and hopes to address any barriers that could occur in testing. It has helped me tremendously on exam content review of questions and has made me aware of my own implicit bias and to be mindful of how I address and acknowledge it in my role as faculty, Chair of DEI, and item writer.
Riley adds that the guide is considered a “living document,” and National Faculty work closely with NBOME staff and leadership to monitor and change it on a frequent basis when new education, environment, ideas, and bias come to light.
Those interested in contributing to NBOME assessments, such as COMLEX-USA and COMAT, may email nationalfaculty@nbome.org.
Additional staff who contributed to the development of the guide include Emmy Bean, MA, editor 1; Jenny Grisolano, MA, associate director of test development; Candice Pernell, MA, associate director for test development; Emily Platt, assistant editor; and Owen Schleif, editor 1.