The establishment of credentialing performance standards provides the decision mechanism for credentialing outcomes. Documentation of credentialing exam standard setting procedures is a key component for establishing validity evidence for a credentialing examination.
Periodic standard setting studies for examinations are required by Standard 17 of the National Commission for Certifying Agencies (NCCA) and Standard 5 of the American Educational Research Association (AERA), National Council on Measurement in Education (NCME), and American Psychological Association (APA) Standards for Educational and Psychological Testing. The frequency with which these analyses are conducted should correspond to how quickly a given profession is perceived to change. It is recommended that testing programs undertake a standard setting study, following a comprehensive professional practice analysis, every four to five years.
In most cases a panel of participants, consisting primarily of subject matter experts, is assembled to provide to the certifying body guidance on the appropriate level of knowledge that must be demonstrated to be considered competent to practice safely in an entry-level environment. The standard setting panel members are selected to reflect the diversity of the profession, including gender and ethnicity, practice setting, geographic region, professional experience, educator/practitioner, etc. The standard setting panel, using one of many methods for setting performance standards, arrives at a single, consensus point on the ability continuum and makes a recommendation to the certifying body for the pass/fail point on the test. The panel is typically convened for a one- to-two day meeting to review the standard setting methodology, to familiarize themselves with the purpose of the test, to conduct an in-depth discussion about the knowledge, skills, and abilities required of an entry level practitioner, and to set about making the judgments that will be used in recommendation of a passing score for the exam. The panel’s passing score recommendation is subject to review, adjustment, and ratification by the governing certification body.
There are many potential methods for determining the passing standard for an examination. The NBCRNA used both the Angoff and Hofstee methods to evaluate the passing standard for the NCE and NSPM. The Modified Angoff Method is a widely-applied, item-centered method for setting criterion passing standards on credentialing examinations. A criterion standard is a fixed standard. Subject matter experts (SMEs) are generally briefed on the Angoff method and allowed to take the test. SMEs are then asked to provide estimates for each question of the proportion of borderline or “minimally acceptable” participants that they would expect to get the question correct. The estimates are generally in decimal form (e.g., 0.6 for item 1: 60% of borderline passing participants would get this question correct). Several rounds, including a practice round, are conducted with SMEs allowed to modify their estimates given different types of information (e.g., actual participant performance information on each question, discussion of outlier ratings). The final recommendation for the cut score is then made aggregating these estimates over all items on the form and all judges.
The Hofstee method collects both criterion and normative information in a quick and efficient manner, asking panel members to estimate ranges for acceptable passing scores and passing rates. Estimates specific to the Hofstee method were collected from panel members after the meeting. The NBCRNA Board reviewed the recommendations made by the standard setting panel and approved the new passing standard in August of 2017, to be effective in January of 2018. Based on analysis presented the Board determined that the passing standard would remain the same as the previous standard.