Standard Setting: Modified Angoff and Hofstee Methods
There are many potential methods for determining the performance standard for an examination. For the CPCA, the NBCRNA elected to use both the Modified Angoff and Hofstee methods, the same methods used for the National Certification Examination (NCE; “the boards”), to establish the performance standard for the CPC Beta Research Study.
The Modified Angoff Method is a widely-applied, item-centered method for setting criterion-referenced standards on credentialing examinations. Subject matter experts (SMEs) provide estimates for each question of the proportion of candidates with minimally acceptable performance they would expect to answer the question correctly. The estimates are generally in p-value type form (e.g., 0.6 for item 1: 60% of just meeting standard participants would get this question correct). Several rounds, including a practice round, are conducted with SMEs allowed to modify their estimates given different types of information (e.g., actual participant performance information on each question, discussion of outlier ratings). The final recommendation for the performance standard is then made aggregating these estimates over all items on the form and all judges.
The Hofstee Method collects both criterion and normative information in a quick and efficient manner, asking panel members to estimate ranges for acceptable performance standards and passing rates. Estimates specific to the Hofstee method were collected from panel members after completion of the Modified Angoff method.
The recommendation of the standard setting Panel was presented to the NBCRNA board, which made the final decision on the performance standard. As applied to the Beta Study participants, this performance standard results in 95% of participants meeting the standard.