CPC Assessment Research Study

The NBCRNA recently conducted both an extensive Beta Research Study on the CPC Assessment (CPCA) involving the participation of more than 1,500 CRNAs, as well as a national benchmarking study. After careful consideration of the results and data from those studies, the NBCRNA Board of Directors made the decision to remove the pass/fail assessment component of the CPC Program at their January 2019 meeting. The CPC Assessment will still be required every eight years, but will remain a performance standard exam only—no pass/fail.

Learn more about new changes to the CPC Assessment – Just announced

The NBCRNA has completed an innovative research study on its computer-based Continued Professional Certification (CPC) Assessment, evaluating testing performance in six varied testing environments. The study provided the data necessary to establish both the testing condition and score standard for the assessment requirement of the NBCRNA’s CPC Program for CRNAs. The Modified Angoff method, a widely used standard-setting approach in credentialing, was used to determine the performance standard. Although testing conditions varied, the percentage of study participants that met the performance standard was 95%, indicating preparedness and mastery of the knowledge necessary for practice in nurse anesthesia.

_...the percentage of study participants that met the determined performance standard was 95%_ (1)

Launched in September 2018, the research study included 1,500 randomly selected CRNAs who volunteered to take the 150-question, four-hour exam in one of the six assigned testing conditions, including with and without resources, and at a testing center versus live, remote proctoring. The study evaluated the testing environments to determine how each may help achieve the goals of the CPC Program in a way that is fair and reliable for all CRNAs taking the assessment requirement which begins in 2020.

To assure fairness to candidates, statistical equating procedures were used to place scores from different testing conditions on the same scale. Based on the findings from this study, the final testing condition that will be used for the CPC Program assessment component for CRNAs beginning in 2020 will be determined in early 2019 and will use the standard also determined from this study. Learn more about the standard setting process or read the full press release on the study.

Press Release on the Study Angoff Method for Standard Setting

Results were mailed to Study participants in mid-December 2018.

The NBCRNA thanks all CRNAs who participated in this innovative research study.

Beta Research Study FAQs

Results

Q I met the Performance Standard on the CPC Assessment. Now what?

First of all, congratulations! You met or exceeded the performance standard for the first CPC Assessment (CPCA). This will count towards your requirement for the first 8-year CPC Program period, so you will not need to take an assessment again for up to another 10 years! Even though you met or exceeded the standard, it is important to point out—for both you and especially your colleagues who may be unclear—this first CPC Assessment is not a pass/fail exam. This is a “performance standard” assessment designed to give CRNAs an evaluation of their performance in each of the four core domains of anesthesia, no matter your practice environment. You currently need to meet the Class A and Class B requirements. The Core Modules are currently optional in this first 4-year period, then required in subsequent 4-year periods beginning in 2020. Find all requirements here on the NBCRNA website.
Q I did not meet the performance standard for the CPC Assessment in the Beta Research Study? Now what?

As a Beta Study participant, if you did not meet the performance standard in any of the four domains, it will be identified on your score report. Although no action is required, you may want to focus future targeted education in any area in where the performance standard was not met. It is important to point out—for both you and your colleagues who may be unclear—the first CPC Assessment that will be taken between 2020-2025 (depending on your cycle) is not a pass/fail exam and you will NOT LOSE CERTIFICATION regardless of performance. However, additional continuing education will need to be completed in any area where the performance standard is not met. Then, as the CPC Program is currently constructed, the passing standard exam (not this performance standard exam) would not need to be taken until 2032/2033—up to another 15 years from now.
Q How was the performance standard set for the CPC Assessment?

The Continued Professional Certification Assessment (CPCA) Beta Research Study test was scored based on candidate performance on all questions. Testing condition variations were incorporated in the scoring process to ensure equivalence and fairness. A numerical estimate of the examinee’s ability was determined using item response theory (IRT), and the examinee’s ability estimate was then transformed to a standardized scale.

The CPC Assessment Beta Research Study used the Modified Angoff and Hofstee methods in determining the performance standard. In the scientific field of standard setting, there are many potential methods for determining the performance standard for an examination. The NBCRNA elected to use both the Angoff and Hofstee methods, the same methods used for the National Certification Examination (NCE), to establish the performance standard for the CPC Assessment Beta Research study.

The Modified Angoff method is a widely-applied, item-centered method for setting criterion-referenced standards on credentialing examinations. Read more about scores and standard setting here.
Q Why is the percentage of those that met the performance standard so high? Was it too easy or worth the effort for CRNAs?

The percentage of examinees that met the performance standard on the CPC Assessment for the Beta Research Study was 95%. By testing standards, this rate indicates that 95% of CRNAs were adequately prepared for the exam and confirms that they possess the mastery of the knowledge determined necessary for practice in nurse anesthesia. Regarding its value, in addition to helping establish a performance standard for all practicing CRNAs no matter the practice setting, Beta Research Study participants now: 1) have experience taking this type of online exam, 2) benefited educationally from preparation for the exam, and 3) for those that did not meet the performance standard, have the information on the domain(s) where the standard was not met so they can better focus their continuing education going forward.

“Although I didn’t really want to take an exam after all these years, I did find it helpful to my practice to (have to) brush up on a few things.”
Q Why did it take so long to get our scores?

The exam scoring and standard setting process is complex. One reason for the waiting period is to enable the NBCRNA to analyze and evaluate each exam to ensure that the results are reliable and fair to those in all testing conditions.
Q How do I read my score report if I was not successful?

The CPC Assessment is based on four core domains of knowledge of nurse anesthesia practice*. The content outline of the CPCA was based on those four domains. One of the benefits of taking the CPCA is that participants are now able to identify the area(s) of their knowledge that can be further strengthened by additional continuing education. Those areas where the performance standard was not met are indicated on the individual score report with a dot. If no dots are present in a given domain on the chart, then the performance standard was met in that/those domain(s).

For participants who did not meet the overall performance standard, there is no prescriptive continuing education required (in any domain) as a result of your performance on the CPCA. Rather, you now have a better roadmap to guide your educational efforts going forward. You will need to take the performance standard assessment along with your peers in your second 4-year CPC Program cycle—by 2024/2025, depending on your past recertification date. See your path on the CPC Calculator and more here on the NBCRNA website.
*Airway Management; Applied Clinical Pharmacology; Physiology and Pathophysiology; and Anesthesia Equipment and Technology.

Confidentiality

Q What is the level of confidentiality?

Data will be kept confidential and de-identified; specific information about name, contact information, ID numbers will be stripped from your record.

Each participant will be given a unique subject identifier in place of their name once the data has been exported from the vendor to the researcher.

No identifying information will be utilized in presentations and publications.
Q Who will my test scores be shared with?

Your name with associated test score will not be shared with anyone, only the data will be used for the purposes of this research study. The data generated may be used in future research and may be accessed by the Institutional Review Board that approved this study for quality assurance and auditing purposes.

CPC Assessment Research Study