Longitudinal Assessment Issues and Considerations

The NBCRNA is continuously evaluating the effectiveness of our credentialing programs. We have listened to your feedback about the Continued Professional Certification Program Assessment (CPCA) and are exploring alternative assessment approaches that promote patient safety and lifelong learning to meet the needs of the CRNA profession. As a result, we are currently conducting a research study to explore the potential of Longitudinal Assessments (LA) as an approach to continuing professional certification.


Note, if you are not selected for the research study you will still have to take the CPCA in its current form during your second-year cycle.

What is Longitudinal Assessment (LA)?

LA is designed to promote learning, retention and transfer of knowledge in the clinical setting. Rather than one comprehensive assessment (as is currently the case with the CPCA), LA delivers shorter, ongoing assessments, with immediate feedback and rationales to increase understanding and reinforce knowledge.

Defining Attribiutes of Longitudinal Assessment

The NBCRNA LA Research Study

The primary purpose of this research study is to determine comparability in performance among participants who (a) take the CPCA in its current form versus (b) take the assessment in an LA configuration (e.g., 30-40 questions per quarter with immediate feedback and rationales). Results from this study will help inform NBCRNA Board of Directors’ decision on whether to change to LA instead of the CPCA requirement.

1,000 CRNAs for this research study were recruited from a call for volunteers that was sent out in February 2022. The research study kicked off on April 4, 2022. Participants selected were representative of the CRNA population, and randomly assigned to one of two conditions: CPCA in its current form or in its LA configuration.

The research study is currently ongoing and will be ending March 31, 2023. It will be followed by a thorough review and analysis of the resulting data to determine the viability, feasibility and acceptability of LA as an alternative to the CPCA.

Eligibility

Research participants were required to be an active CRNA who did not take the CPCA to be eligible and give their consent to be randomly assigned to either the CPCA or the LA group. Regardless of the assigned group, research participants received a price reduction on their assessment and will satisfy the assessment requirement of the CPC Program as long as the performance standard is met. Participants were also required to complete surveys throughout the process and may be asked to participate in a focus group.

Recruitment is now closed for this study. Those selected to participate were notified in March 2022. Note, if you are not selected for the research study you will still have to take the CPCA in its current form during your second-year cycle.

 

NBCRNA Evaluation of LA

The NBCRNA appointed an LA Subcommittee to its Evaluation and Research Advisory Committee, to explore the feasibility and appropriateness of LA for its credentialing programs.

Click the links below for more information: 

Longitudinal Assessment: A Strategy to Improve Continuing Professional Certification 

Concept Analysis of Longitudinal Assessment for Professional Continued Certification

 

LA FAQs

  • Q What is Longitudinal Assessment?

    Longitudinal Assessment (LA) is an alternative form of assessment based on adult learning theory and principles to encourage learning, promote retention, and transfer of knowledge to the clinical setting.

    Rather than one comprehensive assessment (as is currently the case with the CPCA), LAs work by delivering shorter, periodic assessments, with immediate feedback and rationales to help increase understanding and reinforce knowledge toward achieving performance standards. After each question, certificants are asked to rate how confident they are in their answer and how relevant the content assessed is to their clinical practice. This information can be used to determine repeat items to help further promote learning and retention.

    LAs are currently used to, or being investigated, as an alternative to recertification exams by many medical and allied health professional boards, including physician assistants and respiratory therapists.

  • Q What is the purpose of this research study?

    The purpose of this research study is to determine if scores on the CPCA are similar or different among CRNA participants who (a) take the CPCA in its current form (150-items taken at a testing center or online via live remote proctoring) versus (b) taking the CPCA in an LA format (i.e., 30-40 items per quarter with immediate feedback and rationales, along with confidence and relevance ratings for each question). We will also compare differences in CRNAs perceptions about the feasibility and acceptability of the two assessment formats.

  • Q When does the study start and end?

    This is a yearlong study that will start in April 2022 and end in March 2023.

  • Q How can I participate?

    A call for volunteers will be sent out with a link to a survey in February 2022 to CRNAs eligible to take the CPCA. You must complete this survey and give your consent to be randomly assigned to either the CPCA or CPCA-LA group.

    1,000 CRNAs will be randomly selected based on gender, age, years of practice and/or region from those who volunteer to ensure the demographic characteristics of participants matches the CRNA community. Then we will randomly assign 500 participants to take the CPCA and 500 participants to take the CPCA-LA. If you are selected to participate you will not be able to select your group assignment.

  • Q What are the benefits of participating?

    Regardless of being assigned to the CPCA or CPCA-LA group, participants will receive a discount on the cost to take the CPCA and satisfy the Assessment requirement of the CPC Program if you meet the performance standard. Results from this study will help to inform the NBCRNA Board of Directors decision on whether to redesign the CPCA into an LA format.

    Those selected to participate will receive a discount on the cost of the CPCA, which will be $150 instead of the original cost of $295. Participants in both groups will be required to register and pay the assessment fee.

  • Q If selected and assigned to a group, do I have to take both the CPCA and CPCA-LA?

    Regardless of your assigned group, you will not have to take both assessments; therefore, you do not need to take the CPCA if you were assigned to take the CPCA-LA. Conversely, you do not need to take the CPCA-LA if you were assigned to take the CPCA. However, if you are not selected for the research study, you will still have to take the CPCA in its current form during your second-year cycle to meet your CPC Program requirements.

  • Q What if I took the CPCA already or plan to take the CPCA during the February testing window?

    If you have already taken the CPCA in the past, or were a previous research participant in the NBCRNA Beta Research Study in 2018 on the CPCA or the Review and Record Pilot in 2020 on the CPCA, you are not eligible to participate in the study.  

    If you plan to take the CPCA during the February testing window you will not be eligible to participate in this study. 

  • Q Who will be selected and what is involved in the study?

    All eligible CRNAs will be emailed personal invitations to complete a baseline survey to determine eligibility and obtain informed consent. 1,000 CRNAs will be randomly selected from those who volunteer. Participants selected in the study will be randomly assigned to the CPCA or CPCA-LA group and be notified with further instructions. Study participants will need to take the CPCA or CPCA-LA before the study ends. Participants will also be required to complete surveys to share their experience and provide feedback on the assessment formats. You may also be asked to participate in a focus group.

  • Q What do I have to do if selected?

    Each quarter CRNAs assigned to the CPCA-LA group will log into a secure website to answer the 30-40 items for four consecutive quarters. The CPCA-LA is not a proctored assessment, CRNAs may answer the questions all at once or at their own pace and can be taken anywhere that has a stable internet connection. For each item, participants will need to rate their confidence of the answer choice selected and relevance of the item, and will then receive immediate feedback about the incorrect answer with rationales. After completing the quarterly items CRNA participants will complete a short survey to gauge their perceptions about the LA format.

    CRNAs assigned to the CPCA group will sign up to take the CPCA through the NBCRNA portal and may take it either in a test center or remotely during one of the upcoming testing windows during the study duration.  At the end of the assessment, CRNA participants will complete a survey to gauge their perceptions about the CPCA. Once data collection is complete, we’ll analyze the data to determine the viability, feasibility, and acceptability of the LA format.

  • Q Why do you have to randomly assign participants?

    Random assignment helps ensure results of the study are valid and free of bias.

  • Q What are the potential risks?

    Potential risks of participating in this study is possible, including test anxiety while taking the assessments; however, this is minimized since the CPCA, and the CPCA-LA, is not a pass or fail assessment and does not affect your credential as long as you successfully complete any additional continuing education requirements if the performance standard is not met.

  • Q What happens if I do not meet the performance standard on the CPCA or CPCA-LA?

    You will be required to complete at least one Class A credit of your choice in each of the core domains that you are below the performance standard. These credits must be completed before the end of your CPCA cycle.

  • Q Do I have to participate?

    Participating in this research study is voluntary. Your decision whether to participate in this study will not affect your credential or relationship with the NBCRNA.  However, all CRNAs will still be required to complete the CPCA in its current form before the end of your CPC cycle, if not selected to participate in the study.

  • Q What about privacy and confidentiality?

    Your privacy and confidentially are very important to us. Therefore, your personal data will remain confidential and be de-identified in the dataset used for analysis. If you are selected, we will not share your participation in the study with your colleagues or employers, unless you choose to disclose this information. The only people who will have secure access to the data will be the LA platform vendor and the NBCRNA research team who is responsible for collecting and aggregating the data. All data will be transmitted securely and encrypted.

    When the results of the study are discussed or published, only aggregated data will be presented, and no personal information will be shown that would reveal your identity. Only emerging themes from the open-ended questions or focus groups will be reported in our overall findings and any direct quotes taken will be referenced anonymously. We will make the results available on our website at the conclusion of the study.

  • Q Was this research approved by an Institutional Review Board (IRB)?

    This pilot study was approved by the WCG Institutional Review Board as exempt research.

  • Q Has NBCRNA conducted any research or evaluation of LA?

    Yes, the NCBRNA Board of Directors charged the Evaluation and Research Advisory Committee: Longitudinal Assessment Subcommittee to explore the feasibility and appropriateness of LAs for its credentialing programs.

  • Q Where can I read more about LA?

    Two papers written by NBCRNA volunteers have been published:

    1. Longitudinal Assessment: A Strategy to Improve Continuing Professional Certification
    2. Concept Analysis of Longitudinal Assessment for Professional Continued Certification

  • Q Who should I contact if I have more questions?

    Please email research@nbcrna.com if you have any additional questions about this study.