Longitudinal Assessment Modality and Considerations

The NBCRNA is continuously evaluating the effectiveness of our credentialing programs. We have listened to CRNA feedback about the Continued Professional Certification Program Assessment (CPCA) and are continuously exploring alternative assessment approaches that promote patient safety and lifelong learning to meet the needs of the CRNA profession.

The NBCRNA recently completed a research study in March 2023 to explore the potential of Longitudinal Assessments (LA) as an approach to continuing professional certification.

What is Longitudinal Assessment (LA)?

Longitudinal Assessment (LA) is defined as an assessment modality that combines both formative and summative features. The primary purpose of LA in continued certification is to provide a systematic measurement of knowledge, and to this end, LA for nurse anesthesia has been developed based on sound psychometric principles and scientific rigor. A secondary purpose of LA is the solidification of knowledge, which is accomplished through multiple, short assessments delivered over an extended period of time accompanied by ongoing feedback, rationale, and the identification of content areas for improvement. 

The NBCRNA LA Research Study

The primary purpose of the LA Research Study was to determine comparability in performance among participants who (a) take the CPCA in its current form versus (b) take the assessment in an LA configuration (e.g., 30-40 questions per quarter with immediate feedback and rationales).

1,000 CRNAs for this research study were recruited from a call for volunteers that was sent out in February 2022. The research study kicked off on April 4, 2022. Participants selected were representative of the CRNA population, and randomly assigned to one of two conditions: CPCA in its current form or in its LA configuration. The research study ended March 31, 2023.

The undergoing data analysis helped determine the viability, feasibility and acceptability of LA as an alternative to the CPCA.

Research Publication

NBCRNA recently published a study in Practical Assessment, Research, and Evaluation (PARE), titled “ Transforming Assessments of Clinician Knowledge: A Randomized Controlled Trial Comparing Traditional Standardized and Longitudinal Assessment Modalities,” which showcases the benefits of longitudinal assessment (LA) for the continued certification of CRNAs. 

“The findings from this research are critical to NBCRNA’s mission of promoting patient safety through lifelong learning for CRNAs. These findings also helped our Board of Directors to make the strategic decision to move our certification assessment component to an LA format. At NBCRNA, we are committed to ensuring that CRNAs are equipped with the necessary tools to evaluate their anesthesia knowledge – and therefore provide the highest quality care for their patients.” - NBCRNA CEO, John C. Preston, DNSc, CRNA, APRN, FAANA, FNAP, FAAN

Read the full study here.

Read the press release here. 

Findings and Outcomes

Based on the findings of the LA Research Study, the NBCRNA Board of Directors has made a strategic decision to move the assessment component of the CPC Program to an LA format for future CPC Program cycles.

Nothing about the current program is changing – CRNAs will need to finish the requirements of the current CPC Program cycle they are in (Class A, Class B, Core Modules, and the CPCA in its current form if applicable ) before moving into the enhanced CPC Program.

Learn more about LA and the upcoming enhanced CPC Program here. Specific details of the program will be messaged to CRNAs closer to the start date of their enhanced CPC Program cycle.

NBCRNA Evaluation of LA

The NBCRNA appointed an LA Subcommittee to its Evaluation and Research Advisory Committee, to explore the feasibility and appropriateness of LA for its credentialing programs.

Click the links below for more information: 

Longitudinal Assessment to Evaluate Continued Certification and Lifelong Learning in Healthcare Professionals: A Scoping Review

Longitudinal Assessment: A Strategy to Improve Continuing Professional Certification 

Concept Analysis of Longitudinal Assessment for Professional Continued Certification

 

Longitudinal Assessment FAQs

  • Q What is Longitudinal Assessment?

    Longitudinal Assessment (LA) is defined as an assessment modality that combines both formative and summative features. The primary purpose of LA in continued certification is to provide a systematic measurement of knowledge, and to this end, LA for nurse anesthesia has been developed based on sound psychometric principles and scientific rigor. A secondary purpose of LA is the solidification of knowledge, which is accomplished through multiple, short assessments delivered over an extended period of time accompanied by ongoing feedback, rationale, and the identification of content areas for improvement. 

    Defining Attributes of Longitudinal Assessment 

    • Utilize scientific principles to enhance the testing effect of adult learning theory (frequent repetitive testing) 

    • Spaced learning (exposure to materials interspersed with other activities) 

    • Interleaving subject matter (simultaneously presenting several different learning topics) 

    • Provide instant/immediate feedback 

    • Repetitive learning experience 

    • Convenient learning platform 

    • Self-directed 

    Source: Griffis et al, 2021 

    Benefits of Longitudinal Assessment 

    • Supports lifelong learning continuously over the course of a career by providing CNRAs with opportunities to assess their knowledge on a regular basis. 

    • Provides immediate and detailed feedback with rationales. This helps CRNAs identify specific errors and how to correct them. It can also help CRNAs understand the rationales behind the questions, which makes for more effective learning. 

    • Allows convenience of being taken from anywhere and at any time which can be convenient for CRNAs who have busy schedules. 

    • Reduces assessment anxiety since longitudinal assessments are typically shorter and spaced out over time, which gives learners more time to prepare if they choose to do so.  

  • Q What was the purpose of this research study?

    The purpose of the LA Research Study was to determine if scores on the CPCA are similar or different among CRNA participants who (a) take the CPCA in its current form (150-items taken at a testing center or online via live remote proctoring) versus (b) taking the CPCA in an LA format (i.e., 30-40 items per quarter with immediate feedback and rationales, along with confidence and relevance ratings for each question). NBCRNA also compared differences in CRNAs perceptions about the feasibility and acceptability of the two assessment formats.

  • Q When did the study start and end?

    The yearlong study started in April 2022 and ended in March 2023.

  • Q Who was selected and what was involved in the study?

    All eligible CRNAs were emailed personal invitations to complete a baseline survey to determine eligibility and obtain informed consent. 1,000 CRNAs were randomly selected from those who volunteer. Participants selected in the study were randomly assigned to the CPCA or CPCA-LA group and notified with further instructions. Study participants needed to take the CPCA or CPCA-LA before the study ended. Participants were also required to complete surveys to share their experience and provide feedback on the assessment formats. Some were also asked to participate in a focus group.

  • Q What did participants have to do when selected?

    Each quarter CRNAs assigned to the CPCA-LA group logged into a secure website to answer the 30-40 items for four consecutive quarters. The CPCA-LA is not a proctored assessment, CRNAs may answer the questions all at once or at their own pace and can be taken anywhere that has a stable internet connection. For each item, participants needed to rate their confidence of the answer choice selected and relevance of the item, and then received immediate feedback about the incorrect answer with rationales. After completing the quarterly items CRNA participants completed a short survey to gauge their perceptions about the LA format.

    CRNAs assigned to the CPCA group signed up to take the CPCA through the NBCRNA portal.  At the end of the assessment, CRNA participants completed a survey to gauge their perceptions about the CPCA. Once data collection was completed, NBCRNA analyzed the data to determine the viability, feasibility, and acceptability of the LA format.

  • Q Why do you have to randomly assign participants?

    Random assignment helps ensure results of the study are valid and free of bias.

  • Q Was this research approved by an Institutional Review Board (IRB)?

    This pilot study was approved by the WCG Institutional Review Board as exempt research.

  • Q Has NBCRNA conducted any additional research or evaluation of LA?

    Yes, the NBCRNA Board of Directors charged the Evaluation and Research Advisory Committee: Longitudinal Assessment Subcommittee to explore the feasibility and appropriateness of LAs for its credentialing programs.

  • Q Where can I read more about LA?

    Three papers written by NBCRNA volunteers have been published:

    1. Longitudinal Assessment: A Strategy to Improve Continuing Professional Certification
    2. Concept Analysis of Longitudinal Assessment for Professional Continued Certification
    3. Longitudinal Assessment to Evaluate Continued Certification and Lifelong Learning in Healthcare Professionals: A Scoping Review