Longitudinal Assessment Modality and Considerations

The NBCRNA is continuously evaluating the effectiveness of our credentialing programs. We have listened to your feedback about the Continued Professional Certification Program Assessment (CPCA) and are exploring alternative assessment approaches that promote patient safety and lifelong learning to meet the needs of the CRNA profession.

The NBCRNA recently completed a research study in March 2023 to explore the potential of Longitudinal Assessments (LA) as an approach to continuing professional certification.

What is Longitudinal Assessment (LA)?

Longitudinal Assessment (LA) is defined as an assessment modality that combines both formative and summative features. The primary purpose of LA in continued certification is to provide a systematic measurement of knowledge, and to this end, LA for nurse anesthesia has been developed based on sound psychometric principles and scientific rigor. A secondary purpose of LA is the solidification of knowledge, which is accomplished through multiple, short assessments delivered over an extended period of time accompanied by ongoing feedback, rationale, and the identification of content areas for improvement. 

Defining Attributes of Longitudinal Assessment

  • Utilize scientific principles to enhance the testing effect of adult learning theory (frequent repetitive testing)
  • Spaced learning (exposure to materials interspersed with other activities)
  • Interleaving subject matter (simultaneously presenting several different learning topics)
  • Provide instant/immediate feedback
  • Repetitive learning experience
  • Convenient learning platform
  • Self-directed

Source: Griffis et al, 2021

 

The NBCRNA LA Research Study

The primary purpose of the LA Research Study was to determine comparability in performance among participants who (a) take the CPCA in its current form versus (b) take the assessment in an LA configuration (e.g., 30-40 questions per quarter with immediate feedback and rationales).

1,000 CRNAs for this research study were recruited from a call for volunteers that was sent out in February 2022. The research study kicked off on April 4, 2022. Participants selected were representative of the CRNA population, and randomly assigned to one of two conditions: CPCA in its current form or in its LA configuration. The research study ended March 31, 2023.

The undergoing data analysis helped determine the viability, feasibility and acceptability of LA as an alternative to the CPCA.

Based on the findings of the LA Research Study, the NBCRNA Board of Directors has made a strategic decision to move the assessment component of the CPC Program to an LA format for future CPC Program cycles.

Nothing about the current program is changing – CRNAs will need to finish the requirements of the current CPC Program cycle you are in (Class A, Class B, Core Modules, and the CPCA in its current form if applicable ) before moving into LA.

NBCRNA will be messaging specifics and onboarding information about LA to CRNAs in the coming months.

Eligibility

Research participants were required to be an active CRNA who did not take the CPCA to be eligible and give their consent to be randomly assigned to either the CPCA or the LA group. Regardless of the assigned group, research participants received a price reduction on their assessment and satisfied the assessment requirement of the CPC Program as long as the performance standard is met. Participants were also required to complete surveys throughout the process and asked to participate in a focus group.
 

 

NBCRNA Evaluation of LA

The NBCRNA appointed an LA Subcommittee to its Evaluation and Research Advisory Committee, to explore the feasibility and appropriateness of LA for its credentialing programs.

Click the links below for more information: 

Longitudinal Assessment to Evaluate Continued Certification and Lifelong Learning in Healthcare Professionals: A Scoping Review

Longitudinal Assessment: A Strategy to Improve Continuing Professional Certification 

Concept Analysis of Longitudinal Assessment for Professional Continued Certification

 

LA FAQs

  • Q What is Longitudinal Assessment?

    Longitudinal Assessment (LA) is defined as an assessment modality that combines both formative and summative features. The primary purpose of LA in continued certification is to provide a systematic measurement of knowledge, and to this end, LA for nurse anesthesia has been developed based on sound psychometric principles and scientific rigor. A secondary purpose of LA is the solidification of knowledge, which is accomplished through multiple, short assessments delivered over an extended period of time accompanied by ongoing feedback, rationale, and the identification of content areas for improvement. 

  • Q What was the purpose of this research study?

    The purpose of the LA Research Study was to determine if scores on the CPCA are similar or different among CRNA participants who (a) take the CPCA in its current form (150-items taken at a testing center or online via live remote proctoring) versus (b) taking the CPCA in an LA format (i.e., 30-40 items per quarter with immediate feedback and rationales, along with confidence and relevance ratings for each question). NBCRNA also compared differences in CRNAs perceptions about the feasibility and acceptability of the two assessment formats.

  • Q When did the study start and end?

    The yearlong study started in April 2022 and ended in March 2023.

  • Q Who was selected and what was involved in the study?

    All eligible CRNAs were emailed personal invitations to complete a baseline survey to determine eligibility and obtain informed consent. 1,000 CRNAs were randomly selected from those who volunteer. Participants selected in the study were randomly assigned to the CPCA or CPCA-LA group and notified with further instructions. Study participants needed to take the CPCA or CPCA-LA before the study ended. Participants were also required to complete surveys to share their experience and provide feedback on the assessment formats. Some were also asked to participate in a focus group.

  • Q What did participants have to do when selected?

    Each quarter CRNAs assigned to the CPCA-LA group logged into a secure website to answer the 30-40 items for four consecutive quarters. The CPCA-LA is not a proctored assessment, CRNAs may answer the questions all at once or at their own pace and can be taken anywhere that has a stable internet connection. For each item, participants needed to rate their confidence of the answer choice selected and relevance of the item, and then received immediate feedback about the incorrect answer with rationales. After completing the quarterly items CRNA participants completed a short survey to gauge their perceptions about the LA format.

    CRNAs assigned to the CPCA group signed up to take the CPCA through the NBCRNA portal.  At the end of the assessment, CRNA participants completed a survey to gauge their perceptions about the CPCA. Once data collection was completed, NBCRNA analyzed the data to determine the viability, feasibility, and acceptability of the LA format.

  • Q Why do you have to randomly assign participants?

    Random assignment helps ensure results of the study are valid and free of bias.

  • Q Was this research approved by an Institutional Review Board (IRB)?

    This pilot study was approved by the WCG Institutional Review Board as exempt research.

  • Q Has NBCRNA conducted any additional research or evaluation of LA?

    Yes, the NBCRNA Board of Directors charged the Evaluation and Research Advisory Committee: Longitudinal Assessment Subcommittee to explore the feasibility and appropriateness of LAs for its credentialing programs.

  • Q Where can I read more about LA?

    Three papers written by NBCRNA volunteers have been published:

    1. Longitudinal Assessment: A Strategy to Improve Continuing Professional Certification
    2. Concept Analysis of Longitudinal Assessment for Professional Continued Certification
    3. Longitudinal Assessment to Evaluate Continued Certification and Lifelong Learning in Healthcare Professionals: A Scoping Review