Improvings: Implementing a Protocol to Improve Intrapartum Sonography Skills. (IMPROVINgSS)

Prospective observational cohort study on the learning curve for the sonographic assessment of labor progress (fetal head position and station) amongst obstetric caregivers.

Study participants: convenience sample of 20 obstetric caregivers working in the department of obstetrics and gynecology at the University Hospital of Leuven - recruited over a 12-month period.

Format of the training program:

1. Observational phase: the goal is to measure the accuracy of the clinical assessment of fetal head position and station throughout the different stages of labor by caregivers working within the department.
2. Self-study: reading of the 2018 practice guidelines on intra-partum ultrasound by ISUOG and watching the corresponding teaching video.
3. Supervised implementation phase: one-to-one teaching by experienced sonographers, who initially demonstrate, later interactively supervise the quality of the scans acquired by the obstetric caregiver.
4. Proficiency testing: from this point onwards, the trainee independently performs ultrasound scans in labor to asses fetal head position and station. The scans will be repeated by the expert sonographer, the results will be compared (qualitatively and quantitatively) and immediate feedback provided if indicated. Every observation will have a binary outcome (success or failure of trainee acquisition) according to predefined quality assessment criteria. Observations will be continued until proficiency is reached (saturation of the learning curve).

Study Overview

• Status: Recruiting
• Conditions: Obstetric Labor

Detailed Description

1. Participant recruitment: Relevant caregivers will be invited to participate in this trial during their regular daytime shifts on the labor ward. One of the study investigators will explain the goal of the study and they will be given the opportunity to ask questions. Trainees willing to participate will be asked to sign the informed consent form.

   All women admitted on the labor ward in spontaneous labor or for planned induction of labor will be informed about the scope and purpose of this study by either the attending clinicians (obstetric caregivers, trial participants) or study investigators. An information leaflet was designed for this purpose and will be made available to the patients at the time of admission on the labor ward.

2. Trial Procedures:

2.1 Observational Phase: Ultrasound is considered a more reliable tool for assessing labor progress compared to clinical examination. We first want to conduct an internal audit to measure the accuracy of clinical assessments of fetal head position/station throughout labor by caregivers working in our department. For each trainee, 20 observations will be carried out where they will be asked to clinically assess fetal head position and station at different time points in labor. These observations will be carried out in different subjects and only on fetuses > 37 weeks of gestation, in cephalic presentation and with ruptured membranes.

Immediately after their examination, an experienced sonographer will verify their findings by means of an ultrasound scan (=reference standard). Both examinations will be carried out under the same conditions (absence of uterine contractions or maternal pushing efforts) to ensure comparability of findings. Fetal head position will be classified into three categories: occiput anterior, occiput transverse of occiput posterior (similarly for clinical and ultrasound assessment). The distance from the ischial spines (in centimeters) will be used to assess fetal head station clinically. The Angle of Progression (AoP) will be used to assess station by (trans-perineal) ultrasound.

2.2 Training

The goal is to improve ultrasound skills among caregivers working in our department by implementing a standardized training course. This will have the following components:

2.2.1 Self-study: reading of the 2018 practice guidelines on intra-partum ultrasound by ISUOG and watching the corresponding teaching video.

2.2.2 Supervised implementation: during the next round of observations, trainees will be asked to perform ultrasound scans during labor to assess fetal head position and station under direct supervision of an expert sonographer, providing immediate feedback on scanning technique and acquisitioning (probe placement, probe handling, machine settings, etc.). Supervised observations will be carried out until the trainee is deemed "fit" for independent acquisitioning, I.e., a pass/fail system will be used, and the number of supervised observations needed to obtain a pass will be recorded for every trainee.

2.3 Proficiency testing: Similar to the observational phase, where trainees will now be asked to assess fetal head position and station both clinically and by ultrasound. Scans will be repeated by the supervising sonographer and findings will either by concordant or discordant (binary outcome). Cumulative Sum (CUSUM) analysis will be used to monitor the learning curve. We estimate roughly 20 observations will be needed for trainees with prior (obstetrical) ultrasound experience to reach proficiency. In trainees without any previous ultrasound experience (first year residents with no previous exposure), we estimate about 50 observations will be needed.

The following criteria will be taken into consideration for quality assessment of the trainee's performance:

• Fetal head station: AoP acquisition is acceptable when all the following conditions are met:

  1. Correct midsagittal plane: central nucleus of the symphysis joint visible, pubic rami not visible, symphysis pubis as horizontal as possible.
  2. Leading part of the fetal skull visible (with parts of the anterior and posterior tabula).

  3. Caliper placement with the 3-point technique to measure the AoP.

     The following variables will also be recorded:

     • AoP acquisition time: time between probe placement on the skin and obtaining the best image for quantitative assessment (trainee's call to freeze the image for analysis) (seconds).
     • Delta-AoP: absolute difference in the AoP measurement obtained by the trainee and the supervising sonographer (°)
     • Assessment of engagement (yes/no): either concordant or discordant with the supervisor's opinion.
• Fetal head position: sonographic assessment of the trainee will either be concordant or discordant with the expert opinion.

2.4 Kap assessment:

A purposely designed questionnaire will be used to objectify knowledge, attitude and practices regarding intrapartum ultrasound use throughout the different stages of the training program. These KAP assessments will be conducted at the following time points:

1. Prior to the self-study
2. After the self-study and in-person teaching (supervised implementation)
3. When competency/proficiency is declared.

2.5 The following clinical variables will be recorded at every observation throughout the different stages of the training program: Pre-pregnancy BMI, Gestational weight gain (kg), Gestational age (weeks), Epidural Analgesia, Parity (Nulliparous / Multiparous), Delivery mode (first stage cesarean section, 2nd stage cesarean section, spontaneous vaginal delivery, assisted vaginal delivery), Neonatal birth weight (grams), Neonatal head circumference (cm), Stage of labor (first (latent, active) or second), Examination to delivery interval (minutes), Cervical dilatation (cm), Angle of Progression (°), Occiput-Spine-Angle or Chin-Chest-Angle (OSA or CCA) (°), Presence of fetal caput (mm), Presence of fetal skull molding (yes/no).

3. Statistics

3.1 Sample size: This is a pilot study with no previously published data on this topic. We aim to achieve a total sample size of about 20 participants. Previous studies on teaching strategies for point of care ultrasound in obstetrics (not intrapartum ultrasound) were able to demonstrate good efficacy of training programs in similar sample sizes (Shah S, et al. PLoS One. 2020;15(6) - Wanjiku GW, et al. BMJ Health Serv Res. 2018;18(1)).

We aim for the first round of inclusions to take place in 10/2022 with inclusion of approximately 10 trainees. In 02/2022, a second round of inclusions will take place with another estimate of 10 trainees. This will allow us to reach our full sample size by the end of 2023. Observations/Proficiency testing will take place during office hours, 5 days a week. The expected total duration of participation of a subject (trainee) will be approximately 6 months.

3.2 Statistical Analysis

3.2.1 Primary endpoint: Cumulative Sum analysis (CUSUM) was originally developed as a tool for quality control assessment of production pipelines in industry. However, as described by Bolsin and Noyez, it can also be used as an instrument to assess competency among trainees learning a new skill (Bolsin S, Colson M. Int J Qual Heal care. 2000;12(5):433-8. - Noyez L. Heal Care Curr Rev. 2014;02(01):2-4). It allows for an objective assessment of performance accuracy on a case-by-case basis.

A cumulative performance score is typically plotted against the number of observations, so that trends in sequential measurements can be evaluated over time. CUSUM analysis can only be performed with a binary outcome (operation success or failure). In our study, operation success is typically defined as diagnostic agreement with the expert, whereas operation failure is defined as the opposite. Both types of events are assigned a certain value, and the CUSUM score is the running sum of a mixture of increments (with each failure) and decrements (with each success). These values can be plotted on the y-axis against the number of attempts on the x- axis. Horizontal lines are then plotted at regular intervals on the y-axis, defining h1 (the spacing between unacceptable boundary lines) and h0 (the spacing between acceptable boundary lines).

The gradient and overall height of the plot derived provides feedback on performance on a case-by case basis. If the reference value is specified in terms of an acceptable failure rate (see below), the CUSUM score will increase and create a positive slope when operation failure is higher than the target value, whereas operation success will decrease the score and create a negative slope.

Competence can be declared when the graph falls below the lower boundary of h0, i.e., dropping below two consecutive boundary lines; loss of competence is declared when the graph ascends above the upper boundary of h1, i.e. rising above two consecutive boundary lines.

The acceptable and unacceptable failure rate (p0 and p1) should typically be set according to the gold standard (i.e., determined by previous cohort studies or specified by recognized governing bodies).Type 1 and type 2 error rates are typically set at the same value of 0.1 by common convention. This has the advantage of producing identical positive and negative performance thresholds, simplifying CUSUM charts and making interpretation easier.

In our study, we will set the acceptable failure rate for the sonographic assessment of fetal head position and descent at 0.02, based on previous studies by Hassan WA, et al (Ultrasound Obstet Gynecol. 2014;43(2):189-94) and Tang H, et al (Int J Gynecol Obstet. 2021;(0):1-6). The unacceptable failure rate typically ranges between 2 to 5 times the acceptable failure rate (by consensus) and will be set at 0,10 in our study.

3.2.2 Secondary endpoints: Differences in learning curves according to the trainee's experience level (for both primary endpoints), will be assessed by visual inspection of the CUSUM charts.

Failure rates of the clinical assessments during the baseline observational phase will be calculated for every trainee. Descriptive statistics (frequencies, proportions) will be used for this purpose We will try and identify confounding factors associated with correct/incorrect clinical assessment by comparing patient related, neonatal related and obstetrical variables among these groups (as outlined above). The Mann-Whitney U test, unpaired T test, McNemar's test or Chi-square test will be used when relevant. We will use logistic regression to assess associations between demographical and obstetrical parameters and accuracy of clinical examination. Variables associated with accuracy of assessment on univariate analysis will be considered for a multivariable logistic regression model.

For all trainees, Delta-AoP will be plotted against the round of observation to plot for an alternative "learning curve". The same will be done for acquisitioning times. Mean acquisition time between the first and last ten blocks of observations will be compared for each trainee using the Mann-Whitney U test. P values < 0.05 will be considered statistically significant.

4. Ethics and regulatory approvals: The trial will be conducted in compliance with the principles of the Declaration of Helsinki (2013), the principles of GCP and in accordance with all applicable regulatory requirements. This protocol and related documents will be submitted for review to Ethics Committee of the University Hospital Leuven.

The study is conducted only on the basis of prior informed consent by the subjects to participate in the study. The investigator shall obtain a signed informed consent form (ICF) for all patients prior to their enrolment and participation in the study in compliance with all applicable laws, regulations and the approval of the Ethics Committee. The investigator shall retain such ICFs in accordance with the requirements of all applicable regulatory agencies and laws.

The Investigators shall treat all information and data relating to the study disclosed to them on this study as confidential and shall not disclose such information to any third parties or use such information for any purpose other than the performance of the study. The collection, processing and disclosure of personal data, such as patient health and medical information are subject to compliance with applicable personal data protection and the processing of personal data (The General Data Protection Regulation (EU) 2016/679 (GDPR)).

5. Data Handling and Management: The recording system used to collect the data has limited access measures by usernames and passwords. Data will be collected electronically and the saved data will be hosted at UZ Leuven, in the domain allocated to the PI. This is a password-protected domain and access will be only possible to the personnel involved in the study (PI, sub-investigator, administrator). Published results will not contain any personal data that could allow the identification of individual participants.

• Study Type: Observational
• Enrollment (Anticipated): 20

Contacts and Locations

• Study Contact: Name: Bram Packet, MD; Phone Number: 00321648824; Email: bram.packet@kuleuven.be

Study Locations

• Belgium
  • Leuven, Belgium, 3000
    • Recruiting
    • University Hospitals Leuven, department of obstetrics and gynaecology
    • Contact: Bram Packet, MD; Phone Number: 00321648824

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: ADULT; OLDER_ADULT; CHILD
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

Obstetric caregivers working within the department of obstetrics and gynecology at the University Hospital of Leuven.

Description

Inclusion Criteria:

• Obstetric caregivers, willing to participate in the study (to give written informed consent).

Exclusion Criteria:

• Trainees refusing to participate (refusing to give written informed consent).

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Proficiency for the sonographic assessment of fetal head station during labor.	Number of ultrasound examinations a caregiver needs to perform under expert supervision to obtain proficiency - defined as the CUSUM (Cumulative Sum analysis) performance graph dropping below two consecutive boundary lines.	Through study completion, estimated at 12 months.
Proficiency for the sonographic assessment of fetal head position during labor.	Number of ultrasound examinations a caregiver needs to perform under expert supervision to obtain proficiency - defined as the CUSUM (Cumulative Sum analysis) performance graph dropping below two consecutive boundary lines.	Through study completion, estimated at 12 months.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Knowledge, attitude and practices regarding intrapartum ultrasound use.	Changes in KAP (Knowledge, Attitude and Practice) survey scores throughout the completion of the training program. Higher scores indicate better knowledge or a more favorable opinion regarding intrapartum ultrasound use (attitude, practice) - lower scores the opposite.; Knowledge survey: 5 questions on the sonographic assessment of fetal head station (score ranging from 0-5/5), 5 questions on the sonographic assessment of fetal head position (score ranging from 0-5/5).; Attitude: 6 questions, score ranging from 0-6/6.; Practice: 6 questions, score ranging from 0-6/6.; One Likert scale on confidence level regarding sonographic assessment of fetal head station (score ranging 0-5/5, 0 indicating "not confident at all", 5/5 "very confident"); One Likert scale on confidence level regarding sonographic assessment of fetal head position (score ranging 0-5/5, 0 indicating "not confident at all", 5/5 "very confident")	Assessed at three timepoints: once at inclusion, once after the training phase (esimated three months after inclusion) and one final assessment at completion of the training program (when proficiency is declared, estimated at 6 months).
Differences in learning curves according to training level (years of residency / clinical experience).	By visual inspection/comparison of learning curves/CUSUM graphs.	On study completion, estimated at 12 months.
Differences in learning curves for the sonographic assessment of fetal head station vs. fetal head position.	By visual inspection/comparison of learning curves/CUSUM graphs.	On study completion, estimated at 12 months.
Delta-AoP (Angle of Progression)	Differences between the (AoP trainee) and (AoP expert sonographer)/observation longitudinally plotted in time.	On study completion, estimated at 12 months.
Determinants of accuracy clinical assessment fetal head position/station	Comparison clinical variables (demographics, labor or fetal related) between accurate and inaccurate clinical examinations.	On completion of the observational phases by all recruited trainees, estimated at 6 months.

Collaborators and Investigators

• Sponsor: Universitaire Ziekenhuizen KU Leuven
• Investigators: Principal Investigator: Jan Deprest, MD, PhD, UZ Leuven / KU Leuven

Study record dates

Study Major Dates

• Study Start (ACTUAL): July 7, 2022
• Primary Completion (ANTICIPATED): March 31, 2023
• Study Completion (ANTICIPATED): December 30, 2023

Study Registration Dates

• First Submitted: July 1, 2022
• First Submitted That Met QC Criteria: July 18, 2022
• First Posted (ACTUAL): July 19, 2022

Study Record Updates

• Last Update Posted (ACTUAL): October 26, 2022
• Last Update Submitted That Met QC Criteria: October 25, 2022
• Last Verified: September 1, 2022

More Information

Terms related to this study

• Other Study ID Numbers: S66486

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No