Validity of Experts and ScanNav Anatomy PNB When Identifying Sono-anatomical Structures for Ultrasound-Guided Regional Anaesthesia

Anatomical Validity of Experts and ScanNav Anatomy PNB When Identifying Sono-anatomical Structures for Ultrasound-Guided Regional Anaesthesia

This is a multi-centre, prospective, observational study, conducted under the auspices of the University of Oxford to evaluate ultrasound image analysis by human experts and an artificial intelligence device in the context of ultrasound-guided regional anaesthesia (UGRA).

Study Overview

• Status: Active, not recruiting
• Conditions: Ultrasound Imaging of Anatomical Structures
• Intervention / Treatment: Device: Annotate ultrasound scans

Detailed Description

Ethical Approval:

The ultrasound scans/still frame images used in this study have been acquired from an earlier study conducted at Oregon Health & Science University (OHSU) (ClinicalTrials.gov ref: NCT04906018). OHSU Institutional Review Board approval (STUDY00022920) was granted for collection of the ultrasound scans.

Ethical approval for the assessment of these scans by regional anaesthesia experts, has been obtained from Oxford University Research Ethics Committee for Medical Sciences (R75449/RE001).

Block Areas Included:

A Delphi study was undertaken earlier in the research programme: 'International Consensus on the Minimum Anatomical Structures to Identify on Ultrasound for the Performance of Plan A Blocks in Ultrasound-Guided Regional Anaesthesia: A Regional Anaesthesia UK Delphi Study' (work currently unpublished).

As ScanNav Anatomy PNB has undergone development and validation work for all of these blocks apart from the femoral nerve block, the remaining six Plan A block regions will be considered for this project. These regions (and structures assessed) are:

• Interscalene level brachial plexus (Structures: anterior scalene, middle scalene, C5 nerve root, C6 nerve root)
• Axillary level brachial plexus (Structures: axillary artery, axillary vein, conjoint tendon, median nerve, musculocutaneous nerve, radial nerve, ulnar nerve)
• Erector spinae plane (Structures: transverse process, erector spinae muscle group)
• Rectus sheath plane (Structures: rectus abdominis, rectus sheath (anterior layer), rectus sheath (posterior layer), peritoneum)
• Adductor canal (Structures: femoral artery, sartorius, saphenous nerve/nerve complex)
• Popliteal level sciatic nerve (Structures: sciatic nerve)

Standardisation of Ultrasound Scans used for Assessment:

The ultrasound videos and still frames will undergo review during the earlier (OHSU) study, including the following:

1. Does this ultrasound scan obtain the correct view for this block? [Yes/no]
2. Does the video or still frame contain any atypical anatomy? [Yes/no]

The answer for question 1 must be 'Yes' and for question 2 must be 'No' for the scan to be included in the present study.

Selecting Ultrasound Scans for Assessment:

Of the acceptable ultrasound scans, five will be selected (at random) for each region. This will be done for each of the six block regions, providing a total of 30 scans/still frame images overall. A suitable still frame in each scan sequence will be selected by the investigators and saved as an image file in the Portable Network Graphic (PNG) format for later segmentation.

Expert Reviewers:

Approximately 15 UGRA experts will be recruited from centres in the UK. One-three will be recruited from each of seven centres:

• Aneurin Bevan University Health Board
• Guy's & St Thomas' NHS Foundation Trust
• NHS Greater Glasgow & Clyde
• NHS Tayside
• Oxford University Hospitals NHS Foundation Trust
• Imperial College Healthcare NHS Trust
• Royal Cornwall Hospitals NHS Trust

All experts must hold a UK-recognised qualification for independent practice in anaesthesia (i.e., be qualified to perform UGRA independently and supervise others) and be a member of a relevant professional society (e.g., RA-UK, ESRA, ASRA). They must also meet at least three elements of the following criteria:

• Completed advanced training in UGRA or have >10 years of independent practice in UGRA
• Hold a qualification related to UGRA (e.g., EDRA, higher degree or equivalent)
• Regularly delivers direct clinical care using UGRA, including for 'awake' surgery where indicated
• Regularly teaches UGRA in the course of their clinical work, including advanced techniques (Plan B/C/D blocks) where indicated

Defining the Ground Truth:

The UGRA experts will all view each video in turn, on a tablet computer. At the end of each one they will be asked to mark the boundaries of the relevant anatomical structures (listed above) for the corresponding still frame image using the tablet and a stylus.

There are 21 anatomical structures across all the regions assess. Each region will represented by five ultrasound scans. Thus, 105 structures (21 x 5) will each be assessed by around 15 experts. The expert segmentation masks (for each annotated structure) will be processed in the following manner to derive an average mask for each structure:

• Each segmentation mask, ms, for a given structure will be segmented such that ms = f(m) = {1 if mask present 0 if mask not present
• For each pixel, ps, the frames will be summed across all n segmentation masks
• This will give a final mask, mf, where mf = { p : 0 ≤ p ≤ n }
• The mf will then be filtered with a threshold such that mt = f2(mt) = { 1 if pf > t n 0 otherwise

In the above description, ms represents a segmentation mask produced by each expert. Areas enclosed by an expert's segmentation outline will be allocated a numerical value of 1 (0 if not enclosed by an area of segmentation). Each pixel in the final mask (mf) will be summed across all (n) the segmentation masks of the experts, to give a final value of 0 - n. At least 75% of experts must enclose the pixel in their individual segmentation mask for the pixel to be included in the overall ground truth. ≥75% has been chosen to be consistent with the strong recommendations made in the earlier Delphi study. Lastly, mf will then be filtered at this threshold to produce the ground truth mask (mt). The algorithm defined above will be run using Python2 in jupyter-lab ('Project Jupyter').

Variability in Expert Analysis:

The correlation in the expert markings will be compared using the Dice/Intersection over Union (IoU) and Hausdorff co-efficients. Dice/IoU will be used for all structures except those identified by a non-enclosed area, where the Hausdorff co-efficient will be used. These structures are:

• Conjoint tendon
• Transverse process
• Rectus sheath (anterior layer)
• Rectus sheath (posterior layer)
• Peritoneum

Comparison to ScanNav Anatomy PNB:

The ground truth will provide an expert 'gold standard' against which ScanNav Anatomy PNB will be compared. The level of correlation of ScanNav Anatomy PNB highlighting will be compared to the expert ground truth in the same statistical manner as above

• Study Type: Observational
• Enrollment (Estimated): 15

Contacts and Locations

Study Locations

• United Kingdom
  • Wales
    • Cardiff, Wales, United Kingdom, CF10 1DY
      • Intelligent Ultrasound Limited

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

At least 15 UGRA experts will be recruited from seven centres in the UK.

Description

Inclusion Criteria:

Member of a relevant professional society (e.g., RA-UK, ESRA, ASRA) and meets at least three of the following criteria:

• Completed advanced training in UGRA or have >10 years of independent practice in UGRA
• Hold a qualification related to UGRA (e.g., EDRA, higher degree or equivalent)
• Regularly delivers direct clinical care using UGRA, including for 'awake' surgery where indicated
• Regularly teaches UGRA in the course of their clinical work, including advanced techniques (Plan B/C/D blocks) where indicated

Exclusion Criteria:

• none

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Prospective

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

Expert in ultrasound-guided regional anaesthesia; At least 15 UGRA experts, member of a relevant professional society (e.g., RA-UK, ESRA, ASRA) and meeting at least 3 of the following criteria) will be recruited from centres in the UK.; Completed advanced training in UGRA or have >10 years of independent practice in UGRA; Hold a qualification related to UGRA (e.g., EDRA, higher degree or equivalent); Regularly delivers direct clinical care using UGRA, including for 'awake' surgery where indicated; Regularly teaches UGRA in the course of their clinical work, including advanced techniques (Plan B/C/D blocks) where indicated

Device: Annotate ultrasound scans; Each expert will be asked to hand-labelled anatomical structures on relevant ultrasound images

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Experts' variability	Assess variability in identification of key anatomical structures on ultrasound by individual members of the expert panel	2 years
ScanNav anatomy PNB comparison	Compare the identification of anatomical structures on ultrasound images by an artificial intelligence system to the combined assessment of a panel of experts	2 years

Collaborators and Investigators

• Sponsor: IntelligentUltrasound Limited
• Collaborators: University of Oxford

Study record dates

Study Major Dates

• Study Start (Actual): September 1, 2021
• Primary Completion (Estimated): July 1, 2024
• Study Completion (Estimated): July 1, 2024

Study Registration Dates

• First Submitted: July 21, 2021
• First Submitted That Met QC Criteria: July 21, 2021
• First Posted (Actual): July 30, 2021

Study Record Updates

• Last Update Posted (Actual): August 14, 2023
• Last Update Submitted That Met QC Criteria: August 11, 2023
• Last Verified: August 1, 2023

More Information

Terms related to this study

• Other Study ID Numbers: R75449/RE001-1

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No