Transfer Learning of a Neural Network for Robotic Surgical Assessment

Transfer Learning of a Pretrained Preclinical Neural Network for Robotic Surgical Assessment on Limited Clinical Data

The goal of this observational study is to explore how pretrained artificial intelligence (AI) models, trained on preclinical data, can improve the accuracy of action recognition and skills assessment in robot-assisted surgery (RAS) in urological patients by the use of transfer learning. The main questions it aims to answer are:

• Can pretrained AI models accurately assess action recognition and skills assessment in clinical surgeries?
• How do different training approaches of transfer learning affect the performance of the AI models? A baseline model developed from scratch using clinical data will be compared to pretrained models that are (1) directly applied to clinical data (2) fine-tuned by training only some layers of the AI model, and (3) fully retrained to see if these approaches improve performance.

Participants who are robot surgeons will:

• Undergo RAS procedures on patients, with no intervention, where video data will be collected for later action recognition and skills assessment.
• Contribute to model training and evaluation through clinical dataset integration.

Study Overview

• Status: Completed
• Conditions: Robot Surgery
• Intervention / Treatment: Other: observational study

• Study Type: Observational
• Enrollment (Actual): 5

Contacts and Locations

Study Locations

• Denmark
  • North Jutland
    • Aalborg, North Jutland, Denmark, 9000
      • Department of Urology, Aalborg University Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

The study population consisted of robot surgeons who where either experienced or novice (being specialized doctors undergoing surgical fellowship to become robot surgeons).

All procedures where robot-assisted procedures done on patients, who were admitted for treatment at the urological department. The patients also gave their consent regarding data collection. However, the real participant where the robot surgeons.

Description

Inclusion Criteria:

• Robot surgeons who are experienced with more than 100 cases.
• Robot surgical fellows with less than 100 cases.
• Robot surgeons who worked at the urological department of Aalborg University Hospital.

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Experienced robot surgeons; Robot surgeons with 100 or more performed robot surgical cases.	Other: observational study; This was an observational study with no intervention.
Novice robot surgeons; Robot surgeons with less than 100 performed robot surgical cases.	Other: observational study; This was an observational study with no intervention.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy of action recognition using clinical data from scratch	Accuracy of the deep learning algorithm for action recognition, when training the model from scratch using clinical data from robot surgical procedures.	From start to end of a the robot surgical procedure that is being assessed in terms of action recognition.
Accuracy of skills assessment using clinical data from scratch	Accuracy of the deep learning algorithm for skills assessment, when training the model from scratch using clinical data from robot surgical procedures.	From start to end of a the robot surgical procedure that is being assessed in terms of action recognition.
Accuracy of action recognition using the pretrained network directly on clinical data	Accuracy of the pretrained deep learning algorithm for action recognition, when using the model directly on clinical data from robot surgical procedures.	From start to end of a the robot surgical procedure that is being assessed in terms of action recognition.
Accuracy of skills assessment using the pretrained model directly on clinical data	Accuracy of the pretrained deep learning algorithm for skills assessment, when using the model directly on clinical data from robot surgical procedures.	From start to end of a the robot surgical procedure that is being assessed in terms of skills assessment.
K fold accuracies for action recognition and skills assessment for the complete retraining of the pretrained network.	K fold cross-validation accuracies when retraining the complete pretrained model on the clinical data for both action recognition and skills assessment.	From the start to the end of the clinical procedures.
K fold accuracies for action recognition and skills assessment for the partial retraining of the pretrained network.	K fold cross validation accuracies for action recognition and skills assessment for the retraining of the LSTM and dense layers of the pretrained network using clinical data.	From the start to the end of the clinical procedures.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Weighted recall/sensitivity, precision and F1 score for action recognition of the clinical network trained from scratch	Based on the performance of action recognition from the clinical network trained from scratch.	From start to end of a the robot surgical procedure that is being assessed in terms of action recognition.
Weighted recall/sensitivity, precision and F1 score for Skills Assessment of the clinical network trained from scratch	Based on the performance of skills assessment from the clinical network trained from scratch.	From start to end of a the robot surgical procedure that is being assessed in terms of skills assessment..
Predictive certainty of the action recognition and skills assessment of the network trained from scratch on the clinical data.	Predictive certainty with overall mean, minimum and maximum and depicted in probability plots for action recognition and skills assessment of the network trained from scratch on clinical data.	From the start to the end of the clinical procedures.
Predictive certainty of the action recognition and skills assessment of the network partially retrained network.	Predictive certainty with overall mean, minimum and maximum and depicted in probability plots for action recognition and skills assessment of the partially retrained network, where only the LSTM and deep layers of the network was trained on clinical data.	From the start to the end of the clinical procedures.

Collaborators and Investigators

• Sponsor: Aalborg University

Publications and helpful links

Helpful Links

• The URL for the IPD, also the site where all the datasets and codes will be made available as open source.

Study record dates

Study Major Dates

• Study Start (Actual): May 22, 2023
• Primary Completion (Actual): May 26, 2023
• Study Completion (Actual): May 26, 2023

Study Registration Dates

• First Submitted: September 22, 2024
• First Submitted That Met QC Criteria: September 22, 2024
• First Posted (Actual): September 25, 2024

Study Record Updates

• Last Update Posted (Actual): September 26, 2024
• Last Update Submitted That Met QC Criteria: September 24, 2024
• Last Verified: September 1, 2024

More Information

Terms related to this study

• Keywords: deep learning; robot surgery; surgical assessment; transfer learning
• Other Study ID Numbers: 2021-247

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES

IPD Plan Description

The IPD will be shared as anonymous and GDPR secure data on an open access website.

The data will be shared as the anonymized footage of the surgical procedures that the participants made.

• IPD Sharing Time Frame: The IPD and supporting information will be available from the time of submission to the journal, and will be available for an unlimited amount of time.
• IPD Sharing Access Criteria: Everyone who has access to the open source website of GitHub will be able to access the data. And anyone who will have access to the journal will have access to the supporting information.
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; ANALYTIC_CODE

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No