Can the SurgInfoBot Improve the Consent Process for Endoscopy? A Randomised Controlled Trial

Can an Automated Conversational Agent (the SurgInfoBot) Improve the Consent Process for Patients Undergoing Endoscopy? A Randomised Controlled Trial

Human-computer interactions or 'Chatbots' have been utilized in a variety of healthcare settings, including the promotion of positive healthcare behaviors, the deliverance of psychological therapy, and the performance of diagnostic tasks. Standard methods of consenting patients for procedures may not always result in patients being fully informed; a 2004 study of patients undergoing screening sigmoidoscopy demonstrated that 39% of surveyed patients could describe no other indication than doctor recommendation for the procedure after undergoing standard consent.

his study seeks to investigate the usability of a novel chatbot designed to provide peri-procedural information in two endoscopic procedures - diagnostic oesophagogastroduodenoscopy (OGD) and diagnostic colonoscopy. A novel Chatbot - 'SurgInfoBot' has been developed in order to provide real-time, patient-driven peri-procedural information. This study primarily seeks to assess the effect of SurgInfoBot use on patient satisfaction with the consent process in endoscopy. It will also assess the usability of the chatbot according to the system usability scale and test performance according to the as-yet unvalidated Chatbot Usability Questionnaire (CUX). User engagement will be analyzed objectively using stored metrics. Comparison will be made between perceptions of the SurgInfoBot as an information source and other established patient information sources. The potential impact of the SurgInfoBot on peri-procedural anxiety will also be explored.

Study Overview

• Status: Recruiting
• Conditions: Gastro-Intestinal Disorder
• Intervention / Treatment: Other: Access to an automated conversational agent (SurgInfoBot)

Detailed Description

Introduction

Human-computer interactions or 'Chatbots' have been utilized in a variety of healthcare settings, including the promotion of positive healthcare behaviors, the deliverance of psychological therapy, and the performance of diagnostic tasks. The growth of the internet has enabled greater access to information regarding healthcare interventions, the truth and value of which cannot always be verified. The use of Chatbots - online programs which attempt to replicate person to person interactions - in providing healthcare information has been successfully employed by prior studies. Recent studies have investigated the role of chatbots in providing information in the post-operative period, across orthopaedic surgery, vascular surgery and urology. The value of such human-computer interactions in providing pre-procedural information is less well explored.

Standard methods of consenting patients for procedures may not always result in patients being fully informed; a 2004 study of patients undergoing screening sigmoidoscopy demonstrated that 39% of surveyed patients could describe no other indication than doctor recommendation for the procedure after undergoing standard consent. Only 19% of patients mentioned bleeding and perforation as possible complications. This study seeks to investigate the usability of a novel chatbot designed to provide peri-procedural information in two endoscopic procedures - diagnostic oesophagogastroduodenoscopy (OGD) and diagnostic colonoscopy.

A novel Chatbot - 'SurgInfoBot' has been developed in order to provide real-time, patient-driven peri-procedural information. This study primarily seeks to assess the effect of SurgInfoBot use on patient satisfaction with the consent process in endoscopy. It will also assess the usability of the chatbot according to the system usability scale and test performance according to the as-yet unvalidated Chatbot Usability Questionnaire (CUX). User engagement will be analyzed objectively using stored metrics. Comparison will be made between perceptions of the SurgInfoBot as an information source and other established patient information sources. The potential impact of the SurgInfoBot on peri-procedural anxiety will also be explored.

Methods

Trial design

This randomised controlled trial will be reported according to the CONsolidated Standards of Reporting Trials (CONSORT) 2010 guidelines. A single blinded parallel study with 1:1 randomisation will be performed. This study will be conducted at Tallaght University Hospital (Dublin, Ireland).

Participants

All patients undergoing diagnostic or screening colonoscopy or Oesophagogastroduodenoscopy (OGD) will be eligible for inclusion, provided they are aged 18 or over and have capacity to consent to participate in this trial. Patients undergoing sigmoidoscopy procedures will not be included, as the SurgInfoBot does not contain specific information regarding sigmoidoscopy. Patients undergoing planned procedures beyond those required for diagnostics (such as biopsy, campylobacter-like organism testing or polyp retrieval) will not be eligible for inclusion; this includes patients undergoing endoscopic stenting, endoscopic mucosal resection (EMR), planned endoscopic laser ablation, or other interventional procedures. Patients undergoing emergency or inpatient endoscopy will not be eligible for inclusion. This trial did not exclude patients who did not speak English as their first language, though patients requiring an interpreter will not be eligible for inclusion.

Patients were recruited from an academic teaching hospital in West Dublin. Eligible patients will be attending outpatient appointments with a single colorectal and general surgeon (DO K). All endoscopy procedures will be performed in the same recruiting institution.

Interventions

Participants will be randomised to receive either standard consent, or standard consent with SurgInfoBot access. The standard consent procedure is as follows: The rationale, risks, benefits, and alternatives for the intended procedure (diagnostic or screening OGD or colonoscopy) are discussed in the outpatient clinic by a consultant general surgeon (DOK) or members of the surgical team. Patients are given the opportunity to ask questions. All patients receive a written information leaflet, which includes contact details for the endoscopy department should patients wish to ask further questions. Those undergoing colonoscopy receive further instructions along with prescribed colonoscopy 'prep' medication by post. On the day of the procedure, patients are required to sign a standardised consent form.

Patients in the SurgInfoBot arm will be consented as above, but will also granted access to the SurgInfoBot, which they will be invited to use between their outpatient appointment and their endoscopy procedure date. Participants will be provided with an access link and Unique Study Identifier in order to pseudonymise participant data. Participants can access the SurgInfoBot as many times as they wish, and could ask any questions they see fit.

The 'SurgInfoBot' is a fully automated conversational agent, built as a web page that can be accessed via computer or smartphone. The SurgInfoBot was developed within the department of Surgical Affairs, Royal College of Surgeons in Ireland, using the Microsoft Chatbot Framework (Microsoft Corporation, 2019) and hosted by the Azure cloud system (Microsoft corporation, 2019). It is a command-based chatbot which relies on a database of questions and replies to answer user queries. A question bank with associated answers was developed initially by two general surgery residents (BM, CT) using national freely-available online resources where available, and high quality peer-reviewed articles where no national resources addressed the topic of concern. Content was verified for accuracy by a consultant colorectal/ general surgeon (DO K). An iterative process of content generation, usability testing, modification and re-testing was undertaken with healthy volunteers, in order to populate the SurgInfoBot's knowledge database and ensure adequate usability prior to patient use.

Randomisation

Randomisation will be performed using an online randomisation system (https://www.sealedenvelope.com/simple-randomiser/v1/lists) . After undergoing standard endoscopy consenting procedures as above, patients will be asked to consent to take part in this research study. Recruited patients will then be allocated the next available unique study identifier, randomly allocating patients to receive standard further consent information (printed information leaflet) or access to the SurgInfoBot (printed information leaflet and access instructions for the SurgInfoBot online interface). Both groups of patients will also receive a link (via e-mail) to the survey of baseline demographics which will completed before access to the SurgInfoBot is granted. Patients in the SurgInfoBot access arm will also receive an e-mail with their access details.

Baseline data

Baseline demographic information will be recorded. This includes participant gender, age, first language (English or other), level of educational attainment and electronic health literacy as measured using the eHEALS (eHealth Literacy Scale) score.

Primary Outcome Measure and Determination of Sample Size:

Patient satisfaction with the pre-procedural information received will be recorded using a 6-point Likert scale from '- - -' to '+ + +' using the following statement: "Overall, I am satisfied with the information I received before this procedure. This study was powered to detect a 0.5 point increase in satisfaction, based on a mean and standard deviation derived from a previous study utilising this scale by Huber et al. 201213, with a power of 80% and alpha of 0.05. Thirty-one participants were therefore required for recruitment in each group. Based on an anticipated drop-out rate of 20%, we aimed to recruit 70 patients in total (35 in each group).

Secondary Outcome Measures:

Knowledge:

Knowledge was assessed using a 5 item questionnaire using both open and multiple choice questions; participants were asked to identify the procedure they were having performed name the indication for having this procedure performed, identify the nature of sedation given, identify the risks of the procedure, and identify the alternatives of having the procedure performed. Items and responses were reviewed by a general surgery resident (CT) and colorectal/ general surgeon (DOK) for content validity. Knowledge questionnaires were administered immediately after standard consent was provided, and on the day of the procedure. Perceived knowledge was also assessed using a six-point Likert scale, based on participant responses to the statement: "I feel well informed about the planned procedure" from '- - -' to '+ + +'

Anxiety Peri-procedural anxiety was recorded on the day of the procedure using the State-Trait Anxiety Inventory for adults14

SurgInfoBot usability

SurgInfoBot usability was recorded using the validated ChatBot Usability Questionnaire developed by Holmes et al. (2019)15, a usability scale specifically designed for evaluating chatbots in healthcare.

Statistical Analysis

Categorical data will be presented as both absolute and relative frequencies, continuous data by mean and standard deviation. Comparisons between groups will be made using the Chi-Square test and t-tests. An effect size will be calculated for the primary outcome measure (Overall Satisfaction scores) - 95% confidence intervals will also be reported. Multivariate logistic regression will be used to identify predictors of complete satisfaction (defined as an overall satisfaction score of +++). All tests will be two-tailed, with p<0.05 considered statistically significant. All analyses will be performed with SPSS software (IBM, USA).

• Study Type: Interventional
• Enrollment (Anticipated): 70
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Dara O Kavanagh, MCh FRCSI; Phone Number: 2211 0035314142000; Email: dara.kavanagh@tuh.ie

Study Locations

• Ireland
  • Dublin, Ireland
    • Recruiting
    • Tallaght University Hospital
    • Contact: Dara O Kavanagh, MCh FRCSI; Phone Number: 2211 0035314142000; Email: dara.kavanagh@tuh.ie

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• All patients undergoing elective diagnostic or screening colonoscopy or Oesophagogastroduodenoscopy (OGD) are eligible for inclusion, provided they are aged 18 or over and have capacity to consent to participate in this trial.

Exclusion Criteria:

• Patients undergoing sigmoidoscopy procedures will not be included, as the SurgInfoBot does not contain specific information regarding sigmoidoscopy. Patients undergoing planned procedures beyond those required for diagnostics (such as biopsy, campylobacter-like organism testing or polyp retrieval) are not eligible for inclusion; this includes patients undergoing endoscopic stenting, endoscopic mucosal resection (EMR), planned endoscopic laser ablation, or other interventional procedures. Patients undergoing emergency or inpatient endoscopy are not eligible for inclusion. This trial does not exclude patients who dio not speak English as their first language, though patients requiring an interpreter are not eligible for inclusion.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Supportive Care
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Standard Consent; Standard consent procedure: The rationale, risks, benefits, and alternatives for the intended procedure (diagnostic or screening OGD or colonoscopy) will be discussed in the outpatient clinic by a consultant general surgeon (DOK) or members of the surgical team. Patients will be given the opportunity to ask questions. All patients will receive a written information leaflet, which includes contact details for the endoscopy department should patients wish to ask further questions (supplemental data). Those undergoing colonoscopy will receive further instructions along with prescribed colonoscopy 'prep' medication by post. On the day of the procedure, patients are required to sign a standardised consent form.
Experimental: Standard consent + access to an automated conversational agent (SurgInfoBot); Patients in the SurgInfoBot arm will be consented as above, but will also be granted access to the SurgInfoBot, which they will be invited to use between their outpatient appointment and their endoscopy procedure date. Participants will be provided with an access link and Unique Study Identifier in order to pseudonymise participant data. Participants will be able to access the SurgInfoBot as many times as they wish, and can ask any questions they see fit.	Other: Access to an automated conversational agent (SurgInfoBot); See above under arm/group descriptions.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient Satisfaction	6-point Likert scale from '---' to '+++' - 'Overall, I am satisfied with the information I received before this procedure'	Immediate (day of procedure)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Knowledge	a 5 item questionnaire using both open and multiple choice questions	Immediate (Post- standard consent) and at 6 weeks, (pre-endoscopy procedure)
Knowledge (subjective)	Perceived knowledge will also be assessed using a six-point Likert scale, based on participant responses to the statement: "I feel well informed about the planned procedure" from '- - -' to '+ + +'	Immediate (After provision of standard consent) and at 6 weeks (pre-endoscopy procedure)
Peri-procedural anxiety	Peri-procedural anxiety will be recorded on the day of the procedure using the State-Trait Anxiety Inventory for adults	Immediate (Day of procedure)
SurgInfoBot Usability	SurgInfoBot usability will be recorded using the validated ChatBot Usability Questionnaire developed by Holmes et al. (2019), a usability scale specifically designed for evaluating chatbots in healthcare.	6 weeks (Pre-procedure)

Collaborators and Investigators

• Sponsor: Tallaght University Hospital
• Collaborators: Royal College of Surgeons, Ireland
• Investigators: Principal Investigator: Dara O Kavanagh, MCh FRCSI, Tallaght University Hospital

Study record dates

Study Major Dates

• Study Start (Actual): November 29, 2021
• Primary Completion (Anticipated): November 1, 2022
• Study Completion (Anticipated): July 1, 2023

Study Registration Dates

• First Submitted: November 24, 2021
• First Submitted That Met QC Criteria: December 15, 2021
• First Posted (Actual): December 16, 2021

Study Record Updates

• Last Update Posted (Actual): December 16, 2021
• Last Update Submitted That Met QC Criteria: December 15, 2021
• Last Verified: December 1, 2021

More Information

Terms related to this study

• Keywords: Endoscopy; Patient education; Automated conversational agent; Chatbot; Consent
• Additional Relevant MeSH Terms: Gastrointestinal Diseases; Digestive System Diseases; Intestinal Diseases
• Other Study ID Numbers: REC: 2021-11

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No