Effectiveness of a Proficiency-based Progression Communication Training Programme (PROTECT)

Safety Huddle Performance: The Effectiveness of an Interdisciplinary, Proficiency-based, Team Simulation Programme to Improve Communication and Reduce Patient Harm (PROTECT)

Breakdown in communication between healthcare workers leads to significant patient harm on a daily basis, worldwide. The "safety huddle" is being introduced internationally as a means of overcoming this problem. The "safety huddle" is where healthcare workers of all types gather for a 15-minute meeting to voice and address safety concerns for their patients. The safety of the patients depends on how well people communicate with each other at the "safety huddle". As communication skills vary, the research team plans to devise a team training course for healthcare workers of all types that ensures a standard is reached which is as good as teams we know perform well. The research team plans to introduce this training programme to one ward in two different hospitals with the aim of reducing patient harm. The team will compare rates of harm using a method that has been used around the world and involves looking back through some patients' notes for certain clues. The team will do this by choosing notes at random for a 6-month period before the training and again for a 6 month period after the training. The researchers will then see if levels of harm have improved or not with the team training. The researchers will also measure how well the teams conduct the "safety huddle" and if they felt that in general there was more safety awareness on the ward.

Study Overview

• Status: Recruiting
• Conditions: Adverse Events
• Intervention / Treatment: Other: A team, proficiency based progression, simulation training programme

Detailed Description

Effective communication in acute care settings is essential to the provision of safe and reliable patient care. The need for high quality communication skills has become urgent in Irish hospitals as organisations and medical therapies become more complex, specialisation increases, patients have a greater degree of comorbidity and working patterns move to a shift-based pattern (as a result of the European Working Time Directive). As many as 59% of hospital based medical trainees have encountered patient harm which they attribute to a substandard communication, with up to 12% reporting that this harm has been major. In a review undertaken by the National Confidential Enquiry into Patient Outcome and Death of patients who died within 4 days of admission in the UK, poor communication between and within clinical teams was identified as an important issue in 13.5% (267/1983) of the deaths.

A recent review of team training interventions showed that there was overall, moderate-to-high quality evidence suggesting team training can positively impact healthcare team processes and patient outcomes. A metanalysis of team training interventions supports the expanded use of team training and points toward recommendations for optimizing its effectiveness within the healthcare setting. Despite this evidence, communication training for healthcare workers is often either conducted within the boundaries of each discipline or not at all. As part of preparatory work for the National Clinical Guideline on Clinical Communication commissioned by the Department of Health in Ireland, a series of surveys, interviews and focus groups were undertaken across the acute hospital services in Ireland. Results of these surveys indicated that only 2 of 28 health professional training programmes for which information was available, had planned shared learning activities with other disciplines in relation to communication training. Furthermore, in focus groups and interviews undertaken across all disciplines, most participants reported that "they had not ever received formal training" in clinical communication.

Where communication programmes do exist, learner perceptions are frequently used as a marker of effectiveness rather than an objective assessment indicating a quantifiably improved skill set with a demonstrable patient impact.

Proficiency-based progression (PBP) training is an innovative form of outcomes-based training that involves training individuals to achieve a proficiency benchmark. Members of this research team have published extensively on this methodology which has been shown to improve both healthcare worker performance and patient outcomes.

The process involves a thorough task analysis of high performing individuals, teams or services to develop a set of clearly defined metrics. Education and simulation training programmes are then constructed to capture these metrics. In this way, training is honed on aspects that are strategic to high levels of performance to produce a more consistent skill set in the trainee. Practice is "deliberate" as a result of feedback on the metrics, rather than "repeated" which can allow for errors to go unchecked. Assessment consists of comparison against a "proficiency benchmark" which is set as the mean performance as scored on the metrics of real world clinicians who are genuinely good at the task in clinical practice. Results of studies undertaken with this approach on technical skill performance indicate that it is superior to standard simulation courses. The lead knowledge user (DB) and research co-applicant (AGG) have recently published the first randomised trial of its use in non-technical (communication) skills. The primary aim of the study was to determine if the addition of a proficiency-based progression simulation training programme to the national HSE, NEWS e-learning module results in better performance of clinical communication in the context of the deteriorating patient than either the e-learning module alone or in combination with standard simulation. Undergraduate medical and nursing participants were randomised to one of these three groups. Results showed that in a standardised simulated scenario, only 6.9% (2/29) of the e-learning group demonstrated proficiency compared to 13% (3/23) of the standard simulation group and 60% of the PBP group. These encouraging results were obtained despite the fact that the PBP training programme was constrained by the time allowed within the existing curriculum. Of note, work from a pilot for this study conducted the previous year, showed that trainee perceptions of the programme were not related to proficient performance.

The proposal presented here builds on this work and more importantly aims to use patient outcomes as well as healthcare worker performance as the ultimate marker of effectiveness. If this study is successful, the safety huddle training programme will represent an innovative and significant patient safety initiative, developed and designed within the Irish setting and deliverable on any ward/clinical area. It will have a tangible benefit to patients and families. The training programme will be multidisciplinary and represents a ward-based (clinical area), team intervention that will embed the safety huddle into daily practice.

This pragmatic approach by the PROTECT study will facilitate implementation as well as research. It is anticipated by embedding the training at team and ward level, "side to side "spread to other clinical areas in the two study institutions will occur in the months following the study. The study has senior organisational and clinical support at both hospital and regional level to accelerate further roll out to the wider region at the end of the study.

The published results will augment the invesigators' previous work in this area. It will build on this body of knowledge to expand high quality evidence for the Irish context in this domain. This evidence will inform future revisions of national clinical guidelines and training decisions made by training bodies, regulators and health care service providers to encourage a paradigm shift towards validated, team (rather than single discipline) training in the 1-2 years after completion of the study.

• Study Type: Observational
• Enrollment (Anticipated): 60

Contacts and Locations

• Study Contact: Name: Dorothy Breen; Phone Number: 0872430373; Email: dorothy.breen@hse.ie
• Study Contact Backup: Name: Amy Stone; Phone Number: 0857075459; Email: amy.stone@ucc.ie

Study Locations

• Ireland
  • Cork, Ireland, T12 DC4A
    • Recruiting
    • Cork University Hospital
    • Contact: Dorothy Breen; Phone Number: 0872430373; Email: dorothy.breen@hse.ie
    • Contact: Amy Stone; Phone Number: 0857075459; Email: amy.stone@ucc.ie
  • Waterford, Ireland, X91 ER8E
    • Not yet recruiting
    • University Hospital Waterford
    • Contact: Dorothy Breen; Phone Number: 0872430373; Email: dorothy.breen@hse.ie
    • Contact: Amy Stone; Phone Number: 0857075459; Email: amy.stone@ucc.ie
    • Principal Investigator: Peter Neary

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

Members of the multidisciplinary team include nurses, doctors, physios, occupational therapists, pharmacists, dieticians, and occupational therapists etc. The two wards chosen have a similar case mix of patient cohorts to each other. Staff members who are frequently working on either ward 4B or the Cedar ward are to be included in the study population.

Description

Inclusion Criteria:

• Multidisciplinary staff member primarily based on ward 4B, Cork University Hospital
• Multidisciplinary staff member primarily based on the Cedar ward, University Hospital Waterford

Exclusion Criteria:

• Lack of consent.
• Rotating staff who undertake occasional work in the designated wards.

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
4B ward staff; Multidisciplinary staff members from a surgical ward in Cork University Hospital.	Other: A team, proficiency based progression, simulation training programme; The training programme will consist of two components (i) a short online training course including video examples of high performing teams and also poor practice (ii) a ward-based team huddle simulation training and assessment programme. The design and shape of the training and assessment will be based on the development of robust metrics. The design of the metrics and programme will centre around use in the acute hospital setting but will be readily adaptable to the obstetric, paediatric setting. The process of validating the metrics and building the online and simulation course involves the review of videotapes of safety huddle performance.
Cedar ward staff; Multidisciplinary staff members from the Cedar ward in University Hospital Waterford with a similar case mix of patients to ward 4B in CUH.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The primary outcome will be the incidence of adverse events on each ward for a 6 month period before and after the team training intervention.	Adverse events will be documented from random chart review using Institute of Healthcare Improvement Trigger tool methodology including the surgical module before and after the intervention .	Retrospective chart review to be conducted from Feb to Aug 2022.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Huddle Performance as scored on the metrics-based proficiency assessment developed as part of the training programme.	The number of huddle metrics achieved and benchmark assessment by direct observation (at least once/week) and independent videotape review (once/month x 6 months post training)	To be conducted January -June 2022
Safety Culture as measured by the Safety Attitudes Questionnaire before and at 1 and 6 months after training.	The Safety Attitudes Questionnaire will be conducted on both wards before training commencing in July 2021 - Dec 2021 before the intervention and repeated in March/ April 2022 immediately after the training intervention and again in July 2022. Most of the survey items use 5-point agreement scales ("Strongly disagree" to "Strongly agree") or frequency scales ("Never" to "Always") and also include a "Does not apply or Don't know" response option. The survey has a section at the end for open-ended comments.	To be conducted July 21 to July 22
Economic impact as measured by the incremental cost effectiveness ratio (ICER) and budget impact analysis of the intervention.	The cost effectiveness of the intervention will be assessed by comparing incremental costs and effects of the intervention compared to without the intervention. All direct costs associated with the intervention will be identified, measured and valued. In the baseline analysis a cost utility analysis will be performed; wherein effects are estimated in Quality Adjusted Life years. To examine robustness a cost effectiveness analysis will also be performed, whereby effects are measured by reduction in adverse events to determine if additional costs yield additional benefits. A probabilistic sensitivity analysis will be performed to examine uncertainty around the parameters and outputs. Also, a budget impact analysis will be performed.	To be conducted May - September 22
Rates of reported incidents	Number and type of incidents reported on the Irish National Incident Monitoring system for 5 years preceding June 22	To be conducted April 2022 to August 2022
The prevalence of COVID-19 infections and COVID-19 ICU admissions in both hospitals	Measuring the prevalence of COIVD-19 infections and COVID-19 ICU admissions in both hospitals over the duration of the research project, from September 2020 - September 2022.	July 2022
The effect of the safety huddle on creating awareness amongst staff of which patients have been identified as watchers	Examine the effect of the safety huddle in creating awareness of the patients who've been identified as "watchers" on the ward in those staff members who did not participate in the huddle. This will be assessed in one study ward by asking 5 nursing staff members not present at the huddle who the "watchers" reported for that day are. This will be done at two different time points, once in July 2021 and again in July 2022 to compare data before and after the training course.	July 2021 and July 2022

Collaborators and Investigators

• Sponsor: Dorothy Breen
• Collaborators: University of Ulster; Health Research Board, Ireland; Cork University Hospital; Health Service Executive, Ireland; University Hospital Waterford; Orsi Academy; International Society for Quality in Health care; Irish Patients Association
• Investigators: Principal Investigator: Dorothy Breen, Cork University Hospital; Principal Investigator: George Shorten, University College Cork & Cork University Hospital; Principal Investigator: Peter Neary, University Hospital Waterford

Publications and helpful links

General Publications

• Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA 3rd, Ramel S, Smith CD, Arvidsson D. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007 Jun;193(6):797-804. doi: 10.1016/j.amjsurg.2006.06.050.
• Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002 Oct;236(4):458-63; discussion 463-4. doi: 10.1097/00000658-200210000-00008.
• Angelo RL, Ryu RK, Pedowitz RA, Beach W, Burns J, Dodds J, Field L, Getelman M, Hobgood R, McIntyre L, Gallagher AG. A Proficiency-Based Progression Training Curriculum Coupled With a Model Simulator Results in the Acquisition of a Superior Arthroscopic Bankart Skill Set. Arthroscopy. 2015 Oct;31(10):1854-71. doi: 10.1016/j.arthro.2015.07.001. Epub 2015 Sep 2.
• Hughes AM, Gregory ME, Joseph DL, Sonesh SC, Marlow SL, Lacerenza CN, Benishek LE, King HB, Salas E. Saving lives: A meta-analysis of team training in healthcare. J Appl Psychol. 2016 Sep;101(9):1266-304. doi: 10.1037/apl0000120. Epub 2016 Jun 16.
• Kitch BT, Cooper JB, Zapol WM, Marder JE, Karson A, Hutter M, Campbell EG. Handoffs causing patient harm: a survey of medical and surgical house staff. Jt Comm J Qual Patient Saf. 2008 Oct;34(10):563-70. doi: 10.1016/s1553-7250(08)34071-9.
• 2. National Confidential Enquiry into Patient Outcome and Death. Caring to the end? A review of the care of patients who died in hospital within four days of admission. NCEPOD, 2009. www.ncepod.org.uk/2009report2/Downloads/DAH_report.pdf
• Weaver SJ, Dy SM, Rosen MA. Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf. 2014 May;23(5):359-72. doi: 10.1136/bmjqs-2013-001848. Epub 2014 Feb 5.
• 5. National Clinical Effectiveness Committee. Communication (Clinical Handover) in Acute and Children's Hospital Services, National Clinical Guideline No. 11. http://health.gov.ie/wp-content/uploads/2015/12/NCG-No-11-Clinical-Handover-Acute-andChildrens-Hospital-Services-Full-Report.pdf
• Gallagher AG, Satava RM, Shorten GD. Measuring surgical skill: a rapidly evolving scientific methodology. Surg Endosc. 2013 May;27(5):1451-5. doi: 10.1007/s00464-013-2786-x. Epub 2013 Feb 1. No abstract available.
• 7. Gallagher AG, O'Sullivan GC (2011). Fundamentals of surgical simulation; principles & practices: London: Springer Verlag.
• Breen D, O'Brien S, McCarthy N, Gallagher A, Walshe N. Effect of a proficiency-based progression simulation programme on clinical communication for the deteriorating patient: a randomised controlled trial. BMJ Open. 2019 Jul 9;9(7):e025992. doi: 10.1136/bmjopen-2018-025992.
• Kallidaikurichi Srinivasan K, Gallagher A, O'Brien N, Sudir V, Barrett N, O'Connor R, Holt F, Lee P, O'Donnell B, Shorten G. Proficiency-based progression training: an 'end to end' model for decreasing error applied to achievement of effective epidural analgesia during labour: a randomised control study. BMJ Open. 2018 Oct 15;8(10):e020099. doi: 10.1136/bmjopen-2017-020099.
• 13. Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events (Second Edition). IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement; 2009. (Available on www.IHI.org
• 14. The Health Foundation. Measuring safety culture [Internet]. 2011. Available from: https://www.health.org.uk/sites/health/files/MeasuringSafetyCulture.pdf
• Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006 Apr 3;6:44. doi: 10.1186/1472-6963-6-44.

Study record dates

Study Major Dates

• Study Start (Actual): September 30, 2020
• Primary Completion (Anticipated): August 1, 2022
• Study Completion (Anticipated): August 1, 2022

Study Registration Dates

• First Submitted: May 28, 2021
• First Submitted That Met QC Criteria: May 23, 2022
• First Posted (Actual): May 25, 2022

Study Record Updates

• Last Update Posted (Actual): May 25, 2022
• Last Update Submitted That Met QC Criteria: May 23, 2022
• Last Verified: May 1, 2022

More Information

Terms related to this study

• Other Study ID Numbers: PROTECT

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No