Supporting Audit and Feedback to Encourage Vaccine Uptake

Big Data and Little Behaviours: Feedback and Quality Improvement Supports to Primary Care to Facilitate COVID-19 Vaccine Uptake

In Ontario, approximately 6,000 family physicians can access a secure online system that provides a report that lists the COVID-19 vaccination status for each patient in their roster.

This implementation trial tests a practice facilitation intervention that aims to support family physicians to access their vaccination reports and effectively communicate with their unvaccinated patients. The facilitator will help develop action plans and offer a range of options, including co-hosted online town-halls, support for medical office assistants to coordinate patient outreach, and/or connection to trained, volunteer medical students that can help with patient outreach.

Study Overview

• Status: Completed
• Conditions: Vaccine Uptake; Covid19 Vaccination
• Intervention / Treatment: Other: Using Practice Facilitators to Support Physicians

Detailed Description

Our goal is to provide additional resources for family physicians who most need support to effectively engage with unvaccinated patients. Specifically, practice facilitators will use participatory methods incorporating lessons from change management and quality improvement methodology.

Our practice facilitators will help physicians by:

1. Providing technical support to access their vaccination report (or activating a delegate to do so), and to prepare a complete list of their patients that still need to be vaccinated with available contact information.
2. Helping develop an outreach plan with their administrative staff. Outreach can include bulk emails, text messages, phone call, or an invitation to a townhall. They will provide advice and resources for family physicians and their staff in responding to vaccine-related questions. Every eligible physician (i.e., those with access to a vaccine list) in a given practice location will be able to benefit from the support of the practice facilitator.
3. The physicians will also be offered the help of a medical student volunteer to assist with patient engagement. Under the Health Insurance Portability and Accountability Act (HIPAA), Physicians, as the acting health information custodian, can legally assign a custodian to do something on their behalf. The medical student volunteer will enter into a non-disclosure agreement (NDA) to ensure the patient's privacy and confidentiality of vaccination status. The medical student volunteer can help with patient engagement and offer support to family physicians who might not have the time to personally contact those patients who have yet to receive both doses of the COVID-19 vaccine. The medical student volunteers will be provided with a script for making telephone calls and training in Private Health Information, refraining from providing medical advice and the importance of confidentiality. They will also be provided with a list of common questions to address when contacting patients regarding COVID-19 vaccination and resources on how to engage in conversation with individuals who are vaccine hesitant.. A few topics provided to the medical student volunteer prior to contacting patients will include making the case for vaccination, finding out patient concerns, how to counter common vaccine concerns (speed of vaccine development, safety, side effects, misinformation, distrust of science, government and medical community, underlying conditions that can increase vulnerability to vaccine side effects) and why vaccination matters. According to the Canadian Medical Protective Association (CMPA), the medical student will be covered under the insurance of the physician as long as the physician provides permission to contact the patient. The medical student must also record the interaction using the call log datasheet provided by the research team and send this information back to the Family Physicians to be included in their EMR. A call log form will be provided for every volunteer to track their calls and the patients who were contacted.
4. Physicians will also be invited to co-host in a townhall with their unvaccinated patients. Each physician will have the opportunity to organize one town hall for their patients. The facilitator and our research staff will support these town halls, including a template slide deck, and customizable promotional material, all developed in partnership with the Health Commons Solutions Lab. Family physicians will take part in the session so they can address any clinical questions. The family physician will be offered a stipend for their time involved in inviting unvaccinated patients to the sessions and participation. Following the webinar, family physicians will be supported to send targeted letters to each of their as-yet-unvaccinated patients with action plans and supportive materials. Physicians working in communities with community ambassadors (i.e. a lay health advisor) will also have opportunity to include a community ambassador in their town hall. Community ambassadors can co-host the townhall and play an integral piece as they have connections with the community and understand the context and their struggles.

• Study Type: Interventional
• Enrollment (Actual): 600
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Canada
  • Ontario
    • Toronto, Ontario, Canada, M5S1B2
      • Women's College Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• 6,690 family physicians across Ontario who have active (up-to-date passwords) ONE ID accounts

Exclusion Criteria:

• family physicians across Ontario who do not have an active ONE ID account
• family physicians with less than 300 rostered patients
• family physicians with more than 3000 rostered patients

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Intervention group: Practice facilitation and additional support; Ontario Health will send out letters to physicians in the intervention group that will explain to them that they have a large group of eligible and unvaccinated patients and that an initiative is planned to support them in reaching out to those patients, with an embedded evaluation. It will ask them to reach out to the research team to plan a time to access the supports to gather more information, or to opt-out from the evaluation. Specifically, physicians will receive invitations to receive practice facilitation via mail letter and fax, followed by up to five weekly phone calls from a team member at Ontario Health.	Other: Using Practice Facilitators to Support Physicians; Utilize practice facilitators to empower family physicians to contact those who remain unvaccinated have better access to the COVID-19 vaccine.
No Intervention: Control group- No intervention; We choose to include a control group as we do not have the resources to deliver the intervention to the entire physician group. Cluster randomization by primary practice address will limit contamination.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Any vaccine dose during follow-up interval	Count of vaccine doses among rostered patients 12+/100 patients	approximately 4 months (from time of randomization until follow-up completed end of fiscal year, March 31, 2022)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
1st vaccine dose during follow-up interval	patients 12+ eligible not yet vaccinated and received first dose during follow-up interval	4 months
2nd vaccine dose during follow-up interval	Second vaccine doses among rostered patients 12+ during follow-up interval	4 months
3rd vaccine dose during follow-up interval	Third (booster) doses among rostered patients 12+ during follow-up interval	4 months
Pediatric dose during follow-up interval	Vaccine doses among rostered patients aged 5-11 during follow-up interval	4 months

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Any engagement	Number of physicians and practices that agreed to receive support by facilitators	4 months
Medical office assistant engagement	Number of practices in which the facilitator engaged with a medical assistant to arrange patient outreach	4 months
Report data engagement	Number of physicians that logged into their vaccine report with the help of a practice facilitator	4 months
Volunteer engagement	Number of practices that used medical student volunteers to help engage with their unvaccinated patients	4 months
Townhall uptake	Number of patients that attended townhalls with their physician	4 months
Calls by volunteers	Number of medical student volunteers calls made to contact unvaccinated patients	4 months

Collaborators and Investigators

• Sponsor: Women's College Hospital
• Collaborators: Institute for Clinical Evaluative Sciences; Ontario Health
• Investigators: Principal Investigator: Noah Ivers, MD, Women's College Hospital

Publications and helpful links

General Publications

• Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011 Apr 23;6:42. doi: 10.1186/1748-5908-6-42.
• Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999 Sep;89(9):1322-7. doi: 10.2105/ajph.89.9.1322.
• Briss PA, Rodewald LE, Hinman AR, Shefer AM, Strikas RA, Bernier RR, Carande-Kulis VG, Yusuf HR, Ndiaye SM, Williams SM. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. Am J Prev Med. 2000 Jan;18(1 Suppl):97-140. doi: 10.1016/s0749-3797(99)00118-x.
• Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O'Cathain A, Tinati T, Wight D, Baird J. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015 Mar 19;350:h1258. doi: 10.1136/bmj.h1258.
• Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, Augustovski F, Briggs AH, Mauskopf J, Loder E; ISPOR Health Economic Evaluation Publication Guidelines-CHEERS Good Reporting Practices Task Force. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)--explanation and elaboration: a report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value Health. 2013 Mar-Apr;16(2):231-50. doi: 10.1016/j.jval.2013.02.002.
• Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015 May 8;350:h2147. doi: 10.1136/bmj.h2147. No abstract available.
• Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, Grimshaw JM. No more 'business as usual' with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014 Jan 17;9:14. doi: 10.1186/1748-5908-9-14.
• Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012 Jun 13;(6):CD000259. doi: 10.1002/14651858.CD000259.pub3.
• Jarrett C, Wilson R, O'Leary M, Eckersberger E, Larson HJ; SAGE Working Group on Vaccine Hesitancy. Strategies for addressing vaccine hesitancy - A systematic review. Vaccine. 2015 Aug 14;33(34):4180-90. doi: 10.1016/j.vaccine.2015.04.040. Epub 2015 Apr 18.
• Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012 Jan-Feb;10(1):63-74. doi: 10.1370/afm.1312.
• Atkins L, Francis J, Islam R, O'Connor D, Patey A, Ivers N, Foy R, Duncan EM, Colquhoun H, Grimshaw JM, Lawton R, Michie S. A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implement Sci. 2017 Jun 21;12(1):77. doi: 10.1186/s13012-017-0605-9.
• COVID19Tracker.ca: Ontario Vaccination Data, 2021
• Semeniuk I: How many COVID-19 vaccine doses could make the difference between a fourth wave in Canada and no wave at all? New models offer four scenarios, The Globe and Mail, 2021
• Gilkey MB, Calo WA, Moss JL, Shah PD, Marciniak MW, Brewer NT. Provider communication and HPV vaccination: The impact of recommendation quality. Vaccine. 2016 Feb 24;34(9):1187-92. doi: 10.1016/j.vaccine.2016.01.023. Epub 2016 Jan 24.
• Lu PJ, Yankey D, Fredua B, O'Halloran AC, Williams C, Markowitz LE, Elam-Evans LD. Association of Provider Recommendation and Human Papillomavirus Vaccination Initiation among Male Adolescents Aged 13-17 Years-United States. J Pediatr. 2019 Mar;206:33-41.e1. doi: 10.1016/j.jpeds.2018.10.034. Epub 2018 Nov 15.
• Cataldi JR, Kerns ME, O'Leary ST. Evidence-based strategies to increase vaccination uptake: a review. Curr Opin Pediatr. 2020 Feb;32(1):151-159. doi: 10.1097/MOP.0000000000000843.
• Dahrouge S, Glazier R, Hogg W: Defining measures of panel size. Primary health care system (PHCS) Program, 2010
• Ontario College of Family Physicians: With Vaccines Coming, Family Doctors Could Jab Millions More If Fully Involved, 2021
• Anderson EL. Recommended solutions to the barriers to immunization in children and adults. Mo Med. 2014 Jul-Aug;111(4):344-8.
• SAGE Working Group: Report of the SAGE working group on vaccine hesitancy. Geneva: World Health Organization, 2014
• Fu LY, Zimet GD, Latkin CA, Joseph JG. Associations of trust and healthcare provider advice with HPV vaccine acceptance among African American parents. Vaccine. 2017 Feb 1;35(5):802-807. doi: 10.1016/j.vaccine.2016.12.045. Epub 2017 Jan 4.
• BeWORKS Research Group: COVID-19 Vaccine Hesitancy: A Behavioural Lens on a Critical Problem, 2021
• Dube E, Gagnon D, MacDonald NE; SAGE Working Group on Vaccine Hesitancy. Strategies intended to address vaccine hesitancy: Review of published reviews. Vaccine. 2015 Aug 14;33(34):4191-203. doi: 10.1016/j.vaccine.2015.04.041. Epub 2015 Apr 18.
• Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, Everitt H, Kennedy A, Qureshi N, Rogers A, Peacock R, Murray E. Achieving change in primary care--effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open. 2015 Dec 23;5(12):e009993. doi: 10.1136/bmjopen-2015-009993.
• Bordley WC, Chelminski A, Margolis PA, Kraus R, Szilagyi PG, Vann JJ. The effect of audit and feedback on immunization delivery: a systematic review. Am J Prev Med. 2000 May;18(4):343-50. doi: 10.1016/s0749-3797(00)00126-4.
• Desveaux L, Ivers NM, Devotta K, Ramji N, Weyman K, Kiran T. Unpacking the intention to action gap: a qualitative study understanding how physicians engage with audit and feedback. Implement Sci. 2021 Feb 17;16(1):19. doi: 10.1186/s13012-021-01088-1.
• Dube E, Gagnon D, Vivion M. Optimizing communication material to address vaccine hesitancy. Can Commun Dis Rep. 2020 Feb 6;46(2-3):48-52. doi: 10.14745/ccdr.v46i23a05. eCollection 2020 Feb 6.
• Glazier R, Hutchison B, Kopp A: Comparison of family health teams to other Ontario primary care models 2004/05 to 2011/12. 2015, 2019
• Jonah L, Pefoyo AK, Lee A, Hader J, Strasberg S, Kupets R, Chiarelli AM, Tinmouth J. Evaluation of the effect of an audit and feedback reporting tool on screening participation: The Primary Care Screening Activity Report (PCSAR). Prev Med. 2017 Mar;96:135-143. doi: 10.1016/j.ypmed.2016.12.002. Epub 2016 Dec 5.
• Witmer A, Seifer SD, Finocchio L, Leslie J, O'Neil EH. Community health workers: integral members of the health care work force. Am J Public Health. 1995 Aug;85(8 Pt 1):1055-8. doi: 10.2105/ajph.85.8_pt_1.1055.
• Eng E, Young R: Lay health advisors as community change agents. Family & Community Health: The Journal of Health Promotion & Maintenance, 1992
• Eng E, Hatch JW: Networking between agencies and black churches: the lay health advisor model. Prevention in Human Services 10:123-146, 1991
• Wallerstein N, Bernstein E. Introduction to community empowerment, participatory education, and health. Health Educ Q. 1994 Summer;21(2):141-8. doi: 10.1177/109019819402100202. No abstract available.
• Koch E, Thompson A, Keegan P: Community health workers: a leadership brief on preventive health programs. Washington, DC: The CivicHealth Institute at Codman Square Health Center and the Harrison Institute for Public Law at Georgetown University Law Center and the Center for Policy Alternatives, 1998
• Kegler MC, Stern R, Whitecrow-Ollis S, Malcoe LH. Assessing lay health advisor activity in an intervention to prevent lead poisoning in Native American children. Health Promot Pract. 2003 Apr;4(2):189-96. doi: 10.1177/1524839902250774.
• Agency for Healthcare Research and Quality: Engaging Primary Care Practices in Quality Improvement: Strategies for Practice Facilitators, 2015
• Higgins T, Schottenfeld L, Crossen J: Primary Care Practice Facilitation Curriculum (Module 25). 2015
• Shelton RC, Chambers DA, Glasgow RE. An Extension of RE-AIM to Enhance Sustainability: Addressing Dynamic Context and Promoting Health Equity Over Time. Front Public Health. 2020 May 12;8:134. doi: 10.3389/fpubh.2020.00134. eCollection 2020.
• Glasgow RE, Askew S, Purcell P, Levine E, Warner ET, Stange KC, Colditz GA, Bennett GG. Use of RE-AIM to Address Health Inequities: Application in a low-income community health center based weight loss and hypertension self-management program. Transl Behav Med. 2013 Jun 1;3(2):200-210. doi: 10.1007/s13142-013-0201-8.
• Wolfenden L, Foy R, Presseau J, Grimshaw JM, Ivers NM, Powell BJ, Taljaard M, Wiggers J, Sutherland R, Nathan N, Williams CM, Kingsland M, Milat A, Hodder RK, Yoong SL. Designing and undertaking randomised implementation trials: guide for researchers. BMJ. 2021 Jan 18;372:m3721. doi: 10.1136/bmj.m3721.
• Weijer C, Grimshaw JM, Eccles MP, McRae AD, White A, Brehaut JC, Taljaard M; Ottawa Ethics of Cluster Randomized Trials Consensus Group. The Ottawa Statement on the Ethical Design and Conduct of Cluster Randomized Trials. PLoS Med. 2012;9(11):e1001346. doi: 10.1371/journal.pmed.1001346. Epub 2012 Nov 20.
• Canadian Institutes of Health Research NSaERCoC, and Social Sciences and Humanities Research Council of Canada T-CPS: Ethical Conduct for Research Involving Humans, December 2014
• Matheson FI, Dunn JR, Smith KL, Moineddin R, Glazier RH. Development of the Canadian Marginalization Index: a new tool for the study of inequality. Can J Public Health. 2012 Apr 30;103(8 Suppl 2):S12-6. doi: 10.1007/BF03403823.
• Zou GY, Donner A. Extension of the modified Poisson regression model to prospective studies with correlated binary data. Stat Methods Med Res. 2013 Dec;22(6):661-70. doi: 10.1177/0962280211427759. Epub 2011 Nov 8.
• Rixon L, Baron J, McGale N, Lorencatto F, Francis J, Davies A. Methods used to address fidelity of receipt in health intervention research: a citation analysis and systematic review. BMC Health Serv Res. 2016 Nov 18;16(1):663. doi: 10.1186/s12913-016-1904-6.
• Toomey E, Hardeman W, Hankonen N, Byrne M, McSharry J, Matvienko-Sikar K, Lorencatto F. Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions. Health Psychol Behav Med. 2020 Mar 12;8(1):132-151. doi: 10.1080/21642850.2020.1738935.
• Presseau J, McCleary N, Lorencatto F, Patey AM, Grimshaw JM, Francis JJ. Action, actor, context, target, time (AACTT): a framework for specifying behaviour. Implement Sci. 2019 Dec 5;14(1):102. doi: 10.1186/s13012-019-0951-x.
• Desveaux L, Saragosa M, Russell K, McCleary N, Presseau J, Witteman HO, Schwalm JD, Ivers NM. How and why a multifaceted intervention to improve adherence post-MI worked for some (and could work better for others): an outcome-driven qualitative process evaluation. BMJ Open. 2020 Sep 3;10(9):e036750. doi: 10.1136/bmjopen-2019-036750.
• Solar O, Irwin A: A conceptual framework for action on the social determinants of health, WHO Document Production Services, 2010
• Dean HD, Williams KM, Fenton KA. From theory to action: applying social determinants of health to public health practice. Public Health Rep. 2013 Nov;128 Suppl 3(Suppl 3):1-4. doi: 10.1177/00333549131286S301. No abstract available.
• Ramey J, Boren T, Cuddihy E, et al: Does think aloud work? how do we know? Presented at the CHI '06 Extended Abstracts on Human Factors in Computing Systems, Montréal, Québec, Canada, 2006
• Crawshaw J KK, Castillo G, van Allen Z, Grimshaw JM, Presseau J: Factors affecting COVID-19 vaccination acceptance and uptake among the general public: a living behavioural science evidence synthesis (v2, May 31st, 2021). 2021
• Braun V, Clarke V: Using thematic analysis in psychology. Qualitative research in psychology 3:77-101, 2006
• Glick HA, Doshi JA, Sonnad SS, et al: Economic evaluation in clinical trials, OUP Oxford, 2014

Study record dates

Study Major Dates

• Study Start (Actual): November 15, 2021
• Primary Completion (Actual): March 15, 2022
• Study Completion (Actual): November 15, 2022

Study Registration Dates

• First Submitted: October 5, 2021
• First Submitted That Met QC Criteria: October 18, 2021
• First Posted (Actual): October 29, 2021

Study Record Updates

• Last Update Posted (Actual): April 24, 2024
• Last Update Submitted That Met QC Criteria: April 22, 2024
• Last Verified: April 1, 2024

More Information

Terms related to this study

• Keywords: Covid19; Vaccine Hesitancy; Audit and Feedback; Vaccine Uptake; Vaccine Confidence; Community Outreach
• Additional Relevant MeSH Terms: Coronavirus Infections; Coronaviridae Infections; Nidovirales Infections; RNA Virus Infections; Virus Diseases; Infections; Respiratory Tract Infections; Respiratory Tract Diseases; Pneumonia, Viral; Pneumonia; Lung Diseases; COVID-19
• Other Study ID Numbers: 2021-0082-E

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: Data analysis will take place at the Institute for Clinical Evaluative Sciences (ICES). Ontario Health will provide ICES with a list of physicians that were in either the control or intervention group. This list will be transferred to ICES using a secure portal process (axway). Data will be captured from administrative databases (including COVaxON registry data) held at ICES. Data will be linked at the physician level using their encrypted CPSO (College of Physicians and Surgeons of Ontario) number.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No