Effect of Peer Benchmarking on Specialist Electronic Consult Performance in a Los Angeles Safety-Net: a Cluster Randomized Trial

Behavioral Economics and eConsult Steering Committee, Daniella Meeker, Mark W Friedberg, Tara K Knight, Jason N Doctor, Dina Zein, Nancy Cayasso-McIntosh, Noah J Goldstein, Craig R Fox, Jeffrey A Linder, Stephen D Persell, Stanley Dea, Paul Giboney, Hal F Yee, Behavioral Economics and eConsult Steering Committee, Daniella Meeker, Mark W Friedberg, Tara K Knight, Jason N Doctor, Dina Zein, Nancy Cayasso-McIntosh, Noah J Goldstein, Craig R Fox, Jeffrey A Linder, Stephen D Persell, Stanley Dea, Paul Giboney, Hal F Yee

Abstract

Background: Since the advent of COVID-19, accelerated adoption of systems that reduce face-to-face encounters has outpaced training and best practices. Electronic consultations (eConsults), structured communications between PCPs and specialists regarding a case, have been effective in reducing face-to-face specialist encounters. As the health system rapidly adapts to multiple new practices and communication tools, new mechanisms to measure and improve performance in this context are needed.

Objective: To test whether feedback comparing physicians to top performing peers using co-specialists' ratings improves performance.

Design: Cluster-randomized controlled trial PARTICIPANTS: Eighty facility-specialty clusters and 214 clinicians INTERVENTION: Providers in the feedback arms were sent messages that announced their membership in an elite group of "Top Performers" or provided actionable recommendations with feedback for providers that were "Not Top Performers."

Main measures: The primary outcomes were changes in peer ratings in the following performance dimensions after feedback was received: (1) elicitation of information from primary care practitioners; (2) adherence to institutional clinical guidelines; (3) agreement with peer's medical decision-making; (4) educational value; (5) relationship building.

Key results: Specialists showed significant improvements on 3 of the 5 consultation performance dimensions: medical decision-making (odds ratio 1.52, 95% confidence interval 1.08-2.14, p<.05), educational value (1.86, 1.17-2.96) and relationship building (1.63, 1.13-2.35) (both p<.01).

Conclusions: The pandemic has shed light on clinicians' commitment to professionalism and service as we rapidly adapt to changing paradigms. Interventions that appeal to professional norms can help improve the efficacy of new systems of practice. We show that specialists' performance can be measured and improved with feedback using aspirational norms.

Trial registration: clinicaltrials.gov NCT03784950.

Keywords: behavioral economics; electronic consultation; peer comparison; quality of care; specialty care.

Conflict of interest statement

Hal Yee is an advisor to RubiconMD. All other authors declare that they do not have a conflict of interest.

© 2021. Society of General Internal Medicine.

Figures

Figure 1
Figure 1
CONSORT diagram. Specialty-affiliation clusters (m=80) were each randomized to one of three arms, ultimately comprising 214 randomized specialists.
Figure 2
Figure 2
Adjusted odds of improvement after feedback. The odds ratio for improvement for each rated dimension after feedback.

References

    1. Huang X, Rosenthal MB. Transforming Specialty Practice — The Patient-Centered Medical Neighborhood. New England Journal of Medicine. 2014;370(15):1376–1379. doi: 10.1056/nejmp1315416.
    1. Forrest CB, Glade GB, Baker AE, Bocian A, von Schrader S, Starfield B. Coordination of specialty referrals and physician satisfaction with referral care. Arch Pediatr Adolesc Med. 2000;154(5):499–506. doi: 10.1001/archpedi.154.5.499.
    1. Liddy C, Afkham A, Drosinis P, Joschko J, Keely E. Impact of and Satisfaction with a New eConsult Service: A Mixed Methods Study of Primary Care Providers. J Am Board Fam Med. 2015;28(3):394–403. doi: 10.3122/jabfm.2015.03.140255.
    1. Keely E, Canning S, Saloojee N, Afkham A, Liddy C. Improving Access to Gastroenterologist Using eConsultation: A Way to Potentially Shorten Wait Times. J Can Assoc Gastroenterol. 2018;1(3):124–128. doi: 10.1093/jcag/gwy017.
    1. Schettini P, Shah KP, O’Leary CP, et al. Keeping care connected: e-Consultation program improves access to nephrology care. J Telemed Telecare. Published online January 1, 2017:1357633X17748350.
    1. Gleason N, Prasad PA, Ackerman S, et al. Adoption and impact of an eConsult system in a fee-for-service setting. Healthc (Amst). 2017;5(1-2):40–45. doi: 10.1016/j.hjdsi.2016.05.005.
    1. Lee MS, Ray KN, Mehrotra A, Giboney P, Yee HF, Jr, Barnett ML. Primary Care Practitioners’ Perceptions of Electronic Consult Systems: A Qualitative Analysis. JAMA Intern Med. 2018;178(6):782–789. doi: 10.1001/jamainternmed.2018.0738.
    1. Naseriasl M, Adham D, Janati A. E-referral Solutions: Successful Experiences, Key Features and Challenges- a Systematic Review. Mater Sociomed. 2015;27(3):195–199. doi: 10.5455/msm.2015.27.195-199.
    1. Barnett ML, Yee HF, Jr, Mehrotra A, Giboney P. Los Angeles Safety-Net Program eConsult System Was Rapidly Adopted And Decreased Wait Times To See Specialists. Health Aff. 2017;36(3):492–499. doi: 10.1377/hlthaff.2016.1283.
    1. Chen AH, Murphy EJ, Yee HF. eReferral — A New Model for Integrated Care. New England Journal of Medicine. 2013;368(26):2450–2453. doi: 10.1056/nejmp1215594.
    1. Hefzy EM, Wegdan AA, Abdel Wahed WY. Hospital outpatient clinics as a potential hazard for healthcare associated infections. J Infect Public Health. 2016;9(1):88–97. doi: 10.1016/j.jiph.2015.06.015.
    1. Brisson GE. Colleagues Unknown — How Peer Evaluation Could Enhance the Referral Process. New England Journal of Medicine. 2019;381(14):1303–1305. doi: 10.1056/nejmp1905524.
    1. Pham HH, O’Malley AS, Bach PB, Saiontz-Martinez C, Schrag D. Primary care physicians’ links to other physicians through Medicare patients: the scope of care coordination. Ann Intern Med. 2009;150(4):236–242. doi: 10.7326/0003-4819-150-4-200902170-00004.
    1. Garrick TR, Stotland NL. How to write a psychiatric consultation. Am J Psychiatry. 1982;139(7):849–855. doi: 10.1176/ajp.139.7.849.
    1. Tran C, Archibald D, Humphrey-Murto S, Liddy C, Keely E. What makes a high-quality electronic consultation (eConsult)? A nominal group study. J Telemed Telecare. Published online February 4, 2019:1357633X18822885.
    1. O’Malley AS, Reschovsky JD. Referral and consultation communication between primary care and specialist physicians: finding common ground. Arch Intern Med. 2011;171(1):56–65. doi: 10.1001/archinternmed.2010.480.
    1. Liddy C, Abu-Hijleh T, Joschko J, Archibald D, Keely E. eConsults and Learning Between Primary Care Providers and Specialists. Fam Med. 2019;51(7):567–573. doi: 10.22454/FamMed.2019.407574.
    1. Janssen M, Sagasser MH, Laro EAM, de Graaf J, Scherpbier-de Haan ND. Learning intraprofessional collaboration by participating in a consultation programme: what and how did primary and secondary care trainees learn? BMC Med Educ. 2017;17(1):125. doi: 10.1186/s12909-017-0961-9.
    1. Meijer LJ, de Groot E, Blaauw-Westerlaken M, Damoiseaux RAMJ. Intraprofessional collaboration and learning between specialists and general practitioners during postgraduate training: a qualitative study. BMC Health Serv Res. 2016;16(a):376. doi: 10.1186/s12913-016-1619-8.
    1. Thaler RH, Sunstein CR. Nudge: Improving Decisions about Health, Wealth and Happiness. 2009.
    1. Goldstein NJ, Cialdini RB, Griskevicius V. A Room with a Viewpoint: Using Social Norms to Motivate Environmental Conservation in Hotels. J Consum Res. 2008;35(3):472–482. doi: 10.1086/586910.
    1. Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA. 2001;285(22):2871–2879. doi: 10.1001/jama.285.22.2871.
    1. Schultz PW, Wesley Schultz P, Nolan JM, Cialdini RB, Goldstein NJ, Griskevicius V. The Constructive, Destructive, and Reconstructive Power of Social Norms. Psychological Science. 2007;18(5):429–434. doi: 10.1111/j.1467-9280.2007.01917.x.
    1. Walther JB. Computer-Mediated Communication: Impersonal, Interpersonal, and Hyperpersonal Interaction. Communic Res. 1996;23(1):3–43. doi: 10.1177/009365096023001001.
    1. Eskreis-Winkler L, Milkman KL, Gromet DM, Duckworth AL. A large-scale field experiment shows giving advice improves academic outcomes for the advisor. Proc Natl Acad Sci U S A. 2019;116(30):14808–14810. doi: 10.1073/pnas.1908779116.
    1. Song H, Tucker AL, Murrell KL, Vinson DR. Closing the Productivity Gap: Improving Worker Productivity Through Public Relative Performance Feedback and Validation of Best Practices. Manage Sci. Published online June 2017. doi:10.1287/mnsc.2017.2745
    1. Meeker D, Linder JA, Fox CR, et al. Effect of Behavioral Interventions on Inappropriate Antibiotic Prescribing Among Primary Care Practices: A Randomized Clinical Trial. JAMA. 2016;315(6):562–570. doi: 10.1001/jama.2016.0275.
    1. Bond AM, Volpp KG, Emanuel EJ, et al. Real-time Feedback in Pay-for-Performance: Does More Information Lead to Improvement? J Gen Intern Med. 2019;34(9):1737–1743. doi: 10.1007/s11606-019-05004-8.
    1. Frankel RM, Tilden VP, Suchman A. Physicians’ Trust in One Another. JAMA. 2019;321(14):1345. doi: 10.1001/jama.2018.20569.
    1. Englis BG, Solomon MR. To be and not to be: Lifestyle imagery, reference groups, and the clustering of America. J Advert. 1995;24(1):13–28. doi: 10.1080/00913367.1995.10673465.
    1. Berger J, Heath C. Where Consumers Diverge from Others: Identity Signaling and Product Domains. J Consum Res. 2007;34(2):121–134. doi: 10.1086/519142.
    1. Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–340. doi: 10.1002/1098-240X(200008)23:4<334::AID-NUR9>;2-G.
    1. Soni SM, Giboney P, Yee HF., Jr Development and Implementation of Expected Practices to Reduce Inappropriate Variations in Clinical Practice. JAMA. 2016;315(20):2163–2164. doi: 10.1001/jama.2016.4255.
    1. Donner A, Klar N. Design and Analysis of Cluster Randomization Trials in Health Research. Wiley; 2010.
    1. Davidow SL, Sheth J, Sixta CS, Thomas-Hemak L. Closing the Referral Loop: Improving Ambulatory Referral Management, Electronic Health Record Connectivity, and Care Coordination Processes. J Ambul Care Manage. 2018;41(4):240–249. doi: 10.1097/JAC.0000000000000247.
    1. Patel MP, Schettini P, O’Leary CP, Bosworth HB, Anderson JB, Shah KP. Closing the Referral Loop: an Analysis of Primary Care Referrals to Specialists in a Large Health System. J Gen Intern Med. 2018;33(5):715–721. doi: 10.1007/s11606-018-4392-z.
    1. Liao JM, Navathe AS, Press MJ. Medicare’s Approach to Paying for Services That Promote Coordinated Care. JAMA. Published online December 13, 2018. doi:10.1001/jama.2018.19315
    1. EHRN. Epic Health Research Network. Accessed October 28, 2020.
    1. The Impact of the COVID-19 Pandemic on Outpatient Care: Visits Return to Prepandemic Levels, but Not for All Providers and Patients. doi:10.26099/41xy-9m57
    1. COVID-19 E-Consult Resources. Published March 24, 2020. Accessed May 23, 2021.

Source: PubMed

Sponzoři a spolupracovníci

Zdravotní podmínky

Drogové intervence

3
Předplatit