Effect of Smartphone App-Based Education on Clinician Prescribing Habits in a Learning Health Care System: A Randomized Cluster Crossover Trial

Matthew D McEvoy, Mary Lynn Dear, Reagan Buie, David A Edwards, Tyler W Barrett, Brian Allen, Amy C Robertson, Leslie C Fowler, Cassandra Hennessy, Bonnie M Miller, Kim V Garvey, Robert P Bland 3rd, Geoffrey M Fleming, Don Moore, Todd W Rice, Gordon R Bernard, Christopher J Lindsell, Vanderbilt Learning Healthcare System Platform Investigators and the Vanderbilt Committee on Opioid Monitoring and Stewardship, Robert Dittus, Shon Dwyer, Robert Freundlich, Cheryl Gatto, Frank Harrell, Paul Harris, Tina Hartert, Jim Hayman, Catherine Ivory, Kevin Johnson, Ruth Kleinpell, Sunil Kripalani, Lee Ann Liska, Patrick Luther, Jay Morrison, Thomas Nantais, Mariann Piano, Jill Pulley, Kris Rehm, Russell Rothman, Matthew Semler, Robin Steaban, Consuelo Wilkins, Adam Wright, Autumn Zuckerman, Michael Costello, R Scott Frankenfield, Alexander Hawkins, Sterling Haring, Jason Hessler, Elizabeth Huggins, Carin McAbee, Puneet Mishra, Hayley Rector, Jennifer Slayton, Irving Zamora, Matthew D McEvoy, Mary Lynn Dear, Reagan Buie, David A Edwards, Tyler W Barrett, Brian Allen, Amy C Robertson, Leslie C Fowler, Cassandra Hennessy, Bonnie M Miller, Kim V Garvey, Robert P Bland 3rd, Geoffrey M Fleming, Don Moore, Todd W Rice, Gordon R Bernard, Christopher J Lindsell, Vanderbilt Learning Healthcare System Platform Investigators and the Vanderbilt Committee on Opioid Monitoring and Stewardship, Robert Dittus, Shon Dwyer, Robert Freundlich, Cheryl Gatto, Frank Harrell, Paul Harris, Tina Hartert, Jim Hayman, Catherine Ivory, Kevin Johnson, Ruth Kleinpell, Sunil Kripalani, Lee Ann Liska, Patrick Luther, Jay Morrison, Thomas Nantais, Mariann Piano, Jill Pulley, Kris Rehm, Russell Rothman, Matthew Semler, Robin Steaban, Consuelo Wilkins, Adam Wright, Autumn Zuckerman, Michael Costello, R Scott Frankenfield, Alexander Hawkins, Sterling Haring, Jason Hessler, Elizabeth Huggins, Carin McAbee, Puneet Mishra, Hayley Rector, Jennifer Slayton, Irving Zamora

Abstract

Importance: Effective methods for engaging clinicians in continuing education for learning-based practice improvement remain unknown.

Objective: To determine whether a smartphone-based app using spaced education with retrieval practice is an effective method to increase evidence-based practice.

Design, setting, and participants: A prospective, unblinded, single-center, crossover randomized clinical trial was conducted at a single academic medical center from January 6 to April 24, 2020. Vanderbilt University Medical Center clinicians prescribing intravenous fluids were invited to participate in this study.

Interventions: All clinicians received two 4-week education modules: 1 on prescribing intravenous fluids and 1 on prescribing opioid and nonopioid medications (counterbalancing measure), over a 12-week period. The order of delivery was randomized 1:1 such that 1 group received the fluid management module first, followed by the pain management module after a 4-week break, and the other group received the pain management module first, followed by the fluid management module after a 4-week break.

Main outcomes and measures: The primary outcome was evidence-based clinician prescribing behavior concerning intravenous fluids in the inpatient setting and pain medication prescribing on discharge from the hospital.

Results: A total of 354 participants were enrolled and randomized, with 177 in group 1 (fluid then pain management education) and 177 in group 2 (pain management then fluid education). During the overall study period, 16 868 questions were sent to 349 learners, with 11 783 (70.0%) being opened: 10 885 (92.4%) of those opened were answered and 7175 (65.9%) of those answered were answered correctly. The differences between groups changed significantly over time, indicated by the significant interaction between educational intervention and time (P = .002). Briefly, at baseline evidence-concordant IV fluid ordered 7.2% less frequently in group 1 than group 2 (95% CI, -19.2% to 4.9%). This was reversed after training at 4% higher (95% CI, -8.2% to 16.0%) in group 1 than group 2, a more than doubling in the odds of evidence-concordant ordering (OR, 2.56, 95% CI, 0.80-8.21). Postintervention, all gains had been reversed with less frequent ordering in group 1 than group 2 (-9.5%, 95% CI, -21.6% to 2.7%). There was no measurable change in opioid prescribing behaviors at any time point.

Conclusions and relevance: In this randomized clinical trial, use of smartphone app learning modules resulted in statistically significant short-term improvement in some prescribing behaviors. However, this effect was not sustained over the long-term. Additional research is needed to understand how to sustain improvements in care delivery as a result of continuous professional development at the institutional level.

Trial registration: ClinicalTrials.gov Identifier: NCT03771482.

Conflict of interest statement

Conflict of Interest Disclosures: Dr McEvoy has received consulting fees from Takeda Pharmaceuticals for service on a scientific advisory board unrelated to the topic of this study and research funding from Edwards Lifesciences, Cheetah Medical, GE Foundation, and the Tennessee Department of Health, all unrelated to this study. Dr Barrett is partially funded by the National Institutes of Health National Institute on Drug Abuse CTN0099 ED INNOVATION study and the Centers for Disease Control and Prevention and Tennessee Department of Health Services unrelated to the topic of this study. Dr. Rice has received consulting fees from Cumberland Pharmaceuticals Inc as the director of Medical Affairs, Sanofi Inc as a member of a data safety monitoring board, and Cytovale Inc as a consultant, all unrelated to the topic of this study.

Figures

Figure 1.. Study Design
Figure 1.. Study Design
Group 1 received the fluid management module followed by the pain management module. Group 2 received the pain management module followed by the fluid management module.
Figure 2.. Participant Flow Diagram
Figure 2.. Participant Flow Diagram
All clinicians were analyzed by their treatment assignment.

References

    1. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510-520. doi:10.1258/jrsm.2011.110180
    1. Rothman BS, Gupta RK, McEvoy MD. Mobile technology in the perioperative arena: rapid evolution and future disruption. Anesth Analg. 2017;124(3):807-818. doi:10.1213/ANE.0000000000001858
    1. McEvoy MD, Dear ML, Buie R, et al. ; Vanderbilt Learning Healthcare System Investigators . Embedding learning in a learning health care system to improve clinical practice. Acad Med. 2021;96(9):1311-1314. doi:10.1097/ACM.0000000000003969
    1. Kerfoot BP, Baker H. An online spaced-education game for global continuing medical education: a randomized trial. Ann Surg. 2012;256(1):33-38. doi:10.1097/SLA.0b013e31825b3912
    1. Al-Azri H, Ratnapalan S. Problem-based learning in continuing medical education: review of randomized controlled trials. Can Fam Physician. 2014;60(2):157-165.
    1. Shaw T, Long A, Chopra S, Kerfoot BP. Impact on clinical behavior of face-to-face continuing medical education blended with online spaced education: a randomized controlled trial. J Contin Educ Health Prof. 2011;31(2):103-108. doi:10.1002/chp.20113
    1. Curran VR, Fleet LJ, Kirby F. A comparative evaluation of the effect of Internet-based CME delivery format on satisfaction, knowledge and confidence. BMC Med Educ. 2010;10:10. doi:10.1186/1472-6920-10-10
    1. Mazzoleni MC, Rognoni C, Finozzi E, et al. . Usage and effectiveness of e-learning courses for continuous medical education. Stud Health Technol Inform. 2009;150:921-925.
    1. Casebeer L, Engler S, Bennett N, et al. . A controlled trial of the effectiveness of internet continuing medical education. BMC Med. 2008;6:37. doi:10.1186/1741-7015-6-37
    1. Sargeant J, Curran V, Jarvis-Selinger S, et al. . Interactive on-line continuing medical education: physicians’ perceptions and experiences. J Contin Educ Health Prof. 2004;24(4):227-236. doi:10.1002/chp.1340240406
    1. Roediger HL III, Butler AC. The critical role of retrieval practice in long-term retention. Trends Cogn Sci. 2011;15(1):20-27. doi:10.1016/j.tics.2010.09.003
    1. Moore DE Jr, Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1-15. doi:10.1002/chp.20001
    1. Tabibian B, Upadhyay U, De A, Zarezade A, Schölkopf B, Gomez-Rodriguez M. Enhancing human learning via spaced repetition optimization. Proc Natl Acad Sci U S A. 2019;116(10):3988-3993. doi:10.1073/pnas.1815156116
    1. Weidman J, Baker K. The cognitive science of learning: concepts and strategies for the educator and learner. Anesth Analg. 2015;121(6):1586-1599. doi:10.1213/ANE.0000000000000890
    1. Martinelli SM, Isaak RS, Schell RM, Mitchell JD, McEvoy MD, Chen F. Learners and Luddites in the twenty-first century: bringing evidence-based education to anesthesiology. Anesthesiology. 2019;131(4):908-928. doi:10.1097/ALN.0000000000002827
    1. Phillips JL, Heneka N, Bhattarai P, Fraser C, Shaw T. Effectiveness of the spaced education pedagogy for clinicians’ continuing professional development: a systematic review. Med Educ. 2019;53(9):886-902. doi:10.1111/medu.13895
    1. Larsen DP, Butler AC, Roediger HL III. Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Med Educ. 2009;43(12):1174-1181. doi:10.1111/j.1365-2923.2009.03518.x
    1. Cervero RM, Gaines JK. The impact of CME on physician performance and patient health outcomes: an updated synthesis of systematic reviews. J Contin Educ Health Prof. 2015;35(2):131-138. doi:10.1002/chp.21290
    1. Kerfoot BP, Lawler EV, Sokolovskaya G, Gagnon D, Conlin PR. Durable improvements in prostate cancer screening from online spaced education a randomized controlled trial. Am J Prev Med. 2010;39(5):472-478. doi:10.1016/j.amepre.2010.07.016
    1. Magarik M, Fowler LC, Robertson A, Ehrenfeld JM, McEvoy MD, Deitte LA. There’s an app for that: a case study on the impact of spaced education on ordering CT examinations. J Am Coll Radiol. 2019;16(3):360-364. doi:10.1016/j.jacr.2018.10.024
    1. Kerfoot BP, Baker HE, Koch MO, Connelly D, Joseph DB, Ritchey ML. Randomized, controlled trial of spaced education to urology residents in the United States and Canada. J Urol. 2007;177(4):1481-1487. doi:10.1016/j.juro.2006.11.074
    1. Kerfoot BP, DeWolf WC, Masser BA, Church PA, Federman DD. Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial. Med Educ. 2007;41(1):23-31. doi:10.1111/j.1365-2929.2006.02644.x
    1. Kerfoot BP, Armstrong EG, O’Sullivan PN. Interactive spaced-education to teach the physical examination: a randomized controlled trial. J Gen Intern Med. 2008;23(7):973-978. doi:10.1007/s11606-008-0533-0
    1. Kerfoot BP. Learning benefits of on-line spaced education persist for 2 years. J Urol. 2009;181(6):2671-2673. doi:10.1016/j.juro.2009.02.024
    1. Matzie KA, Kerfoot BP, Hafler JP, Breen EM. Spaced education improves the feedback that surgical residents give to medical students: a randomized trial. Am J Surg. 2009;197(2):252-257. doi:10.1016/j.amjsurg.2008.01.025
    1. Kerfoot BP, Baker H. An online spaced-education game to teach and assess residents: a multi-institutional prospective trial. J Am Coll Surg. 2012;214(3):367-373. doi:10.1016/j.jamcollsurg.2011.11.009
    1. Kerfoot BP, Turchin A, Breydo E, Gagnon D, Conlin PR. An online spaced-education game among clinicians improves their patients’ time to blood pressure control: a randomized controlled trial. Circ Cardiovasc Qual Outcomes. 2014;7(3):468-474. doi:10.1161/CIRCOUTCOMES.113.000814
    1. Barrett TW, McEvoy MD, Fowler LC, et al. . Impact of an asynchronous spaced education learning intervention on emergency medicine clinician opioid prescribing. Cureus. 2021;13(9):e18165. doi:10.7759/cureus.18165
    1. Semler MW, Self WH, Wanderer JP, et al. ; SMART Investigators and the Pragmatic Critical Care Research Group . Balanced crystalloids versus saline in critically ill adults. N Engl J Med. 2018;378(9):829-839. doi:10.1056/NEJMoa1711584
    1. Kooken J, Welsh ME, McCoach DB, et al. . Test order in teacher-rated behavior assessments: is counterbalancing necessary? Psychol Assess. 2017;29(1):98-109. doi:10.1037/pas0000314
    1. Edwards DA, Hedrick TL, Jayaram J, et al. ; POQI-4 Working Group . American Society for Enhanced Recovery and Perioperative Quality Initiative joint consensus statement on perioperative management of patients on preoperative opioid therapy. Anesth Analg. 2019;129(2):553-566. doi:10.1213/ANE.0000000000004018
    1. Self WH, Semler MW, Wanderer JP, et al. ; SALT-ED Investigators . Balanced crystalloids versus saline in noncritically ill adults. N Engl J Med. 2018;378(9):819-828. doi:10.1056/NEJMoa1711586
    1. Kent ML, Hurley RW, Oderda GM, et al. ; POQI-4 Working Group . American Society for Enhanced Recovery and Perioperative Quality Initiative-4 joint consensus statement on persistent postoperative opioid use: definition, incidence, risk factors, and health care system initiatives. Anesth Analg. 2019;129(2):543-552. doi:10.1213/ANE.0000000000003941
    1. Wu CL, King AB, Geiger TM, et al. ; Fourth Perioperative Quality Initiative Workgroup . American Society for Enhanced Recovery and Perioperative Quality Initiative joint consensus statement on perioperative opioid minimization in opioid-naïve patients. Anesth Analg. 2019;129(2):567-577. doi:10.1213/ANE.0000000000004194
    1. Chawla LS, Bellomo R, Bihorac A, et al. ; Acute Disease Quality Initiative Workgroup 16. . Acute kidney disease and renal recovery: consensus report of the Acute Disease Quality Initiative (ADQI) 16 Workgroup. Nat Rev Nephrol. 2017;13(4):241-257. doi:10.1038/nrneph.2017.2
    1. Chawla LS, Ince C, Chappell D, et al. ; ADQI XII Fluids Workgroup . Vascular content, tone, integrity, and haemodynamics for guiding fluid therapy: a conceptual approach. Br J Anaesth. 2014;113(5):748-755. doi:10.1093/bja/aeu298
    1. Shaw AD, Raghunathan K, Peyerl FW, Munson SH, Paluszkiewicz SM, Schermer CR. Association between intravenous chloride load during resuscitation and in-hospital mortality among patients with SIRS. Intensive Care Med. 2014;40(12):1897-1905. doi:10.1007/s00134-014-3505-3
    1. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010
    1. Miller BM, Moore DE Jr, Stead WW, Balser JR. Beyond Flexner: a new model for continuous learning in the health professions. Acad Med. 2010;85(2):266-272. doi:10.1097/ACM.0b013e3181c859fb
    1. Lindsell CJ, Gatto CL, Dear ML, et al. . Learning from what we do, and doing what we learn: a learning health care system in action. Acad Med. 2021;96(9):1291-1299. doi:10.1097/ACM.0000000000004021
    1. Milani RV, Lavie CJ, Dornelles AC. The impact of achieving perfect care in acute coronary syndrome: the role of computer assisted decision support. Am Heart J. 2012;164(1):29-34. doi:10.1016/j.ahj.2012.04.004
    1. Cox ZL, Nelsen CL, Waitman LR, McCoy JA, Peterson JF. Effects of clinical decision support on initial dosing and monitoring of tobramycin and amikacin. Am J Health Syst Pharm. 2011;68(7):624-632. doi:10.2146/ajhp100155
    1. Ehrenfeld JM, McEvoy MD, Furman WR, Snyder D, Sandberg WS. Automated near-real-time clinical performance feedback for anesthesiology residents: one piece of the milestones puzzle. Anesthesiology. 2014;120(1):172-184. doi:10.1097/ALN.0000000000000071
    1. Ivers N, Jamtvedt G, Flottorp S, et al. . Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6(6):CD000259. doi:10.1002/14651858.CD000259.pub3
    1. Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867-874. doi:10.1001/jama.282.9.867
    1. Auerbach L, Santen SA, Cutrer WB, Daniel M, Wilson-Delfosse AL, Roberts NK. The educators’ experience: Learning environments that support the master adaptive learner. Med Teach. 2020;42(11):1270-1274. doi:10.1080/0142159X.2020.1801998

Source: PubMed

Szponzorok és közreműködők

Egészségi állapot

Kábítószer-beavatkozások

3
Iratkozz fel