Artificial intelligence-assisted clinical decision support for childhood asthma management: A randomized clinical trial

Hee Yun Seol, Pragya Shrestha, Joy Fladager Muth, Chung-Il Wi, Sunghwan Sohn, Euijung Ryu, Miguel Park, Kathy Ihrke, Sungrim Moon, Katherine King, Philip Wheeler, Bijan Borah, James Moriarty, Jordan Rosedahl, Hongfang Liu, Deborah B McWilliams, Young J Juhn, Hee Yun Seol, Pragya Shrestha, Joy Fladager Muth, Chung-Il Wi, Sunghwan Sohn, Euijung Ryu, Miguel Park, Kathy Ihrke, Sungrim Moon, Katherine King, Philip Wheeler, Bijan Borah, James Moriarty, Jordan Rosedahl, Hongfang Liu, Deborah B McWilliams, Young J Juhn

Abstract

Rationale: Clinical decision support (CDS) tools leveraging electronic health records (EHRs) have been an approach for addressing challenges in asthma care but remain under-studied through clinical trials.

Objectives: To assess the effectiveness and efficiency of Asthma-Guidance and Prediction System (A-GPS), an Artificial Intelligence (AI)-assisted CDS tool, in optimizing asthma management through a randomized clinical trial (RCT).

Methods: This was a single-center pragmatic RCT with a stratified randomization design conducted for one year in the primary care pediatric practice of the Mayo Clinic, MN. Children (<18 years) diagnosed with asthma receiving care at the study site were enrolled along with their 42 primary care providers. Study subjects were stratified into three strata (based on asthma severity, asthma care status, and asthma diagnosis) and were blinded to the assigned groups.

Measurements: Intervention was a quarterly A-GPS report to clinicians including relevant clinical information for asthma management from EHRs and machine learning-based prediction for risk of asthma exacerbation (AE). Primary endpoint was the occurrence of AE within 1 year and secondary outcomes included time required for clinicians to review EHRs for asthma management.

Main results: Out of 555 participants invited to the study, 184 consented for the study and were randomized (90 in intervention and 94 in control group). Median age of 184 participants was 8.5 years. While the proportion of children with AE in both groups decreased from the baseline (P = 0.042), there was no difference in AE frequency between the two groups (12% for the intervention group vs. 15% for the control group, Odds Ratio: 0.82; 95%CI 0.374-1.96; P = 0.626) during the study period. For the secondary end points, A-GPS intervention, however, significantly reduced time for reviewing EHRs for asthma management of each participant (median: 3.5 min, IQR: 2-5), compared to usual care without A-GPS (median: 11.3 min, IQR: 6.3-15); p<0.001). Mean health care costs with 95%CI of children during the trial (compared to before the trial) in the intervention group were lower than those in the control group (-$1,036 [-$2177, $44] for the intervention group vs. +$80 [-$841, $1000] for the control group), though there was no significant difference (p = 0.12). Among those who experienced the first AE during the study period (n = 25), those in the intervention group had timelier follow up by the clinical care team compared to those in the control group but no significant difference was found (HR = 1.93; 95% CI: 0.82-1.45, P = 0.10). There was no difference in the proportion of duration when patients had well-controlled asthma during the study period between the intervention and the control groups.

Conclusions: While A-GPS-based intervention showed similar reduction in AE events to usual care, it might reduce clinicians' burden for EHRs review resulting in efficient asthma management. A larger RCT is needed for further studying the findings.

Trial registration: ClinicalTrials.gov Identifier: NCT02865967.

Conflict of interest statement

No authors have competing interests.

Figures

Fig 1. Study design for assessing the…
Fig 1. Study design for assessing the comparative effectiveness of A-GPS intervention via an RCT with stratified randomization (see S1 File. The Design and Implementation of Asthma-Guidance and Prediction System (A-GPS) for the definition for each stratum for details).
Fig 2. A-GPS report for individual patients…
Fig 2. A-GPS report for individual patients provided to Primary Care Providers (PCPs) quarterly.
Fig 3. Consort flow diagram for subject…
Fig 3. Consort flow diagram for subject enrollment and random allocation of subjects in each stratum to intervention vs control group (see Fig 1 for randomization, intervention and follow up).
Fig 4. Comparisons of the frequency of…
Fig 4. Comparisons of the frequency of asthma exacerbation (the primary endpoint) between the intervention and control group (Fig 4A for overall intervention and control group and Fig 4B for individual strata) during the 12 months prior to the trial and the 12-month trial.
A. Comparison between overall intervention and control group. B. Comparison between intervention and control group at pre-trial and trial by stratum.

References

    1. Martin-Sanchez F, Verspoor K: Big data in medicine is driving big changes. Yearb Med Inform 2014, 9(1):14–20.
    1. Gardner RL, Cooper E, Haskell J, Harris DA, Poplau S, Kroth PJ, et al.: Physician stress and burnout: the impact of health information technology. Journal of the American Medical Informatics Association: JAMIA 2019, 26(2):106–114. doi: 10.1093/jamia/ocy145
    1. EHRs shoudl be a tool not a task []
    1. Linzer M, Bitton A, Tu SP, Plews-Ogan M, Horowitz KR, Schwartz MD, et al.: The End of the 15–20 Minute Primary Care Visit. Journal of general internal medicine 2015, 30(11):1584–1586. doi: 10.1007/s11606-015-3341-3
    1. Holmgren AJ, Downing NL, Bates DW, Shanafelt TD, Milstein A, Sharp CD, et al.: Assessment of Electronic Health Record Use Between US and Non-US Health Systems. JAMA Internal Medicine 2020.
    1. Dzau VJ, McClellan MB, McGinnis JM, Burke SP, Coye MJ, Diaz A, et al.: Vital Directions for Health and Health Care: Priorities From a National Academy of Medicine Initiative. JAMA 2017, 317(14):1461–1470. doi: 10.1001/jama.2017.1964
    1. Bell LM, Grundmeier R, Localio R, Zorc J, Fiks AG, Zhang X, et al.: Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics 2010, 125(4):e770–777. doi: 10.1542/peds.2009-1385
    1. Shiffman RN, Freudigman M, Brandt CA, Liaw Y, Navedo DD: A guideline implementation system using handheld computers for office management of asthma: effects on adherence and patient outcomes. Pediatrics 2000, 105(4 Pt 1):767–773. doi: 10.1542/peds.105.4.767
    1. McCowan C, Neville RG, Ricketts IW, Warner FC, Hoskins G, Thomas GE: Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Medical informatics and the Internet in medicine 2001, 26(3):191–201. doi: 10.1080/14639230110067890
    1. Li A, Walling J, Ahn S, Kotliarov Y, Su Q, Quezado M, et al.: Unsupervised analysis of transcriptomic profiles reveals six glioma subtypes. Cancer Res 2009, 69(5):2091–2099. doi: 10.1158/0008-5472.CAN-08-2100
    1. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al.: Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. Jama 2016, 316(22):2402–2410. doi: 10.1001/jama.2016.17216
    1. Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA: The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med 2018, 24(11):1716–1720. doi: 10.1038/s41591-018-0213-5
    1. Center for Devices and Radiological Health FaDAF: Software as a Medical Device (SAMD): Clinical Evaluation. In. Edited by (FDA) FaDA, vol. FDA-2016-D-2483; 2020.
    1. Angus DC: Randomized Clinical Trials of Artificial Intelligence. JAMA 2020, 323(11):1043–1045. doi: 10.1001/jama.2020.1039
    1. Wijnberge M, Geerts BF, Hol L, Lemmers N, Mulder MP, Berge P, et al.: Effect of a Machine Learning–Derived Early Warning System for Intraoperative Hypotension vs Standard Care on Depth and Duration of Intraoperative Hypotension During Elective Noncardiac Surgery: The HYPE Randomized Clinical Trial. JAMA 2020, 323(11):1052–1060. doi: 10.1001/jama.2020.0592
    1. Assurance NCfQ: Technical Specifications for Health Plans. The Healthcare Effectiveness Data and Information Set (HEDIS), vol. 2; 2016.
    1. National Asthma E, Prevention P: Expert Panel Report 3 (EPR-3): Guidelines for the Diagnosis and Management of Asthma-Summary Report 2007. J Allergy Clin Immunol 2007, 120(5 Suppl):S94–138.
    1. Bisgaard H, Szefler S: Prevalence of asthma-like symptoms in young children. Pediatr Pulmonol 2007, 42(8):723–728. doi: 10.1002/ppul.20644
    1. Molis WE, Bagniewski S, Weaver AL, Jacobson RM, Juhn YJ: Timeliness of diagnosis of asthma in children and its predictors. Allergy 2008, 63(11):1529–1535. doi: 10.1111/j.1398-9995.2008.01749.x
    1. Yoo KH, Johnson SK, Voigt RG, Campeau LJ, Yawn BP, Juhn YJ: Characterization of asthma status by parent report and medical record review. The Journal of allergy and clinical immunology 2007, 120(6):1468–1469. doi: 10.1016/j.jaci.2007.09.008
    1. Juhn Y, Kung A, Voigt R, Johnson S: Characterisation of children’s asthma status by ICD-9 code and criteria-based medical record review. Primary care respiratory journal: journal of the General Practice Airways Group 2011, 20(1):79–83. doi: 10.4104/pcrj.2010.00076
    1. Yunginger JW, Reed CE, O’Connell EJ, Melton L, O’Fallon WM, Silverstein MD: A community-based study of the epidemiology of asthma. Am Rev Respir Dis 1992, 146(4):888–894. doi: 10.1164/ajrccm/146.4.888
    1. Ducharme FM, Dell SD, Radhakrishnan D, Grad RM, Watson WTA, Yang CL, et al.: Diagnosis and management of asthma in preschoolers: A Canadian Thoracic Society and Canadian Paediatric Society position paper. Canadian Respiratory Journal: Journal of the Canadian Thoracic Society 2015, 22(3):135–143.
    1. Wi CI, Sohn S, Rolfes MC, Seabright A, Ryu E, Voge G, et al.: Application of a Natural Language Processing Algorithm to Asthma Ascertainment. An Automated Chart Review. American journal of respiratory and critical care medicine 2017, 196(4):430–437. doi: 10.1164/rccm.201610-2006OC
    1. Wu ST, Juhn YJ, Sohn S, Liu H: Patient-level temporal aggregation for text-based asthma status ascertainment. Journal of the American Medical Informatics Association: JAMIA 2014, 21(5):876–884. doi: 10.1136/amiajnl-2013-002463
    1. Wu ST, Sohn S, Ravikumar KE, Wagholikar K, Jonnalagadda SR, Liu H, et al.: Automated chart review for asthma cohort identification using natural language processing: an exploratory study. Annals of allergy, asthma & immunology: official publication of the American College of Allergy, Asthma, & Immunology 2013, 111(5):364–369.
    1. Yunginger JW, Reed CE, O’Connell EJ, Melton LJ, 3rd, O’Fallon WM, Silverstein MD: A community-based study of the epidemiology of asthma. Incidence rates, 1964–1983. Am Rev Respir Dis 1992, 146(4):888–894. doi: 10.1164/ajrccm/146.4.888
    1. Beard CM, Yunginger JW, Reed CE, O’Connell EJ, Silverstein MD: Interobserver variability in medical record review: an epidemiological study of asthma. J Clin Epidemiol 1992, 45(9):1013–1020. doi: 10.1016/0895-4356(92)90117-6
    1. Hunt LW Jr., Silverstein MD, Reed CE, O’Connell EJ, O’Fallon WM, Yunginger JW: Accuracy of the death certificate in a population-based study of asthmatic patients. Jama 1993, 269(15):1947–1952.
    1. Silverstein MD, Reed CE, O’Connell EJ, Melton LJ, 3rd, O’Fallon WM, Yunginger JW: Long-term survival of a cohort of community residents with asthma. N Engl J Med 1994, 331(23):1537–1541. doi: 10.1056/NEJM199412083312301
    1. Bauer BA, Reed CE, Yunginger JW, Wollan PC, Silverstein MD: Incidence and outcomes of asthma in the elderly. A population-based study in Rochester, Minnesota. Chest 1997, 111(2):303–310. doi: 10.1378/chest.111.2.303
    1. Silverstein MD, Yunginger JW, Reed CE, Petterson T, Zimmerman D, Li JT, et al.: Attained adult height after childhood asthma: effect of glucocorticoid therapy. J Allergy Clin Immunol 1997, 99(4):466–474. doi: 10.1016/s0091-6749(97)70072-1
    1. Juhn YJ, Qin R, Urm S, Katusic S, Vargas-Chanes D: The influence of neighborhood environment on the incidence of childhood asthma: a propensity score approach. J Allergy Clin Immunol 2010, 125(4):838–843 e832. doi: 10.1016/j.jaci.2009.12.998
    1. Juhn YJ, Sauver JS, Katusic S, Vargas D, Weaver A, Yunginger J: The influence of neighborhood environment on the incidence of childhood asthma: a multilevel approach. Soc Sci Med 2005, 60(11):2453–2464. doi: 10.1016/j.socscimed.2004.11.034
    1. Juhn YJ, Weaver A, Katusic S, Yunginger J: Mode of delivery at birth and development of asthma: A population-based cohort study. J Allergy Clin Immun 2005, 116(3):510–516. doi: 10.1016/j.jaci.2005.05.043
    1. Yawn BP, Yunginger JW, Wollan PC, Reed CE, Silverstein MD, Harris AG: Allergic rhinitis in Rochester, Minnesota residents with asthma: frequency and impact on health care charges. J Allergy Clin Immunol 1999, 103(1 Pt 1):54–59. doi: 10.1016/s0091-6749(99)70525-7
    1. Seol HY, Rolfes MC, Chung W, Sohn S, Ryu E, Park MA, et al.: Expert artificial intelligence-based natural language processing characterises childhood asthma. BMJ Open Respiratory Research 2020, 7(1). doi: 10.1136/bmjresp-2019-000524
    1. Seol HY, Sohn S, Liu H, Wi CI, Ryu E, Park MA, et al.: Early Identification of Childhood Asthma: The Role of Informatics in an Era of Electronic Health Records. Frontiers in pediatrics 2019, 7:113. doi: 10.3389/fped.2019.00113
    1. Juhn YJ: Risks for infection in patients with asthma (or other atopic conditions): Is asthma more than a chronic airway disease? The Journal of allergy and clinical immunology 2014, 134(2):247–257 e243. doi: 10.1016/j.jaci.2014.04.024
    1. St Sauver JL, Grossardt BR, Yawn BP, Melton LJ, 3rd, Pankratz JJ, Brue SM, et al.: Data resource profile: the Rochester Epidemiology Project (REP) medical records-linkage system. Int J Epidemiol 2012, 41(6):1614–1624. doi: 10.1093/ije/dys195
    1. St. Sauver JL, Grossardt BR, Leibson CL, Yawn BP, Melton Iii LJ, Rocca WA: Generalizability of Epidemiological Findings and Public Health Decisions: An Illustration From the Rochester Epidemiology Project. Mayo Clinic Proceedings 2012, 87(2):151–160. doi: 10.1016/j.mayocp.2011.11.009
    1. Yawn BP, Wollan P, Kurland M, Scanlon P: A longitudinal study of the prevalence of asthma in a community population of school-age children. Journal of Pediatrics 2002, 140(5):576–581.
    1. Zhong W, Finnie DM, Shah ND, Wagie AE, St. Sauver JL, Jacobson DJ, et al.: Effect of Multiple Chronic Diseases on Health Care Expenditures in Childhood. J Prim Care Community Health 2015, 6(1):2–9. doi: 10.1177/2150131914540916
    1. Busse WW, Morgan WJ, Taggart V, Togias A : Asthma outcomes workshop: Overview. The Journal of allergy and clinical immunology 2012, 129(3):S1–S8. doi: 10.1016/j.jaci.2011.12.985
    1. Lee MS, Ray KN, Mehrotra A, Giboney P, Yee HF Jr, Barnett ML: Primary care practitioners’ perceptions of electronic consult systems: A qualitative analysis. JAMA Internal Medicine 2018. doi: 10.1001/jamainternmed.2018.0738
    1. Visscher SL, Naessens JM, Yawn BP, Reinalda MS, Anderson SS, Borah BJ: Developing a standardized healthcare cost data warehouse. BMC Health Serv Res 2017, 17(1):396. doi: 10.1186/s12913-017-2327-8
    1. Schatz M, Sorkness CA, Li JT, Marcus P, Murray JJ, Nathan RA, et al.: Asthma Control Test: reliability, validity, and responsiveness in patients not previously followed by asthma specialists. J Allergy Clin Immunol 2006, 117(3):549–556. doi: 10.1016/j.jaci.2006.01.011
    1. Cajigal S, Wells KE, Peterson EL, Ahmedani BK, Yang JJ, Kumar R, et al.: Predictive Properties of the Asthma Control Test and Its Component Questions for Severe Asthma Exacerbations. The journal of allergy and clinical immunology In practice 2017, 5(1):121–127.e122. doi: 10.1016/j.jaip.2016.06.025
    1. Murphy KR, Zeiger RS, Kosinski M, Chipps B, Mellon M, Schatz M, et al.: Test for respiratory and asthma control in kids (TRACK): a caregiver-completed questionnaire for preschool-aged children. J Allergy Clin Immunol 2009, 123(4):833–839.e839. doi: 10.1016/j.jaci.2009.01.058
    1. Zeiger RS, Mellon M, Chipps B, Murphy KR, Schatz M, Kosinski M, et al.: Test for Respiratory and Asthma Control in Kids (TRACK): Clinically meaningful changes in score. The Journal of allergy and clinical immunology 2011, 128(5):983–988. doi: 10.1016/j.jaci.2011.08.010
    1. Findley S, Rosenthal M, Bryant-Stephens T, Damitz M, Lara M, Mansfield C, et al.: Community-based care coordination: practical applications for childhood asthma. Health Promot Pract 2011, 12(6 Suppl 1):52s–62s. doi: 10.1177/1524839911404231
    1. Gorin AA, Lenz EM, Cornelius T, Huedo-Medina T, Wojtanowski AC, Foster GD: Randomized Controlled Trial Examining the Ripple Effect of a Nationally Available Weight Management Program on Untreated Spouses. Obesity (Silver Spring, Md) 2018, 26(3):499–504. doi: 10.1002/oby.22098
    1. Jackson SE, Steptoe A, Wardle J: The influence of partner’s behavior on health behavior change: the English Longitudinal Study of Ageing. JAMA Intern Med 2015, 175(3):385–392. doi: 10.1001/jamainternmed.2014.7554
    1. White E, Hurlich M, Thompson RS, Woods MN, Henderson MM, Urban N, et al.: Dietary Changes Among Husbands of Participants in a Low-Fat Dietary Intervention. American Journal of Preventive Medicine 1991, 7(5):319–325.
    1. Katheryn Courville PD: How a smartphone application changes the behavior, thinking and attitudes of interprofessional team members: finding, mechanisms using realist evaluaiton. In: 2018 Clinical Informatics Conference: 2018; Scottsdale, AZ; 2018.
    1. Kaur H, Sohn S, Wi CI, Ryu E, Park MA, Bachman K, et al.: Automated chart review utilizing natural language processing algorithm for asthma predictive index. BMC pulmonary medicine 2018, 18(1):34. doi: 10.1186/s12890-018-0593-9
    1. Lindgren P, Johnson J, Williams A, Yawn B, Pratt GC: Asthma exacerbations and traffic: examining relationships using link-based traffic metrics and a comprehensive patient database. Environ Health 2016, 15(1):102. doi: 10.1186/s12940-016-0184-2
    1. Sbihi H, Koehoorn M, Tamburic L, Brauer M : Asthma Trajectories in a Population-based Birth Cohort. Impacts of Air Pollution and Greenness. American journal of respiratory and critical care medicine 2017, 195(5):607–613. doi: 10.1164/rccm.201601-0164OC

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner