Investigating the Potential for Clinical Decision Support in Sub-Saharan Africa With AFYA (Artificial Intelligence-Based Assessment of Health Symptoms in Tanzania): Protocol for a Prospective, Observational Pilot Study

Marcel Schmude, Nahya Salim, Hila Azadzoy, Mustafa Bane, Elizabeth Millen, Lisa O'Donnell, Philipp Bode, Ewelina Türk, Ria Vaidya, Stephen Gilbert, Marcel Schmude, Nahya Salim, Hila Azadzoy, Mustafa Bane, Elizabeth Millen, Lisa O'Donnell, Philipp Bode, Ewelina Türk, Ria Vaidya, Stephen Gilbert

Abstract

Background: Low- and middle-income countries face difficulties in providing adequate health care. One of the reasons is a shortage of qualified health workers. Diagnostic decision support systems are designed to aid clinicians in their work and have the potential to mitigate pressure on health care systems.

Objective: The Artificial Intelligence-Based Assessment of Health Symptoms in Tanzania (AFYA) study will evaluate the potential of an English-language artificial intelligence-based prototype diagnostic decision support system for mid-level health care practitioners in a low- or middle-income setting.

Methods: This is an observational, prospective clinical study conducted in a busy Tanzanian district hospital. In addition to usual care visits, study participants will consult a mid-level health care practitioner, who will use a prototype diagnostic decision support system, and a study physician. The accuracy and comprehensiveness of the differential diagnosis provided by the diagnostic decision support system will be evaluated against a gold-standard differential diagnosis provided by an expert panel.

Results: Patient recruitment started in October 2021. Participants were recruited directly in the waiting room of the outpatient clinic at the hospital. Data collection will conclude in May 2022. Data analysis is planned to be finished by the end of June 2022. The results will be published in a peer-reviewed journal.

Conclusions: Most diagnostic decision support systems have been developed and evaluated in high-income countries, but there is great potential for these systems to improve the delivery of health care in low- and middle-income countries. The findings of this real-patient study will provide insights based on the performance and usability of a prototype diagnostic decision support system in low- or middle-income countries.

Trial registration: ClinicalTrials.gov NCT04958577; https://ichgcp.net/clinical-trials-registry/NCT04958577.

International registered report identifier (irrid): DERR1-10.2196/34298.

Keywords: Africa; artificial intelligence; chatbot; clinical decision support systems; decision support; diagnosis; diagnostic decision support systems; differential diagnosis; health app; low income; middle income; prototype; symptom assessment; user centered design; user centred design.

Conflict of interest statement

Conflicts of Interest: MS, HA, LO, EM, PB, ET, RV and SG are or were employees, contractors, or equity holders in Ada Health GmbH. All should be considered to have an interest in Ada Health GmbH. HA is a director of the Ada Health Foundation GmbH. The Ada Health GmbH research team has received research grant funding from Fondation Botnar and the Bill & Melinda Gates Foundation.

©Marcel Schmude, Nahya Salim, Hila Azadzoy, Mustafa Bane, Elizabeth Millen, Lisa O’Donnell, Philipp Bode, Ewelina Türk, Ria Vaidya, Stephen Gilbert. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 07.06.2022.

Figures

Figure 1
Figure 1
Screenshots showing the user interface of the tool in this study.
Figure 2
Figure 2
The patient journey in the study. Step 1: using the study tool, a differential diagnosis list is created by the study health care practitioner (clinical officer or assistant medical officer). Step 2: using a structured electronic case report form (eCRF), the patient consults with the usual health care practitioner for the determination of a diagnosis. Step 3: using a structured eCRF, the patient consults with a study physician to confirm the findings in step 2 with higher objectivity and a gold standard diagnosis. DDL: differential diagnosis list; pDDSS: prototype diagnostic decision support system.
Figure 3
Figure 3
Comparison of study arms in stage 1. pDDSS: prototype diagnostic decision support system; DDL: differential diagnosis list.
Figure 4
Figure 4
Data flowchart. DDL: differential diagnosis list; DDSS: diagnostic decision support system.

References

    1. Global health workforce shortage to reach 12.9 million in coming decades. World Health Organization. [2022-05-12]. .
    1. Di Giorgio L, Evans DK, Lindelow M, Nguyen SN, Svensson J, Wane W, Welander Tärneberg Anna. Analysis of clinical knowledge, absenteeism and availability of resources for maternal and child health: a cross-sectional quality of care study in 10 African countries. BMJ Glob Health. 2020 Dec;5(12):e003377. doi: 10.1136/bmjgh-2020-003377. bmjgh-2020-003377
    1. Service Delivery Indicators Health Survey 2014 - Harmonized Public Use Data. World Bank. [2022-05-12]. .
    1. Kruk ME, Gage AD, Joseph NT, Danaei G, García-Saisó Sebastián, Salomon JA. Mortality due to low-quality health systems in the universal health coverage era: a systematic analysis of amenable deaths in 137 countries. The Lancet. 2018 Nov 17;392(10160):2203–2212. doi: 10.1016/S0140-6736(18)31668-4. S0140-6736(18)31668-4
    1. Agarwal S, Glenton C, Tamrat T, Henschke N, Maayan N, Fønhus MS, Mehl GL, Lewin S. Decision-support tools via mobile devices to improve quality of care in primary healthcare settings. Cochrane Database Syst Rev. 2021 Jul 27;7:CD012944. doi: 10.1002/14651858.CD012944.pub2.
    1. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012 Jul 3;157(1):29–43. doi: 10.7326/0003-4819-157-1-201207030-00450.1206700
    1. Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H. Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ. 2009 Jul 7;181(1-2):37–44. doi: 10.1503/cmaj.081272. 181/1-2/37
    1. Devaraj S, Sharma SK, Fausto DJ, Viernes S, Kharrazi H. Barriers and Facilitators to Clinical Decision Support Systems Adoption: A Systematic Review. J Bus Adm Res. 2014 Jul 24;3(2):36. doi: 10.5430/jbar.v3n2p36.
    1. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi: 10.1038/s41746-020-0221-y. 221
    1. Riches N, Panagioti M, Alam R, Cheraghi-Sohi S, Campbell S, Esmail A, Bower P. The Effectiveness of Electronic Differential Diagnoses (DDX) Generators: A Systematic Review and Meta-Analysis. PLoS One. 2016 Mar 8;11(3):e0148991. doi: 10.1371/journal.pone.0148991. PONE-D-15-38539
    1. Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med. 2008 May 01;121(5 Suppl):S2–23. doi: 10.1016/j.amjmed.2008.01.001.S0002-9343(08)00040-5
    1. Graber MA, VanScoy D. How well does decision support software perform in the emergency department? Emerg Med J. 2003 Sep;20(5):426–8. doi: 10.1136/emj.20.5.426.
    1. Bond WF, Schwartz LM, Weaver KR, Levick D, Giuliano M, Graber ML. Differential diagnosis generators: an evaluation of currently available computer programs. J Gen Intern Med. 2012 Feb;27(2):213–219. doi: 10.1007/s11606-011-1804-8.
    1. Martinez-Franco AI, Sanchez-Mendiola M, Mazon-Ramirez JJ, Hernandez-Torres I, Rivero-Lopez C, Spicer T, Martinez-Gonzalez A. Diagnostic accuracy in Family Medicine residents using a clinical decision support system (DXplain): a randomized-controlled trial. Diagnosis (Berl) 2018 Jun 27;5(2):71–76. doi: 10.1515/dx-2017-0045.dx-2017-0045
    1. Evidence-Informed Policy Making in the United Republic of Tanzania: Setting REACH-policy initiative priorities for 2008 - 2010. National Institute for Medical Research. [2022-05-13]. .
    1. Kapologwe NA, Meara JG, Kengia JT, Sonda Y, Gwajima D, Alidina S, Kalolo A. Development and upgrading of public primary healthcare facilities with essential surgical services infrastructure: a strategy towards achieving universal health coverage in Tanzania. BMC Health Serv Res. 2020 Mar 17;20(1):218. doi: 10.1186/s12913-020-5057-2. 10.1186/s12913-020-5057-2
    1. Health Sector Strategic Plan July 2021 – June 2026. Mwanza Intervention Trials Unit. [2022-05-12]. .
    1. Lassi Zohra S, Cometto Giorgio, Huicho Luis, Bhutta Zulfiqar A. Quality of care provided by mid-level health workers: systematic review and meta-analysis. Bull World Health Organ. 2013 Nov 01;91(11):824–833I. doi: 10.2471/BLT.13.118786. BLT.13.118786
    1. Rick T, Moshi D. The Tanzanian assistant medical officer. JAAPA. 2018 Apr;31(4):43–47. doi: 10.1097/01.JAA.0000531051.04879.59.01720610-201804000-00008
    1. Millen E, Salim N, Azadzoy H, Bane MM, O'Donnell L, Schmude M, Bode P, Tuerk E, Vaidya R, Gilbert SH. Study protocol for a pilot prospective, observational study investigating the condition suggestion and urgency advice accuracy of a symptom assessment app in sub-Saharan Africa: the AFYA-'Health' Study. BMJ Open. 2022 Apr 11;12(4):e055915. doi: 10.1136/bmjopen-2021-055915. bmjopen-2021-055915
    1. Chan A, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, Hróbjartsson A, Mann H, Dickersin K, Berlin JA, Doré CJ, Parulekar WR, Summerskill WSM, Groves T, Schulz KF, Sox HC, Rockhold FW, Rennie D, Moher D. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013 Feb 5;158(3):200–207. doi: 10.7326/0003-4819-158-3-201302050-00583.1556168
    1. Gosbee J. Human factors engineering and patient safety. Qual Saf Health Care. 2002 Dec;11(4):352–354. doi: 10.1136/qhc.11.4.352.
    1. Health. Powered by Ada. Ada Health GmbH. [2022-05-12].
    1. Miller S, Gilbert S, Virani V, Wicks P. Patients' Utilization and Perception of an Artificial Intelligence-Based Symptom Assessment and Advice Technology in a British Primary Care Waiting Room: Exploratory Pilot Study. JMIR Hum Factors. 2020 Jul 10;7(3):e19713. doi: 10.2196/19713. v7i3e19713
    1. Cotte F, Mueller T, Gilbert S, Blümke Bibiana, Multmeier J, Hirsch MC, Wicks P, Wolanski J, Tutschkow D, Schade Brittinger C, Timmermann L, Jerrentrup A. Safety of Triage Self-assessment Using a Symptom Assessment App for Walk-in Patients in the Emergency Care Setting: Observational Prospective Cross-sectional Study. JMIR Mhealth Uhealth. 2022 Mar 28;10(3):e32340. doi: 10.2196/32340. v10i3e32340
    1. Scheder-Bieschin J, Blümke Bibiana, de Buijzer E, Cotte F, Echterdiek F, Nacsa J, Ondresik M, Ott M, Paul G, Schilling T, Schmitt A, Wicks P, Gilbert S. Improving Emergency Department Patient-Physician Conversation Through an Artificial Intelligence Symptom-Taking Tool: Mixed Methods Pilot Observational Study. JMIR Form Res. 2022 Feb 07;6(2):e28199. doi: 10.2196/28199. v6i2e28199
    1. Timiliotis J, Blümke Bibiana, Serfözö Peter Daniel, Gilbert S, Ondrésik Marta, Türk Ewelina, Hirsch MC, Eckstein J. A Novel Diagnostic Decision Support System for Medical Professionals: Prospective Feasibility Study. JMIR Form Res. 2022 Mar 24;6(3):e29943. doi: 10.2196/29943. v6i3e29943
    1. Ronicke S, Hirsch MC, Türk Ewelina, Larionov K, Tientcheu D, Wagner AD. Can a decision support system accelerate rare disease diagnosis? Evaluating the potential impact of Ada DX in a retrospective study. Orphanet J Rare Dis. 2019 Mar 21;14(1):69. doi: 10.1186/s13023-019-1040-6. 10.1186/s13023-019-1040-6
    1. Willmen T, Völkel Lukas, Ronicke S, Hirsch MC, Kaufeld J, Rychlik RP, Wagner AD. Health economic benefits through the use of diagnostic support systems and expert knowledge. BMC Health Serv Res. 2021 Sep 09;21(1):947. doi: 10.1186/s12913-021-06926-y. 10.1186/s12913-021-06926-y
    1. Hirsch MC, Ronicke S, Krusche M, Wagner AD. Rare diseases 2030: how augmented AI will support diagnosis and treatment of rare diseases in the future. Ann Rheum Dis. 2020 Jun;79(6):740–743. doi: 10.1136/annrheumdis-2020-217125. annrheumdis-2020-217125
    1. Gilbert S, Mehl A, Baluch A, Cawley C, Challiner J, Fraser H, Millen E, Montazeri M, Multmeier J, Pick F, Richter C, Türk Ewelina, Upadhyay S, Virani V, Vona N, Wicks P, Novorol C. How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs. BMJ Open. 2020 Dec 16;10(12):e040269. doi: 10.1136/bmjopen-2020-040269. bmjopen-2020-040269
    1. Gilbert S, Fenech Matthew, Upadhyay Shubhanan, Wicks Paul, Novorol Claire. Quality of condition suggestions and urgency advice provided by the Ada symptom assessment app evaluated with vignettes optimised for Australia. Aust J Prim Health. 2021 Oct;27(5):377–381. doi: 10.1071/PY21032.PY21032
    1. Ceney A, Tolond S, Glowinski A, Marks B, Swift S, Palser T. Accuracy of online symptom checkers and the potential impact on service utilisation. PLoS One. 2021 Jul 15;16(7):e0254088. doi: 10.1371/journal.pone.0254088. PONE-D-20-21627
    1. Nateqi J, Lin S, Krobath H, Gruarin S, Lutz T, Dvorak T, Gruschina A, Ortner R. [From symptom to diagnosis-symptom checkers re-evaluated : Are symptom checkers finally sufficient and accurate to use? An update from the ENT perspective] HNO. 2019 May;67(5):334–342. doi: 10.1007/s00106-019-0666-y.10.1007/s00106-019-0666-y
    1. Moreno Barriga E, Pueyo Ferrer I, Sánchez Sánchez Miquel, Martín Baranera Montserrat, Masip Utset J. [A new artificial intelligence tool for assessing symptoms in patients seeking emergency department care: the Mediktor application] Emergencias. 2017;29(6):391–396.
    1. Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci. 2011 Oct;4(5):332–7. doi: 10.1111/j.1752-8062.2011.00347.x.
    1. Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, Robson R, Thabane M, Giangregorio L, Goldsmith CH. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010 Jan 06;10:1. doi: 10.1186/1471-2288-10-1. 1471-2288-10-1
    1. Measuring digital development: Facts and figures 2021. International Telecommunication Union. [2022-05-12]. .
    1. Aranda-Jan CB, Mohutsiwa-Dibe N, Loukanova S. Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health. 2014 Feb 21;14:188. doi: 10.1186/1471-2458-14-188. 1471-2458-14-188
    1. Schwalbe N, Wahl B. Artificial intelligence and the future of global health. Lancet. 2020 May 16;395(10236):1579–1586. doi: 10.1016/S0140-6736(20)30226-9. S0140-6736(20)30226-9
    1. Bellemo V, Lim ZW, Lim G, Nguyen QD, Xie Y, Yip MYT, Hamzah H, Ho J, Lee XQ, Hsu W, Lee ML, Musonda L, Chandran M, Chipalo-Mutati G, Muma M, Tan GSW, Sivaprasad S, Menon G, Wong TY, Ting DSW. Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study. Lancet Digit Health. 2019 May;1(1):e35–e44. doi: 10.1016/S2589-7500(19)30004-4. S2589-7500(19)30004-4
    1. Kwizera A, Kissoon N, Musa N, Urayeneza O, Mujyarugamba P, Patterson AJ, Harmon L, Farmer JC, Dünser Martin W, Meier J, “Sepsis in Resource-Limited Nations” Task Force of the Surviving Sepsis Campaign A Machine Learning-Based Triage Tool for Children With Acute Infection in a Low Resource Setting. Pediatr Crit Care Med. 2019 Dec;20(12):e524–e530. doi: 10.1097/PCC.0000000000002121.00130478-201912000-00029
    1. Semigran HL, Linder JA, Gidengil C, Mehrotra A. Evaluation of symptom checkers for self diagnosis and triage: audit study. BMJ. 2015 Jul 08;351:h3480. doi: 10.1136/bmj.h3480.

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner