Study protocol for a pilot prospective, observational study investigating the condition suggestion and urgency advice accuracy of a symptom assessment app in sub-Saharan Africa: the AFYA-'Health' Study

Elizabeth Millen, Nahya Salim, Hila Azadzoy, Mustafa Miraji Bane, Lisa O'Donnell, Marcel Schmude, Philipp Bode, Ewelina Tuerk, Ria Vaidya, Stephen Henry Gilbert, Elizabeth Millen, Nahya Salim, Hila Azadzoy, Mustafa Miraji Bane, Lisa O'Donnell, Marcel Schmude, Philipp Bode, Ewelina Tuerk, Ria Vaidya, Stephen Henry Gilbert

Abstract

Introduction: Due to a global shortage of healthcare workers, there is a lack of basic healthcare for 4 billion people worldwide, particularly affecting low-income and middle-income countries. The utilisation of AI-based healthcare tools such as symptom assessment applications (SAAs) has the potential to reduce the burden on healthcare systems. The purpose of the AFYA Study (AI-based Assessment oF health sYmptoms in TAnzania) is to evaluate the accuracy of the condition suggestions and urgency advice provided by a user on a Swahili language Ada SAA.

Methods and analysis: This study is designed as an observational prospective clinical study. The setting is a waiting room of a Tanzanian district hospital. It will include patients entering the outpatient clinic with various conditions and age groups, including children and adolescents. Patients will be asked to use the SAA before proceeding to usual care. After usual care, they will have a consultation with a study-provided physician. Patients and healthcare practitioners will be blinded to the SAA's results. An expert panel will compare the Ada SAA's condition suggestions and urgency advice to usual care and study provided differential diagnoses and triage. The primary outcome measures are the accuracy and comprehensiveness of the Ada SAA evaluated against the gold standard differential diagnoses.

Ethics and dissemination: Ethical approval was received by the ethics committee (EC) of Muhimbili University of Health and Allied Sciences with an approval number MUHAS-REC-09-2019-044 and the National Institute for Medical Research, NIMR/HQ/R.8c/Vol. I/922. All amendments to the protocol are reported and adapted on the basis of the requirements of the EC. The results from this study will be submitted to peer-reviewed journals, local and international stakeholders, and will be communicated in editorials/articles by Ada Health.

Trial registration number: NCT04958577.

Keywords: GENERAL MEDICINE (see Internal Medicine); Health informatics; PAEDIATRICS.

Conflict of interest statement

Competing interests: EM, HA, LO, MS, PB, ET and SG are employees or company directors of Ada Health GmbH and some of the listed hold stock options in the company. RV is a former employee of Ada Health GmbH. HA is a director of the Ada Health Foundation gGmbH. The Ada Health GmbH research team has received research grant funding from Foundation Botnar and the Bill & Melinda Gates Foundation.

© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Figures

Figure 1
Figure 1
The left panel shows the Ada starting screen in English, and the right panel shows the screen in Swahili. After the starting screen, the user is guided through a series of questions about their presenting complaint and symptoms. The Ada app is currently available in seven languages.
Figure 2
Figure 2
The patient journey in the study.
Figure 3
Figure 3
Data flow chart.
Figure 4
Figure 4
Gold standard urgency advice levels.

References

    1. Chambers D, Cantrell AJ, Johnson M, et al. . Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review. BMJ Open 2019;9:e027743. 10.1136/bmjopen-2018-027743
    1. Mcdaid D, Park A-L. Online health: Untangling the web 2011.
    1. Van Riel N, Auwerx K, Debbaut P, et al. . The effect of Dr Google on doctor-patient encounters in primary care: a quantitative, observational, cross-sectional study. BJGP Open 2017;1:bjgpopen17X100833. 10.3399/bjgpopen17X100833
    1. Aboueid S, Liu RH, Desta BN, et al. . The use of artificially intelligent Self-Diagnosing digital platforms by the general public: Scoping review. JMIR Med Inform 2019;7:e13445. 10.2196/13445
    1. Meyer AND, Giardina TD, Spitzmueller C, et al. . Patient perspectives on the usefulness of an artificial Intelligence–Assisted symptom Checker: cross-sectional survey study. J Med Internet Res 2020;22:e14679. 10.2196/14679
    1. Miller S, Gilbert S, Virani V, et al. . Patients' utilization and perception of an artificial Intelligence-Based symptom assessment and advice technology in a British primary care waiting room: exploratory pilot study. JMIR Hum Factors 2020;7:e19713. 10.2196/19713
    1. Fraser H, Coiera E, Wong D. Safety of patient-facing digital symptom checkers. Lancet 2018;392:2263–4. 10.1016/S0140-6736(18)32819-8
    1. Gilbert S, Mehl A, Baluch A, et al. . How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? a clinical vignettes comparison to GPs. BMJ Open 2020;10:e040269. 10.1136/bmjopen-2020-040269
    1. Hill MG, Sim M, Mills B. The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia. Med J Aust 2020;212:514–9. 10.5694/mja2.50600
    1. Morse KE, Ostberg NP, Jones VG, et al. . Use characteristics and triage acuity of a digital symptom Checker in a large integrated health system: population-based descriptive study. J Med Internet Res 2020;22:e20549. 10.2196/20549
    1. Schwalbe N, Wahl B. Artificial intelligence and the future of global health. Lancet 2020;395:1579–86. 10.1016/S0140-6736(20)30226-9
    1. World Health Organization . Tracking universal health coverage: 2017 global monitoring report, 2017. Available: [Accessed 28 Jun 2021].
    1. World Development Report . Governance and the law. world bank, 2017. Available: [Accessed 28 Jun 2021].
    1. Weinberg J. Symptom tracker apps: the future of healthcare? Raconteur 2020.
    1. Fry MW, Saidi S, Musa A, et al. . "Even though I am alone, I feel that we are many" - An appreciative inquiry study of asynchronous, provider-to-provider teleconsultations in Turkana, Kenya. PLoS One;15:e0238806. 10.1371/journal.pone.0238806
    1. Winn AN, Somai M, Fergestrom N, et al. . Association of use of online symptom Checkers with patients' plans for seeking care. JAMA Netw Open 2019;2:e1918561. 10.1001/jamanetworkopen.2019.18561
    1. Moreno Barriga E, Pueyo Ferrer I, Sánchez Sánchez M, et al. . [A new artificial intelligence tool for assessing symptoms in patients seeking emergency department care: the Mediktor application]. Emergencias 2017;29:391–6.
    1. Semigran HL, Linder JA, Gidengil C, et al. . Evaluation of symptom checkers for self diagnosis and triage: audit study. BMJ 2015;351:h3480. 10.1136/bmj.h3480
    1. Chan A-W, Tetzlaff JM, Altman DG, et al. . SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 2013;158:200–7. 10.7326/0003-4819-158-3-201302050-00583
    1. Health . Powered by Ada. Ada. Available: [Accessed 28 Jun 2021].
    1. et alHennemann S, Kuhn S, Witthöft M. Can an App-Based symptom Checker accurately detect mental disorders: a comparative study in psychotherapy outpatients. JMIR preprints, 2021. Available: [Accessed 20 Dec 2021].
    1. Scheder-Bieschin J, Bluemke B, de BE. Improving emergency department patient-doctor conversation through an artificial intelligence symptom taking tool: an action-oriented design pilot study. medRxiv 2020.
    1. Moore CG, Carter RE, Nietert PJ, et al. . Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci 2011;4:332–7. 10.1111/j.1752-8062.2011.00347.x
    1. Thabane L, Ma J, Chu R, et al. . A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010;10:1. 10.1186/1471-2288-10-1
    1. Ronicke S, Hirsch MC, Türk E, et al. . Can a decision support system accelerate rare disease diagnosis? evaluating the potential impact of Ada DX in a retrospective study. Orphanet J Rare Dis 2019;14:69. 10.1186/s13023-019-1040-6
    1. Nateqi J, Lin S, Krobath H, et al. . Vom symptom Zur diagnose – Tauglichkeit von Symptom-Checkern. HNO 2019;67:334–42. 10.1007/s00106-019-0666-y
    1. Bellemo V, Lim ZW, Lim G, et al. . Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study. Lancet Digit Health 2019;1:e35–44. 10.1016/S2589-7500(19)30004-4
    1. Thomford NE, Bope CD, Agamah FE, et al. . Implementing artificial intelligence and digital health in resource-limited settings? top 10 lessons we learned in congenital heart defects and cardiology. OMICS 2020;24:264–77. 10.1089/omi.2019.0142
    1. Kabaria CW, Molteni F, Mandike R, et al. . Mapping intra-urban malaria risk using high resolution satellite imagery: a case study of Dar ES Salaam. Int J Health Geogr 2016;15:26. 10.1186/s12942-016-0051-y
    1. Kwizera A, Kissoon N, Musa N, et al. . A machine Learning-Based triage tool for children with acute infection in a low resource setting. Pediatr Crit Care Med 2019;20:1–30. 10.1097/PCC.0000000000002121
    1. Lee V, Dunsmuir D, Businge S, et al. . Evaluation of a digital triage platform in Uganda: a quality improvement initiative to reduce the time to antibiotic administration. PLoS One 2020;15:e0240092. 10.1371/journal.pone.0240092

Source: PubMed

Sponsorzy i współpracownicy

Warunki medyczne

Interwencje lekowe

3
Subskrybuj