Clinic Waiting Room-based Study of Swahili Language Artificial Intelligence-driven Symptom Assessments in Tanzanian Primary Health Care Facilities (AFYA)

Clinic Waiting Room-based Pilot of Swahili Language Artificial Intelligence-driven Symptom Assessments in Primary Health Care Facilities, Dar es Salaam, Tanzania

This study will evaluate the accuracy of the condition suggestions and urgency advice of the Swahili language Ada symptom assessment application (SAA), when symptoms are input by a lay-person user and a medical professional; these SAA results will then be compared to the condition suggestions and urgency advice of different tiers of doctors and a "gold standard" created by a panel.

Study Overview

• Status: Active, not recruiting
• Conditions: Neurological Conditions; Mental Health Conditions; Abdominal Pain/Gastrointestinal Issues; Lower Respiratory System Conditions; Upper Respiratory System Conditions; Ophthalmology Conditions; Orthopedic Conditions; Cardiovascular System Conditions; Genitourinary System Conditions; ENT Conditions; Dermatology Conditions; Gynecology/Obstetrics Conditions
• Intervention / Treatment: Device: Ada Waiting Room App (WRA) and Ada Diagnostic Support Application (HDA)

Detailed Description

There will be two different wings of this study: (1) where a lay-person inputs their symptoms into the Swahili language Ada symptom assessment application (SAA); (2) where the usual care doctor asks a lay-person (the patient) the same questions from the SAA and determines their own condition suggestions based off this questioning and the SAA's condition suggestions. In both wings, the patient will be seeing their usual care doctor for a diagnosis and then a study-provided physician for a diagnosis. Separately, the doctors will list a ranking of other considered condition suggestions (diagnoses) and urgency advice, which will not be shared with the patient. These condition suggestions and urgency advice will be compared to a "gold standard", which will be determined by a high level panel for each patient's case, based on the doctor notes and physical examination results of each patient's visit.

• Study Type: Observational
• Enrollment (Actual): 109

Contacts and Locations

Study Locations

• Tanzania
  • Dar Es Salaam, Tanzania
    • Mbagala Rangi Tatu Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 2 months and older (Child, Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Probability Sample

Study Population

Patients will only be included if they are waiting in the clinic waiting room of the hospital where this study will take place. Inclusion of patients will be monitored throughout the study in order to ensure recruitment of a study sample of patients with a comprehensive spectrum of symptoms constellations and conditions: this is to ensure that this pilot study tests the performance of SAA on a broad range of scenarios. Study recruitment will be carried out to a target of enrolling between 2-5 patients for 10 different body system categories (including at least one adult and one child in each category).

Description

Inclusion Criteria:

• All patients who enter the clinic and that are willing/able to provide consent will be included, except those who fall under certain conditions listed below

Exclusion Criteria:

• Patients with severe injury/illness requiring immediate treatment
• Patients with traumatic injury (many of these patients require minimal anamnesis, and it is not rational to include them in a pilot study)
• Patients incapable of completing a health assessment (e.g. due to illiteracy, mental impairment or inebriation or other incapacity).

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Wing A (Ada WRA); Patients in this group will be filling out the Ada assessment themselves and then heading to a normal consultation by first the usual care doctor and then the study-provided physician.	Device: Ada Waiting Room App (WRA) and Ada Diagnostic Support Application (HDA); This device is a medical device but will be used only observationally--this device takes a patient's symptoms into account by asking questions and gives a ranked list of likely conditions that the patient might have based on the answers to these questions. The assessment report also includes urgency advice levels, that is, which care the patient should proceed to based on the condition suggestions. The WRA is used by the patient directly and the HDA is used by the doctor to ask the questions to the patient.
Wing B (Ada HDA); Patients in this group will be asked questions from the Ada assessment by a health care practitioner and then heading to a normal consultation by first the usual care doctor and then the study-provided physician.	Device: Ada Waiting Room App (WRA) and Ada Diagnostic Support Application (HDA); This device is a medical device but will be used only observationally--this device takes a patient's symptoms into account by asking questions and gives a ranked list of likely conditions that the patient might have based on the answers to these questions. The assessment report also includes urgency advice levels, that is, which care the patient should proceed to based on the condition suggestions. The WRA is used by the patient directly and the HDA is used by the doctor to ask the questions to the patient.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy and Comprehensiveness measures of Ada WRA and HDA	The condition-suggestion accuracy and comprehensiveness of the SAA, evaluated against the gold-standard differential diagnosis determined by the review panel, reported in the context of the accuracy of the usual care health practitioner.	through study completion, an average of 4 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Comparison of usual care doctor condition suggestions	In Wing B only, while using the Ada HDA, the difference between the usual care doctor's condition suggestion accuracy and urgency advice will be compared between (1) the study arm where they determine a ranked list of condition suggestions before seeing Ada's list and then will choose a new ranked list based on viewing Ada's list, and (2) the study arm where after using the HDA, the doctor will see Ada's ranked list of condition suggestions and then choose to use Ada's full list or add some condition suggestions of their own. The accuracy of these will then also be compared to the study-provided physician and the gold standard panel.	through study completion, an average of 4 months
Urgency advice accuracy	The urgency advice accuracy of the SAA, evaluated against the gold-standard triage levels determined by the review panel, reported in the context of the accuracy of the usual care health practitioner.	through study completion, an average of 4 months
Questionnaire data insights	Qualitative data on the usability, usefulness and acceptance of the SAA	through study completion, an average of 4 months

Collaborators and Investigators

• Sponsor: Ada Health GmbH
• Collaborators: Muhimbili University of Health and Allied Sciences

Publications and helpful links

General Publications

• Schmude M, Salim N, Azadzoy H, Bane M, Millen E, O'Donnell L, Bode P, Turk E, Vaidya R, Gilbert S. Investigating the Potential for Clinical Decision Support in Sub-Saharan Africa With AFYA (Artificial Intelligence-Based Assessment of Health Symptoms in Tanzania): Protocol for a Prospective, Observational Pilot Study. JMIR Res Protoc. 2022 Jun 7;11(6):e34298. doi: 10.2196/34298.
• Millen E, Salim N, Azadzoy H, Bane MM, O'Donnell L, Schmude M, Bode P, Tuerk E, Vaidya R, Gilbert SH. Study protocol for a pilot prospective, observational study investigating the condition suggestion and urgency advice accuracy of a symptom assessment app in sub-Saharan Africa: the AFYA-'Health' Study. BMJ Open. 2022 Apr 11;12(4):e055915. doi: 10.1136/bmjopen-2021-055915.

Study record dates

Study Major Dates

• Study Start (Actual): August 3, 2021
• Primary Completion (Actual): December 17, 2021
• Study Completion (Anticipated): June 1, 2022

Study Registration Dates

• First Submitted: June 30, 2021
• First Submitted That Met QC Criteria: July 9, 2021
• First Posted (Actual): July 12, 2021

Study Record Updates

• Last Update Posted (Actual): May 17, 2022
• Last Update Submitted That Met QC Criteria: May 16, 2022
• Last Verified: May 1, 2022

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Pathologic Processes; Pain; Neurologic Manifestations; Signs and Symptoms, Digestive; Disease; Abdominal Pain
• Other Study ID Numbers: AFYA

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No