Evaluation of Clinical Impacts and Costs of eHealth in Rwanda

Evaluation of the Clinical Impacts and Costs of eHealth in Rwanda Using Innovative Frameworks and Local Capacity Building

This study will estimate the impact of a suite of clinical decision-support tools on structural, process, and clinical outcomes related to HIV care. The "enhanced EMR" package under investigation will include EMR monitoring tools, data quality control procedures and support, patient reports, alerts, and reminders about patient care. This intervention will be delivered by the Ministry of Health and Rwanda Biomedical Centre and monitored by the study team led by University of Rwanda's School of Public Health and Brown University.

Study Overview

• Status: Unknown
• Conditions: Clinical Decision Support System; HIV/AIDS and Infections; Electronic Medical Records
• Intervention / Treatment: Other: Experimental: Intervention 1 (Int1); Other: Experimental: Intervention 2 (Int2); Other: Experimental: Intervention 3 (Int3)

Detailed Description

Motivation of the study A previous cross sectional analysis of the national HIV program in Rwanda, described the HIV care continuum as a "multitrajectory pathway" with many opportunities for patients to exit and return to care between diagnosis and viral suppression. The authors concluded that the weakest point in the continuum is the transition from diagnosis to linkage to care where only half of newly diagnosed patients link to care within 6 months of receiving their diagnosis.

This study also estimated that 82.2 percent of patients on ART achieve viral suppression. Overall, half of the HIV-positive population in Rwanda in 2013 was assumed to be virally suppressed. This estimate of viral suppression is based on an analysis of EMR data for a subset of 21,995 patients. Correspondence with one of the study authors clarified that 9,680 of these patients were eligible for viral load testing, and 3,066 of eligible patients had recorded viral load data. This suggests that two-thirds of patients eligible for viral load testing do not have viral load results recorded in the EMR. The study do not estimate any type of treatment failure (virologic, immunologic, clinical), and investigators are not aware of any such estimates for Rwanda.

Studies in Botswana, Malawi, Uganda, South Africa, and Cameroon found that 15 to 25 percent of patients had recorded plasma HIV RNA concentrations in excess of 400 copies per mL within 3 years of starting first-line ART. More recently, kenyan study found that among a large cohort of Kenyan patients on ART, 11.6 percent had evidence of immunological treatment failure during the 12-month study period.

In the Kenya study, investigators randomised 7 of 13 clinics using EMRs to an intervention group that received alerts and reminders about immunological treatment failure. The rate of appropriate clinical action in response to treatment failure increased from 30 percent in the control group to 54 percent in the intervention group. The authors also reported a 72 percent relative reduction in the time from the detection of treatment failure to appropriate clinical action. Investigators did not estimate the impact of the CDSS on treatment outcomes such as viral suppression and survival.

With the proposed study in Rwanda, investigators see an opportunity to use low-cost decision support tools to increase the rate of linkage to care from diagnosis, improve data quality and completeness for laboratory data such as viral load, demonstrate the efficacy of these decision support tools for prompting timely clinical intervention following treatment failure, and demonstrate that early intervention can lead to positive clinical outcomes for patients.

Intended/potential use of study findings The study findings will inform the Rwandan government on the performance, clinical impact and costs of the systems they have been implementing, and should help them decide on future eHealth investments for a variety of locations. The results will also help to inform such investments in a wide range of other low and middle income countries managing HIV and other diseases.

Design/locations Investigators will conduct a cluster-randomized trial to estimate the treatment effect of the enhanced EMR packages on structural, process, and clinical outcomes related to HIV care in Rwanda.

Research questions and outcomes

Investigators will ask four primary research questions about the effect of the decision support intervention on process, structural, and clinical outcomes:

1. Do alerts and reminders improve the linkage from HIV testing to care?

   Outcomes:

   a. Rate of linkage to care among HIV-positive patients within 3 months after diagnosis b. Time from HIV+ test result to linkage to care

2. Do alerts and reminders improve the quality and completeness of routine lab results in the EMR?

   Outcomes:

   1. Percent of patients on ART completing their 6th month of treatment who have viral load results recorded in the EMR within 2 months of this initial milestone.
   2. Percent of patients on ART who get an annual VL test and have the results recorded in the EMR within 2 months of this annual milestone.
3. Do alerts and reminders following treatment failure detected by CD4 or viral load improve clinical action?

Outcomes:

1. Percent of ART patients who have a recorded clinical action within 1 month of detected treatment failure

2. Time from treatment failure to recorded clinical action

   4. Do alerts and reminders following treatment failure detected by CD4 or viral load improve therapeutic outcomes such as viral suppression?

   Outcome:

   Percent of patients who experience treatment failure who are fully suppressed 4 months after the point of failure

   Hypotheses With the proposed study in Rwanda, investigators hypothesise that low-cost decision support tools can increase the rate of linkage to care from diagnosis, improve data quality and completeness for laboratory data such as viral load and CD4, and timely clinical intervention following treatment failure.

   Investigators will implement several levels of randomisation to answer different research questions mentioned above.

   I. Do alerts and reminders improve the linkage from HIV testing to care? Randomize included facilities to two arms: Intervention 1 (Int1) and Control (Ctrl1). Facilities assigned to the Ctrl1 will not receive any additional equipment, software tools, training or other forms of support. Facilities assigned to the enhanced package for Int1 will receive alerts and reminders to promote linkage from diagnosis to care.

   II. Do alerts and reminders improve the quality and completeness of lab results in the EMR? Randomize the Intervention 1 group into two additional arms: Intervention 2 (Int2) and Control (Ctrl2). Facilities assigned to Int2 will also receive alerts and reminders to improve lab reporting as part of their enhanced package.

   III. Do alerts and reminders following treatment failure detected by CD4 or viral load improve clinical action? Randomize the Intervention 2 group into two additional arms: Intervention 3 (Int3) or Control (Ctrl3). Facilities assigned to Int3 will also receive alerts and reminders to improve clinical response to the detection of treatment failure as part of their enhanced package.

   IV. Do alerts and reminders following treatment failure detected by CD4 or viral load improve clinical outcomes such as viral suppression? (no additional randomisation)

   Investigators believe that this cascading randomisation is needed because interventions designed to improve services at the beginning of the HIV care continuum could have downstream effects that might make it challenging to estimate the effect of each additional intervention in isolation. For instance, providing facilities with tools to improve the linkage from HIV testing to care (Int1) could improve a facility's data capture more generally and potentially improve ordering and recording of lab results (Int2), which would bias the results. Therefore, investigators propose to randomise to Int2 from within the subset of facilities assigned to Int1.

   For 90% power with alpha of 0.05, an ICC of 0.15, equal allocation to the final study arms, and 10 patients per cluster who experience treatment failure during the study, investigators could detect a shift in the percentage of patients who achieve viral suppression following treatment failure of 30 percentage points from 30% to 60%. These numbers are minimum targets and the investigators plan to enrol more sites if feasible to increase the power of the study.

   Definition of Primary Outcomes and Patient Cohorts

   1a. Rate of linkage to care among HIV-positive patients Cohort: Every new adult patient (18 or older) who tests positive for HIV from the start of the trial through month 9. Outcomes for last "enrolled" patients measured in study month 12.

   Baseline situation: a study in Rwanda reported that 50% of diagnosed cases were linked to care within 3 months.

   Impact: Shift proportion from 50% to 75%

   1. b. Time from HIV+ test result to linkage to care

      Cohort: All adults with HIV positive test results recorded in the EMR at a study facility. Same timeline as 1a.

      Endpoint: Linked to care at a study facility within 3 months (N3 N4) Baseline situation: No data Impact: 50% decrease

   2. a. percentage of ART patients have viral load results in EMR (initial)

   Cohort: Every existing ART patient who completes their 6th month of treatment from the start of the trial until study month 10. Outcomes for last "enrolled" patients measured in study month 12.

   Baseline situation: Based on data presented in one study done in Rwanda and correspondence with one of the study authors, investigators estimate that approximately two-thirds of patients eligible for viral load testing do not have viral load results recorded in the EMR.

   Impact: 30% increase

   2b. Percentage of ART patients have viral load results in EMR (annual)

   Cohort: Every existing ART patient who completes 12 months of treatment (annual) from the start of the trial until study month 10. Outcomes for last "enrolled" patients measured in study month 12.

   Baseline situation: Same as 2a Impact: 30% increase

   3a. Percentage of ART patients with treatment failure experience clinical action Cohort: Every existing ART patient who has been on ART for at least 12 months and experiences treatment failure between the start of the sub-trial and study month 11. Outcomes for last "enrolled" patients measured in study month 12.

   Baseline situation: No data Impact: 50% increase

   3b. Time from detection of treatment failure to clinical action

   Cohort: Every existing ART patient who has been on ART for at least 18 months and experiences treatment failure between the start of the trial and study month 11.

   Endpoint: Time in days from treatment failure (N6e) to recorded clinical action.

   Baseline situation: No data Impact: 50% decrease in time from treatment failure to clinical action

   4. Percentage of patients who experience treatment failure who are fully suppressed 4 months after the point of failure Cohort: Every existing ART patient who has been on ART for at least 12 months and experiences treatment failure between the start of the sub-trial and study month 8. Outcomes for last "enrolled" patients measured in study month 12.

   Baseline situation: Assumed to be 30% in power calculation Impact: 30 percentage points from 30% to 60%

   Analysis

   Investigators will analyse the data using individual-level and cluster-level approaches:

   Individual-level

   Investigators will estimate intent-to-treat (ITT) treatment effects via logistic regression of the primary outcomes on cluster assignment to treatment (see contrasts in Table 1) blocking strata, and a vector of facility-level and patient-level baseline covariates. Standard errors will be clustered at the facility-level. Investigators will run sensitivity analyses with multilevel modelling approaches. Investigators will also use Kaplan-Meier methods to calculate time-to-event; to test the null hypothesis that there is no difference between the survival curves, investigators will use the log rank test.

   Cluster-level Investigators will estimate the ITT treatment effects via ordinary least squares regression of the primary outcomes on cluster assignment to treatment (see contrasts in Table 1) blocking strata, and a vector of facility-level covariates.

   All research questions, hypotheses and study endpoints recorded here have been approved by the IRBs in Rwanda and at CDC prior to 1/1/2018.

   Data Management

   All study facilities will have EMR systems by design. Therefore, most data will be collected by facility staff via routine care procedures. To gain access to this data, investigators will create automated scripts that create a study ID for each patient and extract de-identified data from the EMR. MOH EMR specialists will review the scripts to ensure that data are properly de-identified.

• Study Type: Interventional
• Enrollment (Actual): 112
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Rwanda
  • Kigali, Rwanda, 250
    • School of Public Health

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• Is a health center with an average (3 month) monthly volume of 50-700 patients
• Is owned and operated by the public sector or faith-based institutions
• Has a power source
• Has network connectivity
• Has at least 3 computers and 1 printer

Exclusion criteria:

• District hospitals (typically with high patient volume)
• Privately owned facilities
• Facilities operated by Partners in Health (who already run a version of the intervention)
• Facilities that only offer PMTCT services
• Facilities that run OpenMRS version 1.9 (rather than 1.6)

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

6

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Intervention 1 (Int1); Facilities assigned to the enhanced package for Int1 will receive alerts and reminders to promote linkage of HIV positives from diagnosis to care.	Other: Experimental: Intervention 1 (Int1); This intervention will consist of the following additions to the EMR package. A link on the clinician's homepage to enrol a new HIV+ patient in the EMR which will open a form for (1) entering patient demographics (2) adding the contact home address or description of area, phone number (if available), (3) the peer educator contacts (4) recording the HIV+ result and date. A report will be added that is run every week to identify HIV+ patients not linked to care. The patients identified will be checked with paper records to ensure they have definitely not visited, then contacted after one, 2 weeks and 4 weeks if he/she did not show up. After two attempted contacts, if the patient is not yet linked to care he/she will be visited at home by the health facility social worker using routine home visits by health care providers.; Other Names: Alerts and reminders to improve the linkage from HIV testing to care
No Intervention: Control 1 (Ctrl1); Facilities assigned to the Ctrl1 will not receive any additional equipment, software tools, training or other forms of support.
Experimental: Intervention 2 (Int2); Randomise the Intervention 1 group into two additional arms: Intervention 2 (Int2) and Control (Ctrl2). Facilities assigned to Int2 will also receive alerts and reminders to improve lab reporting as part of their enhanced package.	Other: Experimental: Intervention 1 (Int1); This intervention will consist of the following additions to the EMR package. A link on the clinician's homepage to enrol a new HIV+ patient in the EMR which will open a form for (1) entering patient demographics (2) adding the contact home address or description of area, phone number (if available), (3) the peer educator contacts (4) recording the HIV+ result and date. A report will be added that is run every week to identify HIV+ patients not linked to care. The patients identified will be checked with paper records to ensure they have definitely not visited, then contacted after one, 2 weeks and 4 weeks if he/she did not show up. After two attempted contacts, if the patient is not yet linked to care he/she will be visited at home by the health facility social worker using routine home visits by health care providers.; Other Names: Alerts and reminders to improve the linkage from HIV testing to care; Other: Experimental: Intervention 2 (Int2); The data on availability of VL results in the EMR will come from a SQL statement to query the OpenMRS database. An alert will be fired if the patient has been enrolled for 8 months or more and does not have a viral load result in the EMR. The alert will be displayed on the patient summary and on the consult sheets, with text requesting the clinician orders a VL.; Other Names: Alerts and reminders to improve the quality and completeness of lab results in the EMR
No Intervention: Control 2 (Ctrl2); Facilities assigned to the Ctrl2 will not receive any additional equipment, software tools, training or other forms of support to improve lab reporting as part of their enhanced EMR.
Experimental: Intervention 3 (Int3); Randomise the Intervention 2 group into two additional arms: Intervention 3 (Int3) or Control (Ctrl3). Facilities assigned to Int3 will receive alerts and reminders to improve clinical response to the detection of treatment failure as part of their enhanced package.	Other: Experimental: Intervention 1 (Int1); This intervention will consist of the following additions to the EMR package. A link on the clinician's homepage to enrol a new HIV+ patient in the EMR which will open a form for (1) entering patient demographics (2) adding the contact home address or description of area, phone number (if available), (3) the peer educator contacts (4) recording the HIV+ result and date. A report will be added that is run every week to identify HIV+ patients not linked to care. The patients identified will be checked with paper records to ensure they have definitely not visited, then contacted after one, 2 weeks and 4 weeks if he/she did not show up. After two attempted contacts, if the patient is not yet linked to care he/she will be visited at home by the health facility social worker using routine home visits by health care providers.; Other Names: Alerts and reminders to improve the linkage from HIV testing to care; Other: Experimental: Intervention 2 (Int2); The data on availability of VL results in the EMR will come from a SQL statement to query the OpenMRS database. An alert will be fired if the patient has been enrolled for 8 months or more and does not have a viral load result in the EMR. The alert will be displayed on the patient summary and on the consult sheets, with text requesting the clinician orders a VL.; Other Names: Alerts and reminders to improve the quality and completeness of lab results in the EMR; Other: Experimental: Intervention 3 (Int3); The data on VL results in the EMR showing detectable virus will come from a SQL statement to query the OpenMRS database. An alert will be fired if the patient has been enrolled for at least 12 months and the VL result in the EMR shows > 1000 copies/mm3. The alert will be displayed on the patient summary and on the consult sheets requesting actions to address treatment failure (change first line medication, start second line medication, repeat VL, counselling on treatment adherence). A report will also be added to regularly check for patients with high viral load.; Other Names: Alerts and reminders following treatment failure detected by CD4 or viral load improve clinical action
No Intervention: Control (Ctrl3); Facilities assigned to Ctrl3 will not receive alerts and reminders to improve clinical response to the detection of treatment failure as part of their enhanced package.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Rate of linkage to care among HIV-positive patients	Denominator: All adults (18 or older) with HIV positive test results recorded in the EMR at a study facility. Patients who die in the time between receiving a positive test result and the outcome measurement at 3 months will be excluded. Numerator: Subset of these patients who are linked to care at a study facility within 3 months	12 months
Percentage of ART patients have viral load results in EMR (initial)	Denominator: Adult patients on ART completing their 6th month of treatment, thus becoming eligible for viral load monitoring. Numerator: Subset of these patients with VL results in the EMR 2 months after becoming eligible for testing	10 months
Percentage of ART patients with treatment failure experience clinical action	Denominator: Adult patients who have been on ART for at least 12 months and experience treatment failure: Virologic (viral load ≥ 1000 copies/ml); Immunological (>50% change in CD4 from highest previous value) Numerator: Subset of these patients who have a recorded clinical action in response to treatment failure within 1 month of the detected treatment failure.	12 months
Percentage of patients who experience treatment failure who are fully suppressed 4 months after the point of failure	Denominator: Adult patients who have been on ART for at least 12 months (first eligible for VL testing at 6 months, first expected result 8 months, retest after 4 months) and were found to have possible treatment failure. Numerator: Subset of these patients who are fully suppressed (viral load < 1000 copies /ml) 4 months after the point of treatment failure.	12 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Time from HIV+ test result to linkage to care	All adults with HIV positive test results recorded in the EMR at a study who are linked to care at a study facility within 3 months	3 months
Percentage of ART patients have viral load results in EMR (annual)	Denominator: Adult patients on ART with at least 12th months of treatment, thus becoming eligible for annual viral load monitoring. Numerator: Subset of these patients with VL results in the EMR 2 months after becoming eligible for testing	12 months
Time from detection of treatment failure to clinical action	Every existing ART patient who has been on ART for at least 18 months and experiences treatment failure between the start of the trial and study month 11	11 months

Collaborators and Investigators

• Sponsor: National University, Rwanda
• Collaborators: University of Pittsburgh; Brown University; Centers for Disease Control and Prevention; Ministry of Health, Rwanda; Partners in Health; Rwanda Biomedical Centre; Innovative Support to Emergencies Diseases and Disasters; Jembi Health Systems
• Investigators: Principal Investigator: Fraser HAMISH, MBChB, Brown University: hamish_fraser@brown.edu; Principal Investigator: Jeanine CONDO, MD, PhD, University of Rwanda

Publications and helpful links

General Publications

• Rosen S, Fox MP. Retention in HIV care between testing and treatment in sub-Saharan Africa: a systematic review. PLoS Med. 2011 Jul;8(7):e1001056. doi: 10.1371/journal.pmed.1001056. Epub 2011 Jul 19.
• Allen C, Jazayeri D, Miranda J, Biondich PG, Mamlin BW, Wolfe BA, Seebregts C, Lesh N, Tierney WM, Fraser HS. Experience in implementing the OpenMRS medical record system to support HIV treatment in Rwanda. Stud Health Technol Inform. 2007;129(Pt 1):382-6.
• Amoroso CL, Akimana B, Wise B, Fraser HS. Using electronic medical records for HIV care in rural Rwanda. Stud Health Technol Inform. 2010;160(Pt 1):337-41.
• Driessen J, Cioffi M, Alide N, Landis-Lewis Z, Gamadzi G, Gadabu OJ, Douglas G. Modeling return on investment for an electronic medical record system in Lilongwe, Malawi. J Am Med Inform Assoc. 2013 Jul-Aug;20(4):743-8. doi: 10.1136/amiajnl-2012-001242. Epub 2012 Nov 9.
• Mamlin BW, Biondich PG, Wolfe BA, Fraser H, Jazayeri D, Allen C, Miranda J, Tierney WM. Cooking up an open source EMR for developing countries: OpenMRS - a recipe for successful collaboration. AMIA Annu Symp Proc. 2006;2006:529-33.
• Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C, Abu-Hanna A, de Keizer N. The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: a systematic review. Int J Med Inform. 2012 Oct;81(10):e83-92. doi: 10.1016/j.ijmedinf.2012.07.010. Epub 2012 Aug 24.
• Oluoch T, Katana A, Kwaro D, Santas X, Langat P, Mwalili S, Muthusi K, Okeyo N, Ojwang JK, Cornet R, Abu-Hanna A, de Keizer N. Effect of a clinical decision support system on early action on immunological treatment failure in patients with HIV in Kenya: a cluster randomised controlled trial. Lancet HIV. 2016 Feb;3(2):e76-84. doi: 10.1016/S2352-3018(15)00242-8. Epub 2015 Dec 17.
• Nsanzimana S, Kanters S, Remera E, Forrest JI, Binagwaho A, Condo J, Mills EJ. HIV care continuum in Rwanda: a cross-sectional analysis of the national programme. Lancet HIV. 2015 May;2(5):e208-15. doi: 10.1016/S2352-3018(15)00024-7. Epub 2015 Mar 27.

Study record dates

Study Major Dates

• Study Start (Actual): September 15, 2018
• Primary Completion (Anticipated): July 15, 2020
• Study Completion (Anticipated): July 30, 2020

Study Registration Dates

• First Submitted: February 21, 2020
• First Submitted That Met QC Criteria: February 21, 2020
• First Posted (Actual): February 25, 2020

Study Record Updates

• Last Update Posted (Actual): February 28, 2020
• Last Update Submitted That Met QC Criteria: February 26, 2020
• Last Verified: February 1, 2020

More Information

Terms related to this study

• Additional Relevant MeSH Terms: RNA Virus Infections; Virus Diseases; Infections; Blood-Borne Infections; Communicable Diseases; Sexually Transmitted Diseases, Viral; Sexually Transmitted Diseases; Lentivirus Infections; Retroviridae Infections; Immunologic Deficiency Syndromes; Immune System Diseases; Slow Virus Diseases; HIV Infections; Acquired Immunodeficiency Syndrome
• Other Study ID Numbers: GH000782

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Public data sets will be accessible and shared to anyone who needs them upon writing request letter to the RBC HIV division through Principal Investigator and get approval written letter to access data.
• IPD Sharing Time Frame: 1 month
• IPD Sharing Access Criteria: Interest in working on similar area; Writing a letter of request; Sign data sharing agreement
• IPD Sharing Supporting Information Type: SAP; ANALYTIC_CODE

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No