Assessment of a Novel Algorithm System for Polypharmacy Patient Medication Safety and Consultation

A Retrospective, Comparative Study, for the Assessment of a Novel System, an Algorithm for Medication Safety and Consultation for Chronic Polypharmacy Patients

The aim of this study is to determine if insights based on the analysis of historical data of multidrug patients' electronic health records, by a novel system and algorithm, is noninferior to a clinical pharmacist analysis and insights.

Multidrug patients, also known as polypharmacy patients often suffer from adverse drug reactions (ADRs). In routine practice the clinical pharmacist helps prevent ADRs by comprehensive medication review, identifying drug related risks and problems and providing recommendations. The analysis of multidrug patients is highly complex and time consuming due to the large amount of multifactorial data of chronic multidrug patients' medications, symptoms, comorbidities and age-related issues. The MDI system is a tool for analyzing and providing insights on polypharmacy data [including electronic health records (EHRs) and claims data] to help clinicians evaluate complex medical records and ensure optimal and personal treatment recommendations.

After initial training of the MDI system on historical real-life patient EHRs, the MDI system and the clinical pharmacist reviewed patient EHR data from another patient cohort according to five categories: 1) duplication of therapy, 2) age-related issues, 3) incorrect dose, 4) current side effects and 5) future side effects risks. The insights of this assessment were recorded on patient conclusion sheets and adjudicated by an external judging committee, comprised of two senior academic clinical pharmacists. The judging committee were blinded to the source of the conclusion sheets.

Diagnostic accuracy parameters: agreement, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the MDI system and the clinical pharmacist were assessed. The gold standard of the diagnostic accuracy analysis was the judging committee.

Assuming that the total agreement is 5% higher for the MDI System, with a non-inferiority margin of 5%, α level of 5%, statistical power of 90%, and an expected standard deviation of 15%, the minimum sample size is about 20 cases. The achieved recruitment level was more than twice as much in the actual clinical trial.

Study Overview

• Status: Completed
• Conditions: Polypharmacy Patients
• Intervention / Treatment: Other: Multi-drug analysis of polypharmacy treatment

Detailed Description

This is an observational, retrospective study performed on deidentified electronic health records (EHRs). The study includes two stages: A pilot study for training and verification of a novel system; and a clinical validation study for evaluating the MDI system performance in comparison to the clinical pharmacist.

The main aim of the study is to determine the noninferiority of a novel system performance in comparison to the clinical pharmacist, for the creation of insights based on the analysis of multidrug patients' EHRs. The quality and accuracy of the analysis and insights is adjudicated by an external judging committee (the gold standard), comprised of two (2) senior academic clinical pharmacists (Pharm.D, Ph.D).

The clinical site's Information systems division scanned for EHRs of multidrug patients according to extraction criteria. The raw data of the patients' records were reviewed by the principal investigator (PI). Medical coding definitions of the extraction criteria were modified according to discrepancies found by the PI. The patients' deidentified EHRs were randomized to a training cohort for the training phase and to the clinical validation cohort.

Stage 1 of the trial was for training the MDI system on real-life patient EHRs. Deidentified EHRs of the training cohort were uploaded to the MDI System platform, which calculates each patient's risk for drug induced problems according to five categories: 1) duplication of therapy, 2) age-related issues, 3) incorrect dose, 4) current side effects and 5) future side effects risks. The MDI System output report is reviewed by the clinical team. The algorithms are modified based on the system-wide conclusions and the risk calculation model.

Stage 2 of the trial was for the clinical validation of the MDI system. The MDI system and clinical pharmacists independently evaluated the clinical cohort EHRs and reported insights on conclusion sheets according to the five categories: 1) duplication of therapy, 2) age-related issues, 3) incorrect dose, 4) current side effects and 5) future side effects risks. An external judging committee comprised of two (2) senior clinical pharmacy doctors, who were blinded to the source of the conclusion sheets, adjudicated the MDI system and clinical pharmacist insights.

Agreement is defined as an identical conclusion between the MDI system/clinical pharmacist's conclusion on any of the categories / problems documented on the patient's conclusion sheet and the judging committee conclusion (the gold standard). The judges were also instructed to add any finding that was missing in their professional opinion from the documented patient conclusion sheets. Agreement of a conclusion with the judging committee's conclusion is marked as true positive. Nonagreement of a conclusion and the judging committee's conclusion is marked as false positive. A conclusion variable found missing by the judging committee is marked as false negative. True negatives are defined as agreement between a conclusion regarding a missing finding in one of the categories.

The sample size for the clinical validation of the MDI System was based on the primary endpoint of non-inferiority of the diagnostic total agreement of the MDI System compared to the diagnostic total agreement of the Clinical Pharmacists for the total of the Combined Categories. The gold standard was the Judging Committee. Assuming that the total agreement is 5% higher for the MDI System, with a non-inferiority margin of 5%, α level of 5%, statistical power of 90%, and an expected standard deviation of 15%, the minimum sample size is about 20 cases. The achieved recruitment level was more than twice as much in the actual clinical trial.

Confidence intervals of 95% (95%CI) were calculated according to the T distribution under the non-inferiority assumption that the total agreement of the MDI algorithm system will not be inferior than the agreement accuracy of the clinical pharmacist. If the 95% CI limit of the percentage differences did not exceed 5% in favor of the clinical pharmacist, the non-inferiority hypothesis was confirmed. The calculation was performed for the primary endpoint (total of the five categories) and the secondary endpoints for each individual category.

The discrete variables assessed in the categories: duplication of therapy, age-related issues and incorrect dose were medications. The discrete variables assessed in the categories: current side effects and future side effects were the individual side effects. Diagnostic accuracy of the MDI System and the Clinical Pharmacists were assessed by the Judging Committee (the "gold standard").

For each subject, total agreement, PPV, NPV, sensitivity and specificity were calculated for all the combined categories together and for each separate category. The diagnostic accuracy of the total subjects is presented as mean, 95%CI of the mean, standard deviation (Std), median, minimum and maximum.

In addition, overall by-line (regardless of subject) validity parameters (overall agreement, PPV, NPV, Sensitivity and Specificity) were analyzed using frequency, percentages and 95% CI.

• Study Type: Observational
• Enrollment (Actual): 91

Contacts and Locations

Study Locations

• Israel
  • Jerusalem, Israel, 9112001
    • Hadassah Ein Kerem

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 41 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Patients aged ≥45 years, that were prescribed at least six (6) medications on hospital admission for one of the following medical conditions: type 2 diabetes mellitus, hypertension, hyperlipidemia, ischemic heart disease, s/p cerebrovascular accident (CVA), depression, dementia, neuropathy, and related symptoms and disease states [sleep disorder, pain, diabetic neuropathy, gastrointestinal (GI) discomfort].

Patients were excluded from the study if they had one of the following disease states: oncologic disease, dialysis / end-stage renal disease (ESRD), epilepsy, ventilated and sedated, hyperparathyroidism or were breastfeeding or pregnant.

Description

Inclusion Criteria:

1. Men or women aged 45 and over (inclusive) at the time of admission to hospital. Hospitalized at Hadassah Hospital from 2010 to 2017 in one of the following departments: Internal Medicine, Cardiology, Orthopedics, Neurology, Rehabilitation.
2. Patients who have available hospitalization summary information available that contains the content required to complete the indicated fields (according to the fields listed in Appendix 1 - Patient Sheet).
3. The patient's chronic medications at admission to hospital are from the following pharmacological families / medications: angiotensin converting enzyme inhibitors (ACEI), angiotensin II receptor blockers (ARB), Beta Blockers, calcium channel blockers (CCB), Alpha blockers, Potassium-sparing diuretics, diuretics loop, diuretics thiazide, Anti platelets agents anticoagulants, Metformin Insulin, sulfonylurea (SU) / repaglinide glucagon like peptide 1 (GLP1) analogues, dipeptidyl peptidase IV (DPP-4) inhibitors, sodium-glucose transport protein 2 (SGLT2) inhibitor, statin Fibrates, benzodiazapines / Z-drugs, Mirtazapine, selective serotonin reuptake inhibitors (SSRIs) / norepinephrine reuptake inhibitors (SNRIs), tricyclic antidepressants (TCA), Antispasmodic / Anticholinergic, Antiepileplic agents) Gabapentin, Pregabalin, Primidone) , Opioids, proton-pump inhibitors (PPIs), histamine 2 (H2) blockers.

4. Take at least six (6) chronic medications from the above list at the time of admission to the hospital (the list may also include medications that have been discontinued on admission to the hospital).

   -

Exclusion Criteria:

1. Oncology patients
2. Dialysis patients
3. Epileptic patients
4. Respiratory and anesthetized
5. Breastfeeding patients
6. Pregnant patients
7. Patients with hyperparathyroidism

Study Plan

How is the study designed?

Design Details

• Observational Models: Other
• Time Perspectives: Retrospective

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Clinical pharmacists group; A group of patient files that were evaluated by the clinical pharmacists, Insights Insights were categorized according to five categories: 1) duplication of therapy, 2) age-related issues, 3) incorrect dose, 4) current side effects and 5) future side effects risks. An external judging committee comprised of two (2) senior clinical pharmacy doctors, who were blinded to the source of the conclusion sheets, adjudicated the MDI system and clinical pharmacist insights.	Other: Multi-drug analysis of polypharmacy treatment; Multi-drug analysis to assess and provide insights for lowering the risk of ADRs and increasing the benefit of polypharmacy treatment
MDI system group; A group of patient files that were evaluated by the MDI system. Insights were categorized according to five categories: 1) duplication of therapy, 2) age-related issues, 3) incorrect dose, 4) current side effects and 5) future side effects risks. An external judging committee comprised of two (2) senior clinical pharmacy doctors, who were blinded to the source of the conclusion sheets, adjudicated the MDI system and clinical pharmacist insights.	Other: Multi-drug analysis of polypharmacy treatment; Multi-drug analysis to assess and provide insights for lowering the risk of ADRs and increasing the benefit of polypharmacy treatment

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Primary Outcome Measure: Noninferiority of agreement for combined categories	The overall mean percentage of agreement between the MDI system/clinical pharmacists and the judging committee (gold standard) for the combined categories	90 days

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Secondary Outcome: Noninferiority of agreement for separate categories	The mean percentage of agreement between the MDI system/clinical pharmacists and the judging committee (gold standard) for the separate categories	90 days
Secondary Outcome: Noninferiority of PPV, NPV, sensitivity and specificity for the combined categories	PPV, NPV, sensitivity and specificity for the MDI system/clinical pharmacists according to the judging committee (gold standard) for the combined categories	90 days
Secondary Outcome: Noninferiority of PPV, NPV, sensitivity and specificity for the separate categories	PPV, NPV, sensitivity and specificity for the MDI system/clinical pharmacists according to the judging committee (gold standard) for the separate categories	90 days

Collaborators and Investigators

• Sponsor: MDI Health
• Collaborators: Hadassah Medical Organization
• Investigators: Principal Investigator: Sigal Shafran, Dr., Hadassah Medical Organization

Publications and helpful links

General Publications

• Masnoon N, Shakib S, Kalisch-Ellett L, Caughey GE. What is polypharmacy? A systematic review of definitions. BMC Geriatr. 2017 Oct 10;17(1):230. doi: 10.1186/s12877-017-0621-2.
• Parameswaran Nair N, Chalmers L, Peterson GM, Bereznicki BJ, Castelino RL, Bereznicki LR. Hospitalization in older patients due to adverse drug reactions -the need for a prediction tool. Clin Interv Aging. 2016 May 2;11:497-505. doi: 10.2147/CIA.S99097. eCollection 2016.
• Fabbri E, Zoli M, Gonzalez-Freire M, Salive ME, Studenski SA, Ferrucci L. Aging and Multimorbidity: New Tasks, Priorities, and Frontiers for Integrated Gerontological and Clinical Research. J Am Med Dir Assoc. 2015 Aug 1;16(8):640-7. doi: 10.1016/j.jamda.2015.03.013. Epub 2015 May 7.
• Johnell K, Klarin I. The relationship between number of drugs and potential drug-drug interactions in the elderly: a study of over 600,000 elderly patients from the Swedish Prescribed Drug Register. Drug Saf. 2007;30(10):911-8. doi: 10.2165/00002018-200730100-00009.
• Syrowatka A, Song W, Amato MG, Foer D, Edrees H, Co Z, Kuznetsova M, Dulgarian S, Seger DL, Simona A, Bain PA, Purcell Jackson G, Rhee K, Bates DW. Key use cases for artificial intelligence to reduce the frequency of adverse drug events: a scoping review. Lancet Digit Health. 2022 Feb;4(2):e137-e148. doi: 10.1016/S2589-7500(21)00229-6. Epub 2021 Nov 23.

Study record dates

Study Major Dates

• Study Start (Actual): February 13, 2021
• Primary Completion (Actual): July 7, 2021
• Study Completion (Actual): July 7, 2021

Study Registration Dates

• First Submitted: October 18, 2022
• First Submitted That Met QC Criteria: October 18, 2022
• First Posted (Actual): October 21, 2022

Study Record Updates

• Last Update Posted (Actual): October 23, 2023
• Last Update Submitted That Met QC Criteria: October 19, 2023
• Last Verified: October 1, 2023

More Information

Terms related to this study

• Keywords: Clinical pharmacists; Polypharmacy; Drug interactions; Side effects; Multi-morbidity; Adverse Drug Reaction; Drug safety; Drug related problems; Drug induced disease; Multi-drug treatments; Comprehensive medication reviews; Drug effects; Drug related risks
• Other Study ID Numbers: HMO-0353-19

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No