Natural Language Processing for Screening Opioid Misuse

The Evaluation of a Real-time Natural Language Processing Decision Support Tool for Screening Opioid Misuse With Addiction Consult Intervention for Hospitalized Adults

This is a clinical study to implement and evaluate a hospital-wide, operational intervention for a real-time natural language processing (NLP)-driven clinical decision support (CDS) tool, called Substance Misuse Algorithm for Referral to Treatment Using Artificial Intelligence (SMART-AI). The SMART-AI CDS tool will be evaluated via implementation in the UW Health electronic health record (EHR). The CDS tool is meant for screening inpatient adults for opioid misuse as part of a best practice alert to nurses and providers for addiction consult service needs.

Study Overview

• Status: Completed
• Conditions: Opioid Use Disorder; Opioid Misuse
• Intervention / Treatment: Other: Opioid Misuse Screening with an Addiction Consult Service

Detailed Description

The clinical narrative in the electronic health record (EHR) carries valuable information for predictive analytics, but its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing open-source NLP pipelines to provide interoperable and standardized CDS at the bedside for health operations.

Enterprise Analytics and Applied Data Science at UWHealth have helped the health system grow its IT infrastructure to support a data-driven Learning Health System (LHS) capable of Artificial Intelligence (AI)-augmented decision-making at the bedside.

This is a clinical study to implement and evaluate a hospital-wide, operational intervention for a real-time NLP-driven CDS tool, called Substance Misuse Algorithm for Referral to Treatment Using Artificial Intelligence (SMART-AI). The SMART-AI CDS tool will be evaluated via implementation in the UW Health electronic health record. The CDS tool is meant for screening inpatient adults for opioid misuse as part of a best practice alert to nurses and providers for addiction consult service needs. The tool is part of a quality improvement initiative with approvals from hospital committees, including the Clinical AI and Predictive Analytics Committee. The primary outcome was the percentage of inpatients who screened positive (or would have screened positive) based on the NLP CDS tool who received an addiction consult with any of the following interventions: (1) receipt of opioid use intervention or motivational interviewing (MI); (2) receipt of medication-assisted treatment (MAT); and/or (3) referral to substance use disorder treatment. The primary outcome will be reported as a percentage in the pre- and post-intervention periods and consisted of substance use screening and treatment service engagement for hospitalized patients screened for opioid misuse. Secondary outcomes included the 30-day unplanned hospital readmission rate. Criteria for unplanned hospital readmissions were adopted from the Centers for Medicare & Medicaid Services.

Substance misuse is a common problem in hospitalized patients associated with poor health outcomes, but it is not prioritized and frequently unaddressed during routine care. Current approaches for screening at UW are not done and other health systems use structured diagnostic interviews that require additional staffing and effort during clinical care. Important details about substance use are captured in the clinical notes of the electronic health record but the data are difficult to mine and analyze. Natural language processing and machine learning can be trained to identify relevant findings in the notes to automatically screen patients with substance misuse.

The Investigators trained a convolutional neural network to screen and identify alcohol misuse, opioid misuse, and non-opioid drug misuse with high accuracy using ICD diagnostic codes and admission notes collected during clinical care. The derived algorithm is called Substance Misuse Algorithm for Referral to Treatment Using Artificial Intelligence (SMART-AI). The screening tool uses methods in natural language processing to screen hospitalized patients to prioritize care focused on their substance misuse. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3922677

SMART AI is more accurate than traditional rule-based systems. It uses machine learning and real-time data feeds to continuously monitor the electronic health record (EHR) data and stratify hospitalized patients by risk of unhealthy substance use.

UWHealth is an early adopter of Artificial Intelligence (AI)-driven clinical decision support (CDS) tools and plans to implement SMART-AI for screening unhealthy opioid use. The embedded EHR workflow also allows for an automated data transformation into our health system's data warehouse for analyzing measures to support a Plan Do Study Act (PDSA) operation, an integral component to a Learning Health System (LHS).

Enterprise Analytics and Applied Data Science at UWHealth have helped the health system grow its IT infrastructure to support a data-driven LHS capable of AI-augmented decision-making at the bedside and SMART-AI is one of the first use-cases for screening using natural language processing.

Aim 1: Examine the treatment effect of SMART-AI on patient health outcomes using a pragmatic clinical rollout design.

Study Design: The day of switching on the SMART-AI tool will mark the start of the implementation period. The tool will be evaluated in a PDSA cycle over several months of rollout to examine the primary outcome of addiction consults. The SMART-AI study intervention sample consisted of all hospitalized patients who screened positive for opioid misuse from the NLP CDS tool. The primary effectiveness measure was the percentage of hospitalized patients in the NLP CDS intervention sample who were screened positive for opioid misuse and who received an intervention by the inpatient addiction consult service. A control sample was derived by retrospectively applying the NLP CDS tool to all inpatient EHR records for the two years before the present study initiation in March 2023. Hospitalized patients who screened positive retrospectively under the NLP CDS tool will form the usual care control group.

• Study Type: Observational
• Enrollment (Actual): 47502

Contacts and Locations

Study Locations

• United States
  • Wisconsin
    • Madison, Wisconsin, United States, 53792
      • University of Wisconsin Hospital (UW Health)

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Adults hospitalized at University of Wisconsin Hospital (UW Health)

Description

Inclusion Criteria:

• Adults hospitalized at University of Wisconsin Hospital (UW Health)

Exclusion Criteria:

-

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Pre-Intervention Period: Usual Care with Ad-Hoc Addiction Consults; UW Hospital launched an Addiction Medicine inpatient consult service in 1991 to address the high prevalence of substance use disorders in hospitalized adults. Currently, a single screening item queries 'marijuana or other recreational drug use,' but no formal screening process was in place specifically targeting opioid misuse. For patients at risk of an opioid use disorder, the practice was ad-hoc consultations at the discretion of the primary provider.
Post-Intervention Period: Artificial intelligence-driven clinical decision support; The technical architecture that enabled the real-time, NLP CDS tool incorporated industry-leading and emerging technological capabilities. The NLP CDS infrastructure exports the notes from the EHR, organizes them and feeds them into an NLP pipeline, inputed the processed text features into the opioid screener deep learning model, and delivered the resultant scores back to the bedside electronic health record as a best practice alert.	Other: Opioid Misuse Screening with an Addiction Consult Service; Opioid Misuse Screening with an addiction consult service for brief intervention/motivational interviewing (MI), medication assisted treatment (MAT), or referral to substance use treatment as an outpatient.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Percentage of inpatients who screened positive (or would have screened positive) based on the NLP CDS tool who received an addiction consult	Percentage of inpatients who screened positive (or would have screened positive) based on the NLP CDS tool who received an addiction consult with any of the following interventions: (1) receipt of opioid use intervention or motivational interviewing (MI); (2) receipt of medication-assisted treatment (MAT); and/or (3) referral to substance use disorder treatment.	Up to 6 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
30-day unplanned hospital readmission rate	Criteria for unplanned hospital readmissions were adopted from the Centers for Medicare & Medicaid Services.	Up to 6 months

Collaborators and Investigators

• Sponsor: University of Wisconsin, Madison
• Investigators: Principal Investigator: Majid Afshar, University of Wisconsin, Madison

Publications and helpful links

General Publications

• Afshar M, Sharma B, Dligach D, Oguss M, Brown R, Chhabra N, Thompson HM, Markossian T, Joyce C, Churpek MM, Karnik NS. Development and multimodal validation of a substance misuse algorithm for referral to treatment using artificial intelligence (SMART-AI): a retrospective deep learning study. Lancet Digit Health. 2022 Jun;4(6):e426-e435. doi: 10.1016/S2589-7500(22)00041-3.
• Sharma B, Dligach D, Swope K, Salisbury-Afshar E, Karnik NS, Joyce C, Afshar M. Publicly available machine learning models for identifying opioid misuse from the clinical notes of hospitalized patients. BMC Med Inform Decis Mak. 2020 Apr 29;20(1):79. doi: 10.1186/s12911-020-1099-y.
• Afshar M, Adelaine S, Resnik F, Mundt MP, Long J, Leaf M, Ampian T, Wills GJ, Schnapp B, Chao M, Brown R, Joyce C, Sharma B, Dligach D, Burnside ES, Mahoney J, Churpek MM, Patterson BW, Liao F. Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults. JMIR Med Inform. 2023 Apr 20;11:e44977. doi: 10.2196/44977.
• Afshar M, Resnik F, Joyce C, Oguss M, Dligach D, Burnside ES, Sullivan AG, Churpek MM, Patterson BW, Salisbury-Afshar E, Liao FJ, Goswami C, Brown R, Mundt MP. Clinical implementation of AI-based screening for risk for opioid use disorder in hospitalized adults. Nat Med. 2025 Apr 3. doi: 10.1038/s41591-025-03603-z. Online ahead of print.

Helpful Links

• Github for SMART AI

Study record dates

Study Major Dates

• Study Start (Actual): March 6, 2023
• Primary Completion (Actual): November 1, 2023
• Study Completion (Actual): December 31, 2023

Study Registration Dates

• First Submitted: February 6, 2023
• First Submitted That Met QC Criteria: February 24, 2023
• First Posted (Actual): February 27, 2023

Study Record Updates

• Last Update Posted (Actual): April 23, 2025
• Last Update Submitted That Met QC Criteria: April 18, 2025
• Last Verified: January 1, 2024

More Information

Terms related to this study

• Keywords: Quality Improvement; electronic health record (EHR); Natural Language Processing (NLP); clinical decision support (CDS)
• Additional Relevant MeSH Terms: Narcotic-Related Disorders; Mental Disorders; Substance-Related Disorders; Chemically-Induced Disorders; Opioid-Related Disorders; Physiological Effects of Drugs; Central Nervous System Depressants; Peripheral Nervous System Agents; Analgesics; Sensory System Agents; Narcotics; Analgesics, Opioid
• Other Study ID Numbers: 2022-0384; Pulmonary Medicine (Other Identifier: UW Madison)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No