Using natural language processing of clinical text to enhance identification of opioid-related overdoses in electronic health records data

Abstract

Purpose: To enhance automated methods for accurately identifying opioid-related overdoses and classifying types of overdose using electronic health record (EHR) databases.

Methods: We developed a natural language processing (NLP) software application to code clinical text documentation of overdose, including identification of intention for self-harm, substances involved, substance abuse, and error in medication usage. Using datasets balanced with cases of suspected overdose and records of individuals at elevated risk for overdose, we developed and validated the application using Kaiser Permanente Northwest data, then tested portability of the application using Kaiser Permanente Washington data. Datasets were chart-reviewed to provide a gold standard for comparison and evaluation of the automated method.

Results: The method performed well in identifying overdose (sensitivity = 0.80, specificity = 0.93), intentional overdose (sensitivity = 0.81, specificity = 0.98), and involvement of opioids (excluding heroin, sensitivity = 0.72, specificity = 0.96) and heroin (sensitivity = 0.84, specificity = 1.0). The method performed poorly at identifying adverse drug reactions and overdose due to patient error and fairly at identifying substance abuse in opioid-related unintentional overdose (sensitivity = 0.67, specificity = 0.96). Evaluation using validation datasets yielded significant reductions, in specificity and negative predictive values only, for many classifications mentioned above. However, these measures remained above 0.80, thus, performance observed during development was largely maintained during validation. Similar results were obtained when evaluating portability, although there was a significant reduction in sensitivity for unintentional overdose that was attributed to missing text clinical notes in the database.

Conclusions: Methods that process text clinical notes show promise for improving accuracy and fidelity at identifying and classifying overdoses according to type using EHR data.

Trial registration: ClinicalTrials.gov NCT02667197.

Keywords: electronic health records; methods; natural language processing; opioid overdose; pharmacoepidemiology.

Conflict of interest statement

This project was conducted as part of a Food and Drug Administration (FDA)‐required postmarketing study of extended‐release and long‐acting opioid analgesics (https://www.fda.gov/downloads/Drugs/DrugSafety/InformationbyDrugClass/UCM484415.pdf), funded by the Opioid Postmarketing Consortium (OPC). The OPC is comprised of companies that hold NDAs of extended‐release and long‐acting opioid analgesics and at the time of study conduct included the following companies: Allergan; Assertio Therapeutics, Inc.; BioDelivery Sciences, Inc.; Collegium Pharmaceutical, Inc.; Daiichi Sankyo, Inc.; Egalet Corporation; Endo Pharmaceuticals, Inc.; Hikma Pharmaceuticals USA Inc.; Janssen Pharmaceuticals, Inc.; Mallinckrodt Inc.; Pernix Therapeutics Holdings, Inc.; Pfizer, Inc.; Purdue Pharma, LP.

© 2019 The Authors Pharmacoepidemiology and Drug Safety Published by John Wiley & Sons Ltd.

Figures

Figure 1

The overdose rule tree. A “rule tree” specifies a complex set of constraints that must be matched in the encounter data (text data within a single note, in this case) to generate a positive classification using the MediClass system. A rule tree is rooted by a single rule. A rule tree can be used to define a class alone or in combination with other rule trees.Each node in the tree is either a single rule (marked with a version number and shown in bold font) or it is one or more unified medical language system (UMLS) concepts (shown as plain font labels with no version numbers).Terms (not shown here) are child nodes of concepts, which help define how a concept is matched, using linguistic manipulations, against a sequence of tokens found in the text data. Terms are provided by the UMLS, as well as by custom additions found through trial and error in the development process, and constitute a lexicon of clinical expressions grouped by the concepts that they represent. Also not shown are proximity and ordering constraints, which govern relationships between concepts that are grouped by a rule. For example, a proximity constraint enforces a maximum allowable distance between any tokens (linguistic primitives of the text note) that participate in the identification of concepts within a rule.Every rule has one or more child nodes—each child node is connected to its parent by either the Boolean AND relation (shown with a solid line) or the Boolean OR relation (shown with a dashed line). For the parent node to match the data, all of the AND children and at least one of the OR children must match the data.Rules can take the following modifier: ! = Boolean NOT (ie, the reported match status of the rule is inverted from what is determined by normal match criteria for the rule). Concepts can take the following modifiers, which define constraints on how terms are matched in the text data: [−] = “negated form only” (ie, only terms of the concept asserted as negative in the text will create a match) [+] = “positive form only” (ie, only terms of the concept not asserted as negative in the text will create a match)