Identifying and classifying opioid-related overdoses: A validation study

Carla A Green, Nancy A Perrin, Brian Hazlehurst, Shannon L Janoff, Angela DeVeaugh-Geiss, David S Carrell, Carlos G Grijalva, Caihua Liang, Cheryl L Enger, Paul M Coplan, Carla A Green, Nancy A Perrin, Brian Hazlehurst, Shannon L Janoff, Angela DeVeaugh-Geiss, David S Carrell, Carlos G Grijalva, Caihua Liang, Cheryl L Enger, Paul M Coplan

Abstract

Purpose: The study aims to develop and validate algorithms to identify and classify opioid overdoses using claims and other coded data, and clinical text extracted from electronic health records using natural language processing (NLP).

Methods: Primary data were derived from Kaiser Permanente Northwest (2008-2014), an integrated health care system (~n > 475 000 unique individuals per year). Data included International Classification of Diseases, Ninth Revision (ICD-9) codes for nonfatal diagnoses, International Classification of Diseases, Tenth Revision (ICD-10) codes for fatal events, clinical notes, and prescription medication records. We assessed sensitivity, specificity, positive predictive value, and negative predictive value for algorithms relative to medical chart review and conducted assessments of algorithm portability in Kaiser Permanente Washington, Tennessee State Medicaid, and Optum.

Results: Code-based algorithm performance was excellent for opioid-related overdoses (sensitivity = 97.2%, specificity = 84.6%) and classification of heroin-involved overdoses (sensitivity = 91.8%, specificity = 99.0%). Performance was acceptable for code-based suicide/suicide attempt classifications (sensitivity = 70.7%, specificity = 90.5%); sensitivity improved with NLP (sensitivity = 78.7%, specificity = 91.0%). Performance was acceptable for the code-based substance abuse-involved classification (sensitivity = 75.3%, specificity = 79.5%); sensitivity improved with the NLP-enhanced algorithm (sensitivity = 80.5%, specificity = 76.3%). The opioid-related overdose algorithm performed well across portability assessment sites, with sensitivity greater than 96% and specificity greater than 84%. Cross-site sensitivity for heroin-involved overdose was greater than 87%, specificity greater than or equal to 99%.

Conclusions: Code-based algorithms developed to detect opioid-related overdoses and classify them according to heroin involvement perform well. Algorithms for classifying suicides/attempts and abuse-related opioid overdoses perform adequately for use for research, particularly given the complexity of classifying such overdoses. The NLP-enhanced algorithms for suicides/suicide attempts and abuse-related overdoses perform significantly better than code-based algorithms and are appropriate for use in settings that have data and capacity to use NLP.

Trial registration: ClinicalTrials.gov NCT02667197.

Keywords: abuse; algorithms; heroin; methods; opioid overdose; pharmacoepidemiology; suicide.

Conflict of interest statement

This project was conducted as part of a Food and Drug Administration (FDA)‐required postmarketing study of extended‐release and long‐acting opioid analgesics (https://www.fda.gov/downloads/Drugs/DrugSafety/InformationbyDrugClass/UCM484415.pdf) funded by the Opioid Postmarketing Consortium (OPC). The OPC is composed of companies that hold NDAs of extended‐release and long‐acting opioid analgesics and, at the time of publication, they included the following companies: Allergan; Assertio Therapeutics, Inc; BioDelivery Sciences, Inc; Collegium Pharmaceutical, Inc; Daiichi Sankyo, Inc; Egalet Corporation; Endo Pharmaceuticals, Inc; Hikma Pharmaceuticals USA Inc; Janssen Pharmaceuticals, Inc; SpecGX, LLC; Pernix Therapeutics Holdings, Inc; Pfizer, Inc; and Purdue Pharma, LP. The study was designed in collaboration between OPC members and independent investigators with input from FDA. Investigators maintained intellectual freedom in terms of publishing final results. This study was registered with ClinicalTrials.gov as study NCT02667197 on January 28, 2016. All authors received research funding from the OPC. Drs Green and Perrin and Ms Janoff received prior funding from Purdue Pharma, LP to carry out related research. Dr Green provided research consulting to the OPC. Kaiser Permanente Center for Health Research (KPCHR) staff had primary responsibility for study design, though OPC members provided comments on the protocol. The protocol and statistical analysis plan were reviewed by FDA, revised following review, and then approved. All algorithm development and validation analyses were conducted by KPCHR; analyses of algorithm portability were completed by each participating site. KPCHR staff made all final decisions regarding publication and content, though OPC members reviewed and provided comments on the manuscript. Drs DeVeaugh‐Geiss and Coplan were employees of Purdue Pharma, LP at the time of the study. Dr Carrell has received funding from Pfizer Inc and Purdue Pharma, LP to carry out related research. Dr Grijalva has served as a consultant for Pfizer, and Merck for unrelated work; he has also received funding from NIH, AHRQ, CDC, FDA, and Sanofi‐Pasteur. Drs Liang and Enger are employees of Optum.

© 2019 The Authors. Pharmacoepidemiology and Drug Safety Published by John Wiley & Sons Ltd.

References

    1. Substance Abuse and Mental Health Services Administration . Results from the 2012 National Survey on Drug Use and Health: mental health findings . NSDUH Series H‐47, HHS Publication No. (SMA) 13‐4805 Rockville, MD: Office of Applied Studies;2013.
    1. Centers for Disease Control and Prevention . Vital signs: overdoses of prescription opioid pain relievers—United States, 1999–2008. MMWR. 2011;60(43):1487‐1492.
    1. Warner M, Chen LH, Makuc DM, Anderson RN, Minino AM. Drug poisoning deaths in the United States, 1980‐2008. NCHS Data Brief. 2011;(81):1‐8.
    1. Mack KA. Drug‐induced deaths—United States, 1999–2010. MMWR Surveill Summ. 2013;62(Suppl 3):161‐163.
    1. Rudd RA, Paulozzi LJ, Bauer MJ, et al. Increases in heroin overdose deaths—28 states, 2010 to 2012. MMWR. 2014;63(39):849‐854.
    1. Rudd RA, Aleshire N, Zibbell JE, Gladden M. Increases in drug and opioid overdose deaths—United States, 2000–2014. Morb Mortal Wkly Rep. 2016;64(Early Release):1‐5.
    1. Rudd RA, Seth P, David F, Scholl L. Increases in drug and opioid‐involved overdose deaths—United States, 2010–2015. MMWR Morb Mortal Wkly Rep. 2016;65(5051):1445‐1452.
    1. US Department of Health and Human Services . Statement of the U.S. Surgeon General VADM Jerome M. Adams, MD, MPH, regarding the President's action on opioids; 10/26/2017. . Accessed 11/27/2017.
    1. Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain—United States, 2016. MMWR Recomm Rep. 2016;65(1):1‐49.
    1. Chou R, Fanciullo GJ, Fine PG, et al. Clinical guidelines for the use of chronic opioid therapy in chronic noncancer pain. J Pain. 2009;10(2):113‐130.
    1. Argoff CE, Kahan M, Sellers EM. Preventing and managing aberrant drug‐related behavior in primary care: systematic review of outcomes evidence. J Opioid Manag. 2014;10(2):119‐134.
    1. Garcia MM, Angelini MC, Thomas T, Lenz K, Jeffrey P. Implementation of an opioid management initiative by a state Medicaid program. J Manag Care Pharm. 2014;20(5):447‐454.
    1. Coplan PM, Kale H, Sandstrom L, Landau C, Chilcoat HD. Changes in oxycodone and heroin exposures in the National Poison Data System after introduction of extended‐release oxycodone with abuse‐deterrent characteristics. Pharmacoepidemiol Drug Saf. 2013;22(12):1274‐1282.
    1. Severtson SG, Bartelson BB, Davis JM, et al. Reduced abuse, therapeutic errors, and diversion following reformulation of extended‐release oxycodone in 2010. J Pain. 2013;14(10):1122‐1130.
    1. Butler SF, Cassidy TA, Chilcoat H, et al. Abuse rates and routes of administration of reformulated extended‐release oxycodone: initial findings from a sentinel surveillance sample of individuals assessed for substance abuse treatment. J Pain. 2013;14(4):351‐358.
    1. Nelson LS, Perrone J. Curbing the opioid epidemic in the United States: the risk evaluation and mitigation strategy (REMS). JAMA. 2012;308(5):457‐458.
    1. Porada S. REMS: red tape, or a remedy for opioid abuse? J Fam Pract. 2011;60(9 Suppl):S55‐S62.
    1. Sloan PA. Opioid risk management: understanding FDA mandated risk evaluation and mitigation strategies (REMS). J Opioid Manag. 2009;5(3):131‐133.
    1. Stanos S. Evolution of opioid risk management and review of the classwide REMS for extended‐release/long‐acting opioids. Physician Sports Med. 2012;40(4):12‐20.
    1. Arnold RM, Han PK, Seltzer D. Opioid contracts in chronic nonmalignant pain management: objectives and uncertainties. Am J Med. 2006;119(4):292‐296.
    1. Federation of State Medical Boards of the United States I . Model guidelines for the use of controlled substances for the treatment of pain. Euless, TX2000.
    1. Han H, Kass PH, Wilsey BL, Li CS. Increasing trends in schedule II opioid use and doctor shopping during 1999‐2007 in California. Pharmacoepidemiol Drug Saf. 2014;23(1):26‐35.
    1. Weiner SG, Griggs CA, Mitchell PM, et al. Clinician impression versus prescription drug monitoring program criteria in the assessment of drug‐seeking behavior in the emergency department. Ann Emerg Med. 2013;62(4):281‐289.
    1. Chakravarthy B, Shah S, Lotfipour S. Prescription drug monitoring programs and other interventions to combat prescription opioid abuse. West J Emerg Med. 2012;13(5):422‐425.
    1. Dunn KM, Saunders KW, Rutter CM, et al. Opioid prescriptions for chronic pain and overdose: a cohort study. Ann Intern Med. 2010;152(2):85‐92.
    1. Green CA, Perrin NA, Janoff SL, Campbell CI, Chilcoat HD, Coplan PM. Assessing the accuracy of opioid overdose and poisoning codes in diagnostic information from electronic health records, claims data, and death records. Pharmacoepidemiol Drug Saf. 2017;26(5):509‐517.
    1. Rowe C, Vittinghoff E, Santos GM, Behar E, Turner C, Coffin PO. Performance measures of diagnostic codes for detecting opioid overdose in the emergency department. Acad Emerg Med. 2017;24(4):475‐483.
    1. Chung CP, Callahan ST, Cooper WO, et al. Development of an algorithm to identify serious opioid toxicity in children. BMC Res Notes. 2015;8(1):293.
    1. Efron B, Johnstone I, Hastie T, Tibshirani R. Least angle regression. Ann Stat. 2002;32(2):407‐499.
    1. Tibshirani R. Regression shrinkage and selection via the lasso. J Royal Stat Soc B. 1996;58(1):267‐288.
    1. Rokach L, Maimon O. Data Mining with Decision Trees: Theory and Applications. Singapore: Word Scientific Publishing Co., Inc; 2008.
    1. Simon GE, Johnson E, Lawrence JM, et al. Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records. Am J Psychiatry. 2018: 10.1176/appi.ajp.2018.17101167;175(10):951‐960.
    1. Hazlehurst B, Green CA, Perrin NA, et al. Using natural language processing of clinical text to enhance the identification of opioid‐related overdoses in electronic health records data. Pharmacoepidemiol Drug Saf. 2018;28(8):1143‐1151.
    1. Green CA, Hazlehurst B, Brandes J, et al. Development of an algorithm to identify inpatient opioid‐related overdoses and oversedation using electronic data. Pharmacoepidemiol Drug Saf. 2018;28(8):1138‐1142.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever