NLM Scrubber: NLM s Software Application to De-identify Clinical Text Documents

NLM Scrubber: NLM's Software Application to De-identify Clinical Text Documents

Background: Electronic health records contain a vast amount of data about diseases and treatments. Researchers could use this data to test their ideas, but they would need to use records from more than just their own group of patients. But access to those records is restricted to ensure patient privacy.

U.S. National Library of Medicine (NLM) has created a computer tool called NLM Scrubber. This program recognizes and deletes personal information from health records. The researchers who developed this program now need access to the original records. This will allow them to see how well the program removes personal information from patient records and how they can make it more accurate.

Objectives:

To find ways to improve clinical text de-identification.

Eligibility:

No new participants. Researchers will review data that have already been collected.

Design:

Researchers will collect a random sample of reports. These will be from different doctors in different fields.

Researchers will manually remove personal information from the records.

Researchers will also automatically remove personal information from original records using NLM-Scrubber.

Researchers will compare the results of the computer program versus the manual changes. They will note when the program has not been removing personal information correctly. They will also note when the program has been deleting nonpersonal health information incorrectly.

Researchers will use the results to revise the program. They will keep testing it until the de-identification process is complete.

Study Overview

• Status: Enrolling by invitation
• Conditions: Personally Identifiable Information

Detailed Description

This study is about the quality assessment, improvement, and monitoring of an automatic clinical text de-identification software application called NLM Scrubber, which has been developed at the National Library of Medicine (NLM). The application has been developed so that clinical reports can be used in secondary scientific studies (i.e., for secondary use) without breaching patient privacy. Research on methods for protecting patient privacy and on the development of NLM Scrubber have been conducted by following the guidelines of and in compliance with HIPAA and the Privacy Act.

In order to further develop and improve NLM Scrubber and assess its de-identification performance effectively, the investigators require the original / unredacted samples from all potential clinical report types and sources. To this end, NLM investigators have been

collaborating with entities within NIH, namely, NIH Clinical Center, BTRIS, and NCI as well as outside entities, Kentucky State Registry administered by University of Kentucky and researchers from the University of Pittsburgh, who stated their interest in integrating NLM

Scrubber to their application called Text Information Extraction System. These entities collect samples of various types of clinical reports for assessing and improving NLM Scrubber performance. However we also need access to the original data in order to assess

potential problems and improve the accuracy of NLM Scrubber.

• Study Type: Observational
• Enrollment (Estimated): 50000

Contacts and Locations

Study Locations

• United States
  • Maryland
    • Bethesda, Maryland, United States
      • National Library of Medicine

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 1 day and older (Child, Adult, Older Adult)
• Accepts Healthy Volunteers: No

• Sampling Method: Probability Sample

Study Population

Everybody for whom a clinical narrative report is created.

Description

• No new participant enrollment. Researchers will review data that have already been collected.

Study Plan

How is the study designed?

Design Details

• Observational Models: Other
• Time Perspectives: Retrospective

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort
1; Everybody for whom a clinical narrative report is created.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The rate of de-identification of PII	HIPAA Privacy Rule defines 18 types of personally identifying information, that need to be de-identified, which include personal names, addresses, significant dates, numeric identifiers (such as social security number). Our annotators label those words and numbers creating a gold standard and NLM-Scrubber tries to recognize and eliminate all of them. The rate of de-identification of PII refers to success of this outcome measure.	01/01/2017-01/31/2027

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The rate of erroneously redacted clinical information	While NLM-Scrubber tries to eliminate only PII elements while preserving non-identifying study data, it inadvertently deletes some of the non-identifying study data elements (non-protected health information) as well. The rate of erroneously redacted clinical information refers to the failure of NLM-Scrubber in preserving non-identifying health information.	01/01/2017-01/31/2027

Collaborators and Investigators

• Sponsor: National Library of Medicine (NLM)
• Collaborators: National Institutes of Health Clinical Center (CC); National Cancer Institute (NCI)
• Investigators: Principal Investigator: Mehmet M Kayaalp, Ph.D., National Library of Medicine (NLM)

Publications and helpful links

General Publications

• Kayaalp M. Patient Privacy in the Era of Big Data. Balkan Med J. 2018 Jan 20;35(1):8-17. doi: 10.4274/balkanmedj.2017.0966. Epub 2017 Sep 13.
• Kayaalp M, Browne AC, Dodd ZA, Sagan P, McDonald CJ. De-identification of Address, Date, and Alphanumeric Identifiers in Narrative Clinical Reports. AMIA Annu Symp Proc. 2014 Nov 14;2014:767-76. eCollection 2014.
• Kayaalp M, Browne AC, Callaghan FM, Dodd ZA, Divita G, Ozturk S, McDonald CJ. The pattern of name tokens in narrative clinical text and a comparison of five systems for redacting them. J Am Med Inform Assoc. 2014 May-Jun;21(3):423-31. doi: 10.1136/amiajnl-2013-001689. Epub 2013 Sep 11.

Study record dates

Study Major Dates

• Study Start: May 25, 2016
• Primary Completion (Estimated): January 31, 2027
• Study Completion (Estimated): January 31, 2027

Study Registration Dates

• First Submitted: June 9, 2016
• First Submitted That Met QC Criteria: June 9, 2016
• First Posted (Estimated): June 10, 2016

Study Record Updates

• Last Update Posted (Actual): April 6, 2026
• Last Update Submitted That Met QC Criteria: April 3, 2026
• Last Verified: December 16, 2025

More Information

Terms related to this study

• Keywords: Natural History; Address
• Other Study ID Numbers: 999916122; 16-LM-N122

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: We receive patient data, protected health information (PHI), from our collaborating data sources with the promise that we would protect PHI to the full extent and not share it with third parties.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No