Deep Learning-Based Natural Language Processing for Screening Psychiatric Patients

Abstract

The introduction of pre-trained language models in natural language processing (NLP) based on deep learning and the availability of electronic health records (EHRs) presents a great opportunity to transfer the "knowledge" learned from data in the general domain to enable the analysis of unstructured textual data in clinical domains. This study explored the feasibility of applying NLP to a small EHR dataset to investigate the power of transfer learning to facilitate the process of patient screening in psychiatry. A total of 500 patients were randomly selected from a medical center database. Three annotators with clinical experience reviewed the notes to make diagnoses for major/minor depression, bipolar disorder, schizophrenia, and dementia to form a small and highly imbalanced corpus. Several state-of-the-art NLP methods based on deep learning along with pre-trained models based on shallow or deep transfer learning were adapted to develop models to classify the aforementioned diseases. We hypothesized that the models that rely on transferred knowledge would be expected to outperform the models learned from scratch. The experimental results demonstrated that the models with the pre-trained techniques outperformed the models without transferred knowledge by micro-avg. and macro-avg. F-scores of 0.11 and 0.28, respectively. Our results also suggested that the use of the feature dependency strategy to build multi-labeling models instead of problem transformation is superior considering its higher performance and simplicity in the training process.

Keywords: deep learning; natural language processing; patient screening; psychiatric diagnoses; text classification.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2021 Dai, Su, Lee, Zhang, Wang, Kuo and Wu.

Figures

Figure 1

Distributions of the five disorders in the training and test sets.

Figure 2

Flowchart of the screening process of the neural networks based on the two methods: problem transformation and feature dependency.

Figure 3

Deep learning text classification models developed in this study.

Figure 4

Micro- and macro-average F-scores on the test set of all developed models. “Linear (Mix)” signifies the linear model that considers the BH and PME sections in combination, whereas the “Linear (Sep)” model considers them separately. The figure also includes the performance of BERT* listed in Table 2 as a baseline (BERT-fine tuning) for comparison.

Figure 5

Performance comparison of the top performing models selected from the developed network architectures. PT and FD denote problem transformation and feature dependency, respectively. The fine tuning approaches (ROBERTa*_PT/FD and Distill*_PT) with the best micro- and macro-avg. F-scores are also included for comparison.

Figure 6

Important terms used by the HAN model for classifying a patient with major depressive disorder. The font size of a word indicates the attention score. Words with higher scores appear in larger font sizes. The rectangle shown in front of each sentence indicates the importance (attention score) of the sentence. Larger rectangles signify more important sentences.

Figure 7

Comparison of the results of the pre-trained model and random initialization. Each sub-figure compares the results of HAN-glove_FD (upper) and HAN-rand_FD (lower). (A) TP vs. FN; (B) TP vs. TP; (C) FP vs. FP.

Figure 8

Comparison of the top-30 similar words for “depressed,” “manic,” “hallucination,” and “zyprexa” regarding the randomly initialized embedding of HAN-rand (left) and the pre-trained word embedding of the HAN-glove model (right).