Real-time computer-aided diagnosis of focal pancreatic masses from endoscopic ultrasound imaging based on a hybrid convolutional and long short-term memory neural network model

Abstract

Differential diagnosis of focal pancreatic masses is based on endoscopic ultrasound (EUS) guided fine needle aspiration biopsy (EUS-FNA/FNB). Several imaging techniques (i.e. gray-scale, color Doppler, contrast-enhancement and elastography) are used for differential diagnosis. However, diagnosis remains highly operator dependent. To address this problem, machine learning algorithms (MLA) can generate an automatic computer-aided diagnosis (CAD) by analyzing a large number of clinical images in real-time. We aimed to develop a MLA to characterize focal pancreatic masses during the EUS procedure. The study included 65 patients with focal pancreatic masses, with 20 EUS images selected from each patient (grayscale, color Doppler, arterial and venous phase contrast-enhancement and elastography). Images were classified based on cytopathology exam as: chronic pseudotumoral pancreatitis (CPP), neuroendocrine tumor (PNET) and ductal adenocarcinoma (PDAC). The MLA is based on a deep learning method which combines convolutional (CNN) and long short-term memory (LSTM) neural networks. 2688 images were used for training and 672 images for testing the deep learning models. The CNN was developed to identify the discriminative features of images, while a LSTM neural network was used to extract the dependencies between images. The model predicted the clinical diagnosis with an area under curve index of 0.98 and an overall accuracy of 98.26%. The negative (NPV) and positive (PPV) predictive values and the corresponding 95% confidential intervals (CI) are 96.7%, [94.5, 98.9] and 98.1%, [96.81, 99.4] for PDAC, 96.5%, [94.1, 98.8], and 99.7%, [99.3, 100] for CPP, and 98.9%, [97.5, 100] and 98.3%, [97.1, 99.4] for PNET. Following further validation on a independent test cohort, this method could become an efficient CAD tool to differentiate focal pancreatic masses in real-time.

Conflict of interest statement

The authors have read the journal’s policy and the authors of this manuscript have the following competing interests: SU is an unpaid consultant for INNES Worldwide. There are no patents, products in development or marketing products to declare. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Figures

Fig 1

EUS imaging of a pseudotumoral chronic pancreatitis in (A) gray scale. (B) elastography. (C) color Doppler. (D) contrast enhancement–arterial phase. (E) contrast enhancement–venous phase.

Fig 2

EUS imaging of a neuroendocrine tumor in (A) gray scale. (B) elastography. (C) color Doppler. (D) contrast enhancement–arterial phase. (E) contrast enhancement–venous phase.

Fig 3

EUS imaging of a pancreatic ductal adenocarcinoma in (A) gray scale. (B) elastography. (C) color Doppler. (D) contrast enhancement–arterial phase. (E) contrast enhancement–venous phase.

Fig 4. The architecture of the CNN-LSTM model.

Fig 5. The comparison between the accuracy of the training and the testing datasets.

Fig 6. The comparison between the loss of the training and validation datasets.

Fig 7. The ROC curves computed for CNN-LSTM method.

Fig 8. The precision/recall curves computed for CNN-LSTM method.