Software for automated classification of probe-based confocal laser endomicroscopy videos of colorectal polyps

Abstract

Aim: To support probe-based confocal laser endomicroscopy (pCLE) diagnosis by designing software for the automated classification of colonic polyps.

Methods: Intravenous fluorescein pCLE imaging of colorectal lesions was performed on patients undergoing screening and surveillance colonoscopies, followed by polypectomies. All resected specimens were reviewed by a reference gastrointestinal pathologist blinded to pCLE information. Histopathology was used as the criterion standard for the differentiation between neoplastic and non-neoplastic lesions. The pCLE video sequences, recorded for each polyp, were analyzed off-line by 2 expert endoscopists who were blinded to the endoscopic characteristics and histopathology. These pCLE videos, along with their histopathology diagnosis, were used to train the automated classification software which is a content-based image retrieval technique followed by k-nearest neighbor classification. The performance of the off-line diagnosis of pCLE videos established by the 2 expert endoscopists was compared with that of automated pCLE software classification. All evaluations were performed using leave-one-patient-out cross-validation to avoid bias.

Results: Colorectal lesions (135) were imaged in 71 patients. Based on histopathology, 93 of these 135 lesions were neoplastic and 42 were non-neoplastic. The study found no statistical significance for the difference between the performance of automated pCLE software classification (accuracy 89.6%, sensitivity 92.5%, specificity 83.3%, using leave-one-patient-out cross-validation) and the performance of the off-line diagnosis of pCLE videos established by the 2 expert endoscopists (accuracy 89.6%, sensitivity 91.4%, specificity 85.7%). There was very low power (< 6%) to detect the observed differences. The 95% confidence intervals for equivalence testing were: -0.073 to 0.073 for accuracy, -0.068 to 0.089 for sensitivity and -0.18 to 0.13 for specificity. The classification software proposed in this study is not a "black box" but an informative tool based on the query by example model that produces, as intermediate results, visually similar annotated videos that are directly interpretable by the endoscopist.

Conclusion: The proposed software for automated classification of pCLE videos of colonic polyps achieves high performance, comparable to that of off-line diagnosis of pCLE videos established by expert endoscopists.

Trial registration: ClinicalTrials.gov NCT00874263.

Keywords: Colorectal neoplasia; Computer-aided diagnosis; Content-based image retrieval; Nearest neighbor classification software; Probe-based confocal laser endomicroscopy.

Figures

Figure 1

Imaging modalities of colonic polyps. A: Setup of probe-based confocal laser endomicroscopy (pCLE) imaging system (Cellvizio, Mauna Kea Technologies); B: Endoscopic image of tubular adenoma, and the pCLE miniprobe; C: An image of the pCLE video sequence; D: A pCLE mosaic image built with the video mosaicing tool; E: Histopathology image.

Figure 2

Adjusting bag-of-visual-words technique for probe-based confocal laser endomicroscopy video retrieval. A: Neoplastic probe-based confocal laser endomicroscopy mosaic obtained with non-rigid registration; B: Colored visual words mapped to the disk regions of radius 60 pixels in the mosaic image; C: Overlap scores of the local regions in the mosaic space, computed from the translation results of mosaicing.

Figure 3

Pipeline of the probe-based confocal laser endomicroscopy retrieval-based software classification framework. From the acquisition of the probe-based confocal laser endomicroscopy (pCLE) video query by the Cellvizio system to the on-line automated diagnosis estimation.

Figure 4

Typical results of automated probe-based confocal laser endomicroscopy video retrieval. The probe-based confocal laser endomicroscopy (pCLE) videos are represented by mosaic images; they are annotated with their histopathology diagnosis. Video queries are highlighted in gray and followed by their 3 most similar videos. Automated software classification (hyperplastic vs neoplastic) of query videos is based on the votes of the similar videos. With respect to histopathology, both the automated software classification and the pCLE diagnosis established by expert endoscopists are correct for these queries.

Figure 5

Results of automated probe-based confocal laser endomicroscopy video retrieval represented as mosaics. With respect to histology: the automated software classification is correct for video query Q6 but incorrect for video queries Q5 and Q7, whereas the off-line diagnosis of probe-based confocal laser endomicroscopy videos established by the expert endoscopists is correct for video queries Q5 but incorrect for video queries Q6 and Q7 (for which this disagreement is marked by *).