Dermoscopy diagnosis of cancerous lesions utilizing dual deep learning algorithms via visual and audio (sonification) outputs: Laboratory and prospective observational studies

B N Walker, J M Rehg, A Kalra, R M Winters, P Drews, J Dascalu, E O David, A Dascalu, B N Walker, J M Rehg, A Kalra, R M Winters, P Drews, J Dascalu, E O David, A Dascalu

Abstract

Background: Early diagnosis of skin cancer lesions by dermoscopy, the gold standard in dermatological imaging, calls for a diagnostic upscale. The aim of the study was to improve the accuracy of dermoscopic skin cancer diagnosis through use of novel deep learning (DL) algorithms. An additional sonification-derived diagnostic layer was added to the visual classification to increase sensitivity.

Methods: Two parallel studies were conducted: a laboratory retrospective study (LABS, n = 482 biopsies) and a non-interventional prospective observational study (OBS, n = 63 biopsies). A training data set of biopsy-verified reports, normal and cancerous skin lesions (n = 3954), were used to develop a DL classifier exploring visual features (System A). The outputs of the classifier were sonified, i.e. data conversion into sound (System B). Derived sound files were analyzed by a second machine learning classifier, either as raw audio (LABS, OBS) or following conversion into spectrograms (LABS) and by image analysis and human heuristics (OBS). The OBS criteria outcomes were System A specificity and System B sensitivity as raw sounds, spectrogram areas or heuristics.

Findings: LABS employed dermoscopies, half benign half malignant, and compared the accuracy of Systems A and B. System A algorithm resulted in a ROC AUC of 0.976 (95% CI, 0.965-0.987). Secondary machine learning analysis of raw sound, FFT and Spectrogram ROC curves resulted in AUC's of 0.931 (95% CI 0.881-0.981), 0.90 (95% CI 0.838-0.963) and 0.988 (CI 95% 0.973-1.001), respectively. OBS analysis of raw sound dermoscopies by the secondary machine learning resulted in a ROC AUC of 0.819 (95% CI, 0.7956 to 0.8406). OBS image analysis of AUC for spectrograms displayed a ROC AUC of 0.808 (CI 95% 0.6945 To 0.9208). By applying a heuristic analysis of Systems A and B a sensitivity of 86% and specificity of 91% were derived in the clinical study.

Interpretation: Adding a second stage of processing, which includes a deep learning algorithm of sonification and heuristic inspection with machine learning, significantly improves diagnostic accuracy. A combined two-stage system is expected to assist clinical decisions and de-escalate the current trend of over-diagnosis of skin cancer lesions as pathological. FUND: Bostel Technologies. Trial Registration clinicaltrials.gov Identifier: NCT03362138.

Keywords: Artificial intelligence; Deep learning; Dermoscopy; Melanoma; Skin cancer; Sonification; Telemedicine.

Copyright © 2019 The Author(s). Published by Elsevier B.V. All rights reserved.

Figures

Fig. 1
Fig. 1
Visual representations of sonification audio files.
Fig. 2
Fig. 2
Receiver Operating Characteristic Curves and Area Under the Curve for the Secondary Machine Learning System Applied to Sonification Outputs (a-c), and DL Classifier (System A) in (d).
Fig. 3
Fig. 3
Prospective observational study Example Images.
Fig. 4
Fig. 4
Prospective observational study Output Examples.

References

    1. Schadendorf D., van Akkooi A.C.J., Berking C. Melanoma. Lancet. 2018 Sep 15;392(10151):971–984.
    1. Carrera C., Marchetti M.A., Dusza S.W. Validity and reliability of dermoscopic criteria used to differentiate nevi from melanoma: a web-based international dermoscopy society study. JAMA Dermatol. 2016 Jul 1;152(7):798–806.
    1. Tschandl P., Hofmann L., Fink C., Kittler H., Haenssle H.A. Melanomas vs. nevi in high-risk patients under long-term monitoring with digital dermatoscopy: do melanomas and nevi already differ at baseline? J. Eur. Acad. Dermatol. Venereol. 2017 Jun;31(6):972–977.
    1. Matsumoto M., Secrest A., Anderson A. Estimating the cost of skin cancer detection by dermatology providers in a large health care system. J. Am. Acad. Dermatol. 2018 Apr;78(4):701–709.
    1. Waldmann A., Nolte S., Geller A.C. Frequency of excisions and yields of malignant skin tumors in a population-based screening intervention of 360,288 whole-body examinations. Arch. Dermatol. 2012;148(8):903–910.
    1. Winkelmann R.R., Farberg A.S., Glazer A.M. Integrating Skin Cancer-Related Technologies into Clinical Practice. Dermatol. Clin. 2017 Oct;35(4):565–576.
    1. Brunssen A., Waldmann A., Eisemann N., Katalinic A. Impact of skin cancer screening and secondary prevention campaigns on skin cancer incidence and mortality: A systematic review. J. Am. Acad. Dermatol. 2017 Jan;76(1):129–139.
    1. Gulshan V., Peng L., Coram M. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016 Dec 13;316(22):2402–2410.
    1. Chilamkurthy S., Ghosh R., Tanamala S. Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet. 2018 Oct 11;(18):31643–31645. pii: S0140-6736.
    1. Codella N., Cai J., Abedini M., Garnavi R., Halpern A., Smith J.R. Deep learning, sparse coding, and SVM for melanoma recognition in dermoscopy images. In: Zhou L., Wang L., Wang Q., Shi Y., editors. Machine Learning in Medical Imaging. MLMI 2015. Lecture Notes in Computer Science. Vol. 9352. Springer; Cham: 2015.
    1. Takuya Yoshida M. Emre Celebi, Schaefer Gerald, Iyatomi H. Simple and effective pre-processing for automated melanoma discrimination based on cytological findings. BigData. 2016:3439–3442.
    1. Dubus G., Bresin. A Systematic Review of Mapping Strategies for the Sonification of Physical Quantities. PLos One. 2013 Dec;17(8):e82491.
    1. Ioffe S., Szegedy C. Proceedings of the 32nd International Conference on Machine Learning (ICML) Vol. 37. Lille; France: 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift.
    1. Codella N.C.F., Gutman D., Celebi E. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC) arXiv. 2017
    1. Argenziano G., Soyer H.P., De Giorgi V., Piccolo D., Carli P., Delfino M. EDRA Medical Publishing & New Media; 2002. Dermoscopy: A Tutorial.
    1. Jia Y., Shelhamer E., Donahue J. Proceedings of ACM International Conference Multimed. 2014. Caffe: convolutional architecture for fast feature embedding; pp. 675–678.
    1. Russakovsky O., Deng J., Su H. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 2015;115(3):211–252. (Yu L, Chen H, Dou Q, Qin J, Heng PA. Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks.IEEE Trans Med Imaging. 2017 Apr;36(4):994–1004.
    1. Li X., Zhao L., Wei L., Yang M.H. Deep saliency: multi-task deep neural network model for salient object detection. IEEE Trans. Image Process. 2016 Aug;25(8):3919–3930.
    1. Walker B.N., Nees M.A. Theory of sonification. In: Hermann T., Hunt A., Neuhoff J., editors. The Sonification Handbook. Logos Publishing House; Berlin, Germany: 2011. pp. 9–39. [ISBN 978-3-8325-2819-5]
    1. Celebi M.E., Kingravi H.A., Vela P.A. A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst. Appl. 2013;40(1):200–210.
    1. Malvehy J., Hauschild A., Curiel-Lewandrowski C. Clinical performance of the Nevisense system in cutaneous melanoma detection: an international, multicentre, prospective and blinded clinical trial on efficacy and safety. Br. J. Dermatol. 2014 Nov;171(5):1099–1107.
    1. Fleming N.H., Egbert B.M., Kim J., Swetter S.M. Reexamining the threshold for reexcision of histologically transected dysplastic nevi. JAMA Dermatol. 2016;152(12):1327–1334.
    1. Han S.S., Kim M.S., Lim W., Park G.H., Park I., Chang S.E. Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. J Invest Dermatol. 2018 Jul;138(7):1529–1538.
    1. Haenssle H.A., Fink C., Schneiderbauer R. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann. Oncol. 2018 Aug 1;29(8):1836–1842.
    1. Poveda J., O'Sullivan M., Popovici E., Temko A. Portable neonatal EEG monitoring and sonification on an Android device. Conf Proc IEEE Eng Med Biol Soc. 2017 Jul;2017:2018–2021.
    1. Tschandl P., Rosendahl C., Akay B.N. Expert-level diagnosis of nonpigmented skin cancer by combined convolutional neural networks. JAMA Dermatol. 2019;155(1):58–65.
    1. Melamed R.D., Aydin I.T., Rajan G.S. Genomic characterization of dysplastic nevi unveils implications for diagnosis of melanoma. J Invest Dermatol. 2017;137(4):905.
    1. Elmore J.G., Barnhill R.L., Elder D.E. Pathologists' diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study. BMJ. 2017 Jun 28;j2813:357.
    1. Navarrete-Dechent C., Dusza S.W., Liopyris K., Marghoob A.A., Halpern A.C., Marchetti M.A. Automated dermatological diagnosis: hype or reality? J Invest Dermatol. 2018 Oct;138(10):2277–2279.
    1. Gendreau J.L., Gemelas J., Wang M., Capu Unimaged melanomas in store-and-forward teledermatology. Telemed. J. E Health. 2017 Jun;23(6):517–520.

Source: PubMed

Sponsorzy i współpracownicy

Warunki medyczne

Interwencje lekowe

3
Subskrybuj