Improving Skin cancer Management with ARTificial Intelligence (SMARTI): protocol for a preintervention/postintervention trial of an artificial intelligence system used as a diagnostic aid for skin cancer management in a specialist dermatology setting

Claire Felmingham, Samantha MacNamara, William Cranwell, Narelle Williams, Miki Wada, Nikki R Adler, Zongyuan Ge, Alastair Sharfe, Adrian Bowling, Martin Haskett, Rory Wolfe, Victoria Mar, Claire Felmingham, Samantha MacNamara, William Cranwell, Narelle Williams, Miki Wada, Nikki R Adler, Zongyuan Ge, Alastair Sharfe, Adrian Bowling, Martin Haskett, Rory Wolfe, Victoria Mar

Abstract

Introduction: Convolutional neural networks (CNNs) can diagnose skin cancers with impressive accuracy in experimental settings, however, their performance in the real-world clinical setting, including comparison to teledermatology services, has not been validated in prospective clinical studies.

Methods and analysis: Participants will be recruited from dermatology clinics at the Alfred Hospital and Skin Health Institute, Melbourne. Skin lesions will be imaged using a proprietary dermoscopic camera. The artificial intelligence (AI) algorithm, a CNN developed by MoleMap Ltd and Monash eResearch, classifies lesions as benign, malignant or uncertain. This is a preintervention/postintervention study. In the preintervention period, treating doctors are blinded to AI lesion assessment. In the postintervention period, treating doctors review the AI lesion assessment in real time, and have the opportunity to then change their diagnosis and management. Any skin lesions of concern and at least two benign lesions will be selected for imaging. Each participant's lesions will be examined by a registrar, the treating consultant dermatologist and later by a teledermatologist. At the conclusion of the preintervention period, the safety of the AI algorithm will be evaluated in a primary analysis by measuring its sensitivity, specificity and agreement with histopathology where available, or the treating consultant dermatologists' classification. At trial completion, AI classifications will be compared with those of the teledermatologist, registrar, treating dermatologist and histopathology. The impact of the AI algorithm on diagnostic and management decisions will be evaluated by: (1) comparing the initial management decision of the registrar with their AI-assisted decision and (2) comparing the benign to malignant ratio (for lesions biopsied) between the preintervention and postintervention periods.

Ethics and dissemination: Human Research Ethics Committee (HREC) approval received from the Alfred Hospital Ethics Committee on 14 February 2019 (HREC/48865/Alfred-2018). Findings from this study will be disseminated through peer-reviewed publications, non-peer reviewed media and conferences.

Trial registration number: NCT04040114.

Keywords: adult dermatology; dermatological tumours; dermatology.

Conflict of interest statement

Competing interests: VM is supported by an NHMRC Early Career Fellowship. VM reports personal fees from Novartis, personal fees from Bristol-Myers-Squibb, personal fees from Merck, outside the submitted work. MH reports personal fees from MoleMap Ltd, during the conduct of the study; and is a shareholder in MoleMap Ltd. AB reports personal fees from MoleMap Ltd, during the conduct of the study; personal fees from Molemap Ltd, outside the submitted work; and is a shareholder in Molemap Ltd. AS reports personal fees from MoleMap Ltd, during the conduct of the study; personal fees from Molemap Ltd, outside the submitted work. ZG reports personal fees from MoleMap Ltd. NW and SM are former employees of the Cancer Collaborative Trials Group contracted to implement the SMARTI Study—Melanoma and Skin Cancer Trials (MASC Trials) Ltd. CF is supported by a Monash University Research Training Program Scholarship. RW, NA, WC and MW have nothing to disclose. The study is sponsored by Monash University and endorsed by MASC Trials Ltd.

© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Figures

Figure 1
Figure 1
The Skin cancer Management with ARTificial Intelligence computer display: participant avatar indicating the lesion location.
Figure 2
Figure 2
The Skin cancer Management with ARTificial Intelligence computer display: clinician diagnosis and management plan entry, where: ‘Diagnosis 1’ is the clinician’s initial assessment; ‘Assessment’ is the artificial intelligence (AI) algorithm’s classification; ‘Diagnosis 2’ is the clinician’s AI-assisted assessment and ‘Action Plans’ detail the recommended and final agreed-upon plan.
Figure 3
Figure 3
Participant flow chart. AI, artificial intelligence; eCRF, electronic Case Report Form.

References

    1. Apalla Z, Lallas A, Sotiriou E, et al. . Epidemiological trends in skin cancer. Dermatol Pract Concept 2017;7:1–6. 10.5826/dpc.0702a01
    1. Leiter U, Eigentler T, Garbe C. Epidemiology of skin cancer. Adv Exp Med Biol 2014;810:120–40. 10.1007/978-1-4939-0437-2_7
    1. Schadendorf D, van Akkooi ACJ, Berking C, et al. . Melanoma. Lancet 2018;392:971–84. 10.1016/S0140-6736(18)31559-9
    1. Perera E, Gnaneswaran N, Staines C, et al. . Incidence and prevalence of non-melanoma skin cancer in Australia: a systematic review. Australas J Dermatol 2015;56:258–67. 10.1111/ajd.12282
    1. Australian Institute of Health and Welfare . Cancer in Australia, 2019.
    1. Kozera EK, Yang A, Murrell DF. Patient and practitioner satisfaction with tele-dermatology including Australia's Indigenous population: a systematic review of the literature. Int J Womens Dermatol 2016;2:70–3. 10.1016/j.ijwd.2016.06.004
    1. Gershenwald JE, Scolyer RA, Hess KR, et al. . Melanoma staging: evidence-based changes in the American joint Committee on cancer eighth edition cancer staging manual. CA Cancer J Clin 2017;67:472–92. 10.3322/caac.21409
    1. Gilmore SJ. Automated decision support in melanocytic lesion management. PLoS One 2018;13:e0203459. 10.1371/journal.pone.0203459
    1. Tschandl P, Rinner C, Apalla Z, et al. . Human-computer collaboration for skin cancer recognition. Nat Med 2020;26:1229–34. 10.1038/s41591-020-0942-0
    1. Mar VJ, Soyer HP. Artificial intelligence for melanoma diagnosis: how can we deliver on the promise? Ann Oncol 2018;29:1625–8. 10.1093/annonc/mdy193
    1. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25:44–56. 10.1038/s41591-018-0300-7
    1. Esteva A, Kuprel B, Novoa RA, et al. . Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542:115–8. 10.1038/nature21056
    1. Haenssle HA, Fink C, Schneiderbauer R, et al. . Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol 2018;29:1836–42. 10.1093/annonc/mdy166
    1. Tschandl P, Rosendahl C, Akay BN, et al. . Expert-level diagnosis of nonpigmented skin cancer by combined Convolutional neural networks. JAMA Dermatol 2019;155:58–65. 10.1001/jamadermatol.2018.4378
    1. Marchetti MA, Codella NCF, Dusza SW, et al. . Results of the 2016 international skin imaging collaboration International Symposium on biomedical imaging challenge: comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images. J Am Acad Dermatol 2018;78:270–7. 10.1016/j.jaad.2017.08.016
    1. Fujisawa Y, Otomo Y, Ogata Y, et al. . Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis. Br J Dermatol 2019;180:373–81. 10.1111/bjd.16924
    1. Han SS, Kim MS, Lim W, et al. . Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. J Invest Dermatol 2018;138:1529–38. 10.1016/j.jid.2018.01.028
    1. Brinker TJ, Hekler A, Enk AH, et al. . Deep neural networks are superior to dermatologists in melanoma image classification. Eur J Cancer 2019;119:11–17. 10.1016/j.ejca.2019.05.023
    1. Brinker TJ, Hekler A, Enk AH, et al. . A convolutional neural network trained with dermoscopic images performed on PAR with 145 dermatologists in a clinical melanoma image classification task. Eur J Cancer 2019;111:148–54. 10.1016/j.ejca.2019.02.005
    1. Brinker TJ, Hekler A, Enk AH, et al. . Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur J Cancer 2019;113:47–54. 10.1016/j.ejca.2019.04.001
    1. Yu C, Yang S, Kim W, et al. . Acral melanoma detection using a convolutional neural network for Dermoscopy images. PLoS One 2018;13:e0193321. 10.1371/journal.pone.0193321
    1. Du-Harpur X, Watt FM, Luscombe NM, et al. . What is AI? applications of artificial intelligence to dermatology. Br J Dermatol 2020.
    1. Navarrete-Dechent C, Dusza SW, Liopyris K, et al. . Automated Dermatological Diagnosis: Hype or Reality? J Invest Dermatol 2018;138:2277–9. 10.1016/j.jid.2018.04.040
    1. Tschandl P, Codella N, Akay BN, et al. . Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study. Lancet Oncol 2019;20:938–47. 10.1016/S1470-2045(19)30333-X
    1. Haenssle HA, Fink C, Toberer F, et al. . Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions. Ann Oncol 2020;31:137–43. 10.1016/j.annonc.2019.10.013
    1. Nagendran M, Chen Y, Lovejoy CA, et al. . Artificial intelligence versus clinicians: systematic review of design, reporting Standards, and claims of deep learning studies. BMJ 2020;368:m689. 10.1136/bmj.m689
    1. Hajian-Tilaki K. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med 2013;4:627–35.
    1. Cancer Council Australia Keratinocyte Cancers Guideline Working Party . Clinical practice guidelines for keratinocyte cancer. Sydney: Cancer Council Australia.
    1. Cancer Council Australia Melanoma Guidelines Working Party . Clinical practice guidelines for the diagnosis and management of melanoma. Sydney: Cancer Council Australia, 2016.
    1. Wada M, Ge Z, Gilmore SJ, et al. . Use of artificial intelligence in skin cancer diagnosis and management. Med J Aust 2020;213:256–9. 10.5694/mja2.50759
    1. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. 10.2307/2529310

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe