Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology

L Drukker, J A Noble, A T Papageorghiou, L Drukker, J A Noble, A T Papageorghiou

Abstract

Artificial intelligence (AI) uses data and algorithms to aim to draw conclusions that are as good as, or even better than, those drawn by humans. AI is already part of our daily life; it is behind face recognition technology, speech recognition in virtual assistants (such as Amazon Alexa, Apple's Siri, Google Assistant and Microsoft Cortana) and self-driving cars. AI software has been able to beat world champions in chess, Go and recently even Poker. Relevant to our community, it is a prominent source of innovation in healthcare, already helping to develop new drugs, support clinical decisions and provide quality assurance in radiology. The list of medical image-analysis AI applications with USA Food and Drug Administration or European Union (soon to fall under European Union Medical Device Regulation) approval is growing rapidly and covers diverse clinical needs, such as detection of arrhythmia using a smartwatch or automatic triage of critical imaging studies to the top of the radiologist's worklist. Deep learning, a leading tool of AI, performs particularly well in image pattern recognition and, therefore, can be of great benefit to doctors who rely heavily on images, such as sonologists, radiographers and pathologists. Although obstetric and gynecological ultrasound are two of the most commonly performed imaging studies, AI has had little impact on this field so far. Nevertheless, there is huge potential for AI to assist in repetitive ultrasound tasks, such as automatically identifying good-quality acquisitions and providing instant quality assurance. For this potential to thrive, interdisciplinary communication between AI developers and ultrasound professionals is necessary. In this article, we explore the fundamentals of medical imaging AI, from theory to applicability, and introduce some key terms to medical professionals in the field of ultrasound. We believe that wider knowledge of AI will help accelerate its integration into healthcare. © 2020 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.

© 2020 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.

Figures

Figure 1
Figure 1
Graphic representation of artificial intelligence. (a) Human neural network architecture and its resemblance to a deep artificial neural network. (b) Relationship between artificial intelligence, machine learning and deep learning. ARDA, automated retinal disease assessment (Appendix S1).

References

    1. United Kingdom Engineering and Physical Sciences Research Council . Artificial intelligence technologies. .
    1. Turing AM. I–Computing Machinery and Intelligence. Mind 1950; LIX: 433–460.
    1. McCarthy J, Minsky M, Rochester N, Shannon C. A proposal for the dartmouth summer research project on artificial intelligence, August 1955.
    1. Bory P. Deep new: The shifting narratives of artificial intelligence from Deep Blue to AlphaGo. Convergence 2019; 25: 627–642.
    1. Abramoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI‐based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 2018; 1: 39.
    1. Wang P, Berzin TM, Glissen Brown JR, Bharadwaj S, Becq A, Xiao X, Liu P, Li L, Song Y, Zhang D, Li Y, Xu G, Tu M, Liu X. Real‐time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; 68: 1813–1819.
    1. Rezaii N, Walker E, Wolff P. A machine learning approach to predicting psychosis using semantic density and latent content analysis. NPJ Schizophr 2019; 5: 9.
    1. Artzi NS, Shilo S, Hadar E, Rossman H, Barbash‐Hazan S, Ben‐Haroush A, Balicer RD, Feldman B, Wiznitzer A, Segal E. Prediction of gestational diabetes based on nationwide electronic health records. Nat Med 2020; 26: 71–76.
    1. Makino M, Yoshimoto R, Ono M, Itoko T, Katsuki T, Koseki A, Kudo M, Haida K, Kuroda J, Yanagiya R, Saitoh E, Hoshinaga K, Yuzawa Y, Suzuki A. Artificial intelligence predicts the progression of diabetic kidney disease using big data machine learning. Sci Rep 2019; 9: 11862.
    1. Annarumma M, Withey SJ, Bakewell RJ, Pesce E, Goh V, Montana G. Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks. Radiology 2019; 291: 196–202.
    1. Titano JJ, Badgeley M, Schefflein J, Pain M, Su A, Cai M, Swinburne N, Zech J, Kim J, Bederson J, Mocco J, Drayer B, Lehar J, Cho S, Costa A, Oermann EK. Automated deep‐neural‐network surveillance of cranial images for acute neurologic events. Nat Med 2018; 24: 1337–1341.
    1. Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health‐care professionals in detecting diseases from medical imaging: a systematic review and meta‐analysis. Lancet Digit Health 2019; 1: e271–e297.
    1. Marcus GM. The Apple Watch can detect atrial fibrillation: so what now? Nat Rev Cardiol 2020; 17: 135–136.
    1. Ianevski A, Giri AK, Gautam P, Kononov A, Potdar S, Saarela J, Wennerberg K, Aittokallio T. Prediction of drug combination effects with a minimal set of experiments. Nat Mach Intell 2019; 1: 568–577.
    1. Wang X, Terashi G, Christoffer CW, Zhu M, Kihara D. Protein Docking Model Evaluation by 3D Deep Convolutional Neural Networks. Bioinformatics 2020; 36: 2113–2118.
    1. Zhdanov A, Atluri S, Wong W, Vaghei Y, Daskalakis ZJ, Blumberger DM, Frey BN, Giacobbe P, Lam RW, Milev R, Mueller DJ, Turecki G, Parikh SV, Rotzinger S, Soares CN, Brenner CA, Vila‐Rodriguez F, McAndrews MP, Kleffner K, Alonso‐Prieto E, Arnott SR, Foster JA, Strother SC, Uher R, Kennedy SH, Farzan F. Use of Machine Learning for Predicting Escitalopram Treatment Outcome From Electroencephalography Recordings in Adult Patients With Depression. JAMA Netw Open 2020; 3: e1918377.
    1. Langlotz CP, Allen B, Erickson BJ, Kalpathy‐Cramer J, Bigelow K, Cook TS, Flanders AE, Lungren MP, Mendelson DS, Rudie JDJR. A roadmap for foundational research on artificial intelligence in medical imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019; 291: 781–791.
    1. Safavi KC, Khaniyev T, Copenhaver M, Seelen M, Zenteno Langle AC, Zanger J, Daily B, Levi R, Dunn P. Development and Validation of a Machine Learning Model to Aid Discharge Processes for Inpatient Surgical Care. JAMA Netw Open 2019; 2: e1917221.
    1. Chang PJ. Moving Artificial Intelligence from Feasible to Real: Time to Drill for Gas and Build Roads. Radiology 2020; 294: 432–433.
    1. Majkowska A, Mittal S, Steiner DF, Reicher JJ, McKinney SM, Duggan GE, Eswaran K, Cameron Chen PH, Liu Y, Kalidindi SR, Ding A, Corrado GS, Tse D, Shetty S. Chest Radiograph Interpretation with Deep Learning Models: Assessment with Radiologist‐adjudicated Reference Standards and Population‐adjusted Evaluation. Radiology 2020; 294: 421–431.
    1. Timmerman D, Verrelst H, Bourne TH, De Moor B, Collins WP, Vergote I, Vandewalle J. Artificial neural network models for the preoperative discrimination between malignant and benign adnexal masses. Ultrasound Obstet Gynecol 1999; 13: 17–25.
    1. Eriksson LSE, Epstein E, Testa AC, Fischerova D, Valentin L, Sladkevicius P, Franchi D, Fruhauf F, Fruscio R, Haak LA, Opolskiene G, Mascilini F, Alcazar JL, Van Holsbeke C, Chiappa V, Bourne T, Lindqvist PG, Van Calster B, Timmerman D, Verbakel JY, Van den Bosch T, Wynants L. Ultrasound‐based risk model for preoperative prediction of lymph‐node metastases in women with endometrial cancer: model‐development study. Ultrasound Obstet Gynecol 2020; 56: 443–452.
    1. Huang YL, Chen HY. Computer‐aided diagnosis of urodynamic stress incontinence with vector‐based perineal ultrasound using neural networks. Ultrasound Obstet Gynecol 2007; 30: 1002–1006.
    1. van den Noort F, van der Vaart CH, Grob ATM, van de Waarsenburg MK, Slump CH, van Stralen M. Deep learning enables automatic quantitative assessment of puborectalis muscle and urogenital hiatus in plane of minimal hiatal dimensions. Ultrasound Obstet Gynecol 2019; 54: 270–275.
    1. Huang YL, Kuo SJ, Chang CS, Liu YK, Moon WK, Chen DR. Image retrieval with principal component analysis for breast cancer diagnosis on various ultrasonic systems. Ultrasound Obstet Gynecol 2005; 26: 558–566.
    1. Kuo SJ, Hsiao YH, Huang YL, Chen DR. Classification of benign and malignant breast tumors using neural networks and three‐dimensional power Doppler ultrasound. Ultrasound Obstet Gynecol 2008; 32: 97–102.
    1. Huang YL, Chen DR, Jiang YR, Kuo SJ, Wu HK, Moon WK. Computer‐aided diagnosis using morphological features for classifying breast lesions on ultrasound. Ultrasound Obstet Gynecol 2008; 32: 565–572.
    1. Neocleous AC, Syngelaki A, Nicolaides KH, Schizas CN. Two‐stage approach for risk estimation of fetal trisomy 21 and other aneuploidies using computational intelligence systems. Ultrasound Obstet Gynecol 2018; 51: 503–508.
    1. Bonet‐Carne E, Palacio M, Cobo T, Perez‐Moreno A, Lopez M, Piraquive JP, Ramirez JC, Botet F, Marques F, Gratacos E. Quantitative ultrasound texture analysis of fetal lungs to predict neonatal respiratory morbidity. Ultrasound Obstet Gynecol 2015; 45: 427–433.
    1. Bahado‐Singh RO, Sonek J, McKenna D, Cool D, Aydas B, Turkoglu O, Bjorndahl T, Mandal R, Wishart D, Friedman P, Graham SF, Yilmaz A. Artificial intelligence and amniotic fluid multiomics: prediction of perinatal outcome in asymptomatic women with short cervix. Ultrasound Obstet Gynecol 2019; 54: 110–118.
    1. Tsur A, Batsry L, Toussia‐Cohen S, Rosenstein MG, Barak O, Brezinov Y, Yoeli‐Ullman R, Sivan E, Sirota M, Druzin ML, Stevenson DK, Blumenfeld YJ, Aran D. Development and validation of a machine‐learning model for prediction of shoulder dystocia. Ultrasound Obstet Gynecol 2020; 56: 588–596.
    1. Jugovic D, Tumbri J, Medic M, Jukic MK, Kurjak A, Arbeille P, Salihagic‐Kadic A. New Doppler index for prediction of perinatal brain damage in growth‐restricted and hypoxic fetuses. Ultrasound Obstet Gynecol 2007; 30: 303–311.
    1. Papageorghiou AT, Kemp B, Stones W, Ohuma EO, Kennedy SH, Purwar M, Salomon LJ, Altman DG, Noble JA, Bertino E, Gravett MG, Pang R, Cheikh Ismail L, Barros FC, Lambert A, Jaffer YA, Victora CG, Bhutta ZA, Villar J, International Fetal and Newborn Growth Consortium for the 21st Century (INTERGROWTH‐21st) . Ultrasound‐based gestational‐age estimation in late pregnancy. Ultrasound Obstet Gynecol 2016; 48: 719–726.
    1. Xie HN, Wang N, He M, Zhang LH, Cai HM, Xian JB, Lin MF, Zheng J, Yang YZ. Using deep‐learning algorithms to classify fetal brain ultrasound images as normal or abnormal. Ultrasound Obstet Gynecol 2020; 56: 579–587.
    1. Gobet F. The Future of Expertise: The Need for a Multidisciplinary Approach. Journal of Expertise 2018; 1: 107–113.
    1. Littmann M, Selig K, Cohen‐Lavi L, Frank Y, Hönigschmid P, Kataka E, Mösch A, Qian K, Ron A, Schmid S, Sorbie A, Szlak L, Dagan‐Wiener A, Ben‐Tal N, Niv MY, Razansky D, Schuller BW, Ankerst D, Hertz T, Rost B. Validity of machine learning in biology and medicine increased through collaborations across fields of expertise. Nat Mach Intell 2020; 2: 18–24.
    1. American College of Radiology Data Science Institute . FDA Cleared AI Algorithms. [Accessed May 7th, 2020].
    1. Food and Drug Administration . FDA Authorizes Marketing of First Cardiac Ultrasound Software That Uses Artificial Intelligence to Guide User . 2020.
    1. Harris S. Signify Research. Artificial Inelligence in Medical Imaging to Top $2 Billion by 2023. [Accessed March 2nd, 2020].
    1. Ghorbani A, Ouyang D, Abid A, He B, Chen JH, Harrington RA, Liang DH, Ashley EA, Zou JY. Deep learning interpretation of echocardiograms. NPJ Digit Med 2020; 3: 10.
    1. Liu S, Wang Y, Yang X, Lei B, Liu L, Li SX, Ni D, Wang T. Deep Learning in Medical Ultrasound Analysis: A Review. Engineering 2019; 5: 261–275.
    1. Ouyang D, He B, Ghorbani A, Yuan N, Ebinger J, Langlotz CP, Heidenreich PA, Harrington RA, Liang DH, Ashley EA, Zou JY. Video‐based AI for beat‐to‐beat assessment of cardiac function. Nature 2020; 580: 252–256.
    1. Sharma H, Droste R, Chatelain P, Drukker L, Papageorghiou AT, Noble JA. Spatio‐Temporal Partitioning And Description Of Full‐Length Routine Fetal Anomaly Ultrasound Scans. Proc IEEE Int Symp Biomed Imaging 2019; 16: 987–990.
    1. Baumgartner CF, Kamnitsas K, Matthew J, Smith S, Kainz B, Rueckert D. Real‐Time Standard Scan Plane Detection and Localisation in Fetal Ultrasound Using Fully Convolutional Neural Networks In Medical Image Computing and Computer‐Assisted Intervention – MICCAI 2016. Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W. (eds). Springer International Publishing: Cham, 2016; 203–211.
    1. Chen H, Wu L, Dou Q, Qin J, Li S, Cheng J, Ni D, Heng P. Ultrasound Standard Plane Detection Using a Composite Neural Network Framework. IEEE Trans Cybern 2017; 47: 1576–1586.
    1. Singhal N, Mukherjee S, Perrey C. Automated assessment of endometrium from transvaginal ultrasound using Deep Learned Snake. Presented at 2017 IEEE 14th International Symposium on Biomedical Imaging, 2017; 283–286.
    1. Allen B, Dreyer K, McGinty GD. Integrating Artificial Intelligence Into Radiologic Practice: A Look to the Future. J Am Coll Radiol 2020; 17: 280–283.
    1. Frank MR, Autor D, Bessen JE, Brynjolfsson E, Cebrian M, Deming DJ, Feldman M, Groh M, Lobo J, Moro E, Wang D, Youn H, Rahwan I. Toward understanding the impact of artificial intelligence on labor. Proc Natl Acad Sci USA 2019; 116: 6531–6539.
    1. Topol EJ. Chapter six: Doctors and Patterns In: Deep medicine: how artificial intelligence can make healthcare human again (first edn). Basic Books: New York, 2019; 111–135.
    1. Recht M, Bryan RN. Artificial Intelligence: Threat or Boon to Radiologists? J Am Coll Radiol 2017; 14: 1476–1480.
    1. Mazurowski MA. Artificial Intelligence May Cause a Significant Disruption to the Radiology Workforce. J Am Coll Radiol 2019; 16: 1077–1082.
    1. Noseworthy J. The Future of Care ‐ Preserving the Patient‐Physician Relationship. N Engl J Med 2019; 381: 2265–2269.
    1. Waring L, Miller PK, Sloane C, Bolton G. Charting the practical dimensions of understaffing from a managerial perspective: The everyday shape of the UK's sonographer shortage. Ultrasound 2018; 26: 206–213.
    1. Nelson CA, Butte AJ, Baranzini SE. Integrating biomedical research and electronic health records to create knowledge‐based biologically meaningful machine‐readable embeddings. Nat Commun 2019; 10: 3045.
    1. Kansagra AP, Yu JP, Chatterjee AR, Lenchik L, Chow DS, Prater AB, Yeh J, Doshi AM, Hawkins CM, Heilbrun ME, Smith SE, Oselkin M, Gupta P, Ali S. Big Data and the Future of Radiology Informatics. Acad Radiol 2016; 23: 30–42.
    1. Chiappa V, Bogani G, Ditto A, Martinelli F, Murru G, Raspagliesi F. OP07.10: Artificial intelligence weights the importance of clinical and sonographic factors predicting nodal metastasis in endometrial cancer. Ultrasound Obstet Gynecol 2019; 54 ( S1 ): 107–107.
    1. Babic B, Gerke S, Evgeniou T, Cohen IG. Algorithms on regulatory lockdown in medicine. Science 2019; 366: 1202–1204.
    1. Wiens J, Price WN, Sjoding MW. Diagnosing bias in data‐driven algorithms for healthcare. Nat Med 2020; 26: 25–26.
    1. Drukker L, Droste R, Chatelain P, Noble JA, Papageorghiou AT. Expected‐value bias in routine third‐trimester growth scans. Ultrasound Obstet Gynecol 2020; 55: 375–382.
    1. Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, Topol EJ, Ioannidis JPA, Collins GS, Maruthappu M. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ 2020; 368: m689.
    1. Brundage M, Avin S, Wang J, Belfield H, Krueger G, Hadfield G, Khlaaf H, Yang J, Toner H, Fong R, Maharaj T, Koh PW, Hooker S, Leung J, Trask A, Bluemke E, Lebensold J, O'Keefe C, Koren M, Ryffel T, Rubinovitz JB, Besiroglu T, Carugati F, Clark J, Eckersley P, de Haas S, Johnson M, Laurie B, Ingerman A, Krawczuk I, Askell A, Cammarota R, Lohn A, Krueger D, Stix C, Henderson P, Graham L, Prunkl C, Martin B, Seger E, Zilberman N, Ó hÉigeartaigh S, Kroeger F, Sastry G, Kagan R, Weller A, Tse B, Barnes E, Dafoe A, Scharre P, Herbert‐Voss A, Rasser M, Sodhani S, Flynn C, Gilbert TK, Dyer L, Khan S, Bengio Y, Anderljung M. Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. April 2020.
    1. Institute for Ethics in AI. University of Oxford. [Accessed May 7th, 2020].
    1. Awad E, Dsouza S, Kim R, Schulz J, Henrich J, Shariff A, Bonnefon JF, Rahwan I. The Moral Machine experiment. Nature 2018; 563: 59–64.
    1. Char DS, Shah NH, Magnus D. Implementing Machine Learning in Health Care ‐ Addressing Ethical Challenges. N Engl J Med 2018; 378: 981–983.
    1. Mittelstadt B. Principles alone cannot guarantee ethical AI. Nat Mach Intell 2019; 1: 501–507.
    1. Geis JR, Brady AP, Wu CC, Spencer J, Ranschaert E, Jaremko JL, Langer SG, Borondy Kitts A, Birch J, Shields WF, van den Hoven van Genderen R, Kotter E, Wawira Gichoya J, Cook TS, Morgan MB, Tang A, Safdar NM, Kohli M. Ethics of Artificial Intelligence in Radiology: Summary of the Joint European and North American Multisociety Statement. Radiology 2019; 293: 436–440.
    1. Price WN 2nd, Gerke S, Cohen IG. Potential Liability for Physicians Using Artificial Intelligence. JAMA 2019. DOI: 10.1001/jama.2019.15064.
    1. Liu Y, Chen PC, Krause J, Peng L. How to Read Articles That Use Machine Learning: Users' Guides to the Medical Literature. JAMA 2019; 322: 1806–1816.
    1. CONSORT‐AI and SPIRIT‐AI Steering Group . Reporting guidelines for clinical trials evaluating artificial intelligence interventions are needed. Nat Med 2019; 25: 1467–1468.
    1. Abu‐Rustum RS, Abuhamad AZ. Fetal imaging: past, present, and future. A journey of marvel. BJOG 2018; 125: 1568.
    1. The Alan Turing Institute . Frequently Asked Questions. [Accessed December 20th, 2019].

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다