Concordance Study Between IBM Watson for Oncology and Clinical Practice for Patients with Cancer in China

Abstract

Background: IBM Watson for Oncology (WFO), which can use natural language processing to evaluate data in structured and unstructured formats, has begun to be used in China. It provides physicians with evidence-based treatment options and ranks them in three categories for treatment decision support. This study was designed to examine the concordance between the treatment recommendation proposed by WFO and actual clinical decisions by oncologists in our cancer center, which would reflect the differences of cancer treatment between China and the U.S.

Patients and methods: Retrospective data from 362 patients with cancer were ingested into WFO from April 2017 to October 2017. WFO recommendations were provided in three categories: recommended, for consideration, and not recommended. Concordance was analyzed by comparing the treatment decisions proposed by WFO with those of the multidisciplinary tumor board. Concordance was achieved when the oncologists' treatment decisions were in the recommended or for consideration categories in WFO.

Results: Ovarian cancer showed the highest concordance, which was 96%. Lung cancer and breast cancer obtained a concordance of slightly above 80%. The concordance of rectal cancer was 74%, whereas colon cancer and cervical cancer showed the same concordance of 64%. In particular, the concordance of gastric cancer was very low, only 12%, and 88% of cases were under physicians choice.

Conclusion: Different cancer types showed different concordances, and only gastric cancers were significantly less likely to be concordant. Incidence and pharmaceuticals may be the major cause of discordance. To be comprehensively and rapidly applied in China, WFO needs to accelerate localization. ClinicalTrials.gov Identifier: NCT03400514.

Implications for practice: IBM Watson for Oncology (WFO) has begun to be used in China. In this study, concordance was examined between the treatment recommendation proposed by WFO and clinical decisions for 362 patients in our cancer center, which could reflect the differences of cancer treatment between China and the U.S. Different cancer types showed different concordances, and only gastric cancers were significantly less likely to be concordant. Incidence and pharmaceuticals may be the major causes of discordance. To be comprehensively and rapidly applied in China, WFO needs to accelerate localization. This study may have a significant effect on application of artificial intelligence systems in China.

Keywords: Artificial Intelligence; China; Concordance; Watson for Oncology.

Conflict of interest statement

Disclosures of potential conflicts of interest may be found at the end of this article.

© AlphaMed Press 2018.

Figures

Figure 1.

CONSORT diagram. Abbreviation: WFO, Watson for Oncology.

Figure 2.

Treatment concordance of different cancer types between Watson for Oncology (WFO) and our cancer center. Treatment was considered concordant if the delivered treatment was rated as either recommended or for consideration by WFO and discordant if the delivered treatment was either not recommended by WFO or was physician's choice (not included in WFO). Concordance of lung cancer by histology (A) and tumor stage (B). Concordance of breast cancer by histology (C) and tumor stage (D). Concordance of gastric cancer (E), colon cancer (F), rectal cancer (G), cervical cancer (H), ovarian cancer (I), and overall (J). Abbreviations: NSCLC, non‐small cell lung cancer; SCLC, small cell lung cancer; TNBC, triple‐negative breast cancer.

Figure 3.

Distribution of actual clinical decisions for gastric cancer.

Figure 4.

Treatment regimens produced by Watson for Oncology (WFO) for a patient. WFO treatment regimens of the first diagnosis (A) and after clinical therapy (B).