A holistic comparative analysis of diagnostic tests for urothelial carcinoma: a study of Cxbladder Detect, UroVysion® FISH, NMP22® and cytology based on imputation of multiple datasets

Vivienne Breen, Nikola Kasabov, Ashish M Kamat, Elsie Jacobson, James M Suttie, Paul J O'Sullivan, Laimonis Kavalieris, David G Darling, Vivienne Breen, Nikola Kasabov, Ashish M Kamat, Elsie Jacobson, James M Suttie, Paul J O'Sullivan, Laimonis Kavalieris, David G Darling

Abstract

Background: Comparing the relative utility of diagnostic tests is challenging when available datasets are small, partial or incomplete. The analytical leverage associated with a large sample size can be gained by integrating several small datasets to enable effective and accurate across-dataset comparisons. Accordingly, we propose a methodology for a holistic comparative analysis and ranking of cancer diagnostic tests through dataset integration and imputation of missing values, using urothelial carcinoma (UC) as a case study.

Methods: Five datasets comprising samples from 939 subjects, including 89 with UC, where up to four diagnostic tests (cytology, NMP22®, UroVysion® Fluorescence In-Situ Hybridization (FISH) and Cxbladder Detect) were integrated into a single dataset containing all measured records and missing values. The tests were firstly ranked using three criteria: sensitivity, specificity and a standard variable (feature) ranking method popularly known as signal-to-noise ratio (SNR) index derived from the mean values for all subjects clinically known to have UC versus healthy subjects. Secondly, step-wise unsupervised and supervised imputation (the latter accounting for the 'clinical truth' as determined by cystoscopy) was performed using personalized modelling, k-nearest-neighbour methods, multiple logistic regression and multilayer perceptron neural networks. All imputation models were cross-validated by comparing their post-imputation predictive accuracy for UC with their pre-imputation accuracy. Finally, the post-imputation tests were re-ranked using the same three criteria.

Results: In both measured and imputed data sets, Cxbladder Detect ranked higher for sensitivity, and urine cytology a higher specificity, when compared with other UC tests. Cxbladder Detect consistently ranked higher than FISH and all other tests when SNR analyses were performed on measured, unsupervised and supervised imputed datasets. Supervised imputation resulted in a smaller cross-validation error. Cxbladder Detect was robust to imputation showing a 2% difference in its predictive versus clinical accuracy, outperforming FISH, NMP22 and cytology.

Conclusion: All data analysed, pre- and post-imputation showed that Cxbladder Detect had higher SNR and outperformed all other comparator tests, including FISH. The methodology developed and validated for comparative ranking of the diagnostic tests for detecting UC, may be further applied to other cancer diagnostic datasets across population groups and multiple datasets.

Figures

Fig. 1
Fig. 1
Ranking of tests in a univariate mode using SNR on the integrated dataset before imputation
Fig. 2
Fig. 2
Rankings of tests for the integrated dataset after a supervised and b unsupervised imputation
Fig. 3
Fig. 3
Comparisons after a supervised and b unsupervised imputation in two-dimensional contour plots of sensitivity and specificity

References

    1. Nur U, Shack LG, Rachet B, Carpenter JR, Coleman MP. Modelling relative survival in the presence of incomplete data: a tutorial. Int J Epidemiol. 2010;39:118–28. doi: 10.1093/ije/dyp309.
    1. He Y, Yucel R, Zaslavsky AM. Misreporting, missing data and multiple imputation: improving accuracy of cancer registry databases. Chance (NY) 2008;21:55–8.
    1. He Y, Zaslavsky AM, Harrington DP, Catalano P, Landrum MB. Multiple imputation in a large-scale complex survey: a practical guide. Stat Methods Med Res. 2010;19:653–70. doi: 10.1177/0962280208101273.
    1. Eisemann N, Waldmann A, Katalinic A. Imputation of missing values of tumour stage in population-based cancer registration. BMC Med Res Methodol. 2011;11:129–41. doi: 10.1186/1471-2288-11-129.
    1. Guzel C, Kaya M, Yildiz O, Bilge HS. Breast cancer diagnosis based on naïve bayes machine learning with KNN missing data imputation. AWERProcedia Inf Technol Comput Sci. 2010;4:401–7.
    1. Li Y, Willer C, Sanna S, Abecasis G. Genotype imputation. Annu Rev Genomics Hum Genet. 2009;10:387–406. doi: 10.1146/annurev.genom.9.081307.164242.
    1. Kasabov N, editor. Springer handbook of bio-neuroinformatics. Heidelbeg: Springer; 2014.
    1. Rubin DB. Multiple imputation for non-response in surveys. New York: Wiley; 1987.
    1. Rubin DB. Multiple imputation after 18+ years. J Am Stat Assoc. 1996;91:473–89. doi: 10.1080/01621459.1996.10476908.
    1. Little RJA, Rubin DB. Statistical analysis with missing data. 2. Hoboken: Wiley; 2002.
    1. Su YS, Gelman A, Hill J, Yajima M. Multiple imputation with diagnostics (mi) in R: Opening windows into the black box. J Stat Softw. 2011;45:1–30.
    1. Gilks WR, Richardson S, Spiegelhalter D. Markov chain Monte Carlo in practice. New York: Chapman and Hall; 1996.
    1. Davis R, Jones JS, Barocas DA, Castle EP, Lang EK, Leveillee RJ, et al. Diagnosis, evaluation and follow-up of asymptomatic microhematuria (AMH) in adults: AUA guideline. J Urol. 2012;188(6 Suppl):2473–81. doi: 10.1016/j.juro.2012.09.078.
    1. Raitanen MP, Aine R, Rintala E, Kallio P, Juusela H, Tammela TL, et al. Differences between local and review urinary cytology in diagnosis of bladder cancer. an inter-observer multicenter analysis. Eur Urol. 2002;41:284–9. doi: 10.1016/S0302-2838(02)00006-4.
    1. Grossman HB, Messing E, Soloway M, Tomera K, Katz G, Berger Y, et al. Detection of bladder cancer using a point-of-care proteomic assay. JAMA. 2005;293:810–6. doi: 10.1001/jama.293.7.810.
    1. Sokolova IA, Halling KC, Jenkins RB, Burkhardt HM, Meyer RG, Seelig SA, et al. The development of a multitarget, multicolor fluorescence in situ hybridization assay for the detection of urothelial carcinoma in urine. J Mol Diagn. 2000;2:116–23. doi: 10.1016/S1525-1578(10)60625-3.
    1. Halling KC, Kipp BR. Bladder cancer detection using FISH (UroVysion) Adv Anat Pathol. 2008;15:279–86. doi: 10.1097/PAP.0b013e3181832320.
    1. O’Sullivan P, Sharples K, Dalphin M, Davidson P, Gilling P, Cambridge L, et al. A multigene urine test for the detection and stratification of bladder cancer in patients presenting with hematuria. J Urol. 2012;188:741–7. doi: 10.1016/j.juro.2012.05.003.
    1. Kamat AM, Karam JA, Grossman HB, Kader AK, Munsell M, Dinney CP. Prospective trial to identify optimal cancer surveillance protocol. BJU Int. 2011;108:1119–23. doi: 10.1111/j.1464-410X.2010.10026.x.
    1. Kasabov N. Evolving connectionist systems: the knowledge engineering approach. 2. London: Springer; 2007.
    1. Kasabov N, Hu Y. Integrated optimisation method for personalised modelling and case studies for medical decision support. Int J Funct Inform Personal Med. 2010;3:236–56. doi: 10.1504/IJFIPM.2010.039123.
    1. Hajdinjak T. UroVysion FISH test for detecting urothelial cancers: meta-analysis of diagnostic accuracy and comparison with urinary cytology testing. Urol Oncol. 2008;26:646–51. doi: 10.1016/j.urolonc.2007.06.002.
    1. Dimashkieh H, Wolff DJ, Smith TM, Houser PM, Nietert PJ, Yang J. Evaluation of Urovysion and cytology for bladder cancer detection. Cancer Cytopathol. 2013;121:591–7. doi: 10.1002/cncy.21327.
    1. Sullivan PS, Nooraie F, Sanchez H, Hirschowitz S, Levin M, Rao PN, et al. Comparison of ImmuneCyt, UroVysion and urine cytology in detection of recurrent urothelial carcinoma. Cancer Cytopath. 2009;117:167–73. doi: 10.1002/cncy.20026.
    1. Koppie TM, Bochner BH. Bladder cancer staging. In: Scardino PT, Marston Lineham W, Zelefsky MJ, Vogelzang NJ, editors. Comprehensive textbook of genitourinary oncology. 4. Baltimore: Lipincott Williams Wilson; 2011. pp. 343–8.
    1. Stafford P, Cichacz Z, Woodbury NW, Johnston SA. Immunosignature system for diagnosis of cancer. Proc Natl Acad Sci U S A. 2014;111:E3072–80. doi: 10.1073/pnas.1409432111.

Source: PubMed

Patrocinadores y Colaboradores

Condiciones médicas

Intervenciones de drogas

3
Suscribir