Generating automated kidney transplant biopsy reports combining molecular measurements with ensembles of machine learning classifiers

Jeff Reeve, Georg A Böhmig, Farsad Eskandary, Gunilla Einecke, Gaurav Gupta, Katelynn Madill-Thomsen, Martina Mackova, Philip F Halloran, INTERCOMEX MMDx-Kidney Study Group, Jeff Reeve, Georg A Böhmig, Farsad Eskandary, Gunilla Einecke, Gaurav Gupta, Katelynn Madill-Thomsen, Martina Mackova, Philip F Halloran, INTERCOMEX MMDx-Kidney Study Group

Abstract

We previously reported a system for assessing rejection in kidney transplant biopsies using microarray-based gene expression data, the Molecular Microscope® Diagnostic System (MMDx). The present study was designed to optimize the accuracy and stability of MMDx diagnoses by replacing single machine learning classifiers with ensembles of diverse classifier methods. We also examined the use of automated report sign-outs and the agreement between multiple human interpreters of the molecular results. Ensembles generated diagnoses that were both more accurate than the best individual classifiers, and nearly as stable as the best, consistent with expectations from the machine learning literature. Human experts had ≈93% agreement (balanced accuracy) signing out the reports, and random forest-based automated sign-outs showed similar levels of agreement with the human experts (92% and 94% for predicting the expert MMDx sign-outs for T cell-mediated (TCMR) and antibody-mediated rejection (ABMR), respectively). In most cases disagreements, whether between experts or between experts and automated sign-outs, were in biopsies near diagnostic thresholds. Considerable disagreement with histology persisted. The balanced accuracies of MMDx sign-outs for histology diagnoses of TCMR and ABMR were 73% and 78%, respectively. Disagreement with histology is largely due to the known noise in histology assessments (ClinicalTrials.gov NCT01299168).

Keywords: basic (laboratory) research/science; biopsy; kidney failure/injury; kidney transplantation/nephrology; microarray/gene array; molecular biology; rejection: T cell mediated (TCMR); rejection: antibody-mediated (ABMR).

© 2019 The American Society of Transplantation and the American Society of Transplant Surgeons.

References

REFERENCES

    1. Jameson JL, Longo DL. Precision medicine - personalized, problematic, and promising. N Engl J Med. 2015;372(23):2229-2234.
    1. Eskandary F, Regel H, Baumann L, et al. A randomized, placebo-controlled trial of bortezomib in late antibody-mediated kidney transplant rejection: the BORTEJECT study. J Am Soc Nephrol. 2017;29(2):591-605.
    1. Eskandary F, Jilma B, Mulbacher J, et al. Anti-C1s monoclonal antibody BIVV009 in late antibody-mediated kidney allograft rejection - results from a first-in-patient phase 1 trial. Am J Transplant. 2018;18(4):916-926.
    1. Loupy A, Haas M, Solez K, et al. The Banff 2015 Kidney Meeting Report: current challenges in rejection classification and prospects for adopting molecular pathology. Am J Transplant. 2017;17(1):28-41.
    1. Furness PN, Taub N, Assmann KJ, et al. International variation in histologic grading is large, and persistent feedback does not improve reproducibility. Am J Surg Pathol. 2003;27(6):805-810.
    1. Furness PN, Taub N. International variation in the interpretation of renal transplant biopsies: report of the CERTPAP Project. Kidney Int. 2001;60(5):1998-2012.
    1. Reeve J, Sellares J, Mengel M, et al. Molecular diagnosis of T cell-mediated rejection in human kidney transplant biopsies. Am J Transplant. 2013;13(3):645-655.
    1. Schinstock CA, Sapir-Pichhadze R, Naesens M, et al. Banff survey on antibody-mediated rejection clinical practices in kidney transplantation: diagnostic misinterpretation has potential therapeutic implications. Am J Transplant. 2018;19(1):123-131.
    1. Sellares J, Reeve J, Loupy A, et al. Molecular diagnosis of antibody-mediated rejection in human kidney transplants. Am J Transplant. 2013;13(4):971-983.
    1. Halloran PF, Famulski KS, Reeve J. Molecular assessment of disease states in kidney transplant biopsy samples. Nat Rev Nephrol. 2016;12(9):534-548.
    1. Reeve J, Bohmig GA, Eskandary F, et al. Precision molecular phenotyping of kidney transplant biopsies using archetypal analysis. JCI Insight. 2017;2(12):94197.
    1. Madill-Thomsen KS, Wiggins RC, Eskandary F, Bohmig GA, Halloran PF. The effect of cortex/medulla proportions on molecular diagnoses in kidney transplant biopsies: rejection and injury can be assessed in medulla. Am J Transplant. 2017;17(8):2117-2128.
    1. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. New York, NY: Springer;2009.
    1. Kuhn M, Johnson K. Applied Predictive Modeling, 1st edn. New York, NY: Springer-Verlag; 2013.
    1. James G, Witten D, Hastie T, Tibshirani R. An Introduction to Statistical Learning, 1st edn. New York, NY: Springer-Verlag; 2013.
    1. Halloran PF, Reeve J, INTERCOMEX Study Group. Validating molecular microscope readings and estimating agreement with histology. Am J Transplant. 2018;18[S4]:422-423.
    1. Halloran PF, Venner JM, Madill-Thomsen K, et al. Review: the transcripts associated with organ allograft rejection. Am J Transplant. 2018;18(4):785-795.
    1. Reeve J, Einecke G, Mengel M, et al. Diagnosing rejection in renal transplants: a comparison of molecular- and histopathology-based approaches. Am J Transplant. 2009;9(8):1802-1810.
    1. Halloran PF, Pereira AB, Chang J, et al. Potential impact of microarray diagnosis of T cell-mediated rejection in kidney transplants: the INTERCOM study. Am J Transplant. 2013;13(9):2352-2363.
    1. Halloran PF, Pereira AB, Chang J, et al. Microarray diagnosis of antibody-mediated rejection in kidney transplant biopsies: an international prospective study (INTERCOM). Am J Transplant. 2013;13(11):2865-2874.
    1. Halloran PF, Venner JM, Famulski KS. Comprehensive analysis of transcript changes associated with allograft rejection: combining universal and selective features. Am J Transplant. 2017;17(7):1754-1769.
    1. Halloran PF, Akalin E, Aubert O, et al. Real time central assessment of kidney transplant indication biopsies by microarrays: the INTERCOMEX Study. Am J Transplant. 2017;17(11):2851-2862.
    1. Oza NC, Turner K. Classifier ensembles: select real-world applications. Information Fusion. 2008;9(1):4-20.
    1. Rokach L. Ensemble Methods in Supervised Learning. In: Maimon O, Rokach L, eds. Data Mining and Knowledge Discovery Handbook. Boston, MA: Springer US; 2010:959-979.
    1. Reeve J, Bohmig GA, Eskandary F, et al. Assessing rejection-related disease in kidney transplant biopsies based on archetypal analysis of molecular phenotypes. JCI Insight. 2017;2(12):e94197.
    1. R Core Team. R: A Language and Environment for Statistical Computing. 2018. Vienna, Austria: R Foundation for Statistical Computing; 2015.
    1. Sing T, Sander O, Beerenwinkel N, Lengauer T. ROCR: visualizing classifier performance in R. Bioinformatics. 2005;21(20):3940-3941.
    1. Eugster MJA, Leish F. From Spider-Man to hero --archetypal analysis in R. J Stat Softw. 2009;30(8):1-23.
    1. Le S, Josse J, Husson F. An R package for multivariate analysis. J Stat Softw. 2008;25(1):1-8.
    1. ReporteRs: Microsoft Word and PowerPoint Documents Generation [computer program]. Version 0.8.10 2019.
    1. Kuhn M, Wing J, Weston S, et al. caret: Classification and Regression Training. R Statistical Software [6.0-81]. 2018.
    1. Random Forests for Survival, Regression and Classification (RF-SRC) [computer program]. Version 2.7.0 2018.
    1. Einecke G, Reeve J, Halloran PF. A molecular biopsy test based on arteriolar under-hyalinosis reflects increased probability of rejection related to under-immunosuppression. Am J Transplant. 2017;18(4):821-831.
    1. Mengel M, Chang J, Kayser D, et al. The molecular phenotype of six-week protocol biopsies from human renal allografts: reflections of prior injury but not future course. Am J Transplant. 2011;11(4):708-718.
    1. Eskandary F, Bond G, Kozakowski N, et al. Diagnostic contribution of donor-specific antibody characteristics to uncover late silent antibody-mediated rejection-results of a cross-sectional screening study. Transplant. 2017;101(3):631-641.
    1. Einecke G, Reeve J, Sis B, et al. A molecular classifier for predicting future graft loss in late kidney transplant biopsies. J Clin Investig. 2010;120:1862-1872.
    1. Halloran PF, Chang J, Famulski K, et al. Disappearance of T cell-mediated rejection despite continued antibody-mediated rejection in late kidney transplant recipients. JASN. 2015;26:1711-1720.
    1. Sis B, Jhangri G, Bunnag S, Allanach K, Kaplan B, Halloran PF. Endothelial gene expression in kidney transplants with alloantibody indicates antibody-mediated damage despite lack of C4d staining. Am J Transplant. 2009;9:2312-2323.
    1. Einecke G, Sis B, Reeve J, et al. Antibody-mediated microcirculation injury is the major cause of late kidney transplant failure. Am J Transplant. 2009;9:2520-2531.
    1. Famulski KS, de Freitas DG, Kreepala C, et al. Molecular phenotypes of acute kidney injury in human kidney transplants. JASN. 2012;23:948-958.
    1. Venner JM, Famulski KS, Reeve J, Chang J, Halloran PF. Relationships among injury, fibrosis, and time in human kidney transplants. J Clin Investig Insight. 2016;1:e85323. .

Source: PubMed

Sponsors et collaborateurs

Les conditions médicales

Interventions en matière de drogue

3
S'abonner