Development and performance of CUHAS-ROBUST application for pulmonary rifampicin-resistance tuberculosis screening in Indonesia

Bumi Herman, Wandee Sirichokchatchawan, Sathirakorn Pongpanich, Chanin Nantasenamat, Bumi Herman, Wandee Sirichokchatchawan, Sathirakorn Pongpanich, Chanin Nantasenamat

Abstract

Background and objectives: Diagnosis of Pulmonary Rifampicin Resistant Tuberculosis (RR-TB) with the Drug-Susceptibility Test (DST) is costly and time-consuming. Furthermore, GeneXpert for rapid diagnosis is not widely available in Indonesia. This study aims to develop and evaluate the CUHAS-ROBUST model performance, an artificial-intelligence-based RR-TB screening tool.

Methods: A cross-sectional study involved suspected all type of RR-TB patients with complete sputum Lowenstein Jensen DST (reference) and 19 clinical, laboratory, and radiology parameter results, retrieved from medical records in hospitals under the Faculty of Medicine, Hasanuddin University Indonesia, from January 2015-December 2019. The Artificial Neural Network (ANN) models were built along with other classifiers. The model was tested on participants recruited from January 2020-October 2020 and deployed into CUHAS-ROBUST (index test) application. Sensitivity, specificity, and accuracy were obtained for assessment.

Results: A total of 487 participants (32 Multidrug-Resistant/MDR 57 RR-TB, 398 drug-sensitive) were recruited for model building and 157 participants (23 MDR and 21 RR) in prospective testing. The ANN full model yields the highest values of accuracy (88% (95% CI 85-91)), and sensitivity (84% (95% CI 76-89)) compare to other models that show sensitivity below 80% (Logistic Regression 32%, Decision Tree 44%, Random Forest 25%, Extreme Gradient Boost 25%). However, this ANN has lower specificity among other models (90% (95% CI 86-93)) where Logistic Regression demonstrates the highest (99% (95% CI 97-99)). This ANN model was selected for the CUHAS-ROBUST application, although still lower than the sensitivity of global GeneXpert results (87.5%).

Conclusion: The ANN-CUHAS ROBUST outperforms other AI classifiers model in detecting all type of RR-TB, and by deploying into the application, the health staff can utilize the tool for screening purposes particularly at the primary care level where the GeneXpert examination is not available.

Trial registration: NCT04208789.

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Fig 1. Flowchart of participants recruitment.
Fig 1. Flowchart of participants recruitment.
The figure consists of two diagrams. The upper diagram illustrates the selection of data for model building and the lower diagram for the prospective data collection. Notice that some prospective participants were excluded due to procedural reasons. As 40 people were excluded for showing no growth on Lowenstein Jensen culture after eight weeks. Four in-patient participants were scheduled to have a DST but later pronounced death before DST could take place.
Fig 2. ANN structure of the CUHAS-ROBUST…
Fig 2. ANN structure of the CUHAS-ROBUST model.
This figure depicts the ANN model with 19 parameters, two hidden layers with two nodes in each layer and the blue lines show the weight of bias in each node.

References

    1. WHO. WHO consolidated guidelines on drug-resistant tuberculosis treatment. Geneva: World Health Organization; 2019.
    1. WHO. Global Tuberculosis Report. Geneva: World Health Organization; 2019.
    1. Kaur R, Jindal N, Arora S, Kataria S. Epidemiology of Rifampicin Resistant Tuberculosis and Common Mutations in rpoB Gene of Mycobacterium tuberculosis: A Retrospective Study from Six Districts of Punjab (India) Using Xpert MTB/RIF Assay. J Lab Physicians. 2016;8(2):96–100. 10.4103/0974-2727.180789
    1. Orlando S, Triulzi I, Ciccacci F, Palla I, Palombi L, Marazzi MC, et al.. Delayed diagnosis and treatment of tuberculosis in HIV+ patients in Mozambique: A cost-effectiveness analysis of screening protocols based on four symptom screening, smear microscopy, urine LAM test and Xpert MTB/RIF. PLoS One. 2018;13(7):e0200523. 10.1371/journal.pone.0200523
    1. Joshi B, Lestari T, Graham SM, Baral SC, Verma SC, Ghimire G, et al.. The implementation of Xpert MTB/RIF assay for diagnosis of tuberculosis in Nepal: A mixed-methods analysis. PLOS ONE. 2018;13(8):e0201731. 10.1371/journal.pone.0201731
    1. Meyer AJ, Atuheire C, Worodria W, Kizito S, Katamba A, Sanyu I, et al.. Sputum quality and diagnostic performance of GeneXpert MTB/RIF among smear-negative adults with presumed tuberculosis in Uganda. PloS one. 2017;12(7):e0180572–e. 10.1371/journal.pone.0180572
    1. Kim S. Drug-susceptibility testing in tuberculosis: Methods and reliability of results. The European respiratory journal: official journal of the European Society for Clinical Respiratory Physiology. 2005;25:564–9. 10.1183/09031936.05.00111304
    1. Pearce EC, Woodward JF, Nyandiko WM, Vreeman RC, Ayaya SO. A Systematic Review of Clinical Diagnostic Systems Used in the Diagnosis of Tuberculosis in Children. AIDS Research and Treatment. 2012;2012:401896. 10.1155/2012/401896
    1. Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. Journal of Biomedical Informatics. 2002;35(5):352–9. 10.1016/s1532-0464(03)00034-0
    1. Abiodun OI, Jantan A, Omolara AE, Dada KV, Mohamed NA, Arshad H. State-of-the-art in artificial neural network applications: A survey. Heliyon. 2018;4(11):e00938. 10.1016/j.heliyon.2018.e00938
    1. Martínez D, Heudebert G, Seas C, Henostroza G, Rodriguez M, Zamudio C, et al.. Clinical Prediction Rule for Stratifying Risk of Pulmonary Multidrug-Resistant Tuberculosis. PLoS One. 2010;5(8):e12082. 10.1371/journal.pone.0012082
    1. Évora LHRA Seixas JM, Kritski AL. Neural network models for supporting drug and multidrug resistant tuberculosis screening diagnosis. Neurocomputing. 2017;265:116–26.
    1. Ramalho DMP, Miranda PFC, Andrade MK, Brígido T, Dalcolmo MP, Mesquita E, et al.. Outcomes from patients with presumed drug resistant tuberculosis in five reference centers in Brazil. BMC Infect Dis. 2017;17(1):571. 10.1186/s12879-017-2669-1
    1. Schönfeld N, Bergmann T, Vesenbeckh S, Mauch H, Bettermann G, Bauer TT, et al.. Minimal inhibitory concentrations of first-line drugs of multidrug-resistant tuberculosis isolates. Lung India. 2012;29(4):309–12. 10.4103/0970-2113.102794
    1. Pradipta IS, Forsman LD, Bruchfeld J, Hak E, Alffenaar JW. Risk factors of multidrug-resistant tuberculosis: A global systematic review and meta-analysis. The Journal of infection. 2018;77(6):469–78. 10.1016/j.jinf.2018.10.004
    1. Hata K, Nakagawa T, Mizuno M, Yanagi N, Kitamura H, Hayashi T, et al.. Relationship between smoking and a new index of arterial stiffness, the cardio-ankle vascular index, in male workers: a cross-sectional study. Tob Induc Dis. 2012;10(1):11–. 10.1186/1617-9625-10-11
    1. Coutinho AE, Chapman KE. The anti-inflammatory and immunosuppressive effects of glucocorticoids, recent developments and mechanistic insights. Mol Cell Endocrinol. 2011;335(1):2–13. 10.1016/j.mce.2010.04.005
    1. 2. Classification and Diagnosis of Diabetes: Standards of Medical Care in Diabetes—2020. 2020;43(Supplement 1):S14–S31.
    1. Bhatta L, Leivseth L, Mai X-M, Henriksen AH, Carslake D, Chen Y, et al.. GOLD Classifications, COPD Hospitalization, and All-Cause Mortality in Chronic Obstructive Pulmonary Disease: The HUNT Study. Int J Chron Obstruct Pulmon Dis. 2020;15:225–33. 10.2147/COPD.S228958
    1. Parekh BS, Ou C-Y, Fonjungo PN, Kalou MB, Rottinghaus E, Puren A, et al.. Diagnosis of Human Immunodeficiency Virus Infection. Clin Microbiol Rev. 2018;32(1):e00064–18. 10.1128/CMR.00064-18
    1. Little RR, Rohlfing C, Sacks DB. The National Glycohemoglobin Standardization Program: Over 20 Years of Improving Hemoglobin A(1c) Measurement. Clin Chem. 2019;65(7):839–48. 10.1373/clinchem.2018.296962
    1. Deun A, Hossain M, Gumusboga M, Rieder H. Ziehl-Neelsen staining: Theory and practice. The international journal of tuberculosis and lung disease: the official journal of the International Union against Tuberculosis and Lung Disease. 2008;12:108–10.
    1. Caulfield AJ, Wengenack NL. Diagnosis of active tuberculosis disease: From microscopy to molecular techniques. Journal of Clinical Tuberculosis and Other Mycobacterial Diseases. 2016;4:33–43. 10.1016/j.jctube.2016.05.005
    1. Wáng YXJ, Chung MJ, Skrahin A, Rosenthal A, Gabrielian A, Tartakovsky M. Radiological signs associated with pulmonary multi-drug resistant tuberculosis: an analysis of published evidences. Quant Imaging Med Surg. 2018;8(2):161–73. 10.21037/qims.2018.03.06
    1. Gilpin C, Korobitsyn A, Weyer K. Current tools available for the diagnosis of drug-resistant tuberculosis. Ther Adv Infect Dis. 2016;3(6):145–51. 10.1177/2049936116673553
    1. Singh UB, Pandey P, Mehta G, Bhatnagar AK, Mohan A, Goyal V, et al.. Genotypic, Phenotypic and Clinical Validation of GeneXpert in Extra-Pulmonary and Pulmonary Tuberculosis in India. PloS one. 2016;11(2):e0149258–e. 10.1371/journal.pone.0149258
    1. Phillips PPJ, Mendel CM, Nunn AJ, McHugh TD, Crook AM, Hunt R, et al.. A comparison of liquid and solid culture for determining relapse and durable cure in phase III TB trials for new regimens. BMC Medicine. 2017;15(1):207. 10.1186/s12916-017-0955-9
    1. Varghese B, Al-Omari R, Al-Hajoj S. Inconsistencies in drug susceptibility testing of Mycobacterium tuberculosis: Current riddles and recommendations. International Journal of Mycobacteriology. 2013;2(1):14–7. 10.1016/j.ijmyco.2012.11.003
    1. Lorian V, Lacasse ML. N-Acetyl-L-Cysteine Sputum Homogenization and Its Mechanism of Action on Isolation of Tubercle Bacilli. Diseases of the Chest. 1967;51(3):275–7. 10.1378/chest.51.3.275
    1. Stinson K, Eisenach K, Kayes S, Matsumoto M, Siddiqi S, Nakashima S, et al.. Global Laboratory Initiative a Working Group of the Stop TB Partnership: Mycobacteriology Laboratory Manual. 2014:147.
    1. Hall L, Jude KP, Clark SL, Wengenack NL. Antimicrobial susceptibility testing of Mycobacterium tuberculosis complex for first and second line drugs by broth dilution in a microtiter plate format. J Vis Exp. 2011(52):3094. 10.3791/3094
    1. Saia R, Carta S, Reforgiato Recupero D, Fenu G, Saia M. A Discretized Enriched Technique to Enhance Machine Learning Performance in Credit Scoring 2019.
    1. Al-Shayea Q. Artificial Neural Networks in Medical Diagnosis. Int J Comput Sci Issues. 2011;8:150–4.
    1. Team RDC. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2020.
    1. Günther F, Fritsch S. neuralnet: Training of Neural Networks. R Journal. 2010;2.
    1. Nie F, Zhanxuan H, Li X. An investigation for loss functions widely used in machine learning. Communications in Information and Systems. 2018;18:37–52.
    1. Weber K, Langille J. Improving Classification Accuracy Assessments with Statistical Bootstrap Resampling Techniques. Giscience & Remote Sensing—GISCI REMOTE SENS. 2007;44:237–50.
    1. Blank S. Resampling Stats for Excel. Illinois, USA2019.
    1. Soeroto AY, Lestari BW, Santoso P, Chaidir L, Andriyoko B, Alisjahbana B, et al.. Evaluation of Xpert MTB-RIF guided diagnosis and treatment of rifampicin-resistant tuberculosis in Indonesia: A retrospective cohort study. PloS one. 2019;14(2):e0213017–e. 10.1371/journal.pone.0213017
    1. Hajian-Tilaki K. Sample size estimation in diagnostic test studies of biomedical informatics. Journal of Biomedical Informatics. 2014;48:193–204. 10.1016/j.jbi.2014.02.013
    1. Weng C-H, Huang TC-K, Han R-P. Disease prediction with different types of neural network classifiers. Telematics and Informatics. 2016;33(2):277–92.
    1. Manca DP. Do electronic medical records improve quality of care? Yes. Can Fam Physician. 2015;61(10):846–51.
    1. Alwosheel A, van Cranenburgh S, Chorus CG. Is your dataset big enough? Sample size requirements when using artificial neural networks for discrete choice analysis. Journal of Choice Modelling. 2018;28:167–82.
    1. Alhashimi FH, Khabour OF, Alzoubi KH, Al-Shatnawi SF. Attitudes and beliefs related to reporting alcohol consumption in research studies: a case from Jordan. Pragmat Obs Res. 2018;9:55–61. 10.2147/POR.S172613
    1. Rusanov A, Weiskopf NG, Wang S, Weng C. Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research. BMC Med Inform Decis Mak. 2014;14:51–. 10.1186/1472-6947-14-51
    1. Sturkenboom MGG, Akkerman OW, van Altena R, de Lange WCM, Kosterink JGW, van der Werf TS, et al.. Dosage of isoniazid and rifampicin poorly predicts drug exposure in tuberculosis patients. European Respiratory Journal. 2016;48(4):1237. 10.1183/13993003.00986-2016
    1. Sariko ML, Mpagama SG, Gratz J, Kisonga R, Saidi Q, Kibiki GS, et al.. Glycated hemoglobin screening identifies patients admitted for retreatment of tuberculosis at risk for diabetes in Tanzania. Journal of infection in developing countries. 2016;10(4):423–6. 10.3855/jidc.7324
    1. Allaouzi I, Ahmed M. A 3D-CNN and SVM for Multi-Drug Resistance Detection 2018.
    1. Tatusch M, Conrad S. Detection of Multidrug-Resistant Tuberculosis Using Convolutional Neural Networks and Decision Trees 2018.
    1. Jaeger S, Juarez-Espinosa OH, Candemir S, Poostchi M, Yang F, Kim L, et al.. Detecting drug-resistant tuberculosis in chest radiographs. Int J Comput Assist Radiol Surg. 2018;13(12):1915–25. 10.1007/s11548-018-1857-9
    1. Huda W, Abrahams RB. Radiographic Techniques, Contrast, and Noise in X-Ray Imaging. American Journal of Roentgenology. 2015;204(2):W126–W31. 10.2214/AJR.14.13116
    1. Rohini K, Surekha Bhat M, Srikumar PS, Mahesh Kumar A. Assessment of Hematological Parameters in Pulmonary Tuberculosis Patients. Indian J Clin Biochem. 2016;31(3):332–5. 10.1007/s12291-015-0535-8
    1. Eesa A, Arabo W. A Normalization Methods for Backpropagation: A Comparative Study. Science Journal of University of Zakho. 2017;5:319.
    1. Jin J, Li M, Jin L. Data Normalization to Accelerate Training for Linear Neural Net to Predict Tropical Cyclone Tracks % J Mathematical Problems in Engineering. 2015;2015:8.
    1. Refaeilzadeh P, Tang L, Liu H. Cross-Validation. In: Liu L, ÖZsu MT, editors. Encyclopedia of Database Systems. Boston, MA: Springer US; 2009. p. 532–8.
    1. Kisi O, Uncuoğlu E. Comparison of three back-propagation training algorithms for two case studies. Indian Journal of Engineering and Materials Sciences. 2005;12.
    1. Johnson JM, Khoshgoftaar TM. Survey on deep learning with class imbalance. Journal of Big Data. 2019;6(1):27.
    1. Chawla NV, Bowyer K, Hall L, Kegelmeyer WPJJAIR. SMOTE: Synthetic Minority Over-sampling Technique. 2002;16:321–57.
    1. Dorman SE, Schumacher SG, Alland D, Nabeta P, Armstrong DT, King B, et al.. Xpert MTB/RIF Ultra for detection of Mycobacterium tuberculosis and rifampicin resistance: a prospective multicentre diagnostic accuracy study. Lancet Infect Dis. 2018;18(1):76–84. 10.1016/S1473-3099(17)30691-6
    1. Bobitt J, Aguayo L, Payne L, Jansen T, Schwingel A. Geographic and Social Factors Associated With Chronic Disease Self-Management Program Participation: Going the "Extra-Mile" for Disease Prevention. Prev Chronic Dis. 2019;16:E25–E. 10.5888/pcd16.180385
    1. Willis BH. Empirical evidence that disease prevalence may affect the performance of diagnostic tests with an implicit threshold: a cross-sectional study. 2012;2(1):e000746.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться