Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review

Jordana Leitao, Nikita Desai, Lukasz Aleksandrowicz, Peter Byass, Pierre Miasnikof, Stephen Tollman, Dewan Alam, Ying Lu, Suresh Kumar Rathi, Abhishek Singh, Wilson Suraweera, Faujdar Ram, Prabhat Jha, Jordana Leitao, Nikita Desai, Lukasz Aleksandrowicz, Peter Byass, Pierre Miasnikof, Stephen Tollman, Dewan Alam, Ying Lu, Suresh Kumar Rathi, Abhishek Singh, Wilson Suraweera, Faujdar Ram, Prabhat Jha

Abstract

Background: Computer-coded verbal autopsy (CCVA) methods to assign causes of death (CODs) for medically unattended deaths have been proposed as an alternative to physician-certified verbal autopsy (PCVA). We conducted a systematic review of 19 published comparison studies (from 684 evaluated), most of which used hospital-based deaths as the reference standard. We assessed the performance of PCVA and five CCVA methods: Random Forest, Tariff, InterVA, King-Lu, and Simplified Symptom Pattern.

Methods: The reviewed studies assessed methods' performance through various metrics: sensitivity, specificity, and chance-corrected concordance for coding individual deaths, and cause-specific mortality fraction (CSMF) error and CSMF accuracy at the population level. These results were summarized into means, medians, and ranges.

Results: The 19 studies ranged from 200 to 50,000 deaths per study (total over 116,000 deaths). Sensitivity of PCVA versus hospital-assigned COD varied widely by cause, but showed consistently high specificity. PCVA and CCVA methods had an overall chance-corrected concordance of about 50% or lower, across all ages and CODs. At the population level, the relative CSMF error between PCVA and hospital-based deaths indicated good performance for most CODs. Random Forest had the best CSMF accuracy performance, followed closely by PCVA and the other CCVA methods, but with lower values for InterVA-3.

Conclusions: There is no single best-performing coding method for verbal autopsies across various studies and metrics. There is little current justification for CCVA to replace PCVA, particularly as physician diagnosis remains the worldwide standard for clinical diagnosis on live patients. Further assessments and large accessible datasets on which to train and test combinations of methods are required, particularly for rural deaths without medical attention.

Figures

Figure 1
Figure 1
Classification of verbal autopsy interpretation methods.
Figure 2
Figure 2
Systematic review process of studies assessing the performance of physician-certified verbal autopsy and computer-coded verbal autopsy methods. Search terms used: verbal autopsy, cause of death, validity, validation, performance, accuracy, assessment.
Figure 3
Figure 3
Cause-specific mortality fraction relative error between physician-certified verbal autopsy and InterVA versus reference standards, by cause of death. CSMF error is presented between PCVA and hospital-based deaths, and InterVA-3 and PCVA, from reviewed studies. The bars of the graph are not comparable between PCVA and InterVA-3, as each used a different reference standard. CSMF, cause-specific mortality fraction; PCVA, physician-certified verbal autopsy.

References

    1. Jha P. Counting the dead is one of the world’s best investments to reduce premature mortality. Hypothesis. 2012;10:e3.
    1. Liu L, Johnson HL, Cousens S, Perin J, Scott S, Lawn JE, Rudan I, Campbell H, Cibulskis R, Li M, Mathers C, Black RE. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet. 2012;379:2151–2161. doi: 10.1016/S0140-6736(12)60560-1.
    1. Vogel G. How do you count the dead? Science. 2012;336:1372–1374. doi: 10.1126/science.336.6087.1372.
    1. Hill K, Lopez AD, Shibuya K, Jha P. Interim measures for meeting needs for health sector data: births, deaths, and causes of death. Lancet. 2007;370:1726–1735. doi: 10.1016/S0140-6736(07)61309-9.
    1. Jha P, Gajalakshmi V, Gupta PC, Kumar R, Mony P, Dhingra N, Peto R. RGI-CGHR Prospective Study Collaborators. Prospective study of one million deaths in India: rationale, design, and validation results. PLoS Med. 2006;3:e18. doi: 10.1371/journal.pmed.0030018.
    1. Dhingra N, Jha P, Sharma VP, Cohen AA, Jotkar RM, Rodriguez PS, Bassani DG, Suraweera W, Laxminarayan R, Peto R. Adult and child malaria mortality in India: a nationally representative mortality survey. Lancet. 2010;376:1768–1774. doi: 10.1016/S0140-6736(10)60831-8.
    1. Mozambique National Institute of Statistics, U.S. Census Bureau, MEASURE Evaluation, U.S. Centers for Disease Control and Prevention. Mortality in Mozambique: Results from a 2007–2008 Post-Census Mortality Survey. Chapel Hill, USA: MEASURE Evaluation; 2012. [ ]
    1. Fottrell E, Byass P. Verbal autopsy: methods in transition. Epidemiol Rev. 2010;32:38–55. doi: 10.1093/epirev/mxq003.
    1. Leitao J, Chandramohan D, Byass P, Jakob R, Bundhamcharoen K, Choprapawon C, de Savigny D, Fottrell E, França E, Frøen F, Gewaifel G, Hodgson A, Hounton S, Kahn K, Krishnan A, Kumar V, Masanja H, Nichols E, Notzon F, Rasooly MH, Sankoh O, Spiegel P, Abou Zahr C, Amexo M, Kebede D, Alley WS, Marinho F, Ali M, Loyola E, Chikersal J. et al.Revising the World Health Organization verbal autopsy instrument to facilitate routine cause-of-death monitoring. Glob Health Action. 2013;6:21518.
    1. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. doi: 10.1371/journal.pmed.1000100.
    1. Flaxman AD, Vahdatpour A, Green S, James SL, Murray CJ. Random forests for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2011;9:29. doi: 10.1186/1478-7954-9-29.
    1. James SL, Flaxman AD, Murray CJ. PHMRC: Performance of the Tariff Method: validation of a simple additive algorithm for analysis of verbal autopsies. Popul Health Metr. 2011;9:31. doi: 10.1186/1478-7954-9-31.
    1. Murray CJ, James SL, Birnbaum JK, Freeman MK, Lozano R, Lopez AD. Simplified Symptom Pattern Method for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2011;9:30. doi: 10.1186/1478-7954-9-30.
    1. Lozano R, Freeman MK, James SL, Campbell B, Lopez AD, Flaxman AD, Murray CJ. Population Health Metrics Research Consortium (PHMRC) Performance of InterVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2011;9:50. doi: 10.1186/1478-7954-9-50.
    1. Byass P, Chandramohan D, Clark SJ, D'Ambruoso L, Fottrell E, Graham WJ, Herbst AJ, Hodgson A, Hounton S, Kahn K, Krishnan A, Leitao J, Odhiambo F, Sankoh OA, Tollman SM. Strengthening standardised interpretation of verbal autopsy data: the new InterVA‒4 tool. Glob Health Action. 2012;5:1–8.
    1. Flaxman AD, Vahdatpour A, James SL, Birnbaum JK, Murray CJ. Population Health Metrics Research Consortium (PHMRC) Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2011;9:35. doi: 10.1186/1478-7954-9-35.
    1. Byass P, Huong DL, Minh HV. A probabilistic approach to interpreting verbal autopsies: methodology and preliminary validation in Vietnam. Scand J Public Health Suppl. 2003;62:32–37.
    1. Anker M, Black RE, Coldham C, Kalter HD, Quigley MA, Ross D, Snow RW. A standard verbal autopsy method for investigating causes of death in infants and children. Geneva: World Health Organization; [ ]
    1. Campbell O, Ronsmans C, on behalf of the Maternal Health and Safe Motherhood Programme, World Health Organization. Verbal autopsies for maternal deaths. Geneva: World Health Organization; 2007. [ ]
    1. World Health Organization. Verbal Autopsy Standards: Ascertaining and attributing causes of death. Geneva: World Health Organization; 2007. [ ]
    1. Murray CJ, Lopez AD, Black R, Ahuja R, Ali SM, Basque A, Dandona L, Dantzer E, Das V, Dhingra U, Dutta A, Fawzi W, Flaxman AD, Gómez S, Hernández B, Joshi R, Kalter H, Kumar A, Kumar V, Lozano R, Lucero M, Mehta S, Neal B, Ohno SL, Prasad R, Praveen D, Premji Z, Ramírez-Villalobos D, Remolador H, Riley I. et al.Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets. Popul Health Metr. 2011;9:27. doi: 10.1186/1478-7954-9-27.
    1. Quigley MA, Chandramohan D, Setel P, Binka F, Rodrigues LC. Validity of data-derived algorithms for ascertaining causes of adult death in two African sites using verbal autopsy. Trop Med Int Health. 2000;5:33–39. doi: 10.1046/j.1365-3156.2000.00517.x.
    1. INDEPTH. INDEPTH Standardized Verbal Autopsy questionnaire. 2003. Ghana: INDEPTH Network; 2003. [ ]
    1. SAVVY: Sample Vital Registration with Verbal Autopsy. [ ]
    1. Setel PW, Rao C, Hemed Y, Whiting DR, Yang G, Chandramohan D, Alberti KGMM, Lopez AD. Core verbal autopsy procedures with comparative validation results from two countries. PLoS Med. 2006;3:e268. doi: 10.1371/journal.pmed.0030268.
    1. Chandramohan D, Maude GH, Rodrigues LC, Hayes RJ. Verbal autopsies for adult deaths: their development and validation in a multicentre study. Trop Med Int Health. 1998;3:436–446. doi: 10.1046/j.1365-3156.1998.00255.x.
    1. Maude GH, Ross DA. The effect of different sensitivity, specificity and cause-specific mortality fractions on the estimation of differences in cause-specific mortality rates in children from studies using verbal autopsies. Int J Epidemiol. 1997;26:1097–1106. doi: 10.1093/ije/26.5.1097.
    1. Setel PW, Whiting DR, Hemed Y, Chandramohan D, Wolfson LJ, Alberti KGMM, Lopez AD. Validity of verbal autopsy procedures for determining cause of death in Tanzania. Trop Med Int Health. 2006;11:681–696. doi: 10.1111/j.1365-3156.2006.01603.x.
    1. Yang G, Rao C, Ma J, Wang L, Wan X, Dubrovsky G, Lopez AD. Validation of verbal autopsy procedures for adult deaths in China. Int J Epidemiol. 2006;35:741–748. doi: 10.1093/ije/dyi181.
    1. Polprasert W, Rao C, Adair T, Pattaraarchachai J, Porapakkham Y, Lopez AD. Cause-of-death ascertainment for deaths that occur outside hospitals in Thailand: application of verbal autopsy methods. Popul Health Metr. 2010;8:13. doi: 10.1186/1478-7954-8-13.
    1. Lozano R, Lopez AD, Atkinson C, Naghavi M, Flaxman AD, Murray CJ. Population Health Metrics Research Consortium (PHMRC) Performance of physician-certified verbal autopsies: multisite validation study using clinical diagnostic gold standards. Popul Health Metr. 2011;9:32. doi: 10.1186/1478-7954-9-32.
    1. Gajalakshmi V, Peto R, Kanaka S, Balasubramanian S. Verbal autopsy of 48 000 adult deaths attributable to medical causes in Chennai (formerly Madras), India. BMC Public Health. 2002;2:7. doi: 10.1186/1471-2458-2-7.
    1. Gajalakshmi V, Peto R. Verbal autopsy of 80,000 adult deaths in Tamil nadu. South India. BMC Public Health. 2004;4:47. doi: 10.1186/1471-2458-4-47.
    1. Mayanja BN, Baisley K, Nalweyiso N, Kibengo FM, Mugisha JO, Paal LVD, Maher D, Kaleebu P. Using verbal autopsy to assess the prevalence of HIV infection among deaths in the ART period in rural Uganda: a prospective cohort study, 2006–2008. Popul Health Metr. 2011;9:36. doi: 10.1186/1478-7954-9-36.
    1. Mpimbaza A, Filler S, Katureebe A, Kinara SO, Nzabandora E, Quick L, Ratcliffe A, Wabwire-Mangen F, Chandramohan D, Staedke SG. Validity of verbal autopsy procedures for determining malaria deaths in different epidemiological settings in Uganda. PLoS One. 2011;6:e26892. doi: 10.1371/journal.pone.0026892.
    1. Oti S, Catherine K. Verbal autopsy interpretation: a comparative analysis of the InterVA model versus physician review in determining causes of death in the Nairobi DSS. Popul Heal Metr. 2010;8:21. doi: 10.1186/1478-7954-8-21.
    1. Ramroth H, Lorenz E, Rankin JC, Fottrell E, Yé M, Neuhann F, Ssennono M, Sié A, Byass P, Becher H. Cause of death distribution with InterVA and physician coding in a rural area of Burkina Faso. Trop Med Int Health. 2012;17:904–913. doi: 10.1111/j.1365-3156.2012.02998.x.
    1. Byass P, Kahn K, Fottrell E, Collinson MA, Tollman SM. Moving from data on deaths to public health policy in Agincourt. South Africa: approaches to analysing and understanding verbal autopsy findings. PLoS Med. 2010;7:e1000325. doi: 10.1371/journal.pmed.1000325.
    1. Tensou B, Araya T, Telake DS, Byass P, Berhane Y, Kebebew T, Sanders EJ, Reniers G, Tensou B, Araya T, Telake DS, Byass P, Berhane Y, Kebebew T, Sanders EJ, Reniers G. Evaluating the InterVA model for determining AIDS mortality from verbal autopsies in the adult population of Addis Ababa. Trop Med Int Heal. 2010;15:547–553.
    1. Tadesse S, Tadesse T. Evaluating the performance of interpreting Verbal Autopsy 3.2 model for establishing pulmonary tuberculosis as a cause of death in Ethiopia: a population-based cross-sectional study. BMC Public Health. 2012;12:1039. doi: 10.1186/1471-2458-12-1039.
    1. Byass P, de Courten M, Graham WJ, Laflamme L, McCaw-Binns A, Sankoh OA, Tollman SM, Zaba B. Reflections on the Global Burden of Disease 2010 Estimates. PLoS Med. 2013;10:e1001477. doi: 10.1371/journal.pmed.1001477.
    1. Byass P. Usefulness of the Population Health Metrics Research Consortium gold standard verbal autopsy data for general verbal autopsy methods. BMC Med. 2014;12:23.
    1. SRS Collaborators of the Registrar General Office - Centre for Global Health Research. Health Care Professional’s Manual for Assigning Causes of Death Based on RHIME Household Reports. University of Toronto, Toronto: RGI-CGHR; 2011.
    1. Aleksandrowicz L, Malhotra V, Dikshit R, Gupta PC, Kumar R, Sheth J, Rathi SK, Suraweera W, Miasnikof P, Jotkar R, Sinha D, Awasthi S, Bhatia P, Jha P. Performance criteria for verbal autopsy‒based systems to estimate national causes of death: development and application to the Indian Million Death Study. BMC Med. 2014;12:21.
    1. King G, Lu Y. Verbal autopsy methods with multiple causes of death. Stat Sci. 2008;23:78–91. doi: 10.1214/07-STS247.
    1. Soleman N, Chandramohan D, Shibuya K. Verbal autopsy: current practices and challenges. Bull World Health Organ. 2006;84:239–245. doi: 10.2471/BLT.05.027003.
    1. Quigley MA, Armstrong Schellenberg JR, Snow RW. Algorithms for verbal autopsies: a validation study in Kenyan children. Bull World Health Organ. 1996;74:147–154.
    1. Quigley MA, Chandramohan D, Rodrigues LC. Diagnostic accuracy of physician review, expert algorithms and data-derived algorithms in adult verbal autopsies. Int J Epidemiol. 1999;28:1081–1087. doi: 10.1093/ije/28.6.1081.
    1. Desai N, Aleksandrowicz L, Miasnikof P, Lu Y, Leitao J, Byass P, Tollman S, Mee P, Alam D, Rathi SK, Singh A, Kumar R, Ram F, Jha P. Performance of four computer‒coded verbal autopsy methods for cause of death assignment compared with physician coding on 24,000 deaths in low- and middle‒income countries. BMC Med. 2014;12:20.
    1. Jha P. Reliable direct measurement of causes of death in low- and middle-income countries. BMC Med. 2014;12:19.
    1. Dikshit R, Gupta PC, Ramasundarahettige C, Gajalakshmi V, Aleksandrowicz L, Badwe R, Kumar R, Roy S, Suraweera W, Bray F, Mallath M, Singh PK, Sinha DN, Shet AS, Gelband H, Jha P. Cancer mortality in India: a nationally representative survey. Lancet. 2012;379:1807–1816. doi: 10.1016/S0140-6736(12)60358-4.
    1. Jagnoor J, Suraweera W, Keay L, Ivers RQ, Thakur JS, Jha P. for the Million Death Study Collaborators. Unintentional injury mortality in India, 2005: nationally representative mortality survey of 1.1 million homes. BMC Public Health. 2012;12:487. doi: 10.1186/1471-2458-12-487.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever