Performance characteristics of five immunoassays for SARS-CoV-2: a head-to-head benchmark comparison
National SARS-CoV-2 Serology Assay Evaluation Group, Mark Ainsworth, Monique Andersson, Kathryn Auckland, J Kenneth Baillie, Eleanor Barnes, Sally Beer, Amy Beveridge, Sagida Bibi, Luke Blackwell, Martyna Borak, Abbie Bown, Tim Brooks, Nicola A Burgess-Brown, Susana Camara, Matthew Catton, Kevin K Chau, Thomas Christott, Elizabeth Clutterbuck, Jesse Coker, Richard J Cornall, Stuart Cox, David Crawford-Jones, Derrick W Crook, Silvia D'Arcangelo, Wanwisa Dejnirattsai, Julie M M Dequaire, Stavros Dimitriadis, Kate E Dingle, George Doherty, Christina Dold, Tao Dong, Susanna J Dunachie, Daniel Ebner, Marc Emmenegger, Alexis Espinosa, David W Eyre, Rory Fairhead, Shayan Fassih, Conor Feehily, Sally Felle, Alejandra Fernandez-Cid, Maria Fernandez Mendoza, Thomas H Foord, Thomas Fordwoh, Deborah Fox McKee, John Frater, Veronica Gallardo Sanchez, Nick Gent, Dominique Georgiou, Christopher J Groves, Bassam Hallis, Peter M Hammond, Stephanie B Hatch, Heli J Harvala, Jennifer Hill, Sarah J Hoosdally, Bryn Horsington, Alison Howarth, Tim James, Katie Jeffery, Elizabeth Jones, Anita Justice, Fredrik Karpe, James Kavanagh, David S Kim, Richard Kirton, Paul Klenerman, Julian C Knight, Leonidas Koukouflis, Andrew Kwok, Ullrich Leuschner, Robert Levin, Aline Linder, Teresa Lockett, Sheila F Lumley, Spyridoula Marinou, Brian D Marsden, Jose Martinez, Lucas Martins Ferreira, Lara Mason, Philippa C Matthews, Alexander J Mentzer, Alexander Mobbs, Juthathip Mongkolsapaya, Jordan Morrow, Shubhashish M M Mukhopadhyay, Matthew J Neville, Sarah Oakley, Marta Oliveira, Ashley Otter, Kevin Paddon, Jordan Pascoe, Yanchun Peng, Elena Perez, Prem K Perumal, Timothy E A Peto, Hayleah Pickford, Rutger J Ploeg, Andrew J Pollard, Anastasia Richardson, Thomas G Ritter, David J Roberts, Gillian Rodger, Christine S Rollier, Cathy Rowe, Justine K Rudkin, Gavin Screaton, Malcolm G Semple, Alex Sienkiewicz, Laura Silva-Reyes, Donal T Skelly, Alberto Sobrino Diaz, Lizzie Stafford, Lisa Stockdale, Nicole Stoesser, Teresa Street, David I Stuart, Angela Sweed, Adan Taylor, Hannah Thraves, Hoi P Tsang, Marije K Verheul, Richard Vipond, Timothy M Walker, Susan Wareing, Yolanda Warren, Charlie Wells, Clare Wilson, Kate Withycombe, Rebecca K Young, National SARS-CoV-2 Serology Assay Evaluation Group, Mark Ainsworth, Monique Andersson, Kathryn Auckland, J Kenneth Baillie, Eleanor Barnes, Sally Beer, Amy Beveridge, Sagida Bibi, Luke Blackwell, Martyna Borak, Abbie Bown, Tim Brooks, Nicola A Burgess-Brown, Susana Camara, Matthew Catton, Kevin K Chau, Thomas Christott, Elizabeth Clutterbuck, Jesse Coker, Richard J Cornall, Stuart Cox, David Crawford-Jones, Derrick W Crook, Silvia D'Arcangelo, Wanwisa Dejnirattsai, Julie M M Dequaire, Stavros Dimitriadis, Kate E Dingle, George Doherty, Christina Dold, Tao Dong, Susanna J Dunachie, Daniel Ebner, Marc Emmenegger, Alexis Espinosa, David W Eyre, Rory Fairhead, Shayan Fassih, Conor Feehily, Sally Felle, Alejandra Fernandez-Cid, Maria Fernandez Mendoza, Thomas H Foord, Thomas Fordwoh, Deborah Fox McKee, John Frater, Veronica Gallardo Sanchez, Nick Gent, Dominique Georgiou, Christopher J Groves, Bassam Hallis, Peter M Hammond, Stephanie B Hatch, Heli J Harvala, Jennifer Hill, Sarah J Hoosdally, Bryn Horsington, Alison Howarth, Tim James, Katie Jeffery, Elizabeth Jones, Anita Justice, Fredrik Karpe, James Kavanagh, David S Kim, Richard Kirton, Paul Klenerman, Julian C Knight, Leonidas Koukouflis, Andrew Kwok, Ullrich Leuschner, Robert Levin, Aline Linder, Teresa Lockett, Sheila F Lumley, Spyridoula Marinou, Brian D Marsden, Jose Martinez, Lucas Martins Ferreira, Lara Mason, Philippa C Matthews, Alexander J Mentzer, Alexander Mobbs, Juthathip Mongkolsapaya, Jordan Morrow, Shubhashish M M Mukhopadhyay, Matthew J Neville, Sarah Oakley, Marta Oliveira, Ashley Otter, Kevin Paddon, Jordan Pascoe, Yanchun Peng, Elena Perez, Prem K Perumal, Timothy E A Peto, Hayleah Pickford, Rutger J Ploeg, Andrew J Pollard, Anastasia Richardson, Thomas G Ritter, David J Roberts, Gillian Rodger, Christine S Rollier, Cathy Rowe, Justine K Rudkin, Gavin Screaton, Malcolm G Semple, Alex Sienkiewicz, Laura Silva-Reyes, Donal T Skelly, Alberto Sobrino Diaz, Lizzie Stafford, Lisa Stockdale, Nicole Stoesser, Teresa Street, David I Stuart, Angela Sweed, Adan Taylor, Hannah Thraves, Hoi P Tsang, Marije K Verheul, Richard Vipond, Timothy M Walker, Susan Wareing, Yolanda Warren, Charlie Wells, Clare Wilson, Kate Withycombe, Rebecca K Young
Abstract
Background: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA.
Methods: We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds.
Findings: At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2-94·8) and specificity was 99·9% (99·4-100%); for the DiaSorin assay sensitivity was 96·2% (94·2-97·7) and specificity was 98·9% (98·0-99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8-99·7) and specificity was 99·0% (98·1-99·5); for the Roche assay sensitivity was 97·2% (95·4-98·4) and specificity was 99·8% (99·3-100); and for the Siemens assay sensitivity was 98·1% (96·6-99·1) and specificity was 99·9% (99·4-100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset.
Interpretation: Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health.
Funding: Public Health England and UK National Institute for Health Research.
Copyright © 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license. Published by Elsevier Ltd.. All rights reserved.
Figures
References
- Center for Systems and Science Engineering at Johns Hopkins University COVID-19 dashboard. 2020.
- Amanat F, Stadlbauer D, Strohmeier S. A serological assay to detect SARS-CoV-2 seroconversion in humans. Nat Med. 2020;26:1033–1036.
- Adams ER, Ainsworth M, Anand R. Antibody testing for COVID-19: a report from the National COVID Scientific Advisory Panel. Wellcome Open Res. 2020;5:139.
- Petherick A. Developing antibody tests for SARS-CoV-2. Lancet. 2020;395:1101–1102.
- Karpe F, Vasan SK, Humphreys SM. Cohort profile: the Oxford Biobank. Int J Epidemiol. 2018;47:21g.
- UK Medicines and Healthcare products Regulatory Agency Target product profile: enzyme Immunoassay (EIA) Antibody tests to help determine if people have antibodies to SARS-CoV-2. 2020.
- UK Medicines and Healthcare products Regulatory Agency Data sheet: research reagent for anti-SARS-CoV-2 Ab. April 30, 2020.
- UK Medicines and Healthcare products Regulatory Agency Data sheet: COVID-19 convalescent plasma panel, human. April 30, 2020.
- Brecher SM, Dryjowicz-Burek J, Yu H, Campbell S, Ratcliffe N, Gupta K. Patients with common cold coronaviruses tested negative for IgG antibody to SARS-CoV-2. J Clin Microbiol. 2020;58:e01029–e01120.
- Tang MS, Hock KG, Logsdon NM. Clinical performance of two SARS-CoV-2 serologic assays. Clin Chem. 2020;66:1055–1062.
- Bryan A, Pepper G, Wener MH. Performance characteristics of the Abbott architect SARS-CoV-2 IgG assay and seroprevalence in Boise, Idaho. J Clin Microbiol. 2020;58:e00941–e01020.
- Jääskeläinen A, Kuivanen S, Kekäläinen E. Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation. medRxiv. 2020 doi: 10.1101/2020.05.18.20101618. published online May 22. (preprint)
- Tre-Hardy M, Wilmet A, Beukinga I, Dogne JM, Douxfils J, Blairon L. Validation of a chemiluminescent assay for specific SARS-CoV-2 antibody. Clin Chem Lab Med. 2020;58:1357–1364.
- Bonelli F, Sarasini A, Zierold C. Clinical and analytical performance of an automated serological test that identifies S1/S2 neutralizing IgG in Covid-19 patients semiquantitatively. bioRxiv. 2020 doi: 10.1101/2020.05.19.105445. published online May 20. (preprint)
- Plebani M, Padoan A, Negrini D, Carpinteri B, Sciacovelli L. Diagnostic performances and thresholds: the key to harmonization in serological SARS-CoV-2 assays? Clin Chim Acta. 2020;509:1–7.
- GeurtsvanKessel CH, Okba NMA, Igloi Z. Towards the next phase: evaluation of serological assays for diagnostics and exposure assessment. medRxiv. 2020 doi: 10.1101/2020.04.23.20077156. published online May 5. (preprint)
- Public Health England Evaluation of the Abbott SARS-CoV-2 IgG for the detection of anti-SARS- CoV-2 antibodies. 2020.
- Public Health England Evaluation of Roche Elecsys Anti-SARS-CoV-2 serology assay for the detection of anti-SARS-CoV-2 antibodies. 2020.
- Wrapp D, Wang N, Corbett KS. Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation. Science. 2020;367:1260–1263.
- Emmenegger M, de Cecco E, Lamparter D. Early peak and rapid decline of SARS-CoV-2 seroprevalence in a Swiss metropolitan region. medRxiv. 2020 doi: 10.1101/2020.05.31.20118554. published online Aug 7. (preprint)
- WHO . World Health Organization; Geneva: 2020. Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19)
- Stringhini S, Wisniak A, Piumatti G. Seroprevalence of anti-SARS-CoV-2 IgG antibodies in Geneva, Switzerland (SEROCoV-POP): a population-based study. Lancet. 2020;396:313–319.
- Office for National Statistics Coronavirus (COVID-19) infection survey pilot. 2020.
- Wajnberg A, Amanat F, Firpo A. SARS-CoV-2 infection induces robust, neutralizing antibody responses that are stable for at least three months. medRxiv. 2020 doi: 10.1101/2020.07.14.20151126. published online July 17. (preprint)
- Long QX, Tang XJ, Shi QL. Clinical and immunological assessment of asymptomatic SARS-CoV-2 infections. Nat Med. 2020;26:1200–1204.
- Callow KA, Parry HF, Sergeant M, Tyrrell DA. The time course of the immune response to experimental coronavirus infection of man. Epidemiol Infect. 1990;105:435–446.
- Reed SE. The behaviour of recent isolates of human respiratory coronavirus in vitro and in volunteers: evidence of heterogeneity among 229E-related strains. J Med Virol. 1984;13:179–192.
- Kiyuka PK, Agoti CN, Munywoki PK. Human coronavirus NL63 molecular epidemiology and evolutionary patterns in rural coastal Kenya. J Infect Dis. 2018;217:1728–1739.
- Choe PG, Perera R, Park WB. MERS-CoV antibody responses 1 year after symptom onset, South Korea, 2015. Emerg Infect Dis. 2017;23:1079–1084.
- Mo H, Zeng G, Ren X. Longitudinal profile of antibodies against SARS-coronavirus in SARS patients and their clinical significance. Respirology. 2006;11:49–53.
- Bao L, Deng W, Gao H. Lack of reinfection in rhesus macaques infected with SARS-CoV-2. bioRxiv. 2020 doi: 10.1101/2020.03.13.990226. published online May 1. (preprint)
- Folegatti PM, Ewer KJ, Aley PK. Safety and immunogenicity of the ChAdOx1 nCoV-19 vaccine against SARS-CoV-2: a preliminary report of a phase 1/2, single-blind, randomised controlled trial. Lancet. 2020 doi: 10.1016/S0140-6736(20)31604-4. published online July 20.
Source: PubMed