An immune-based biomarker signature is associated with mortality in COVID-19 patients
Michael S Abers, Ottavia M Delmonte, Emily E Ricotta, Jonathan Fintzi, Danielle L Fink, Adriana A Almeida de Jesus, Kol A Zarember, Sara Alehashemi, Vasileios Oikonomou, Jigar V Desai, Scott W Canna, Bita Shakoory, Kerry Dobbs, Luisa Imberti, Alessandra Sottini, Eugenia Quiros-Roldan, Francesco Castelli, Camillo Rossi, Duilio Brugnoni, Andrea Biondi, Laura Rachele Bettini, Mariella D'Angio', Paolo Bonfanti, Riccardo Castagnoli, Daniela Montagna, Amelia Licari, Gian Luigi Marseglia, Emily F Gliniewicz, Elana Shaw, Dana E Kahle, Andre T Rastegar, Michael Stack, Katherine Myint-Hpu, Susan L Levinson, Mark J DiNubile, Daniel W Chertow, Peter D Burbelo, Jeffrey I Cohen, Katherine R Calvo, John S Tsang, NIAID COVID-19 Consortium, Helen C Su, John I Gallin, Douglas B Kuhns, Raphaela Goldbach-Mansky, Michail S Lionakis, Luigi D Notarangelo, Michael S Abers, Ottavia M Delmonte, Emily E Ricotta, Jonathan Fintzi, Danielle L Fink, Adriana A Almeida de Jesus, Kol A Zarember, Sara Alehashemi, Vasileios Oikonomou, Jigar V Desai, Scott W Canna, Bita Shakoory, Kerry Dobbs, Luisa Imberti, Alessandra Sottini, Eugenia Quiros-Roldan, Francesco Castelli, Camillo Rossi, Duilio Brugnoni, Andrea Biondi, Laura Rachele Bettini, Mariella D'Angio', Paolo Bonfanti, Riccardo Castagnoli, Daniela Montagna, Amelia Licari, Gian Luigi Marseglia, Emily F Gliniewicz, Elana Shaw, Dana E Kahle, Andre T Rastegar, Michael Stack, Katherine Myint-Hpu, Susan L Levinson, Mark J DiNubile, Daniel W Chertow, Peter D Burbelo, Jeffrey I Cohen, Katherine R Calvo, John S Tsang, NIAID COVID-19 Consortium, Helen C Su, John I Gallin, Douglas B Kuhns, Raphaela Goldbach-Mansky, Michail S Lionakis, Luigi D Notarangelo
Abstract
Immune and inflammatory responses to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) contribute to disease severity of coronavirus disease 2019 (COVID-19). However, the utility of specific immune-based biomarkers to predict clinical outcome remains elusive. Here, we analyzed levels of 66 soluble biomarkers in 175 Italian patients with COVID-19 ranging from mild/moderate to critical severity and assessed type I IFN-, type II IFN-, and NF-κB-dependent whole-blood transcriptional signatures. A broad inflammatory signature was observed, implicating activation of various immune and nonhematopoietic cell subsets. Discordance between IFN-α2a protein and IFNA2 transcript levels in blood suggests that type I IFNs during COVID-19 may be primarily produced by tissue-resident cells. Multivariable analysis of patients' first samples revealed 12 biomarkers (CCL2, IL-15, soluble ST2 [sST2], NGAL, sTNFRSF1A, ferritin, IL-6, S100A9, MMP-9, IL-2, sVEGFR1, IL-10) that when increased were independently associated with mortality. Multivariate analyses of longitudinal biomarker trajectories identified 8 of the aforementioned biomarkers (IL-15, IL-2, NGAL, CCL2, MMP-9, sTNFRSF1A, sST2, IL-10) and 2 additional biomarkers (lactoferrin, CXCL9) that were substantially associated with mortality when increased, while IL-1α was associated with mortality when decreased. Among these, sST2, sTNFRSF1A, IL-10, and IL-15 were consistently higher throughout the hospitalization in patients who died versus those who recovered, suggesting that these biomarkers may provide an early warning of eventual disease outcome.
Keywords: COVID-19; Chemokines; Cytokines; Immunology.
Conflict of interest statement
Conflict of interest: SLL and MJD are employees of and own stock in BioAegis Therapeutics, Inc, which is developing recombinant human plasma gelsolin for potential clinical use.
Figures
References
- John Hopkins Coronavirus Resource Center. COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Updated November 30, 2020. Accessed November 30, 2020.
- Guan WJ, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. 2020;382(18):1708–1720. doi: 10.1056/NEJMoa2002032.
- Chen N, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020;395(10223):507–513. doi: 10.1016/S0140-6736(20)30211-7.
- Huang C, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020;395(10223):497–506. doi: 10.1016/S0140-6736(20)30183-5.
- Richardson S, et al. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. JAMA. 2020;323(20):2052–2059. doi: 10.1001/jama.2020.6775.
- Wu C, et al. Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China. JAMA Intern Med. 2020;180(7):934–943. doi: 10.1001/jamainternmed.2020.0994.
- Blanco-Melo D, et al. Imbalanced host response to SARS-CoV-2 drives development of COVID-19. Cell. 2020;181(5):1036–1045. doi: 10.1016/j.cell.2020.04.026.
- Merad M, Martin JC. Pathological inflammation in patients with COVID-19: a key role for monocytes and macrophages. Nat Rev Immunol. 2020;20(6):355–362. doi: 10.1038/s41577-020-0331-4.
- Tay MZ, et al. The trinity of COVID-19: immunity, inflammation and intervention. Nat Rev Immunol. 2020;20(6):363–374. doi: 10.1038/s41577-020-0311-8.
- Hadjadj J, et al. Impaired type I interferon activity and inflammatory responses in severe COVID-19 patients. Science. 2020;369(6504):718–724. doi: 10.1126/science.abc6027.
- Chen G, et al. Clinical and immunological features of severe and moderate coronavirus disease 2019. J Clin Invest. 2020;130(5):2620–2629. doi: 10.1172/JCI137244.
- doi: 10.1101/2020.02.25.20025643. Gong J DH, et al. Correlation analysis between disease severity and inflammation-related parameters in patients with COVID-19 pneumonia [preprint]. Posted on medRxiv February 27, 2020.
- Zhou F, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395(10229):1054–1062. doi: 10.1016/S0140-6736(20)30566-3.
- Del Valle DM, et al. An inflammatory cytokine signature predicts COVID-19 severity and survival. Nat Med. 2020;26(10):1636–1643. doi: 10.1038/s41591-020-1051-9.
- Lucas C, et al. Longitudinal analyses reveal immunological misfiring in severe COVID-19. Nature. 2020;584(7821):463–469. doi: 10.1038/s41586-020-2588-y.
- Arunachalam PS, et al. Systems biological assessment of immunity to mild versus severe COVID-19 infection in humans. Science. 2020;369(6508):1210–1220. doi: 10.1126/science.abc6261.
- Yang Y, et al. Plasma IP-10 and MCP-3 levels are highly associated with disease severity and predict the progression of COVID-19. J Allergy Clin Immunol. 2020;146(1):119–127. doi: 10.1016/j.jaci.2020.04.027.
- Mathew D, et al. Deep immune profiling of COVID-19 patients reveals distinct immunotypes with therapeutic implications. Science. 2020;369(6508):eabc8511.
- Laing AG, et al. A dynamic COVID-19 immune signature includes associations with poor prognosis. Nat Med. 2020;26(10):1623–1635. doi: 10.1038/s41591-020-1038-6.
- Rosenbaum L. Facing Covid-19 in Italy — Ethics, logistics, and therapeutics on the epidemic’s front line. N Engl J Med. 2020;382(20):1873–1875. doi: 10.1056/NEJMp2005492.
- Wei P-F, ed. Diagnosis and treatment protocol for novel coronavirus pneumonia (trial version 7). Chin Med J (Engl). 2020;133(9):1087–1095.
- Elshazli RM, et al. Diagnostic and prognostic value of hematological and immunological markers in COVID-19 infection: a meta-analysis of 6320 patients. PLoS One. 2020;15(8):e0238160. doi: 10.1371/journal.pone.0238160.
- Tian W, et al. Predictors of mortality in hospitalized COVID-19 patients: a systematic review and meta-analysis. J Med Virol. 2020;92(10):1875–1883. doi: 10.1002/jmv.26050.
- Group RC, et al. Dexamethasone in hospitalized patients with Covid-19 — preliminary report [published online July 17, 2020]. N Engl J Med. .
- Laguna-Goya R, et al. IL-6-based mortality risk model for hospitalized patients with COVID-19. J Allergy Clin Immunol. 2020;146(4):799–807. doi: 10.1016/j.jaci.2020.07.009.
- Bedin AS, et al. Monocyte CD169 expression as a biomarker in the early diagnosis of COVID-19 [published online November 18, 2020]. J Infect Dis. .
- doi: 10.1101/2020.03.24.20042655. Zhang D GR, et al. COVID-19 infection induces readily detectable morphological and inflammation-related phenotypic changes in peripheral blood monocytes, the severity of which correlate with patient outcome [preprint]. Posted on medRxiv March 26, 2020.
- Giavridis T, et al. CAR T cell-induced cytokine release syndrome is mediated by macrophages and abated by IL-1 blockade. Nat Med. 2018;24(6):731–738. doi: 10.1038/s41591-018-0041-7.
- Savic S, et al. Moving towards a systems-based classification of innate immune-mediated diseases. Nat Rev Rheumatol. 2020;16(4):222–237. doi: 10.1038/s41584-020-0377-5.
- Wilson JG, et al. Cytokine profile in plasma of severe COVID-19 does not differ from ARDS and sepsis. JCI Insight. 2020;5(17):140289.
- Shakoory B, et al. Interleukin-1 receptor blockade is associated with reduced mortality in sepsis patients with features of macrophage activation syndrome: reanalysis of a prior phase III trial. Crit Care Med. 2016;44(2):275–281. doi: 10.1097/CCM.0000000000001402.
- Dimopoulos G, et al. Favorable anakinra responses in severe Covid-19 patients with secondary hemophagocytic lymphohistiocytosis. Cell Host Microbe. 2020;28(1):117–123. doi: 10.1016/j.chom.2020.05.007.
- Huet T, et al. Anakinra for severe forms of COVID-19: a cohort study. Lancet Rheumatol. 2020;2(7):e393–e400. doi: 10.1016/S2665-9913(20)30164-8.
- Herold T, et al. Elevated levels of IL-6 and CRP predict the need for mechanical ventilation in COVID-19. J Allergy Clin Immunol. 2020;146(1):128–136. doi: 10.1016/j.jaci.2020.05.008.
- Mazzoni A, et al. Impaired immune cell cytotoxicity in severe COVID-19 is IL-6 dependent. J Clin Invest. 2020;130(9):4694–4703. doi: 10.1172/JCI138554.
- Vultaggio A, et al. Prompt predicting of early clinical deterioration of moderate-to-severe COVID-19 patients: usefulness of a combined score using IL-6 in a preliminary study. J Allergy Clin Immunol Pract. 2020;8(8):2575–2581. doi: 10.1016/j.jaip.2020.06.013.
- Lee JS, et al. Immunophenotyping of COVID-19 and influenza highlights the role of type I interferons in development of severe COVID-19. Sci Immunol. 2020;5(49):eabd1554. doi: 10.1126/sciimmunol.abd1554.
- doi: 10.1101/2020.06.13.20127605. Mann ER MM, et al. Longitudinal immune profiling reveals distinct features of COVID-19 pathogenesis [preprint]. Posted on medRxiv June 16, 2020.
- Perlin DS, et al. Levels of the TNF-related cytokine LIGHT Increase in hospitalized COVID-19 patients with cytokine release syndrome and ARDS. mSphere. 2020;5(4):e00699-20.
- Schulte-Schrepping J, et al. Severe COVID-19 is marked by a dysregulated myeloid cell compartment. Cell. 2020;182(6):1419–1440. doi: 10.1016/j.cell.2020.08.001.
- Silvin A, et al. Elevated calprotectin and abnormal myeloid cell subsets discriminate severe from mild COVID-19. Cell. 2020;182(6):1401–1408. doi: 10.1016/j.cell.2020.08.002.
- Skendros P, et al. Complement and tissue factor-enriched neutrophil extracellular traps are key drivers in COVID-19 immunothrombosis. J Clin Invest. 2020;130(11):6151–6157. doi: 10.1172/JCI141374.
- Wilk AJ, et al. A single-cell atlas of the peripheral immune response in patients with severe COVID-19. Nat Med. 2020;26(7):1070–1076. doi: 10.1038/s41591-020-0944-y.
- Legg JP, et al. Type 1 and type 2 cytokine imbalance in acute respiratory syncytial virus bronchiolitis. Am J Respir Crit Care Med. 2003;168(6):633–639. doi: 10.1164/rccm.200210-1148OC.
- Zhao Y, et al. Longitudinal COVID-19 profiling associates IL-1RA and IL-10 with disease severity and RANTES with mild disease. JCI Insight. 2020;5(13):e139834.
- Vestweber D. How leukocytes cross the vascular endothelium. Nat Rev Immunol. 2015;15(11):692–704. doi: 10.1038/nri3908.
- Ackermann M, et al. Pulmonary vascular endothelialitis, thrombosis, and angiogenesis in Covid-19. N Engl J Med. 2020;383(2):120–128. doi: 10.1056/NEJMoa2015432.
- Alves-Filho JC, et al. Interleukin-33 attenuates sepsis by enhancing neutrophil influx to the site of infection. Nat Med. 2010;16(6):708–712. doi: 10.1038/nm.2156.
- Chen KF, et al. Diagnostic accuracy of lipopolysaccharide-binding protein as biomarker for sepsis in adult patients: a systematic review and meta-analysis. PLoS One. 2016;11(4):e0153188. doi: 10.1371/journal.pone.0153188.
- Dubois C, et al. High plasma level of S100A8/S100A9 and S100A12 at admission indicates a higher risk of death in septic shock patients. Sci Rep. 2019;9(1):15660. doi: 10.1038/s41598-019-52184-8.
- Krychtiuk KA, et al. Predictive value of low interleukin-33 in critically ill patients. Cytokine. 2018;103:109–113. doi: 10.1016/j.cyto.2017.09.017.
- Pregernig A, et al. Prediction of mortality in adult patients with sepsis using six biomarkers: a systematic review and meta-analysis. Ann Intensive Care. 2019;9(1):125. doi: 10.1186/s13613-019-0600-1.
- Faustino LD, et al. Interleukin-33 activates regulatory T cells to suppress innate γδ T cell responses in the lung. Nat Immunol. 2020;21(11):1371–1383. doi: 10.1038/s41590-020-0785-3.
- Wang S, et al. S100A8/A9 in inflammation. Front Immunol. 2018;9:1298.
- Piktel E, et al. Plasma gelsolin: indicator of inflammation and its potential as a diagnostic tool and therapeutic target. Int J Mol Sci. 2018;19(9):2516. doi: 10.3390/ijms19092516.
- Ciancanelli MJ, et al. Infectious disease. life-threatening influenza and impaired interferon amplification in human IRF7 deficiency. Science. 2015;348(6233):448–453. doi: 10.1126/science.aaa1578.
- Lim HK, et al. Severe influenza pneumonitis in children with inherited TLR3 deficiency. J Exp Med. 2019;216(9):2038–2056. doi: 10.1084/jem.20181621.
- Hernandez N, et al. Inherited IFNAR1 deficiency in otherwise healthy patients with adverse reaction to measles and yellow fever live vaccines. J Exp Med. 2019;216(9):2057–2070. doi: 10.1084/jem.20182295.
- Hernandez N, et al. Life-threatening influenza pneumonitis in a child with inherited IRF9 deficiency. J Exp Med. 2018;215(10):2567–2585. doi: 10.1084/jem.20180628.
- Duncan CJ, et al. Human IFNAR2 deficiency: lessons for antiviral immunity. Sci Transl Med. 2015;7(307):307ra154. doi: 10.1126/scitranslmed.aac4227.
- Dupuis S, et al. Impaired response to interferon-alpha/beta and lethal viral disease in human STAT1 deficiency. Nat Genet. 2003;33(3):388–391. doi: 10.1038/ng1097.
- Hambleton S, et al. IRF8 mutations and human dendritic-cell immunodeficiency. N Engl J Med. 2011;365(2):127–138. doi: 10.1056/NEJMoa1100066.
- Pozzetto B, et al. Characteristics of autoantibodies to human interferon in a patient with varicella-zoster disease. J Infect Dis. 1984;150(5):707–713. doi: 10.1093/infdis/150.5.707.
- Walter JE, et al. Broad-spectrum antibodies against self-antigens and cytokines in RAG deficiency. J Clin Invest. 2015;125(11):4135–48. doi: 10.1172/JCI80477.
- Uggenti C, et al. Self-awareness: nucleic acid-driven inflammation and the type I interferonopathies. Annu Rev Immunol. 2019;37:247–267. doi: 10.1146/annurev-immunol-042718-041257.
- Channappanavar R, et al. Dysregulated type I interferon and inflammatory monocyte-macrophage responses cause lethal pneumonia in SARS-CoV-infected mice. Cell Host Microbe. 2016;19(2):181–193. doi: 10.1016/j.chom.2016.01.007.
- Kim H, et al. Development of a validated interferon score using NanoString technology. J Interferon Cytokine Res. 2018;38(4):171–185. doi: 10.1089/jir.2017.0127.
- Franco LM, et al. Immune regulation by glucocorticoids can be linked to cell type-dependent transcriptional responses. J Exp Med. 2019;216(2):384–406. doi: 10.1084/jem.20180595.
- Lionakis MS, Kontoyiannis DP. Glucocorticoids and invasive fungal infections. Lancet. 2003;362(9398):1828–1838. doi: 10.1016/S0140-6736(03)14904-5.
- Luo P, et al. Tocilizumab treatment in COVID-19: a single center experience. J Med Virol. 2020;92(7):814–818. doi: 10.1002/jmv.25801.
- Zain Mushtaq M, et al. Outcome of COVID-19 patients with use of tocilizumab: a single center experience. Int Immunopharmacol. 2020;88:106926.
- Arbeev KG, et al. Joint analyses of longitudinal and time-to-event data in research on aging: implications for predicting health and survival. Front Public Health. 2014;2:228.
- Ibrahim JG, et al. Basic concepts and methods for joint models of longitudinal and survival data. J Clin Oncol. 2010;28(16):2796–2801. doi: 10.1200/JCO.2009.25.0654.
- Zhou Z, et al. Heightened innate immune responses in the respiratory tract of COVID-19 patients. Cell Host Microbe. 2020;27(6):883–890. doi: 10.1016/j.chom.2020.04.017.
- Zhang Q, et al. Inborn errors of type I IFN immunity in patients with life-threatening COVID-19. Science. 2020;370(6515):eabd4570. doi: 10.1126/science.abd4570.
- Bastard P, et al. Autoantibodies against type I IFNs in patients with life-threatening COVID-19. Science. 2020;370(6515):eabd4585. doi: 10.1126/science.abd4585.
- Shi H, et al. Neutrophil calprotectin identifies severe pulmonary disease in COVID-19 [published online September 1, 2020]. J Leukoc Biol. .
- Morjaria S, et al. The effect of neutropenia and filgrastim (G-CSF) in cancer patients with COVID-19 infection [preprint]. Posted on medRxiv August 15, 2020.
- Grommes J, Soehnlein O. Contribution of neutrophils to acute lung injury. Mol Med. 2011;17(3–4):293–307.
- Leonard WJ, et al. The γc family of cytokines: basic biology to therapeutic ramifications. Immunity. 2019;50(4):832–850. doi: 10.1016/j.immuni.2019.03.028.
- Cassatella MA, McDonald PP. Interleukin-15 and its impact on neutrophil function. Curr Opin Hematol. 2000;7(3):174–177. doi: 10.1097/00062752-200005000-00008.
- Agouridakis P, et al. Association between increased levels of IL-2 and IL-15 and outcome in patients with early acute respiratory distress syndrome. Eur J Clin Invest. 2002;32(11):862–867. doi: 10.1046/j.1365-2362.2002.01081.x.
- Leahy TR, et al. Interleukin-15 is associated with disease severity in viral bronchiolitis. Eur Respir J. 2016;47(1):212–222. doi: 10.1183/13993003.00642-2015.
- Kandikattu HK, et al. IL-15 immunotherapy is a viable strategy for COVID-19. Cytokine Growth Factor Rev. 2020;54:24–31. doi: 10.1016/j.cytogfr.2020.06.008.
- Griesenauer B, Paczesny S. The ST2/IL-33 axis in immune cells during inflammatory diseases. Front Immunol. 2017;8:475.
- Hoogerwerf JJ, et al. Soluble ST2 plasma concentrations predict mortality in severe sepsis. Intensive Care Med. 2010;36(4):630–637. doi: 10.1007/s00134-010-1773-0.
- Watanabe M, et al. Soluble ST2 as a prognostic marker in community-acquired pneumonia. J Infect. 2015;70(5):474–482. doi: 10.1016/j.jinf.2015.02.004.
- Xia J, et al. Increased IL-33 expression in chronic obstructive pulmonary disease. Am J Physiol Lung Cell Mol Physiol. 2015;308(7):L619–L627. doi: 10.1152/ajplung.00305.2014.
- Portugal CAA, et al. IL-33 and ST2 as predictors of disease severity in children with viral acute lower respiratory infection. Cytokine. 2020;127:154965.
- Giamarellos-Bourboulis EJ, et al. Complex immune dysregulation in COVID-19 patients with severe respiratory failure. Cell Host Microbe. 2020;27(6):992–1000. doi: 10.1016/j.chom.2020.04.009.
- Moratto D, et al. Flow cytometry identifies risk factors and dynamic changes in patients with COVID-19. J Clin Immunol. 2020;40(7):970–973. doi: 10.1007/s10875-020-00806-6.
- doi: 10.1101/2020.05.31.20112979. Neumann J PT, et al. An open resource for T cell phenotype changes in COVID-19 identifies IL-10-producing regulatory T cells as characteristic of severe cases [preprint]. Posted on medRxiv June 2, 2020.
- Chang J, et al. Negative regulation of MyD88-dependent signaling by IL-10 in dendritic cells. Proc Natl Acad Sci U S A. 2009;106(43):18327–18332. doi: 10.1073/pnas.0905815106.
- Chaudhry A, et al. Interleukin-10 signaling in regulatory T cells is required for suppression of Th17 cell-mediated inflammation. Immunity. 2011;34(4):566–578. doi: 10.1016/j.immuni.2011.03.018.
- Bedoya F, et al. Viral antigen induces differentiation of Foxp3+ natural regulatory T cells in influenza virus-infected mice. J Immunol. 2013;190(12):6115–6125. doi: 10.4049/jimmunol.1203302.
- van der Sluijs KF, et al. IL-10 is an important mediator of the enhanced susceptibility to pneumococcal pneumonia after influenza infection. J Immunol. 2004;172(12):7603–9. doi: 10.4049/jimmunol.172.12.7603.
- Joyce DA, et al. Two inhibitors of pro-inflammatory cytokine release, interleukin-10 and interleukin-4, have contrasting effects on release of soluble p75 tumor necrosis factor receptor by cultured monocytes. Eur J Immunol. 1994;24(11):2699–2705. doi: 10.1002/eji.1830241119.
- Dickensheets HL, et al. Interleukin-10 upregulates tumor necrosis factor receptor type-II (p75) gene expression in endotoxin-stimulated human monocytes. Blood. 1997;90(10):4162–4171. doi: 10.1182/blood.V90.10.4162.
- Burbelo PD, et al. Sensitivity in detection of antibodies to nucleocapsid and spike proteins of severe acute respiratory syndrome coronavirus 2 in patients with coronavirus disease 2019. J Infect Dis. 2020;222(2):206–213. doi: 10.1093/infdis/jiaa273.
- R. Bayesian applied regression modeling via Stan. R package version 2211. Accessed November 25, 2020.
- doi: 10.5281/zenodo.1284334. Brilleman SL, et al. Joint longitudinal and time-to-event models via Stan. Paper presented at: StanCon 2018; January 10, 2018; Pacific Grove, California, USA. Accessed November 30, 2020.
- Storey JD, et al. The positive false discovery rate: a Bayesian interpretation and the q-value. Ann Statist. 2003;31(6):2013–2035. doi: 10.1214/aos/1074290335.
- R. qvalue: Q-value estimation for false discovery rate control. R package version 2200. Accessed November 25, 2020.
- van Buuren S, Groothuis-Oudshoorn K. mice: multivariate imputation by chained equations in R. J Stat Softw. 2011;45(3):1–67.
Source: PubMed