Variability in CKD Biomarker Studies: Soluble Urokinase Plasminogen Activator Receptor (suPAR) and Kidney Disease Progression in the Chronic Kidney Disease in Children (CKiD) Study

Abstract

Rationale & objective: Biomarker studies are important for generating mechanistic insight and providing clinically useful predictors of chronic kidney disease (CKD) progression. However, variability across studies can often muddy the evidence waters. Here we evaluated real-world variability in biomarker studies using two published studies, independently conducted, of the novel plasma marker soluble urokinase-type plasminogen activator receptor (suPAR) for predicting CKD progression in children with CKD.

Study design: A comparison of 2 prospective cohort studies.

Setting & participants: 541 children from the Chronic Kidney Disease in Children (CKiD) study, median age 12 years, median glomerular filtration rate (GFR) of 54 mL/min/1.73m2.

Outcome: The first occurrence of either a 50% decline in GFR from baseline or incident end-stage kidney disease.

Analytical approach: The suPAR plasma marker was measured using the Quantikine ELISA immunoassay in the first study and Meso Scale Discovery (MSD) platform in the second. The analytical approaches varied. We used suPAR data from the 2 assays and mimicked each analytical approach in an overlapping subset.

Results: We found that switching assays had the greatest impact on inferences, resulting in a 38% to 66% change in the magnitude of the effect estimates. Covariate and modeling choices resulted in an additional 8% to 40% variability in the effect estimate. The cumulative variability led to different inferences despite using a similar sample of CKiD participants and addressing the same question.

Limitations: The estimated variability does not represent optimal repeatability but instead illustrates real-world variability that may be present in the CKD biomarker literature.

Conclusions: Our results highlight the importance of validation, avoiding conclusions based on P value thresholds, and providing comparable metrics. Further transparency of data and equal weighting of negative and positive findings in explorations of novel biomarkers will allow investigators to more quickly weed out less promising biomarkers.

Keywords: biomarker variability; chronic kidney disease; kidney disease progression; suPAR.

© 2021 The Authors.

Figures

Graphical abstract

Figure 1

Flow chart showing the sample of participants for the published studies A and B, as well as the overlapping sample used for the comparison study. Eligibility criteria are shown for both studies. CKiD, Chronic Kidney Disease in Children cohort.

Figure 2

Comparison of suPAR levels on ELISA vs MSD platforms. The left panel shows the distribution of suPAR measurements from the two suPAR assays. The right panel shows the distribution of the individual differences in measurements between the two assays. Abbreviations: ELISA, enzyme-linked immunosorbent assay (Quantikine); MSD, Meso Scale Discovery; suPAR, soluble urokinase plasminogen activator receptor.

Figure 3

Agreement of suPAR measurements from 2 assays. The left panel shows the deviation of agreement from the line of identity on the natural scale showing both a shift of the central tendency and a slope change. The right panel shows the results from Bland Altman analysis after natural log transformation to normalize distributions showing a bias, difference in the spread of the data and modest linear correlation. Study A measurements were performed with ELISA, and study B measurements were performed with MSD. The bias was estimated as the mean of the differences in measurements. The Pearson correlation and ratio of standard deviations are also shown. Abbreviations: ELISA, enzyme-linked immunosorbent assay (Quantikine); MSD, Meso Scale Discovery; suPAR, soluble urokinase plasminogen activator receptor.

Figure 4

Comparison of dendrograms from hierarchical clustering analysis using complete linkage. Each leaf or line corresponds to 1 observation. Observations that are similar to each other are combined (fused) as the dendrogram flows away from the center. The height of the fusion along the horizontal axis indicates the (dis)similarity between 2 observations. The farther away from the center the fusion occurs, the less similar the observations are. The dendrogram on the left shows relationships between participants’ given values of ELISA suPAR, BMI z score, log2(UPCR), age, eGFR, and BUN. The dendrogram on the right shows relationships between participants’ given values of MSD suPAR, BMI z score, log2(UPCR), age, eGFR, and BUN. Grey lines illustrate how individuals re-sort depending on whether ELISA or MSD is used for suPAR measurement. The quality of the alignment of the 2 trees is indicated by the entanglement. Entanglement is a measure between 1 (full entanglement) and 0 (no entanglement). A lower entanglement coefficient corresponds to a good alignment. Abbreviations: BMI, body mass index; eGFR, estimated glomerular filtration rate; ELISA, enzyme-linked immunosorbent assay (Quantikine); MSD, Meso Scale Discovery; BUN, blood urea nitrogen; suPAR, soluble urokinase plasminogen activator receptor; UPCR, urinary protein-creatinine ratio.

Figure 5

Probability of composite event of end-stage kidney disease or >50% decline in glomerular filtration rate based on quartile (Q) categories. Area of the diamond within each square represents magnitude of the risk of the composite event. Numerators are the number of events and denominators are the number of individuals in each cross category of ELISA and MSD quartile. Abbreviations: ELISA, enzyme-linked immunosorbent assay (Quantikine); MSD, Meso Scale Discovery.