A comparative effectiveness analysis of three continuous glucose monitors

Abstract

Objective: To compare three continuous glucose monitoring (CGM) devices in subjects with type 1 diabetes under closed-loop blood glucose (BG) control.

Research design and methods: Six subjects with type 1 diabetes (age 52 ± 14 years, diabetes duration 32 ± 14 years) each participated in two 51-h closed-loop BG control experiments in the hospital. Venous plasma glucose (PG) measurements (GlucoScout, International Biomedical) obtained every 15 min (2,360 values) were paired in time with corresponding CGM glucose (CGMG) measurements obtained from three CGM devices, the Navigator (Abbott Diabetes Care), the Seven Plus (DexCom), and the Guardian (Medtronic), worn simultaneously by each subject. Errors in paired PG-CGMG measurements and data reporting percentages were obtained for each CGM device.

Results: The Navigator had the best overall accuracy, with an aggregate mean absolute relative difference (MARD) of all paired points of 11.8 ± 11.1% and an average MARD across all 12 experiments of 11.8 ± 3.8%. The Seven Plus and Guardian produced aggregate MARDs of all paired points of 16.5 ± 17.8% and 20.3 ± 18.0%, respectively, and average MARDs across all 12 experiments of 16.5 ± 6.7% and 20.2 ± 6.8%, respectively. Data reporting percentages, a measure of reliability, were 76% for the Seven Plus and nearly 100% for the Navigator and Guardian.

Conclusions: A comprehensive head-to-head-to-head comparison of three CGM devices for BG values from 36 to 563 mg/dL revealed marked differences in performance characteristics that include accuracy, precision, and reliability. The Navigator outperformed the other two in these areas.

Trial registration: ClinicalTrials.gov NCT01161862.

Figures

Figure 1

A: Representative results from one of twelve 48-h closed-loop BG control experiments in one of six subjects showing venous PG concentrations measured every 15 min with the GlucoScout (red symbols) and CGMG values measured approximately every 5 min with the Navigator (black symbols), Seven Plus (blue symbols), and Guardian (green symbols). The timing of six meals is indicated by black triangles. One period of structured exercise at 1600 h (2 h before the fourth meal) is indicated by a gray square. Listed in the legend for each CGM device is the number (N) of glucose values measured, the data reporting percentage (in square brackets), and the MARD averaged over the 48-h period, based on 194, 171, and 180 paired PG–CGMG values for the Navigator, Seven Plus, and Guardian, respectively. B: The 48-h MARDs computed in each of the 12 experiments are shown for each sensor, with the mean and SD of each of those MARDs superimposed on the data for each device.

Figure 2

Clarke error grid analyses of venous plasma glucose (PG) measured by the GlucoScout (A), with venous BG measured by the YSI designated as the reference, and CGMG measured by the Navigator (B), the Seven Plus (C), and the Guardian (D), with venous PG measured by the GlucoScout designated as the reference. A: Based on a total of 597 GlucoScout–YSI glucose pairs, 98.3% of points fell in zone A and the remaining 1.7% fell in zone B. The slope and intercept of the linear least squares fit to these data (solid red line) were 1.02 and −2 mg/dL, respectively. The MARD was 5.1% between GlucoScout PG and YSI BG (after converting the latter to PG with a multiplicative factor of 1.12). B: Based on a total of 2,356 Navigator–GlucoScout pairs, the Navigator achieved 80.6% of points in zone A, 18.3% in zone B, 0% in zone C, and 1.0% in zone D. The slope and intercept of the linear least squares fit to these data (solid black line) were 0.71 and 33 mg/dL, respectively. The Navigator achieved an overall data reporting percentage of 99.8% and a MARD of 11.8 ± 11.1%. C: Based on a total of 1,799 Seven Plus–GlucoScout pairs, the Seven Plus achieved 76.2% of points in zone A, 22.7% in zone B, 0.9% in zone C, and 0.1% in zone D. The slope and intercept of the linear least squares fit to these data (solid blue line) were 1.02 and 1 mg/dL, respectively. The Seven Plus achieved an overall data reporting percentage of 76.2% and a MARD of 16.5 ± 17.8%. D: Based on a total of 2,328 Guardian–GlucoScout pairs, the Guardian achieved 63.7% of points in zone A, 33.2% in zone B, 0.3% in zone C, and 2.1% in zone D. The slope and intercept of the linear least squares fit to these data (solid green line) were 0.77 and 26 mg/dL, respectively. The Guardian achieved an overall data reporting percentage of 98.6% and a MARD of 20.3 ± 18.0%.

Figure 3

A: Distribution, as a function of PG, of the RD between each CGMG measurement and its corresponding PG value (measured with the GlucoScout) for the Navigator (black), Seven Plus (blue), and Guardian (green). B: Histograms in the PG–RD plane for each of the datasets shown above in A. The horizontal line in each panel in A and the line in the PG–RD plane in each panel in B correspond to the MRD for each of the three datasets. C: Distribution, as a function of PG, of the ARD between each CGMG measurement and its corresponding PG value (measured with the GlucoScout) for the Navigator (black), Seven Plus (blue), and Guardian (green). D: Histograms in the PG–ARD plane for each of the datasets shown in C. The horizontal line in each panel in C and the line in the PG–ARD plane in each panel in D correspond to the MARD for each of the three datasets. Note, it can be seen that the data in C and D are derivable by reflecting all negatively valued RD data that fall below the PG axis in A and B to their corresponding positive values above the PG axis. The five largest bins for the Navigator had frequencies of 96, 103, 105, 85, and 81 (all between 0 and 7% ARD) corresponding with PG values of 91–98, 98–105, 105–112, 112–119, and 119–126 mg/dL, respectively. These five bins collectively contain 470 of the 2,356 data points (20%). The remaining bins had fewer than 60 hits each. Of the 2,356 data points, 940 (40%) fell in the bins with 0–7% ARD. The five largest bins for the Seven Plus had frequencies of 46, 48, 46, 43 (all between 0 and 7% ARD), and 44 (between 7 and 14% ARD) corresponding with PG values of 98–105, 105–112, 112–119, 119–126, and 98–105 mg/dL, respectively. These five bins collectively contain 227 of the 1,795 data points (12.6%). The remaining bins had fewer than 40 hits each. Of the 1,795 data points, 565 (31.5%) fell in the bins with 0–7% ARD. The five largest bins for the Guardian had frequencies of 50, 58, 61, 53, and 56 (all between 0 and 7% ARD), corresponding with PG values of 91–98, 98–105, 105–112, 112–119, and 119–126 mg/dL, respectively. These five bins collectively contain 278 of the 2,324 data points (12%). The remaining bins had fewer than 50 hits each. Of the 2,324 data points, 569 (24.5%) fell in the bins with 0–7% ARD.

Figure 4

A–C: The MARD and SD in the MARD corresponding to each PG value from 70 to 320 mg/dL for the Navigator, Seven Plus, and Guardian, respectively. Data points without error bars represent sole values for that particular PG value. D: The MARD and SD in the MARD corresponding to the clinically relevant PG ranges from 70–120, 120–180, 180– 250, and ≥250 mg/dL for the Navigator, Seven Plus, and Guardian. The number (N) of data in each PG range is shown in the corresponding bar for each device. For PG values in the normoglycemic range, from 70 to 120 mg/dL, the MARDs were 9.1 ± 9.3% (N = 899), 16.5 ± 17.8% (N = 677), and 20.0 ± 19.9% (N = 889), for the Navigator, Seven Plus, and Guardian, respectively. Much less reliable, because of the small sample size obtained, are the data corresponding to PG values in the moderate-to-mild hypoglycemic range from 50 to 70 mg/dL (not shown here); in this range, the MARDs were 46 ± 33% (N = 14), 31 ± 25% (N = 11), and 36 ± 40% (N = 14), for the Navigator, Seven Plus, and Guardian, respectively.

Figure 5

The absolute value of the difference between the time rate of change in CGMG and the time rate of change in PG corresponding to eight different ranges in the absolute value of the time rate of change in PG (0–0.25, 0.25–0.5, 0.5–0.75, 0.75–1, 1–1.5, 1.5–2, 2–3, and ≥3 mg/dL/min) for the Navigator, Seven Plus, and Guardian. The number (N) of data in each range is shown below the range label.