CUSUM-Logistic Regression analysis for the rapid detection of errors in clinical laboratory test results

Abstract

Objective: The main drawback of the periodic analysis of quality control (QC) material is that test performance is not monitored in time periods between QC analyses, potentially leading to the reporting of faulty test results. The objective of this study was to develop a patient based QC procedure for the more timely detection of test errors.

Method: Results from a Chem-14 panel measured on the Beckman LX20 analyzer were used to develop the model. Each test result was predicted from the other 13 members of the panel by multiple regression, which resulted in correlation coefficients between the predicted and measured result of >0.7 for 8 of the 14 tests. A logistic regression model, which utilized the measured test result, the predicted test result, the day of the week and time of day, was then developed for predicting test errors. The output of the logistic regression was tallied by a daily CUSUM approach and used to predict test errors, with a fixed specificity of 90%.

Results: The mean average run length (ARL) before error detection by CUSUM-Logistic Regression (CSLR) was 20 with a mean sensitivity of 97%, which was considerably shorter than the mean ARL of 53 (sensitivity 87.5%) for a simple prediction model that only used the measured result for error detection.

Conclusion: A CUSUM-Logistic Regression analysis of patient laboratory data can be an effective approach for the rapid and sensitive detection of clinical laboratory errors.

Keywords: Average of normals; Laboratory test errors; Logistic regression; Quality control.

Published by Elsevier Inc.

Figures

Figure 1. Inter-relationships between test results in Chem-14 panel

(A)Heirarchial based clustering of Chem-14 panel test results. (B)Heat map showing relationship between test results panel based on linear correlation coefficients (R) between individual test pairs. (n=53,607). (C)Stepwise forward multiple regression was used to predict test result from the 13 other tests in the Chem-14 panel. The correlation coefficient (R) for the predicted result based on the multiple regression model versus the measured test result is shown on the Y-axis. (n=179,280).

Figure 2. Effect of time of day and day of the week on mean test results

The hourly mean (Panel A and C) or daily mean (Panel B and D) of each test in the Chem-14 panel was calculated and plotted against time. Tests were grouped into 2 categories based on the differential effect of time on test result distributions (n=179,280)

Figure 3. Difference in error prediction between full and simple CSLR models

A 10% proportional low bias was introduced into reported “good” test results (n=53,607) to simulate “bad” test results and were analysed by the simple (Panel A) and the full (Panel B) CSLR model. The AUC of a ROC plot was calculated for distinguishing good versus bad Alb test results for the full CSLR model. This was done after stepwise inclusion of the four different input variables shown. (n=179,280) (Panel C).

Figure 4. Daily CUSUM scores for albumin

The daily CUSUM score for Alb was calculated for either good Alb (Panel A and B) or bad (10% high bias) Alb test results (Panel C and D) for the simple (Panel A and C) and full CSLR model (Panel B and D). Dark points show CUSUM scores that exceed the cutpoint for error detection. Central line shows mean Cusum score versus test count. (n=1093 days)

Figure 5. Relationship between magnitude of test error and ARL

Good Alb test results were mathematically transformed to simulate various amounts of high bias as shown on the X-axis. The ARL for error detection was calculated for the simple (circle) and full (triangle) models. (Panel A, n=179,280). For each of the indicated tests, the AUC for error prediction was plotted against the ARL for error prediction for high biased tests containing errors shown in Table 1. (Panel B, n=179,280).