A 10-Gene Classifier for Indeterminate Thyroid Nodules: Development and Multicenter Accuracy Study

Abstract

Background: In most of the world, diagnostic surgery remains the most frequent approach for indeterminate thyroid cytology. Although several molecular tests are available for testing in centralized commercial laboratories in the United States, there are no available kits for local laboratory testing. The aim of this study was to develop a prototype in vitro diagnostic (IVD) gene classifier for the further characterization of nodules with an indeterminate thyroid cytology.

Methods: In a first stage, the expression of 18 genes was determined by quantitative polymerase chain reaction (qPCR) in a broad histopathological spectrum of 114 fresh-tissue biopsies. Expression data were used to train several classifiers by supervised machine learning approaches. Classifiers were tested in an independent set of 139 samples. In a second stage, the best classifier was chosen as a model to develop a multiplexed-qPCR IVD prototype assay, which was tested in a prospective multicenter cohort of fine-needle aspiration biopsies.

Results: In tissue biopsies, the best classifier, using only 10 genes, reached an optimal and consistent performance in the ninefold cross-validated testing set (sensitivity 93% and specificity 81%). In the multicenter cohort of fine-needle aspiration biopsy samples, the 10-gene signature, built into a multiplexed-qPCR IVD prototype, showed an area under the curve of 0.97, a positive predictive value of 78%, and a negative predictive value of 98%. By Bayes' theorem, the IVD prototype is expected to achieve a positive predictive value of 64-82% and a negative predictive value of 97-99% in patients with a cancer prevalence range of 20-40%.

Conclusions: A new multiplexed-qPCR IVD prototype is reported that accurately classifies thyroid nodules and may provide a future solution suitable for local reference laboratory testing.

Keywords: gene classifier; in vitro diagnostic test; indeterminate thyroid nodules; qPCR.

Conflict of interest statement

N.M., M.B., G.M., and E.B. are employed by GeneproDX. H.E.G. owns shares at GeneproDX. H.E.G., J.R.M., and S.V. are inventors of Patent WO2014085434 A1. No competing financial interests exist for the remaining authors.

Figures

FIG. 1.

Study design flow diagram.

FIG. 2.

Development of a thyroid genetic classifier (TGC) that effectively classifies indeterminate thyroid nodules (ITN). (A) Differential gene expression between malignant and benign tissue biopsy samples. Gene expression was determined by quantitative polymerase chain reaction (qPCR) in 71 benign and 43 malignant fresh tissue biopsies. To calculate the gene expression for each sample (benign or malignant), the target gene was normalized by two reference genes (Supplementary Table S2). Bars represent the differential gene expression of malignant samples with respect to the average gene expression of benign (*p < 0.05). (B) TGC-3 shows high and reproducible sensitivity and specificity. Comparative performance of three genetic classifiers developed by two different approaches: non-linear discriminant analysis (TGC-1 and TGC-2) and LDA (TGC-3). Values of sensitivity (white circles) and specificity (black circles) are shown for classifiers trained with and without outlier classifying system (OCS). The testing set sensitivity and specificity is shown only for classifiers also trained by the OCS. (C) TGC model. High-level diagram of the final algorithm. Gene expression data were analyzed through three consecutive steps. In step 1, values <5th percentile or >95th percentile were identified for each gene (atypical values). Then, a lineal function integrated the atypical values of each sample obtaining the OCS score. Two cutoff points were set to classify samples with higher or lower OCS scores as malignant or benign, respectively, with 100% of accuracy (step 2). In step 3, samples without atypical values and samples that were not classified in step 2 were classified based on lineal discriminant analysis. Finally, output scores from both, OCS and discriminant analysis, were integrated to assess the performance of the classifiers (step 4).

FIG. 3.

TGC effectively classifies ITN fine-needle aspiration (FNA) biopsy samples. Dispersion graph of TGC scores from FNA training and statistical validation sets. Cutoff score to classify samples as malignant or benign was 0.32. The OCS classified samples with TGC score of 1 as malignant and samples with TGC score of 0 as benign.

FIG. 4.

Bayes' theorem analysis shows a high theoretical performance of the in vitro diagnostic (IVD) TGC. Expected predictive performance of the IVD-TGC and other genetic classifiers (Afirma, ThyGenX/ThyraMIR, ThyroSeq v2, and RosettaGX Reveal) was assessed in a broad cancer prevalence considering the sensitivity and specificity reported in the prototype studies. (A) Estimated negative predictive value (NPV) for IVD-TGC and other molecular tests. A NPV of 95% was set as the minimum value to rule out malignancy. (B) Estimated positive predictive value for IVD-TGC and other molecular tests.