Comparing test-retest reliability of dynamic functional connectivity methods

Abstract

Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC ("brain states"). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.

Copyright © 2017 Elsevier Inc. All rights reserved.

Figures

Figure 1. Reliability of dynamic correlation means and variances from the Kirby data

A) Omnibus reliability of dynamic correlation means and variances across all component pairs, or edges, for sliding windows (SW), tapered sliding windows (TSW) and dynamic conditional correlations (DCC) methods, as measured by the image intra-class correlation (I2C2). The mean I2C2 values across components are represented by blue dots, and the 95% confidence interval (CI) is represented by red bars. B) Edge-wise reliability of dynamic correlation means as measured using the intra-class correlation (ICC). C) Edge-wise reliability of dynamic correlation variances as measured using the ICC. Dynamic correlation means were similarly reliable across estimation methods using both omnibus and edge-wise reliability measures. In contrast, DCC-derived variances were more reliable than SW- and TSW-derived variances.

Figure 2. Comparison of dynamic functional connectivity involving signal and noise components from the Kirby data

A) The relationship between variance of dynamic functional connectivity (FC) of each edge and reliability of that variance estimated using SW, TSW, and DCC methods. B) The relationship between dynamic correlation means and variances for each edge. For both A and B, each point represents a single edge, where red dots indicate edges composed of two signal components and blue dots indicate edges that contain at least one noise component. Compared to dynamic correlation variances derived using the SW and TSW methods, DCC-derived correlation variances for edges involving a noise component appears to shrink more towards zero, thus creating greater separation between signal-signal edges and all other edges. Additionally, for all estimation methods, the variances of dynamic correlations between signal components increased as the absolute value of the dynamic correlation means between signal components decreased.

Figure 3. Edge variances averaged across subjects for each Kirby session and method

The variances of A) SW-, B) TSW-, and C) DCC-derived dynamic correlations for each edge averaged over all 20 subjects for each session. Note that dynamic FC variances are higher for signal-signal edges than for edges involving at least one noise component for all methods. D) DCC-derived dynamic FC variance of signal-signal edges. The functional label assigned to each signal node is indicated using the color code at the bottom right of the figure. [SC: subcortical (mint green); Aud: auditory (aqua); SM: somatomotor (orange); Vis: visual (pink); CC: cognitive control (olive green); DMN: default mode network (grey); Cb: cerebellum (blue); Noise: light purple]. Within both sessions, time-dependent edges between Vis components and both CC and DMN components appeared to be particularly variable (variance values above 0.12). In contrast, edges involving the cerebellum (blue) and sub-cortical structures (light green) showed very little volatility (variance values below 0.08).

Figure 4. Reliability of dynamic correlation means and variances from the HCP data

A) Omnibus reliability of dynamic correlation means and variances across all components pairs obtained using SW methods with varying window lengths of 30, 60, and 120 TRs (SW30, SW 60, and SW120 respectively) and the DCC method. Omnibus reliability is measured using I2C2; the mean I2C2 values across components for each method are represented by blue dots, and the 95% CIs are represented by red bars. B) Edge-wise reliability of dynamic correlation means as measured using the ICC. C) Edge-wise reliability of dynamic correlation variances as measured using the ICC. Dynamic correlation means were similarly reliable across estimation methods using both omnibus and edge-wise reliability measures. In contrast, DCC-derived variances were more reliable than those derived using the SW methods.

Figure 5. Comparison of the dynamic correlation means and variances of each edge from the HCP data

A) The relationship between variance of dynamic functional connectivity (FC) of each edge and reliability of that variance estimated using SW30, SW60, SW120, and DCC methods. B) The relationship between dynamic correlation means and variances for each edge. For both A and B, each point represents a single edge, where red dots indicate edges composed of two signal components and blue dots indicate edges that contain at least one noise component.Compared to dynamic correlation variances derived using the SW methods, DCC-derived correlation variances for edges involving a noise component appears to shrink more towards zero. In addition, for all estimation methods, the variances of dynamic correlations between signal components increased as the absolute value of the dynamic correlation means between signal components decreased.

Figure 6. DCC-derived edge variances averaged across all subjects in each of the four runs from HCP data

HCP data was collected over two visits that occurred on separate days, with two runs collected during each visit. Across sessions, phase encoding directions for the two runs were alternated between right-to-left (RL) and left-to-right (LR) directions. Sessions 1A and 2B indicate runs collected using the RL phase encoding direction, while sessions 1B and 2A indicate runs collected using the LR direction. The functional label assigned to each signal node is indicated using the color code at the bottom of the figure.

Figure 7. Brain states from the Kirby data

Two brain states were identified by k-means clustering the A) SW, B) TSW, and C) DCC output of signal nodes for sessions 1 and 2 separately. Brain states were highly consistent across all estimation methods. The functional label assigned to each signal node is indicated using the color code at the bottom of the figure. [SC: subcortical (mint green); Aud: auditory (aqua); SM: somatomotor (orange); Vis: visual (pink); CC: cognitive control (olive green); DMN: default mode network (grey); Cb: cerebellum (blue)].

Figure 8. Brain-state-derived summary measures for each session and method from the Kirby data

The left column contains box plots of the average time spent in each brain state (dwell time) in TRs for each session estimated using the A) SW, B) TSW, and C) DCC methods. The right column contains box plots of the number of transitions (change points) across subjects. On average, subjects spent more time in State 1 than State 2 across sessions and methods.

Figure 9. SW30-derived brain states averaged across subjects for each of the four HCP sessions

Brain states were identified using the cluster centers from k-means clustering. The functional label assigned to each signal node is indicated using the color code located at the bottom of the figure.

Figure 10. SW60-derived brain states averaged across subjects for each of the four HCP sessions

Brain states were determined using the cluster centers from k-means clustering. The functional label assigned to each signal node is indicated using the color code located at the bottom of the figure.

Figure 11. SW120-derived brain states averaged across subjects for each of the four HCP sessions

Brain states were determined using the cluster centers from k-means clustering. The functional label assigned to each signal node is indicated using the color code located at the bottom of the figure.

Figure 12. DCC-derived brain states averaged across subjects for each of the four HCP runs

Brain states were determined using the cluster centers from k-means clustering. The functional label assigned to each signal node is indicated using the color code located at the bottom of the figure.

Figure 13. Brain-state-derived summary measures for each session and method, from HCP data

Box plots of the dwell time in TRs and the number of change points estimated using the A) SW30, B) SW60, C) SW120, and D) DCC methods.