How to test for phasic modulation of neural and behavioural responses

Abstract

Research on whether perception or other processes depend on the phase of neural oscillations is rapidly gaining popularity. However, it is unknown which methods are optimally suited to evaluate the hypothesized phase effect. Using a simulation approach, we here test the ability of different methods to detect such an effect on dichotomous (e.g., "hit" vs "miss") and continuous (e.g., scalp potentials) response variables. We manipulated parameters that characterise the phase effect or define the experimental approach to test for this effect. For each parameter combination and response variable, we identified an optimal method. We found that methods regressing single-trial responses on circular (sine and cosine) predictors perform best for all of the simulated parameters, regardless of the nature of the response variable (dichotomous or continuous). In sum, our study lays a foundation for optimized experimental designs and analyses in future studies investigating the role of phase for neural and behavioural responses. We provide MATLAB code for the statistical methods tested.

Keywords: EEG; MEG; Neural oscillations; Phase; Simulations; tACS.

Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.

Figures

Fig. 1

Modelling the phase effect. A. Definition of neural parameters. The vertical axis represents the average response (e.g., proportion of hits for a dichotomous response variable, average scalp potential for a continuous response variable) at each simulated phase. This corresponds to model(phase) in Section 2.1.1. B. Simulated values for neural parameters. These parameters produce different effect shapes that can be sinusoidal (total width = 100%, asymmetry = 0) or not. Note that effect size is identical for all of the blue curves shown. For one exemplary combination of asymmetry and total width, all possible effect sizes are shown in brown.

Fig. 2

Illustration of statistical methods. A. Data from 20 virtual participants in one experiment for a given combination of (neural and experimental) parameters and a dichotomous response variable. Note that the preferred phase (the phase yielding the maximum response) differs across participants. B-D. Statistical methods that were included in the present study, divided into three categories. For all methods, the single-subject measure of the hypothesized phase effect is visualized and/or described based on data from one exemplary subject. If methods divide data into phase bins, these are shown with circles. For alignment-based methods (B), the bin used for alignment is shown as an open circle. For all methods, p-values shown were obtained for the group level, by applying the respective method to the data shown in A. The panel illustrating method ITC (D, 3) depicts phase distributions for hit and miss trials (observed data and surrogate distribution). See Section 2.2 for further details.

Fig. 3

Highest possible sensitivity to detect phase effects (i.e. highest d-prime among all methods tested; color-coded) separately for all experimental parameters tested and for the three method categories. Values were averaged across all neural parameters before maximal sensitivity was determined. The confidence intervals of all sensitivities shown are in the range d’±0.12 to d’±0.16 (not shown).

Fig. 4

Sensitivity of the selected parametric alignment-based and parametric regression-based methods to detect phase effects, for various combinations of Nbins, study design, and effect size. Results were averaged across Ntrials as this parameter did not affect the identity of the winning methods. Effect size corresponds to a peak-to-peak modulation of performance for a sinusoidal shape (total width = 100%, asymmetry = 0). Note that MAX-ADJ is not defined for 4 phase bins, and MAX-OPP VS MIN OPP AV is not defined for 4 and 6 phase bins. Confidence intervals are shown by shaded areas – note that these are often too narrow to be visible.

Fig. 5

Sensitivity of the selected permutation-based and parametric regression-based methods to detect phase effects. Other conventions as in Fig. 4.

Fig. 6

Highest possible sensitivity to detect phase effects, separately for all neural parameters and the three method categories tested. The confidence intervals of all sensitivities shown are in the range d’±0.02 (for low effect sizes) to d’±0.12 (for high effect sizes).

Fig. 7

Sensitivity of the selected parametric alignment-based methods to detect phase effects, for combinations of Nbins, total width, and effect size. The black line shows the average sensitivity for LOG REGRESS FISHER and LOG REGRESS PERM which performed equally well. For other conventions, see caption of Fig. 4.

Fig. 8

A. Highest possible sensitivity to detect the phasic modulation of a continuous response variable, separately for different experimental parameters. Note that overall sensitivity (across all parameters) is not comparable with that for the dichotomous response variable (Fig. 3), as it depends on the parameters that characterise the distribution from which single-trial responses are chosen (see Materials and Methods). B,C. Sensitivities separately for the selected parametric alignment-based (B), parametric regression-based (B,C) and permutation-based methods (C) for various simulated parameters (cf. Fig. 4, Fig. 5). Note that LIN REGRESS FISHER, LIN REGRESS PERM, and CIRC-LIN CORR overlap for all parameter combinations shown in panel C. For all other conventions, see caption of Fig. 4.

Fig. 9

Highest possible sensitivity (i.e. highest d-prime among all methods) for split-data methods and other approaches. Results are shown for simulated experiments of 384 trials each (using other Ntrials did not change results) and the dichotomous response variable. Sensitivity was averaged across all other parameter combinations before maximal sensitivity was determined. “Split-data”: The dataset was split, with one part used to identify the preferred phase (the percentage of data used for this purpose is indicated), the other to test for actual phase effects across all phase bins. “Full dataset”: The preferred phase was estimated in the same dataset that is used to test for phase effects, but the phase bin used for alignment was excluded from the test (i.e. the standard alignment-based approach in this paper). “Known beforehand”: The preferred phase was known beforehand (in our case, estimated in another independent dataset of 384 trials) and used to test for phase effects across all phase bins without the necessity of data exclusion. The remaining 2 bars show equivalent sensitivity of the winning methods in the other two categories. Error bars show the upper limits of 95% confidence intervals.