Quantifying 'causality' in complex systems: understanding transfer entropy
Fatimah Abdul Razak, Henrik Jeldtoft Jensen, Fatimah Abdul Razak, Henrik Jeldtoft Jensen
Abstract
'Causal' direction is of great importance when dealing with complex systems. Often big volumes of data in the form of time series are available and it is important to develop methods that can inform about possible causal connections between the different observables. Here we investigate the ability of the Transfer Entropy measure to identify causal relations embedded in emergent coherent correlations. We do this by firstly applying Transfer Entropy to an amended Ising model. In addition we use a simple Random Transition model to test the reliability of Transfer Entropy as a measure of 'causal' direction in the presence of stochastic fluctuations. In particular we systematically study the effect of the finite size of data sets.
Conflict of interest statement
Competing Interests: The authors have declared that no competing interests exist.
Figures
References
- Bak P (1996) How Nature Works: The Science of Self Organized Criticality. New York: Springer-Verlag.
- Christensen K, Moloney RN (2005) Complexity and Criticality. London: Imperial College Press.
- Jensen HJ (1998) Self Organized Criticality: Emergent Complex Behavior in Physical and Biological Systems. Cambridge: Cambridge University Press.
- Pruessner G (2012) Self-Organised Criticality: Theory, Models and Characterisation. Cambridge: Cambridge University Press.
- Jensen HJ (2009) Probability and statistics in complex systems, introduction to. In: Encyclopedia of Complexity and Systems Science. pp. 7024–7025.
- Runge J, Heitzig J, Marwan N, Kurths J (2012) Quantifying causal coupling strength: A lag-specific measure for multivariate time series related to transfer entropy. Phys Rev E 86: 061121.
- Wiener N (1956) I am Mathematician: The later life of a prodigy. Massachusetts: MIT Press.
- Granger CWJ (1969) Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37: 424–438.
- Bressler SL, Seth A (2011) Wiener-granger causality: A well established methodolgy. NeuroImage 58: 323–329.
- Sauer N (2010) Causality and causation: What we learn from mathematical dynamic systems theory. Transactions of the Royal Society of South Africa 65: 65–68.
- Hausman DM (1999) The mathematical theory of causation. Brit J Phil Sci 3: 151–162.
- Friston K (2011) Dynamic causal modeling and Granger causality comments on: The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution. NeuroImage 58: 303–305.
- Vicente R, Wibral M, Lindner M, Pipa G (2011) Transfer entropy: a model-free measure of effective connectivity for the neurosciences. J Comput Neurosci 30: 45–67.
- Martini M, Kranz TA, Wagner T, Lehnertz K (2011) Inferring directional interactions from transient signals with symbolic transfer entropy. Phys Rev E 83: 011919.
- Marschinski R, Kantz H (2002) Analysing the information flow between financial time series: An improved estimator for transfer entropy. Eur Phys J B 30: 275–281.
- Schreiber T (2000) Measuring information transfer. Phys Rev Lett 85: 461–464.
- Kaiser A, Schreiber T (2002) Information transfer in continuous process. Physica D 166: 43–62.
- Pompe B, Runge J (2011) Momentary information transfer as a coupling of measure of time series. Phys Rev E 83: 051122.
- Hlavackova-Schindler K, Paluš M, Vejmelka M, Bhattacharya J (2007) Causality detection based on information-theoretic approachesin time series analysis. PhysicsReport 441: 1–46.
- Vejmelka M, Palus M (2008) Inferring the directionality of coupling with conditional mutual information. Phys Rev E 77: 026214.
- Lungarella M, Ishiguro K, Kuniyoshi Y, Otsu N (2007) Methods for quantifying the causal structure of bivariate time series. J Bifurcation Chaos 17: 903–921.
- Wibral M, Pampu N, Priesemann V, Siebenhuhner F, Seiwert H, et al. (2013) Measuring information-transfer delays. PLoS ONE 8.
- Shannon CE (1948) A mathematical theory of communication. The Bell Systems Technical Journal 27: 379–656, 379-423, 623-656.
- Cover T, Thomas J (1999) Elements of information theory. New York: Wiley.
- Kraskov A, Stögbauer H, Grassberger P (2004) Estimating mutual information. Phys Rev E 69: 066138.
- Nichols JM, Seaver M, Trickey ST (2005) Detecting nonlinearity in structural systems using the transfer entropy. Phys Rev E 72: 046217.
- Li Z, Ouyang G, Li D, Li X (2011) Characterization of the causality between spike trains with permutation conditional mutual information. Phys Rev E 84: 021929.
- Cipra BA (1987) An introduction to the Ising model. The American Mathematical Monthly 94: 937–959.
- Krauth W(2006) Statistical Mechanics: Algorithms and Computations. Oxford: Oxford University Press.
- Norris JR (2008) Markov Chains. Cambridge: Cambridge University Press.
- Witthauer L, Dieterle M (2007). The phase transition of the 2D-Ising model. Available: . (refer to Figure 9).
- Abdul Razak F (2013) Mutual Information based measures on complex interdependent networks of neuro data sets. Ph.D. thesis, Department of Mathematics, Imperial College London.
- Theiler J (1986) Spurious dimension from correlation algorithms applied to limited time-series data. Phys Rev A 34: 2427–2432.
- Papana A, Kugiumtzis D, Larsson PG (2011) Reducing the bias of causality measures. Phys Rev E 83: 036207.
- Palus M, Stefanovska A (2003) Direction of coupling from phases of interacting oscillators: An information-theoretic approach. Phys Rev E 67: 055201.
Source: PubMed