dahlhaus, kurths, maass, timmer - mathematical methods in signal processing and digital image analysis

Here, in-we present multivariate approaches to time series analysis being able to tinguish direct and indirect, in some cases the directions of interactions inlinear as well as nonlinear

Springer Complexity

Springer Complexity is an interdisciplinary program publishing the best research and level teaching on both fundamental and applied aspects of complex systems – cutting acrossall traditional disciplines of the natural and life sciences, engineering, economics, medicine,neuroscience, social and computer science

academic-Complex Systems are systems that comprise many interacting parts with the ability to ate a new quality of macroscopic collective behavior the manifestations of which are the sponta-neous formation of distinctive temporal, spatial or functional structures Models of such systemscan be successfully mapped onto quite diverse “real-life” situations like the climate, the coherentemission of light from lasers, chemical reaction–diffusion systems, biological cellular networks,the dynamics of stock markets and of the Internet, earthquake statistics and prediction, freewaytraffic, the human brain, or the formation of opinions in social systems, to name just some of thepopular applications

gener-Although their scope and methodologies overlap somewhat, one can distinguish the ing main concepts and tools: self-organization, nonlinear dynamics, synergetics, turbulence, dy-namical systems, catastrophes, instabilities, stochastic processes, chaos, graphs and networks,cellular automata, adaptive systems, genetic algorithms and computational intelligence.The two major book publication platforms of the Springer Complexity program are the mono-graph series “Understanding Complex Systems” focusing on the various applications of com-plexity, and the “Springer Series in Synergetics”, which is devoted to the quantitative theoreticaland methodological foundations In addition to the books in these two core series, the programalso incorporates individual titles ranging from textbooks to major reference works

follow-Editorial and Programme Advisory Board

P´eter ´ Erdi

Center for Complex Systems Studies, Kalamazoo College, USA

and Hungarian Academy of Sciences, Budapest, Hungary

Founding Editor: J.A Scott Kelso

Future scientific and technological developments in many fields will necessarilydepend upon coming to grips with complex systems Such systems are complex inboth their composition – typically many different kinds of components interactingsimultaneously and nonlinearly with each other and their environments on multiplelevels – and in the rich diversity of behavior of which they are capable

The Springer Series in Understanding Complex Systems series (UCS) promotesnew strategies and paradigms for understanding and realizing applications of com-plex systems research in a wide variety of fields and endeavors UCS is explicitlytransdisciplinary It has three main goals: First, to elaborate the concepts, methodsand tools of complex systems at all levels of description and in all scientific fields,especially newly emerging areas within the life, social, behavioral, economic, neuro-and cognitive sciences (and derivatives thereof); second, to encourage novel applica-tions of these ideas in various fields of engineering and computation such as robotics,nano-technology and informatics; third, to provide a single forum within which com-monalities and differences in the workings of complex systems may be discerned,hence leading to deeper insight and understanding

UCS will publish monographs, lecture notes and selected edited contributionsaimed at communicating new findings to a large multidisciplinary audience

R Dahlhaus · J Kurths · P Maass · J Timmer

(Eds.)

Mathematical Methods

in Signal Processing

and Digital Image Analysis

With 96 Figures and 20 Tables

Am Neuen Palais 19

14469 PotsdamGermanyjkurths@agnld.uni-potsdam.deJens Timmer

Universit¨at FreiburgZentrum DatenanalyseEckerstr 1

79104 FreiburgGermanyjens.timmer@fdm.uni-freiburg.de

Understanding Complex Systems ISSN: 1860-0832

Library of Congress Control Number: 2007940881

c

 2008 Springer-Verlag Berlin Heidelberg

This work is subject to copyright All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks Duplication of this publication

or parts thereof is permitted only under the provisions of the German Copyright Law of September 9,

1965, in its current version, and permission for use must always be obtained from Springer Violations are liable to prosecution under the German Copyright Law.

The use of general descriptive names, registered names, trademarks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

Cover Design: WMXDesign GmbH, Heidelberg

Printed on acid-free paper

9 8 7 6 5 4 3 2 1

springer.com

Interest in time series analysis and image processing has been growing veryrapidly in recent years Input from different scientific disciplines and new the-oretical advances are matched by an increasing demand from an expandingdiversity of applications Consequently, signal and image processing has beenestablished as an independent research direction in such different areas aselectrical engineering, theoretical physics, mathematics or computer science.This has lead to some rather unstructured developments of theories, meth-ods and algorithms The authors of this book aim at merging some of thesediverging directions and to develop a consistent framework, which combinesthese heterogeneous developments The common core of the different chap-ters is the endavour to develop and analyze mathematically justified methodsand algorithms This book should serve as an overview of the state of the artresearch in this field with a focus on nonlinear and nonparametric models fortime series as well as of local, adaptive methods in image processing

The presented results are in its majority the outcome of the DFG-priorityprogram SPP 1114 “Mathematical methods for time series analysis and digitalimage processing” The starting point for this priority program was the consid-eration, that the next generation of algorithmic developments requires a closecooperation of researchers from different scientific backgrounds Accordingly,this program, which was running for 6 years from 2001 to 2007, encompassedapproximately 20 research teams from statistics, theoretical physics and math-ematics The intensive cooperation between teams from different specializeddisciplines is mirrored by the different chapters of this book, which were jointlywritten by several research teams The theoretical findings are always testedwith applications of different complexity

We do hope and expect that this book serves as a background reference

to the present state of the art and that it sparks exciting and creative newresearch in this rapidly developing field

This book, which concentrates on methodologies related to tion of dynamical systems, non- and semi-parametric models for time series,

identifica-stochastic methods, wavelet or multiscale analysis, diffusion filters and ematical morphology, is organized as follows.

math-The Chap 1 describes recent developments on multivariate time seriesanalysis The results are obtained from combinig statistical methods withthe theory of nonlinear dynamics in order to better understand time seriesmeasured from underlying complex network structures The authors of thischapter emphasize the importance of analyzing the interrelations and causalinfluences between different processes and their application to real-world datasuch as EEG or MEG from neurological experiments The concept of de-termining directed influences by investigating renormalized partial directedcoherence is introduced and analyzed leading to estimators of the strength ofthe effect of a source process on a target process

The development of surrogate methods has been one of the major ing forces in statistical data analysis in recent years The Chap 2 discussesthe mathematical foundations of surrogate data testing and examines thestatistical performance in extensive simulation studies It is shown that theperformance of the test heavily depends on the chosen combination of the teststatistics, the resampling methods and the null hypothesis

driv-The Chap 3 concentrates on multiscale approaches to image processing

It starts with construction principles for multivariate multiwavelets and cludes some wavelet applications to inverse problems in image processing withsparsity constraints The chapter includes the application of these methods toreal life data from industrial partners

in-The investigation of inverse problems is also at the center of Chap 4.Inverse problems in image processing naturally appear as parameter identi-fication problems for certain partial differential equations The applicationstreated in this chapter include the determination of heterogeneous media insubsurface structures, surface matching and morphological image matching

as well as a medically motivated image blending task This chapter includes

a survey of the analytic background theory as well as illustrations of thesespecific applications

Recent results on nonlinear methods for analyzing bivariate coupled tems are summarized in Chap 5 Instead of using classical linear methodsbased on correlation functions or spectral decompositions, the present chap-ter takes a look at nonlinear approaches based on investigating recurrencefeatures The recurrence properties of the underlying dynamical system areinvestigated on different time scales, which leads to a mathematically justifiedtheory for analyzing nonlinear recurrence plots The investigation includes ananalysis of synchronization effects, which have been developed into one of themost powerfull methodologies for analyzing dynamical systems

sys-Chapter 6 takes a new look at strucutred smoothing procedures for ing signals and images Different techniques from stochastic kernel smoother

denois-to anisotropic variational approaches and wavelet based techniques are lyzed and compared The common feature of these methods is their local and

ana-Preface VIIadaptive nature A strong emphasize is given to the comparison with standardmethods.

Chapter 7 presents a novel framework for the detection and accuratequantification of motion, orientation, and symmetry in images and imagesequences It focuses on those aspects of motion and orientation that can-not be handled successfully and reliably by existing methods, for example,motion superposition (due to transparency, reflection or occlusion), illumina-tion changes, temporal and/or spatial motion discontinuities, and dispersivenonrigid motion The performance of the presented algorithms is character-ized and their applicability is demonstrated by several key application areasincluding environmental physics, botany, physiology, medical imaging, andtechnical applications

The authors of this book as well as all participants of the SPP 1114 ematical methods for time series analysis and digital image processing” wouldlike to express their sincere thanks to the German Science Foundation forthe generous support over the last 6 years This support has generated andsparked exciting research and ongoing scientific discussions, it has lead to alarge diversity of scientific publications and – most importantly- has allowed

“Math-us to educate a generation of highly talented and ambitio“Math-us young scientists,which are now spread all over the world Furthermore, it is our great pleasure

to acknowledge the impact of the referees, which accompangnied and shapedthe developments of this priority program during its different phases Finally,

we want to express our gratitude to Mrs Sabine Pfarr, who prepared thismanuscript in an seemingly endless procedure of proof reading, adjusting im-ages, tables, indices and bibliographies while still keeping a friendly level ofcommunication with all authors concerning those nasty details scientist easilyforget

1 Multivariate Time Series Analysis

Bj¨ orn Schelter, Rainer Dahlhaus, Lutz Leistritz, Wolfram Hesse,

B¨ arbel Schack, J¨ urgen Kurths, Jens Timmer, Herbert Witte 1

2 Surrogate Data – A Qualitative and Quantitative Analysis

Thomas Maiwald, Enno Mammen, Swagata Nandi, Jens Timmer 41

3 Multiscale Approximation

Stephan Dahlke, Peter Maass, Gerd Teschke, Karsten Koch,

Dirk Lorenz, Stephan M¨ uller, Stefan Schiffler, Andreas St¨ ampfli,

Herbert Thiele, Manuel Werner 75

4 Inverse Problems and Parameter Identification in Image

Processing

Jens F Acker, Benjamin Berkels, Kristian Bredies, Mamadou S Diallo, Marc Droske, Christoph S Garbe, Matthias Holschneider,

Jaroslav Hron, Claudia Kondermann, Michail Kulesh, Peter Maass,

Nadine Olischl¨ ager, Heinz-Otto Peitgen, Tobias Preusser,

Martin Rumpf, Karl Schaller, Frank Scherbaum, Stefan Turek 111

5 Analysis of Bivariate Coupling by Means of Recurrence

Christoph Bandt, Andreas Groth, Norbert Marwan, M Carmen

Romano, Marco Thiel, Michael Rosenblum, J¨ urgen Kurths 153

6 Structural Adaptive Smoothing Procedures

J¨ urgen Franke, Rainer Dahlhaus, J¨ org Polzehl, Vladimir Spokoiny,

Gabriele Steidl, Joachim Weickert, Anatoly Berdychevski,

Stephan Didas, Siana Halim, Pavel Mr´ azek, Suhasini Subba Rao,

Joseph Tadjuidje 183

X Contents

7 Nonlinear Analysis of Multi-Dimensional Signals

Christoph S Garbe, Kai Krajsek, Pavel Pavlov, Bj¨ orn Andres,

Matthias M¨ uhlich, Ingo Stuke, Cicero Mota, Martin B¨ ohme, Martin

Haker, Tobias Schuchert, Hanno Scharr, Til Aach, Erhardt Barth,

Rudolf Mester, Bernd J¨ ahne 231

Index 289

XII List of Contributors

University of Frankfurt, Frankfurt,

Germany; Federal University of

Amazonas, Manaus, Brazil

Indian Statistical Institute,

New Delhi, India

nandi@isid.ac.in

Nadine Olischl¨ ager

University of Bonn, Bonn, Germany

XIV List of Contributors

Suhasini Subba Rao

University of Texas, Austin,

TX, USA

suhasini@stat.tamu.edu

Joseph Tadjuidje

University of Kaiserslautern,Kaiserslautern, Germanytadjuidj@mathematik.uni-kl.de

Multivariate Time Series Analysis

Bj¨orn Schelter1, Rainer Dahlhaus2, Lutz Leistritz3, Wolfram Hesse3,

B¨arbel Schack3, J¨urgen Kurths4, Jens Timmer1, and Herbert Witte3

Institute for Medical Statistics, Informatics, and Documentation,

University of Jena, Jena, Germany

1.1 Motivation

Nowadays, modern measurement devices are capable to deliver signals with creasing data rates and higher spatial resolutions When analyzing these data,particular interest is focused on disentangling the network structure underly-ing the recorded signals Neither univariate nor bivariate analysis techniquesare expected to describe the interactions between the processes sufficientlywell Moreover, the direction of the direct interactions is particularly impor-tant to understand the underlying network structure sufficiently well Here,

in-we present multivariate approaches to time series analysis being able to tinguish direct and indirect, in some cases the directions of interactions inlinear as well as nonlinear systems

As far as the linear theory is considered, various time series analysis niques have been proposed for the description of interdependencies betweendynamic processes and for the detection of causal influences in multivariatesystems [10, 12, 16, 24, 50, 67] In the frequency domain the interdependen-cies between two dynamic processes are investigated by means of the cross-spectrum and the coherence But an analysis based on correlation or coherence

tech-is often not sufficient to adequately describe interdependencies within a tivariate system As an example, assume that three signals originate from dis-tinct processes (Fig 1.1) If interrelations were investigated by an application

mul-of a bivariate analysis technique to each pair mul-of signals and if a relationshipwas detected between two signals, they would not necessarily be linked di-rectly (Fig 1.1) The interdependence between these signals might also bemediated by the third signal To enable a differentiation between direct andindirect influences in multivariate systems, graphical models applying partialcoherence have been introduced [8, 9, 10, 53, 57]

Besides detecting interdependencies between two signals in a multivariatenetwork of processes, an uncovering of directed interactions enables deeperinsights into the basic mechanisms underlying such networks In the aboveexample, it would be possible to decide whether or not certain processesproject their information onto others or vice versa In some cases both di-rections might be present, possibly in distinct frequency bands The concept

of Granger-causality [17] is usually utilized for the determination of causalinfluences This probabilistic concept of causality is based on the commonsense conception that causes precede their effects in time and is formulated

Fig 1.1 (a) Graph representing the true interaction structure Direct interactions

are only present between signals X1 and X2and X1 and X3; the direct interaction

between X2 and X3 is absent (b) Graph resulting from bivariate analysis, like

cross-spectral analysis From the bivariate analysis it is suggested that all nodes

are interacting with one another The spurious edge between signals X2 and X3 is

mediated by the common influence of X1

in terms of predictability Empirically, Granger-causality is commonly ated by fitting vector auto-regressive models A graphical approach for mod-eling Granger-causal relationships in multivariate processes has been dis-cussed [11, 14] More generally, graphs provide a convenient framework forcausal inference and allow, for example, the discussion of so-called spuriouscausalities due to confounding by unobserved processes [13]

evalu-Measures to detect directed influences in multivariate linear systems thatare addressed in this manuscript are, firstly, the Granger-causality index [24],the directed transfer function [28], and, lastly, partial directed coherence [2].While the Granger-causality index has been introduced for inference of lin-ear Granger-causality in the time domain, partial directed coherence has beensuggested to reveal Granger-causality in the frequency domain based on linearvector auto-regressive models [2, 24, 49, 56, 57, 70, 71] Unlike coherence andpartial coherence analysis, the statistical properties of partial directed coher-ence have only recently been addressed In particular, significance levels fortesting nonzero partial directed coherences at fixed frequencies are now avail-able while they were usually determined by simulations before [2, 61] On theone hand, without a significance level, detection of causal influences becomesmore hazardous for increasing model order as the variability of estimated par-tial directed coherences increases leading to false positive detections On theother hand, a high model order is often required to describe the dependen-cies of a multivariate process examined sufficiently well The derivation ofthe statistics of partial directed coherence suggests a modification with supe-rior properties to some extent that led to the concept of renormalized partialdirected coherence

A comparison of the above mentioned techniques is an indispensable requisite to reveal their specific abilities and limitations Particular properties

pre-4 B Schelter et al.

of these multivariate time series analysis techniques are thereby discussed [70].This provides knowledge about the applicability of certain analysis techniqueshelping to reliably understand the results obtained in specific situations Forinstance, the performance of the linear techniques on nonlinear data whichare often faced in applications is compared Since linear techniques are notdeveloped for nonlinear analysis, this investigation separates the chaff fromthe wheat at least under these circumstances

The second part of this chapter constitutes approaches to nonlinear timeseries analysis Nonlinear systems can show particular behaviors that are im-possible for linear systems [43] Among others, nonlinear systems can syn-chronize Synchronization phenomena have first been observed by Huygens forcoupled self-sustained oscillators The process of synchronization is an adap-tation of certain characteristics of the two processes Huygens has observed

an unison between two pendulum clocks that were mounted to the same wall.The oscillations between the clocks showed a phase difference of 180◦ [4, 42]

A weaker form of synchronization has recently been observed between twocoupled chaotic oscillators These oscillators were able to synchronize theirphases while their amplitudes stay almost uncorrelated [6, 38, 42, 43, 46].Nowadays, several forms of synchronization have been described ranging fromphase synchronization to lag synchronization to almost complete synchroniza-tion [7, 43, 47] Generalized synchronization is characterized by some arbitraryfunction that relates processes to one another [30, 48, 60]

The process of synchronization is necessarily based on self-sustained cillators By construction linear systems are not self-sustained oscillators andtherefore synchronization cannot be observed for those linear systems [58, 72].However, as will be shown, techniques for the analysis of synchronizationphenomena can be motivated and derived based on the linear analysis tech-niques [55]

os-As the mean phase coherence, a measure able to quantify synchronization,

is originally also a bivariate technique, a multivariate extension was highlydesired This issue is related to the problem of disentangling direct and indirectinteractions as discussed in the vicinity of linear time series analysis Twosynchronized oscillators are not necessarily directly coupled One commonlyinfluencing oscillator is sufficient to warrant a spurious coupling between thefirst two Again similar to the linear case, interpretations of results are thushampered if a disentangling was not possible But a multivariate extension ofphase synchronization analysis has been developed A procedure based on thepartial coherence analysis was employed and carried over to the multivariatenonlinear synchronizing systems [55] By means of a simulation study it isshown that the multivariate extension is a powerful technique that allowsdisentangling interactions in multivariate synchronizing systems

The chapter is structured as follows First the linear techniques are duced Their abilities and limitations are discussed in an application to real-world data The occurrence of burst suppression patterns is investigated bymeans of an animal model of anesthetized pigs In the second part, nonlinearsynchronization is discussed First, the mean phase coherence is intuitively

intro-introduced and then mathematically derived from cross-spectral analysis.

A multivariate extension of phase synchronization concludes the second part

of this Chapter

1.3 Mathematical Background

In this section, we summarize the theory of the multivariate linear time ries analysis techniques under investigation, i.e partial coherence and partialphase spectrum (Sect 1.3.1), the Granger-causality index, the partial directedcoherence, and the directed transfer function (Sect 1.3.2) Finally, we brieflyintroduce the concept of directed graphical models (Sect 1.3.3)

se-1.3.1 Non-Parametric Approaches

Partial Coherence and Partial Phase Spectrum

In multivariate dynamic systems, more than two processes are usually served and a differentiation of direct and indirect interactions between theprocesses is desired In the following we consider a multivariate system con-

ob-sisting of n stationary signals X i , i = 1, , n.

Ordinary spectral analysis is based on the spectrum of the process X k

introduced as

S X k X k (ω) =

FT {X k } (ω) FT {X k } ∗ (ω)

where · denotes the expectation value of (·), and FT {·} (ω) the Fourier

transform of (·), and (·) ∗the complex conjugate of (·) Analogously, the

cross-spectrum between two processes X k and X l

S X k X l (ω) =

FT {X k } (ω) FT {X l } ∗ (ω)

and the normalized cross-spectrum, i.e the coherence as a measure of

inter-action between two processes X k and X l

CohX k X l (ω) =  |S X k X l (ω) |

S X k X k (ω) S X l X l (ω) (1.3)are defined The coherence is normalized to [0, 1], whereby a value of one indicates the presence of a linear filter between X k and X land a value of zeroits absence

To enable a differentiation in direct and indirect interactions bivariate herence analysis is extended to partial coherence The basic idea is to subtractlinear influences from third processes under consideration in order to detectdirectly interacting processes The partial cross-spectrum

co-S X X |Z (ω) = S X X (ω) − S X Z (ω)S −1 (ω)S ZX (ω) (1.4)

6 B Schelter et al.

is defined between process X k and process X l, given all the linear information

of the remaining possibly more-dimensional processes Z = {X i |i = k, l} Using

this procedure, the linear information of the remaining processes is subtractedoptimally Partial coherence

are smaller than the critical value s For signals in a narrow frequency band,

a linear phase relationship is thus difficult to detect Moreover, if the two cesses considered were mutually influencing each other, no simple procedureexists to detect the mutual interaction by means of one single phase spectrumespecially for influences in similar frequency bands

pro-Marrying Parents of a Joint Child

When analyzing multivariate systems by partial coherence analysis, an effectmight occur, which might be astonishingly in the first place While bivariatecoherence is non-significant the partial coherence can be significantly differ-

ent from zero This effect is called marrying parents of a joint child and is

explained as follows (compare Fig 1.2):

Imagine that two processes X2 and X3 influence process X1 but do notinfluence each other This is correctly indicated by a zero bivariate coherence

between oscillator X2 and oscillator X3 In contrast to bivariate coherence,

partial coherence between X2 and X3 conditions on X1 To explain the

sig-nificant partial coherence between the processes X2 and X3, the specific case

X1 = X2+ X3 is considered The optimal linear information of X1 in X2 is

1/2 X1= 1/2 (X2+X3) Subtracting this from X2gives 1/2 (X2−X3)

Analo-gously, a subtraction of the optimal linear information 1/2 X1= 1/2 (X2+X3)

from X3 leads to−1/2 (X2− X3) As coherence between 1/2 (X2− X3) and

−1/2 (X2−X3) is one, the partial coherence between X2and X3becomes nificant This effect is also observed for more complex functional relationships

sig-between stochastic processes X1, X2 and X3 The “parents” X2 and X3 are

connected and “married by the common child” X1 The interrelation between

X2 and X3is still indirect, even if the partial coherence is significant In

con-clusion, the marrying parents of a joint child effect should not be identified

as a direct interrelation between the corresponding processes and is detected

by simultaneous consideration of bivariate coherence and partial coherence.Finally we mention that in practice the effect usually is much smaller than

in the above example; e.g if X1 = X2+ X3+ ε with independent random

variables of equal variance, then it can be shown that the partial coherence

is 0.5.

sum of two signals X2 and X3, which are independent processes, i.e the direct

interaction between X2 and X3 is absent (b) Graph resulting from multivariate

analysis From the multivariate analysis it is suggested that all nodes are interacting

with one another The spurious edge between signal X2and X3is due to the so-called

marrying parents of a joint child effect

8 B Schelter et al.

1.3.2 Parametric Approaches

Besides the non-parametric spectral concept introduced in the previous tion, we investigate three parametric approaches to detect the direction ofinteractions in multivariate systems The general concept underlying theseparametric methods is the notion of causality introduced by Granger [17].This causality principle is based on the common sense idea, that a cause mustprecede its effect A possible definition of Granger-causality based on the prin-ciple of predictibilty may be given by the following supposition For dynamic

sec-systems a process X l is said to Granger-cause a process X k, if knowledge of the

past of process X l improves the prediction of the process X k compared to the

knowledge of the past of process X k alone and several other variables underdiscussion In the following we will speak of multivariate Granger-causality ifadditional variables are used or of bivariate Granger-causality if no additionalvariables are used The former corresponds in some sense to partial coherencewhile the latter corresponds in some sense to ordinary coherence A compar-ison of bivariate and multivariate Granger-causality can be found in Eichler,Sect 9.4.4 [15]

Commonly, Granger-causality is estimated by means of vector sive models Since a vector autoregressive process is linear by construction,only linear Granger-causality can be inferred by this methodology In the fol-lowing, we will use the notion causality in terms of linear Granger-causalityalthough not explicitly mentioned

autoregres-The parametric analysis techniques introduced in the following are based

on modeling the multivariate system by stationary n-dimensional vector toregressive processes of order p (VAR[p])

The estimated coefficient matrix elements ˆa kl,r (k, l = 1, , n; r = 1 , p)

themselves or their frequency domain representatives

with the Kronecker symbol (δ kl = 1, if k = l and δ kl = 0, else) contain the

information about the causal influences in the multivariate system The efficient matrices weight the information of the past of the entire multivari-ate system The causal interactions between processes are modeled by theoff-diagonal elements of the matrices The influence of the history of an in-dividual process on the present value is modeled by the diagonal elements

co-For bivariate Granger-causality n is set to 2 and X1(t) and X2(t) are the two

processes under investigation

The estimated covariance matrix ˆΣ of the noise ε(t) = (ε1(t), , ε n (t)) 

contains information about linear instantaneous interactions and therefore,strictly speaking, non-causal influences between processes But changes in thediagonal elements of the covariance matrix, when fitted to the entire systems

as well as the sub-systems, can be utilized to investigate Granger-causal

influ-ences, since the estimated variance of the residuals ε i (t) reflects information

that cannot be revealed by the past of the processes

Following the principle of predictability, basically all multivariate processmodels, which provide a prediction error, may be used for a certain definition

of a Granger-causality index Such models are e.g time-variant autoregressivemodels or self-exciting threshold autoregressive (SETAR) models The firstone results in a definition of a time-variant Granger-causality index, the secondone provides the basis for a state-dependent Granger-causality index

Time-Variant Granger-Causality Index

To introduce a Granger-causality index in the time-domain and to

inves-tigate directed influences from a component X j to a component X i of a

n-dimensional system, n- and (n − 1)-dimensional VAR-models for X i are

considered Firstly, the entire dimensional VAR-model is fitted to the

n-dimensional system, leading to the residual variance ˆΣ i,n (t) = var (ε i,n (t)) Secondly, a (n − 1)-dimensional VAR-model is fitted to a (n − 1)-dimensional

subsystem {X k , k = 1, , n |k = j} of the n-dimensional system, leading to

the residual variance ˆΣ i,n−1 (t) = var

γ i ←j (t) is larger than or equal to zero except for some biased estimation of

parameters For a time-resolved extension of the Granger-causality index, atime-variant VAR-parameter estimation technique is utilized by means of therecursive least square algorithm RLS which is a special approach of adaptivefiltering [35] Consequently, a detection of directed interactions between two

processes X i and X j is possible in the time domain

Here, the time-resolved Granger-causality index is the only analysis nique under investigation reflecting information about multivariate systems

tech-in the time-domatech-in The multivariate extensions of alternative time-domatech-inanalysis techniques, such as the widely used cross-correlation function, areusually also based on operations in the frequency-domain Partial correla-tion functions are commonly estimated by means of estimating partial auto-and cross-spectra Furthermore, complex covariance structures between time

10 B Schelter et al.

lags and processes prevent a decision about statistically significant time lagsobtained by cross-correlation analysis Moreover, high values of the cross-correlation function do not reflect any statistical significance

State-Dependent Granger-Causality Index

Many investigations of interaction networks are based on event-related data.Independent of the used data source – EEG, MEG or functional MRI (fMRI)– this is combined with the processing of transient signals or signals withnonlinear properties Thus, a modeling of the underlying processes by means ofautoregressive processes is questionable and remains controversial A possibleextension of the linear Granger-causality is given by SETAR models whichare suitable to model biomedical signals with transient components or withnonlinear signal properties [32]

Let N > 1 be the dimension of a process X, and let R1, , R K be apartition ofRN Furthermore, let

is called (multivariate) SETAR process with delay d The processes ω (k)

are zero mean uncorrelated noise processes Thus, SETAR processes ize a regime state-depended autoregressive modeling Usually, the partition

real-R1, , R K is defined by a thresholding of each underlying real axis ofRN

Let Ψ −j = (X1, , X j−1 , X j+1 , , X N)T be the reduced vector of the

observed process, where the j -th component of X is excluded Then, two

variances ˆΣ i |Ψ −j (k) and ˆ Σ i |X (k) of prediction errors ω i (k) |Ψ −j with respect

to the reduced process Ψ −j and ω (k) i |X with respect to the full process X

may be estimated for each regime R k , k = 1, , K Clearly, two different

decompositions ofRN have to be considered using a SETAR modeling of Ψ −j and X If X is in the regime R k for any arbitrary k, then the reduced process

Ψ −j is located in the regime defined by the projection of R kto the hyper plane

ofRN , where the j-th component is omitted.

Let I k be the index set, where the full process is located in the regime R k.That is, it holds

Now the relation

I k ⊆n : Ψ n,d (k −j)= 1



(1.16)

is fulfilled for all j Thus, the index set I k may be transferred to Ψ −j, and the

variance of ω i (k −j)|Ψ −j may be substituted by a conditional variance ω (k) i |Ψ −j,

which is estimated by means of I k Now, the following definition of the regime

or state dependent Granger-causality index considers alterations of predictionerrors in each regime separately

ˆ

γ i←j (k) = ln

ˆ

Significance Thresholds for Granger-Causality Index

Basically, Granger-causality is a binary quantity In order to define a binarystate dependent or time-variant Ganger causality a significance threshold is

needed that indicates γ i←j (k) > 0 or γ i←j (t) > 0, respectively Generally, thus

far we do not have the exact distribution of the corresponding test statistics Apossible way out is provided by shuffle procedures To estimate the distribution

under the hypothesis γ i (k) ←j = 0 or γ i←j (t) = 0, respectively, shuffle procedures may be applied In this case, only the j-th component is permitted to be

shuffled; the temporal structure of all other components has to be preserved

In the presence of multiple realizations of the process X which is often

the case dealing with stimulus induced responses in EEG, MEG or fMRIinvestigations, Bootstrap methods may be applied e.g to estimate confidenceintervals Thereby, the single stimulus responses (trials) are considered as i.i.d.random variables [23, 33]

Partial Directed Coherence

As a parametric approach in the frequency-domain, partial directed coherencehas been introduced to detect causal relationships between processes in multi-variate dynamic systems [2] In addition, partial directed coherence accountsfor the entire multivariate system and renders a differentiation between di-rect and indirect influences possible Based on the Fourier transform of thecoefficient matrices (cf 1.11), partial directed coherence

π i ←j (ω) = |A ij (ω) |



between processes X j and X i is defined, where| · | is the absolute value of (·).

Normalized between 0 and 1, a direct influence from process X j to process X i

12 B Schelter et al.

is inferred by a non-zero partial directed coherence π i ←j (ω) To test the

statis-tical significance of non-zero partial directed coherence values in applications

to finite time series, critical values should be used that are for instance troduced in [56] Similarly to the Granger-causality index, a significant causalinfluence detected by partial directed coherence analysis has to be interpreted

in-in terms of lin-inear Granger-causality [17] In the followin-ing in-investigations, rameter matrices have been estimated by means of multivariate Yule-Walkerequations

pa-Renormalized Partial Directed Coherence

Above, partial directed coherence has been discussed and a pointwise nificance level has been introduced in [56] The pointwise significance levelallows identifying those frequencies at which the partial directed coherencediffers significantly from zero, which indicates the existence of a direct influ-ence from the source to the target process More generally, one is interested

sig-in comparsig-ing the strength of directed relationships at different frequencies

or between different pairs of processes Such a quantitative interpretation ofthe partial directed coherence and its estimates, however, is hampered by anumber of problems

(i) The partial directed coherence measures the strength of influences tive to a given signal source This seems counter-intuitive since the truestrength of coupling is not affected by the number of other processesthat are influenced by the source process In particular, adding furtherprocesses that are influenced by the source process decreases the par-tial directed coherence although the quality of the relationship betweensource and target process remains unchanged This property preventsmeaningful comparisons of influences between different source processes

rela-or even between different frequencies as the denominatrela-or in (1.18) variesover frequency

In contrast it is expected that the influence of the source on the targetprocess is diminished by an increasing number of other processes thataffect the target process, which suggests to measure the strength relative

to the target process This leads to the alternative normalizing term



co-(ii) The partial directed coherence is not scale-invariant, that is, it depends

on the units of measurement of the source and the target process In ticular, the partial directed coherence can take values arbitrarily close toeither one or zero if the scale of the target process is changed accordingly.This problem becomes important especially if the involved processes arenot measured on a common scale

par-(iii) When the partial directed coherence is estimated, further problems arisefrom the fact that the significance level depends on the frequency unlikethe significance level for the ordinary coherence derived in Sect 1.3.1 [5]

In particular, the critical values derived from

lim

N →∞ N cov (ˆ a ij (k), ˆ a ij (l)) = Σ ii H jj (k, l) , (1.23)where Σ ii is the variance of noise process ε i in the autoregressive pro-

cess The elements H jj (k, l) are entries of the inverse H = R −1 of the

covariance matrix R of the vector auto-regressive process X However,

as shown below this is not the optimal result that can be obtained

More-over, it can be shown that a χ2-distribution with two degrees of freedom

is obtained

14 B Schelter et al.

(iv) Although the pointwise significance level adapts correctly to the varyinguncertainty in the estimates of the partial directed coherence, this behav-ior shows clearly the need for measures of confidence in order to be able

to compare estimates at different frequencies Without such measures, itremains open how to interpret large peaks that exceed the significancelevel only slightly and how to compare them to smaller peaks that areclearly above the threshold

In summary, this discussion has shown that partial directed coherence as

a measure of the relative strength of directed interactions does not allow clusions on the absolute strength of coupling nor does it suit for comparingthe strength at different frequencies or between different pairs of processes.Moreover, the frequency dependence of the significance level shows that largevalues of the partial directed coherence are not necessarily more reliable thansmaller values, which weakens the interpretability of the partial directed co-herence further In the following, it is shown that these problems may beovercome by a different normalization

con-A New Definition of Partial Directed Coherence: Renormalized Partial Directed Coherence

For the derivation of an alternative normalization, recall that the partial

di-rected coherence is defined in terms of the Fourier transform A ij (ω) in (1.11).

Since this quantity is complex-valued, it is convenient to consider the dimensional vector

A ij (ω) substituted for A ij (ω) is asymptotically normally distributed with

mean Pij (ω) and covariance matrix V ij (ω)/N , where

cos(kω) cos(lω) − cos(kω) sin(lω)

− sin(kω) cos(lω) sin(kω) sin(lω) (1.25)

For p ≥ 2 and ω = 0 mod π, the matrix V ij (ω) is positive definite [56], and

it follows that, for large N , the quantity

N ˆ λ ◦ ij (ω) = N ˆPij (ω) Vij (ω) −1P ˆij (ω) has approximately a noncentral χ2-distribution with two degrees of freedom

and noncentrality parameter N λ ij (ω), where

λ ij (ω) = P ij (ω) Vij (ω) −1Pij (ω).

If p = 1 or ω = 0 mod π, the matrix V ij (ω) has only rank one and thus

is not invertible However, it can be shown that in this case N ˆ λ ◦ ij (ω) with

Vij (ω) −1 being a generalized inverse of Vij (ω) has approximately a tral χ2-distribution with one degree of freedom and noncentrality parameter

noncen-N λ ij (ω) [56].

The parameter λ ij (ω), which is nonnegative and equals zero if and only if

A ij (ω) = 0, determines how much P ij (ω) and thus A ij (ω) differ from zero.

Consequently, it provides an alternative measure for the strength of the effect

of the source process X j on the target process X i

The most important consequence of the normalization by Vij (ω) is that

the distribution of ˆλ ◦ ij (ω) depends only on the parameter λ ij (ω) and the ple size N In particular, it follows that the α-significance level for ˆ λ ◦ ij (ω)

sam-is given by χ2df ,1−α /N and thus is constant unlike in the case of the

par-tial directed coherence Here, χ2

df ,1−α denotes the 1− α quantile of the

χ2-distribution with the corresponding degrees of freedom (2 or 1) More

generally, confidence intervals for parameter λ ij (ω) can be computed;

algo-rithms for computing confidence intervals for the noncentrality parameter of a

noncentral χ2-distribution can be found, for instance, in [29] The properties

of noncentral χ2-distributions (e.g [26]) imply that such confidence intervals

for λ ij (ω) increase monotonically with ˆ λ ◦ ij (ω), that is, large values of the

esti-mates are indeed likely to correspond to strong influences among the processes

Finally, the parameter λ ij (ω) can be shown to be scale-invariant.

With these properties, ˆλ ◦ ij (ω) seems an “ideal” estimator for λ ij (ω)

How-ever, it cannot be computed from data since it depends on the unknown

co-variance matrix Vij (ω) In practice, V ij (ω) needs to be estimated by

substi-tuting estimates ˆ H and ˆΣ for H and Σ in (1.25) This leads to the alternative

estimator

ˆ

λ ij (ω) = ˆPij (ω) V ˆij (ω) −1P ˆij (ω)

It can be shown by Taylor expansion that under the null hypothesis of

λ ij (ω) = 0 this statistic is still χ2-distributed with two respectively one

de-grees of freedom, that is, the α-significance level remains unchanged when

ˆ

λ ◦ ij (ω) is replaced by ˆ λ ij (ω) In contrast, the exact asymptotic distribution

of the new estimator under the alternative is not known Nevertheless, sive simulations have revealed that approximate confidence intervals can beobtained by applying the theoretical results yielded for the “ideal” estimatorˆ

exten-λ ◦ ij (ω) to the practical estimator ˆ λ ij (ω) [54].

Directed Transfer Function

The directed transfer function is an alternative frequency-domain analysistechnique to detect directions of interactions and is again based on the Fouriertransformation of the coefficient matrices (cf (1.11)) The transfer function

H ij (ω) = A −1 (ω) leads to the definition of the directed transfer function [2, 28]

inves-Time-Resolved Extension of Parametric Approaches

In order to detect non-stationary effects in the interrelation structure of themultivariate system, an extension of the parametric approaches is introduced

To this aim a time-resolved parameter estimation technique is utilized TheGranger-causality index has already been introduced as a time resolved pro-cedure applying the recursive least square algorithm [35]

An alternative way to estimate time-resolved parameters in VAR-modelsand to consider explicitly observation noise influence in the multivariate sys-tem is based on time-variant state space models (SSM) [19, 62]

B (t) = B (t − 1) + η (t)

X (t) = B (t − 1) X (t − 1) + ε (t)

Y (t) = C (t) X (t) +  (t)

(1.27)

State space models consist of hidden state equations B(t) and X(t) as well

as observation equations Y (t) The hidden state equation for B(t) includes the parameter matrices a r (t) The observation equation Y (t) takes explicitly account for observation noise (t) For η(t) = 0, the equation for B(t) enables

a detection of time-varying parameters For a numerically efficient procedure

to estimate the parameters in the state space model, the EM-algorithm based

on the extended Kalman filter is used in the following [69]

1.3.3 Directed Graphical Models

Graphical models are a methodology to visualize and reveal relationships inmultivariate systems [11] Such a graph is shown in Fig 1.3 The verticesreflect the processes and the arrows the significant results of the applied anal-ysis technique For example, if partial directed coherences are non-significant

between process X3 and process X4, both processes are identified as not fluencing each other and arrows between the processes in the corresponding

in-Fig 1.3 Directed graph summarizing the interdependence structure for an

exem-plary multivariate system

graphical model are missing In contrast, if partial directed coherence is only

significant for one direction, for example from process X4 to process X2 but

not in the opposite direction, an arrow is drawn from process X4 to process

X2

1.4 Application to Neural Data

In order to examine time-variant causal influences within distinct neural works during defined functional states of brain activity, data obtained from

net-an experimental approach of deep sedation were net-analyzed Burst suppressionpatterns (BSP) in the brain electric activity were used for the analysis Thisspecific functional state was chosen because BSP represent a defined refer-ence point within the stream of changes in electroencephalographic (EEG)properties during sedation [73] leading to secured unconsciousness An anal-ysis of well-described alternating functional states of assumed differences ofsignal transfer in a time frame of seconds is possible It has been shown that

a hyperpolarization block of thalamo-cortical neurons evoked mainly by itated inhibitory input of reticular thalamic nucleus (RTN) activity inducesinhibition of thalamo-cortical volley activity which is reflected by cortical in-terburst activity [27, 63, 64] This in turn is assumed to be responsible fordisconnection of afferent sensory input leading to unconsciousness The role

facil-of burst activity in terms facil-of information transfer remains elusive Therefore,BSP is studied in order to elaborate time and frequency dependent features

of information transfer between intrathalamic, thalamo-cortical and thalamic networks Patterns were induced by propofol infusion in juvenile pigsand derived from cortical and thalamic electrodes

cortico-The analysis was performed to clarify a suggested time-dependent directedinfluence between the above mentioned brain structures known to be essen-tially involved in regulation of the physiological variation in consciousnessduring wakefulness and during sleep [25, 40] as well as responsible to induceunconsciousness during administration of various anesthetic and sedative com-pounds In addition, the alternating occurrence pattern characteristic of burstactivity allowed a triggered analysis of the Granger-causality index Multipletrials enable to use a generalized recursive least square estimator [24, 35],

18 B Schelter et al.

providing a more stable vector auto-regressive parameter estimation and acalculation of a significance level based on these repetitions

1.4.1 Experimental Protocol and Data Acquisition

The investigation was carried out on six female, domestic juvenile pigs(mixed breed, 7 weeks old, 15.1± 1.4 kg body weight (b.w.)) recorded at the

University Hospital of Jena by the group of Dr Reinhard Bauer Deep sedationwith burst suppression patterns was induced by continuous propofol infusion.Initially, 0.9 mg/kg b.w./min of propofol for approximately 7 min were ad-ministered until occurrence of burst suppression patterns (BSP) in occipitalleads [37], followed by a maintenance dose of 0.36 mg/kg b.w./min Ten screwelectrodes at frontal, parietal, central, temporal, and occipital brain regionswere utilized for electrocorticogram (ECoG) recordings For signal analysis arecording from the left parietooccipital cortex (POC) was used Electrodes in-troduced stereotactically into the rostral part of the reticular thalamic nucleus(RTN) and the dorsolateral thalamic nucleus (LD) of the left side were usedfor the electrothalamogram (EThG) recordings (Fig 1.4 (a)) Unipolar signalswere amplified and filtered (12-channel DC, 0.5–1,000 Hz bandpass filter, 50 Hznotch filter; Fa Schwind, Erlangen) before sampled continuously (125 Hz) with

a digital data acquisition system (GJB Datentechnik GmbH, Langewiesen).Four linked screw electrodes inserted into the nasal bone served as reference.ECoG and EThG recordings were checked visually to exclude artifacts

1.4.2 Analysis of Time-Variant and Multivariate Causal Influences Within Distinct Thalamo-Cortical Networks

In order to quantify time-variant and multivariate causal influences in a tinct functional state of general brain activity, a representative example ofdeep sedation is chosen, characterized by existence of burst suppression pat-terns Registrations from both thalamic leads (LD, RTN) and from the pari-etooccipital cortex (POC) have been utilized, which is known to respond earlywith patterns typical for gradual sedation including BSP [37] In the presentapplication, results for the Granger-causality index and the partial directedcoherence are discussed, since a time-resolved extension of partial coherence

dis-is not considered and the directed transfer function approach leads to resultssimilar to partial directed coherence

For partial directed coherence analysis continuous registrations of 384 sduration were utilized to provide an overview of the entire recording (Fig 1.4(b)) For a closer investigation of the burst patterns, the analysis using theGranger-causality index was applied to triggered registrations of 3 s durationeach, i.e 1 s before and 2 s after burst onset (Fig 1.4 (c)) In a total of 66trials, trigger points were identified by visual inspection and were set at theburst onset The deep sedation state was characterized by a distinct BSP inthe POC lead as well as continuous high amplitude and low frequency activity

in both thalamic leads

Fig 1.4 (a) Schematic representation of skull electrode localizations Dots

indi-cate ECoG (electrocorticogram) recordings POC indiindi-cates parietooccipital cortexrecording and is used in the present investigation Additionally, the RTN recording

and LD recording were utilized recorded using EThG (b) 20 s section of continuous original trace and (c) one representative trial of triggered original traces of brain

electrical activity simultaneously recorded from cortical and thalamic structures of

a juvenile pig under propofol-induced deep sedation

For the entire time series of 384 s duration, pairwise partial directed ence analysis was performed to investigate time-varying changes in directedinfluences between both thalamic structures RTN and LD and the parietooc-cipital cortex (POC) The results are shown in Fig 1.5 (a) The graph summa-rizing the influences is given in Fig 1.5 (b) A strong and continuous influence

coher-is observed from both thalamic leads RTN and LD to POC at approximately

in-(< 1 Hz) Both thalamic leads are mutually influencing each other The graph

sum-marizing the results is shown in (b) The dashed arrows correspond to influences for

low frequencies

2 Hz For the opposite direction, the causal influences are restricted to the

low frequency range (<1 Hz) indicated by the dashed arrows in the graph.

Furthermore, a directed influence is strongly indicated between the thalamicleads from LD to RTN, while the opposite direction shows a tendency to lowerfrequencies The time-dependency is more pronounced in the interaction be-tween both thalamic leads

A clearer depiction of the interrelation structures occurring during the gle burst patterns is presented in Fig 1.6 by applying the Granger-causalityindex to segments of 3 s duration For pairwise analysis between the threesignals (Fig 1.6 (a) and (b)), directed influences from both thalamic leads tothe parietooccipital cortex are observed for broad time periods At several,well-defined time points, causal influences are detected for the opposite di-rection and between both thalamic leads (dashed arrows) The interrelationbetween the thalamic leads remains significant for the multivariate analysisgiven in Fig 1.6 (c) and (d) The directed influence from POC to LD and RTN

sin-Fig 1.6 Investigation of directed interrelations during the occurrence of burst

pat-terns using the Granger-causality index in the time domain Gray-colored regions

indicate significant influences (α = 5%, one-sided) When applying pairwise

analy-sis, directed influences from both thalamic leads LD and RTN to the parietooccipital

cortex POC are detected (a) The results are summarized in the graph in (b) The

dashed arrows corresponds to interactions lasting for short time intervals The relation between the thalamic leads remains significant for the multivariate analysis

inter-(c) The directed influence from the parietooccipital cortex POC to the

investi-gated thalamic structures is exclusively sustained at the burst onsets The graph

summarizing the results is given in (d)

is reduced to the burst onsets From RTN and LD to the POC, no significantinterrelation is traceable

Results from the multivariate Granger-causality index cannot be directlycorrelated to the results obtained by the bivariate analysis In particular, themissing interrelation from RTN and LD to POC is difficult to interpret withthe knowledge of the bivariate results One possible explanation might be anadditional but unobserved process commonly influencing the three processes.This assumption is suggested by the results obtained from somatosensoryevoked potential (SEP) analysis (Fig 1.7) In contrast to previous opinions

of a proposed functional disconnection of afferent sensory inputs to cortical networks during interburst periods leading to a functional state ofunconsciousness [1], SEP analysis indicates that even during this particularfunctional state a signal transduction appears from peripheral skin sensors viathalamo-cortical networks up to cortical structures leading to signal process-ing Hence in principle, a subthalamically generated continuous input could beresponsible for the pronounced influence in the low frequency band, as shown

thalamo-by partial directed coherence analysis Such a low frequency component mightnot be observable by the Granger-causality index due to the missing selectivityfor specific frequency bands

22 B Schelter et al.

Fig 1.7 Evoked activity derived from the parietooccipital cortex (POC, upper

panel), rostral part of reticular thalamic nucleus thalamus (RTN, middle panel) anddorsolateral thalamic nucleus (LD, lower panel) owing to bipolar stimulation of thetrigeminal nerve by a pair of hypodermic needles inserted on left side of the outerdisc ridge of the porcine snout (rectangular pulses with constant current, duration

of 70μs, 1 Hz repetition frequency, 100 sweeps were averaged) in order to obtainsomatosensory evoked potentials (SEP) during burst as well as interburst periods.Note a similar signal pattern during burst and interburst periods

Problems in the estimation procedure caused by, for instance, highly related processes or missing of important processes could also explain thiseffect [53] Furthermore, the discrepancies between the bivariate and multi-variate analysis could be due to the nonlinear behavior of the system However,this possibility is not very likely, because spectral properties obtained in par-tial directed coherence analysis do not indicate a highly nonlinear behavior

cor-1.5 Discussion of Applicability of Multivariate Linear Analysis Techniques to Neural Signal Transfer

In the application of methods to neural signal transfer, for example in theanalysis of neural coordination in either the normal or pathological brain, oneshould be aware not only of the potentials but also the limitations of the meth-ods For this purpose, the features of the different analysis techniques wereanalyzed by means of synthetic data simulated by various model systems [70]

On the basis of simulations, the performance of the four investigatedanalysis techniques, i.e partial coherence with its corresponding phase spec-trum (PC), the Granger-causality index (GCI), the directed transfer function

Table 1.1 Summary of the results obtained by the comparison of the four

multi-variate time series analysis techniques To evaluate the performance, five aspects areconsidered The brackets denote some specific limitations

(DTF), and the partial directed coherence (PDC) are summarized with spect to five aspects (Table 1.1) [70], which are important when analyzingdata from unknown processes:

re-• Direct versus indirect interactions: A differentiation between direct and

indirect information transfer in multivariate systems is not possible bymeans of the directed transfer function Therefore, the directed transferfunction is not sensitive in this sense (minus sign in Table 1.1) The re-maining multivariate analysis techniques are in general able to distinguishbetween direct and indirect interactions Thus, the GCI, PDC, and PC aresensitive in distinguishing direct from indirect influences Despite the highsensitivity in general, there might be some situations in which this char-acteristic is restricted, for instance in nonlinear, non-stationary systems

• Direction of influences: All multivariate methods are capable of detecting

the direction of influences Partial coherence in combination with its phasespectrum is limited to high coherence values and to unidirectional influ-ences between the processes This shortcoming of partial coherence andpartial phase spectrum is indicated by the minus sign in Table 1.1

• Specificity in absence of the influences: All four analysis techniques reject

interrelations in the absence of any influence between the processes, flecting the high specificity of the methods For the parametric approachesdirected transfer function and partial directed coherence, a renormaliza-tion of the covariance matrix of the noise in the estimated vector auto-regressive model is required Otherwise spurious interactions are detected

re-A significance level for both techniques should account for this For the nificance level for partial directed coherence, this dependence on the noisevariance is explicitly considered However, the renormalization is necessary

sig-to achieve a balanced average height of values of PDC and DTF in thecase of an absence of an interaction at the corresponding frequency

• Nonlinearity in the data: For the nonlinear coupled stochastic systems with

pronounced frequencies, analysis techniques in the frequency domain arepreferable High model orders are required to describe the nonlinear sys-tem sufficiently with a linear vector auto-regressive model Interpretation

24 B Schelter et al.

of the results obtained by the directed transfer function and the causality index is more complicated since there has been no obvious sig-nificance level The PC, PDC, and the DTF are sensitive in detecting in-teractions in nonlinear multivariate systems The Granger-causality indexdoes not reveal the correct interrelation structure

Granger-• Influences varying with time: The Granger-causality index, directed

trans-fer function, and the time-varying partial directed coherence detect varioustypes of time-varying influences Therefore they are sensitive for time-resolved investigations of non-stationary data

This summary provides an overview of which analysis techniques are propriate for specific applications or problems However, the particular capa-bilities and limitations of a specific analysis technique do not simply point todrawbacks of the method in general If for instance the major task is to detectdirections of influences, the directed transfer function is applicable even if thedifferentiation, for example of direct or indirect interactions, is not possible.Partial coherence as a non-parametric method is robust in detecting re-lationships in multivariate systems Direct and indirect influences can bedistinguished in linear systems and certain nonlinear stochastic system likethe R¨ossler system Since partial coherence is a non-parametric approach, it

ap-is possible to capture these influences without knowledge of the underlyingdynamics Furthermore, the statistical properties are well-known and criti-cal values for a given significance level can be calculated in order to decide

on significant influences This is an important fact especially in applications

to noisy neural signal transfer as measured by e.g electroencephalographyrecordings A drawback is that the direction of relationships can only be de-termined by means of phase spectral analysis If spectral coherence is weak orrestricted to a small frequency range, directions of influences are difficult toinfer by means of partial phase spectral analysis Additionally, mutual interac-tions between two processes are also hardly detectable utilizing partial phasespectra

Defined in the time-domain, the Granger-causality index is favorable insystems where neither specific frequencies nor frequency-bands are exposed

in advance The Granger-causality index utilizes information from the variance matrix Weak interactions or narrow-band interactions are difficult

co-to detect, since they can lead co-to only small changes in the covariance trix The Granger-causality index, estimated by means of the recursive leastsquare algorithm, renders a methodology to trace interdependence structures

ma-in non-stationary data possible This might become important ma-in applications

to brain neural networks, when the time course of transitions in neural dination is of particular interest

coor-By means of the directed transfer function, directions of influences in tivariate dynamical systems are detectable Nevertheless, in contrast to theremaining three analysis techniques, a differentiation between indirect and di-rect influences is in general not possible using the directed transfer function

mul-Analyzing brain networks, at least weakly nonlinear processes might be pected to generate the neural signals In the application to the nonlinearstochastic systems, the directions of the couplings could be observed at theoscillation frequencies The directed transfer function benefits from its prop-erty as an analysis technique in the frequency domain Increasing the order ofthe fitted model system is sufficient to capture the main features of the sys-tem and thus to detect the interdependence structure correctly Nevertheless,

ex-a mex-atrix inversion is required for estimex-ating the directed trex-ansfer function,which might lead to computational challenges especially if high model ordersare necessary In order to detect transitions in the coordination between neuralsignals, the directed transfer function is useful when applying a time-resolvedparameter estimation procedure

In the frequency domain, partial directed coherence is the most powerfulanalysis technique By means of partial directed coherence, direct and indi-rect influences as well as their directions are detectable The investigation

of the paradigmatic model system of coupled stochastic R¨ossler oscillatorshas shown [70], that at least for nonlinearities, coupling directions can be in-ferred by means of partial directed coherence Increasing the order of the fittedmodel is required to describe the nonlinear system by a linear vector auto-regressive model sufficiently However, as the statistical properties of partialdirected coherence and significance levels for the decision of significant influ-ences are known, high model orders of the estimated vector auto-regressivemodel are less problematic Using additionally time-resolved parameter esti-mation techniques, partial directed coherence is applicable to non-stationarysignals Using this procedure, influences in dependence on time and frequencyare simultaneously detectable Since in applications to neural networks it isusually unknown whether there are changes in neural coordination or whethersuch changes are of particular interest, respectively, time-resolved analysistechniques avoid possible false interpretations

The promising results showing that most parametric, linear analysis niques have revealed correct interaction structures in multivariate systems,indicate beneficial applicability to empirical data Electrophysiological signalsfrom thalamic and cortical brain structures representative for key interrela-tions within a network responsible for control and modulation of consciousnesshave been analyzed Data obtained from experimental recordings of deep se-dation with burst suppression patterns were used, which allows usage of datafrom a well-defined functional state including a triggered analysis approach.Partial directed coherence based on state space modeling allows for inference

tech-of the time- and frequency-dependence tech-of the interrelation structure Themechanisms generating burst patterns were investigated in more detail by ap-plying the Granger-causality index Besides a clear depiction of the systemgenerating such burst patterns, the application presented here suggests thattime dependence is not negligible

26 B Schelter et al.

1.6 Nonlinear Dynamics

So far, the linear methodology has been addressed However, the field of linear dynamics has brought to the forefront novel concepts, ideas, and tech-niques to analyze and characterize time series of complex dynamic systems.Especially synchronization analysis to detect interactions between nonlinearself-sustained oscillators has made its way into the daily routine in manyinvestigations [43]

non-Following the observations and pioneering work of Huygens, the process ofsynchronization has been observed in many different systems such as systemsexhibiting a limit cycle or a chaotic attractor Several types of synchronizationhave been observed for these systems ranging from phase synchronization asthe weakest form of synchronization via lag synchronization to generalized orcomplete synchrony [30, 39, 41, 46, 47]

Thereby, phase synchronization analysis has gained particular interestsince it relies only on a weak coupling between the oscillators It has beenshown that some chaotic oscillators are able to synchronize their phases forconsiderably weak coupling between them [46] To quantify the process ofsynchronization, different measures have been proposed [36, 45, 65] Two fre-quently used measures are a measure based on entropy and a measure based

on circular statistics, which is the so called mean phase coherence [36] Bothmeasures quantify the sharpness of peaks in distributions of the phase differ-ences In the following the mean phase coherence is introduced

1.6.1 Self-Sustained Oscillators

While in the framework of linear systems as vector auto-regressive or average processes are of particular interest, in nonlinear dynamics self-sus-tained oscillators play an important role In general these oscillators can beformulated as

moving-˙

whereby X(t) has to be a more dimensional variable to ensure an oscillatory

behavior The external influence as well as the parameters can either be valued or not

vector-Since especially the interaction between processes is considered here, thefollowing description of a system of coupled oscillators

and h2(·) of the coupling can thereby be arbitrary In general it is even

not necessary that it is a function It can as well be a relation However,here only well behaved functions are considered Usually, diffusive coupling is

used, i.e h1(X1(t), X2(t)) = (X2(t) − X1(t)) and h2accordingly For ε i,j= 0the solution of the above system is expected to be a limit cycle for each oscil-lator in the sequel This is to ensure a much simpler mathematical motivation

of phase synchronization

1.7 Phase Synchronization

To describe the interaction between coupled self-sustained oscillators, the

no-tion of phase synchrony has gained particular interest The phase Φ(t) of a

limit cycle (periodic) oscillator is a monotonically increasing function with

Φ(t)| t=pT = p2π = pωT , where p denotes the number of completed cycles, T is the time needed for one complete cycle, and ω the frequency of the oscillator To define the phase also for values of the time t = pT , the following expression is used

h k j(X1, X2) (1.31)

can be obtained in the case of coupled oscillators as introduced above [43]

The superscript k denotes the k-th component of the corresponding vector For small ε i,j the above sum can be approximated by 2π periodic functions

˙

Φ1(t) = ω1+ ε 1,2 H1(Φ1, Φ2) (1.32)

˙

Φ2(t) = ω2+ ε 2,1 H2(Φ2, Φ1) (1.33)which leads to

n ˙ Φ1(t) − m ˙Φ2(t) = nω1− mω2+ ε 1,2 H˜1(Φ1, Φ2)− ε 2,1 H˜2(Φ2, Φ1)

for some integers n and m [43] The difference n ˙ Φ1(t) − m ˙Φ2(t) can be

con-sidered as a generalized phase difference starting from the simplest expression

˙

Φ1(t) − ˙Φ2(t) with n, m = 1.

In the case of ε 1,2 = ε 2,1 and with the notion Φ n,m 1,2 = nΦ1− mΦ2 and

Δω = nω − mω the above differential equation can be written as