Stochastic Controledited Part 1 potx

The Fokker-Planck equation describes the evolution of conditional probability density for given initial states for a Markov process, which satisfies the Itô stochastic differential equat

Stochastic Control

edited by

Chris Myers

SCIYO

Statements and opinions expressed in the chapters are these of the individual contributors and not necessarily those of the editors or publisher No responsibility is accepted for the accuracy of information contained in the published articles The publisher assumes no responsibility for any damage or injury to persons or property arising out of the use of any materials, instructions, methods

or ideas contained in the book

Publishing Process Manager Jelena Marusic

Technical Editor Goran Bajac

Cover Designer Martina Sirotic

Image Copyright Evv, 2010 Used under license from Shutterstock.com

First published September 2010

Printed in India

A free online edition of this book is available at www.sciyo.com

Additional hard copies can be obtained from publication@sciyo.com

Stochastic Control, Edited by Chris Myers

p cm

ISBN 978-953-307-121-3

WHERE KNOWLEDGE IS FREE

free online editions of Sciyo

Books, Journals and Videos can

be found at www.sciyo.com

The Fokker-Planck equation 1

Shambhu N Sharma and Hiren G Patel

The Itô calculus for a noisy dynamical system 21

Shambhu N Sharma

Application of coloured noise as a driving

force in the stochastic differential equations 43

W.M.Charles

Complexity and stochastic synchronization

in coupled map lattices and cellular automata 59

Ricardo López-Ruiz and Juan R Sánchez

Zero-sum stopping game associated with threshold probability 81

Xavier Warin and Stephane Vialle

Exploring Statistical Processes with Mathematica7 125

Fred Spiring

A learning algorithm based on PSO and L-M for parity problem 151

Guangyou Yang, Daode Zhang, and Xinyu Hu

Improved State Estimation of Stochastic Systems

via a New Technique of Invariant Embedding 167

Nicholas A Nechval and Maris Purgailis

Fuzzy identification of discrete time nonlinear stochastic systems 195

Ginalber L O Serra

Contents

Fuzzy frequency response for stochastic

linear parameter varying dynamic systems 217

Carlos C T Ferreira and Ginalber L O Serra

Delay-dependent exponential stability and filtering

for time-delay stochastic systems with nonlinearities 235

Huaicheng Yan, Hao Zhang, Hongbo Shi and Max Q.-H Meng

Optimal filtering for linear states over polynomial observations 261

Joel Perez, Jose P Perez and Rogelio Soto

The stochastic matched filter

and its applications to detection and de-noising 271

Philippe Courmontagne

Wireless fading channel models:

from classical to stochastic differential equations 299

Mohammed Olama, Seddik Djouadi and Charalambos Charalambous

Information flow and causality quantification

in discrete and continuous stochastic systems 329

X San Liang

Reduced-Order LQG Controller

Design by Minimizing Information Loss 353

Suo Zhang and Hui Zhang

The synthesis problem of the optimum control

for nonlinear stochastic structures in the multistructural

systems and methods of its solution 371

Sergey V Sokolov

Optimal design criteria for isolation devices in vibration control 393

Giuseppe Carlo Marano and Sara Sgobba

Sensitivity analysis and stochastic modelling

of the effective properties for reinforced elastomers 411

Marcin Kamiński and Bernd Lauke

Stochastic improvement of structural design 437

Soprano Alessandro and Caputo Francesco

Modelling earthquake ground motions by stochastic method 475

Nelson Lam, John Wilson and Hing Ho Tsang

Quasi-self-similarity for laser-plasma interactions

modelled with fuzzy scaling and genetic algorithms 493

Danilo Rastovic

Efficient Stochastic Simulation to Analyze

Targeted Properties of Biological Systems 505

Hiroyuki Kuwahara, Curtis Madsen, Ivan Mura,

Chris Myers, Abiezer Tejeda and Chris Winstead

Stochastic Decision Support Models and Optimal

Stopping Rules in a New Product Lifetime Testing 533

Nicholas A Nechval and Maris Purgailis

A non-linear double stochastic model

of return in financial markets 559

Vygintas Gontis, Julius Ruseckas and Aleksejus Kononovičius

Mean-variance hedging under partial information 581

M Mania, R Tevzadze and T Toronjadze

Pertinence and information needs of different subjects

on markets and appropriate operative (tactical or strategic)

stochastic control approaches 609

Vladimir Šimović and Vladimir Šimović, j.r.

Fractional bioeconomic systems:

optimal control problems, theory and applications 629

Darya V Filatova, Marek Grzywaczewski and Nikolai P Osmolovskii

Uncertainty presents significant challenges in the reasoning about and controlling of complex dynamical systems To address this challenge, numerous researchers are developing improved methods for stochastic analysis This book presents a diverse collection of some of the latest research in this important area In particular, this book gives an overview of some

of the theoretical methods and tools for stochastic analysis, and it presents the applications of these methods to problems in systems theory, science, and economics

The first section of the book presents theoretical methods and tools for the analysis of stochastic systems The first two chapters by Sharma et al present the Fokker-Planck equation and the Ito calculus In Chapter 3, Charles presents the use of colored noise with stochastic differential equations In Chapter 4, Lopez-Ruiz and Sanchez discuss coupled map lattices and cellular automata In Chapter 5, Ohtsubo presents a game theoretic approach In Chapter 6, Doria presents an approach that uses Hausdorff outer and inner measures In Chapter 7, Warin and Vialle for analysis using distributed algorithms Finally, in Chapter 8, Spiring explores the use

of Mathematica7

The second section of the book presents the application of stochastic methods in systems theory In Chapter 9, Yang et al present a learning algorithm for the parity problem In Chapter 10, Nechval and Pugailis present an improved technique for state estimation In Chapter 11, Serra presents a fuzzy identification method In Chapter 12, Ferreira and Serra present an application of fuzzy methods to dynamic systems The next three chapters by Yan

et al., Perez et al., and Courmontagne explore the problem of filtering for stochastic systems

In Chapter 16, Olama et al look at wireless fading channel models In Chapter 17, Liang considers information flow and causality quantification The last two chapters of this section

by Zhang and Zhang and Sokolov consider control systems

The third section of the book presents the application of stochastic methods to problems in science In Chapter 20, Marano and Sgobba present design criteria for vibration control In Chapter 21, Kaminski and Lauke consider reinforced elastomers In Chapter 22, Alessandro and Francesco discuss structural design In Chapter 23, Lam et al apply stochastic methods to the modeling of earthquake ground motion In Chapter 24, Rastovic addresses laser-plasma interactions Finally, in Chapter 25, Kuwahara et al apply new, efficient stochastic simulation methods to biological systems

Preface

The final section of the book presents the application of stochastic methods to problems in economics In Chapter 26, Nechval and Purgailis consider the problem of determining a products lifetime In Chapter 27, Gontis et al applies a stochastic model to financial markets

In Chapter 28, Mania et al take on the problem of hedging in the market In Chapter 29, Simovic and Simovic apply stochastic control approaches to tactical and strategic operations

in the market Finally, in Chapter 30, Darya et al consider optimal control problems in fractional bio-economic systems

Editor

Chris Myers

University of Utah

U.S.A.

The Fokker-Planck equation

Shambhu N Sharma and Hiren G Patel

X The Fokker-Planck equation

Shambhu N Sharma † and Hiren G Patel‡

Department of Electrical Engineering†

National Institute of Technology, Surat, India

snsvolterra@gmail.com Department of Electrical Engineering‡

National Institute of Technology, Surat, India

hgp@eed.svnit.ac.in

In 1984, H Risken authored a book (H Risken, The Fokker-Planck Equation: Methods of

Solution, Applications, Springer-Verlag, Berlin, New York) discussing the Fokker-Planck

equation for one variable, several variables, methods of solution and its applications,

especially dealing with laser statistics There has been a considerable progress on the topic

as well as the topic has received greater clarity For these reasons, it seems worthwhile again

to summarize previous as well as recent developments, spread in literature, on the topic

The Fokker-Planck equation describes the evolution of conditional probability density for

given initial states for a Markov process, which satisfies the Itô stochastic differential

equation The structure of the Fokker-Planck equation for the vector case is

, ) , , ( ) , ( (

2

1 ) ) , , ( ) , ( ( ) ,

T t

t

x x

t x t x p t x GG tr

x

t x t x p t x f tr t

where f ( t xt, )is the system non-linearity, G ( t xt, )is termed as the process noise

coefficient, and p ( x , t xt0, to)is the conditional probability density The Fokker-Planck

equation, a prediction density evolution equation, has found its applications in developing

prediction algorithms for stochastic problems arising from physics, mathematical control

theory, mathematical finance, satellite mechanics, as well as wireless communications In

this chapter, the Authors try to summarize elementary proofs as well as proofs constructed

from the standard theories of stochastic processes to arrive at the Fokker-Planck equation

This chapter encompasses an approximate solution method to the Fokker-Planck equation

as well as a Fokker-Planck analysis of a Stochastic Duffing-van der Pol (SDvdP) system,

which was recently analysed by one of the Authors

Key words: The Duffing-van der Pol system, the Galerkin approximation, the

Ornstein-Uhlenbeck process, prediction density, second-order fluctuation equations

1

Stochastic Control2

1 Introduction

The stochastic differential equation formalism arises from stochastic problems in diverse

field, especially the cases, where stochastic problems are analysed from the dynamical

systems’ point of view Stochastic differential equations have found applications in

population dynamics, stochastic control, radio-astronomy, stochastic networks, helicopter

rotor dynamics, satellite trajectory estimation problems, protein kinematics, neuronal

activity, turbulence diffusion, stock pricing, seismology, statistical communication theory,

and structural mechanics A greater detail about stochastic differential equations’

applications can be found in Kloeden and Platen (1991) Some of the standard structures of

stochastic differential equations are the Itô stochastic differential equation, the Stratonovich

stochastic differential equation, the stochastic differential equation involving p-differential,

stochastic differential equation in Hida sense, non-Markovian stochastic differential

equations as well as the Ornstein-Uhlenbeck (OU) process-driven stochastic differential

equation The Itô stochastic differential equation is the standard formalism to analyse

stochastic differential systems, since non-Markovian stochastic differential equations can be

re-formulated as the Itô stochastic differential equation using the extended phase space

formulation, unified coloured noise approximation (Hwalisz et al 1989) Stochastic

differential systems can be analysed using the Fokker-Planck equation (Jazwinski 1970) The

Fokker-Planck equation is a parabolic linear homogeneous differential equation of order two

in partial differentiation for the transition probability density The Fokker-Planck operator is

an adjoint operator In literature, the Fokker-Planck equation is also known as the

Kolmogorov forward equation The Kolmogorov forward equation can be proved using

mild regularity conditions involving the notion of drift and diffusion coefficients (Feller

2000) The Fokker-Planck equation, definition of the conditional expectation, and integration

by part formula allow to derive the evolution of the conditional moment In the Risken’s

book, the stochastic differential equation involving the Langevin force was considered and

subsequently, the Fokker-Planck equation was derived The stochastic differential equation

with the Langevin force can be regarded as the white noise-driven stochastic differential

equation, where the input process satisfies wt  0 , wtws   ( t  s ). He considered

the approximate solution methods to the scalar and vector Fokker-Planck equations

involving change of variables, matrix continued-fraction method, numerical integration

method, etc (Risken 1984, p 158) Further more, the laser Fokker-Planck equation was

derived

This book chapter is devoted to summarize alternative approaches to derive the

Fokker-Planck equation involving elementary proofs as well as proofs derived from the Itô

differential rule In this chapter, the Fokker-Planck analysis hinges on the stochastic

differential equation in the Itô sense in contrast to the Langevin sense From the

mathemacians’ point of view, the Itô stochastic differential equation involves rigorous

interpretation in contrast to the Langevin stochastic differential equation On the one hand,

the stochastic differential equation in Itô sense is described as

, ) , ( )

,

dx   on the other, the Langevin stochastic differential

equation assumes the structure x t  f ( xt, t )  G ( xt, t ) wt,where Btandwt are the

Brownian and white noises respectively The white noise can be regarded as an informal

non-existent time derivative Btof the Brownian motion Bt. Kiyoshi Itô, a famous Japanese mathematician, considered the term ' dBt'  B tdtand developed Itô differential rule The results of Itô calculus were published in two seminal papers of Kiyoshi Itô in 1945 The approach of this chapter is different and more exact in contrast to the Risken’s book in the sense that involving the Itô stochastic differential equation, introducing relatively greater discussion on the Kolmogorov forward and Backward equations This chapter discusses a Fokker-Planck analysis of a stochastic Duffing-van der Pol system, an appealing case, from the dynamical systems’ point of view as well

This chapter is organised as follows: (i) section 2 discusses the evolution equation of the prediction density for the Itô stochastic differential equation A brief discussion about approximate methods to the Fokker-Planck equation, stochastic differential equation is also given in section 2 (ii) in section 3, the stochastic Duffing-van der Pol system was analysed to demonstrate a usefulness of the Fokker-Planck equation (iii) Section 4 is about the numerical simulation of the mean and variance evolutions of the SDvdP system Concluding remarks are given in section (5)

2 Evolution of conditional probability density

The Fokker-Planck equation describes the evolution of conditional probability density for given initial states for the Itô stochastic differential system The equation is also known as the prediction density evolution equation, since it can be utilized to develop prediction algorithms, especially where observations are not available at every time instant One of the potential applications of the Fokker-Planck equation is to develop estimation algorithms for the satellite trajectory estimation This chapter summarizes four different proofs to arrive at the Fokker-Planck equation The first two proofs can be regarded as elementary proofs and the last two utilize the Itô differential rule Moreover, the Fokker-Planck equation for the OU process-driven stochastic differential equation is discussed here, where the input process

has non-zero, finite, relatively smaller correlation time

The first proof of this chapter begins with the Chapman-Kolmogorov equation The

Chapman-Kolmogorov equation is a consequence of the theory of the Markov process This plays a key role in proving the Kolmogorov backward equation (Feller 2000) Here, we describe briefly the Chapman-Kolmogorov equation and subsequently, the concept of the conditional probability density as well as transition probability density are introduced to derive the evolution of conditional probability density for the non-Markov process The Fokker-Planck equation becomes a special case of the resulting equation The conditional probability density

).

( )

, ( ) , ( x1 x2 x3 p x1 x2 x3 p x2 x3

Consider the random variables xt1, xt2, xt3at the time instantst1, t t2, 3, where

3 2

t   and take values x1, x2, x3. In the theory of the Markov process, the above can

be re-stated as

) ( ) ( ) , ( x1 x2 x3 p x1 x2 p x2 x3

1 Introduction

The stochastic differential equation formalism arises from stochastic problems in diverse

field, especially the cases, where stochastic problems are analysed from the dynamical

systems’ point of view Stochastic differential equations have found applications in

population dynamics, stochastic control, radio-astronomy, stochastic networks, helicopter

rotor dynamics, satellite trajectory estimation problems, protein kinematics, neuronal

activity, turbulence diffusion, stock pricing, seismology, statistical communication theory,

and structural mechanics A greater detail about stochastic differential equations’

applications can be found in Kloeden and Platen (1991) Some of the standard structures of

stochastic differential equations are the Itô stochastic differential equation, the Stratonovich

stochastic differential equation, the stochastic differential equation involving p-differential,

stochastic differential equation in Hida sense, non-Markovian stochastic differential

equations as well as the Ornstein-Uhlenbeck (OU) process-driven stochastic differential

equation The Itô stochastic differential equation is the standard formalism to analyse

stochastic differential systems, since non-Markovian stochastic differential equations can be

re-formulated as the Itô stochastic differential equation using the extended phase space

formulation, unified coloured noise approximation (Hwalisz et al 1989) Stochastic

differential systems can be analysed using the Fokker-Planck equation (Jazwinski 1970) The

Fokker-Planck equation is a parabolic linear homogeneous differential equation of order two

in partial differentiation for the transition probability density The Fokker-Planck operator is

an adjoint operator In literature, the Fokker-Planck equation is also known as the

Kolmogorov forward equation The Kolmogorov forward equation can be proved using

mild regularity conditions involving the notion of drift and diffusion coefficients (Feller

2000) The Fokker-Planck equation, definition of the conditional expectation, and integration

by part formula allow to derive the evolution of the conditional moment In the Risken’s

book, the stochastic differential equation involving the Langevin force was considered and

subsequently, the Fokker-Planck equation was derived The stochastic differential equation

with the Langevin force can be regarded as the white noise-driven stochastic differential

equation, where the input process satisfies wt  0 , wtws   ( t  s ). He considered

the approximate solution methods to the scalar and vector Fokker-Planck equations

involving change of variables, matrix continued-fraction method, numerical integration

method, etc (Risken 1984, p 158) Further more, the laser Fokker-Planck equation was

derived

This book chapter is devoted to summarize alternative approaches to derive the

Fokker-Planck equation involving elementary proofs as well as proofs derived from the Itô

differential rule In this chapter, the Fokker-Planck analysis hinges on the stochastic

differential equation in the Itô sense in contrast to the Langevin sense From the

mathemacians’ point of view, the Itô stochastic differential equation involves rigorous

interpretation in contrast to the Langevin stochastic differential equation On the one hand,

the stochastic differential equation in Itô sense is described as

, )

, (

)

,

dx   on the other, the Langevin stochastic differential

equation assumes the structure x t  f ( xt, t )  G ( xt, t ) wt,where Btandwt are the

Brownian and white noises respectively The white noise can be regarded as an informal

non-existent time derivative Btof the Brownian motion Bt. Kiyoshi Itô, a famous Japanese mathematician, considered the term ' dBt'  B tdtand developed Itô differential rule The results of Itô calculus were published in two seminal papers of Kiyoshi Itô in 1945 The approach of this chapter is different and more exact in contrast to the Risken’s book in the sense that involving the Itô stochastic differential equation, introducing relatively greater discussion on the Kolmogorov forward and Backward equations This chapter discusses a Fokker-Planck analysis of a stochastic Duffing-van der Pol system, an appealing case, from the dynamical systems’ point of view as well

This chapter is organised as follows: (i) section 2 discusses the evolution equation of the prediction density for the Itô stochastic differential equation A brief discussion about approximate methods to the Fokker-Planck equation, stochastic differential equation is also given in section 2 (ii) in section 3, the stochastic Duffing-van der Pol system was analysed to demonstrate a usefulness of the Fokker-Planck equation (iii) Section 4 is about the numerical simulation of the mean and variance evolutions of the SDvdP system Concluding remarks are given in section (5)

2 Evolution of conditional probability density

The Fokker-Planck equation describes the evolution of conditional probability density for given initial states for the Itô stochastic differential system The equation is also known as the prediction density evolution equation, since it can be utilized to develop prediction algorithms, especially where observations are not available at every time instant One of the potential applications of the Fokker-Planck equation is to develop estimation algorithms for the satellite trajectory estimation This chapter summarizes four different proofs to arrive at the Fokker-Planck equation The first two proofs can be regarded as elementary proofs and the last two utilize the Itô differential rule Moreover, the Fokker-Planck equation for the OU process-driven stochastic differential equation is discussed here, where the input process

has non-zero, finite, relatively smaller correlation time

The first proof of this chapter begins with the Chapman-Kolmogorov equation The

Chapman-Kolmogorov equation is a consequence of the theory of the Markov process This plays a key role in proving the Kolmogorov backward equation (Feller 2000) Here, we describe briefly the Chapman-Kolmogorov equation and subsequently, the concept of the conditional probability density as well as transition probability density are introduced to derive the evolution of conditional probability density for the non-Markov process The Fokker-Planck equation becomes a special case of the resulting equation The conditional probability density

).

( )

, ( ) , ( x1 x2 x3 p x1 x2 x3 p x2 x3

Consider the random variables xt1, xt2, xt3at the time instantst1, t t2, 3, where

3 2

t   and take values x1, x2, x3. In the theory of the Markov process, the above can

be re-stated as

) ( ) ( ) , ( x1 x2 x3 p x1x2 p x2 x3

Stochastic Control4

integrating over the variablex2, we have

, ) ( ) ( ) ( x1 x3 p x1 x2 p x2 x3 dx2

introducing the notion of the transition probability density and time instants

) , ( ) , ( )

, ( x1 x3 q x1 x2 q x2 x3 dx2

Consider the multi-dimensional probability density p ( x1, x2)  p ( x1x2) p ( x2)and

integrating over the variablex2, we have

, ) ( ) ( ) ( x1 p x1 x2 p x2 dx2

or

p ( x1)   qt1,t2( x1, x2) p ( x2) dx2, (1)

where qt1,t2( x1, x2)is the transition probability density and t 1 t2..The transition

probability densityqt1,t2( x1, x2) is the inverse Fourier transform of the characteristic

function iu(x t1 x t2),

2

1 ) ,

2 1

1 x x e Ee du

t t

x

iu

x x n

iu

!

) (

2 1 1

x n

iu e

x

p

n

n t t

n x

x iu

2 2 0

) (

!

) ( ( 2

1 ) (  1 2  1 2

) )

( 2

1 (

2 1 2

1 du x x p x dx e

iu n

n

n t t x

x iu n

dx x p x x x x x

dx x p x x x x x

n x

n

) (

!

1 )

(0

n

) (

!

10

!

1 )

( ) (

1

x n

x p x p

!

1 )

(1

x p x k x n x

as the non-Markov process Consider a Markov process, which satisfies the Itô stochastic differential equation, the evolution of conditional probability density retains only the first two termsk1( x )andk2( x ), which is a direct consequence of the stochastic differential rule for the Itô stochastic differential equation in combination with the definition

).

( )

integrating over the variablex2, we have

, )

( )

( )

( x1x3 p x1x2 p x2 x3 dx2

introducing the notion of the transition probability density and time instants

)

, (

) ,

( )

, ( x1 x3 q x1 x2 q x2 x3 dx2

Consider the multi-dimensional probability density p ( x1, x2)  p ( x1x2) p ( x2)and

integrating over the variablex2, we have

, )

( )

( )

where qt1,t2( x1, x2)is the transition probability density and t 1 t2..The transition

probability densityqt1,t2( x1, x2) is the inverse Fourier transform of the characteristic

function iu(x t1 x t2),

2

1 )

,

2 1

1 x x e Ee du

t t

x

iu

x x

2 1

1 1

x p

x x

n

iu e

x

p

n

n t

t

n x

x iu

2 2

0

) (

!

) (

( 2

1 )

( )

( )

) (

2

1 (

2 1

2

1 du x x p x dx e

iu n

n

n t

t x

x iu

dx x p x x x x x

dx x p x x x x x

n x

n

) (

!

1 )

(0

n

) (

!

10

!

1 )

( ) (

1

x n

x p x p

!

1 )

(1

x p x k x n x

as the non-Markov process Consider a Markov process, which satisfies the Itô stochastic differential equation, the evolution of conditional probability density retains only the first two termsk1( x )andk2( x ), which is a direct consequence of the stochastic differential rule for the Itô stochastic differential equation in combination with the definition

).

( )

Stochastic Control6

leads to the Fokker-Planck equation,

), ( ) ,

( 2

1 ) ( ) , ( )

x

t x g x

p t x f x x

t x GG t

x f

2( ) ( , )(.) 2

1 )(.) ,

of the conditional moment (Jazwinski 1970) The Fokker-Planck equation

is also known as the Kolmogorov Forward equation

The second proof of this chapter begins with the Green function, the Kolmogorov forward

and backward equations involve the notion of the drift and diffusion coefficients as well as

mild regularity conditions (Feller 2000) The drift and diffusion coefficients are regarded as

the system non-linearity and the ‘stochastic perturbation in the variance evolution’

respectively in noisy dynamical system theory Here, we explain briefly about the formalism

associated with the proof of the Kolmogorov forward and backward equations Consider the

Green’s function

ut( x )   qt( x , y ) u0( y ) dy , (5)

where qt( y x , )is the transition probability density, ut(x )is a scalar function, xis the

initial point and y is the final point Equation (5) is modified at the time duration t  has

uth( x )   qth( x , y ) u0( y ) dy (6)

The Chapman-Kolmogorov equation can be stated as

) , ( ) , ( )

, ( x y q x  q  y d 

x u x b t

, b (x )and a (x )are the drift and diffusion coefficients respectively (Feller 2000), and the detailed proof of equation (8) can be found in a celebrated book authored by Feller (2000) For the vector case, the Kolmogorov backward equation can

1 ) ( ) ( )

j i

t ij

i

t i

t

x x

x u x a x

x u x b t

x u

where the summation is extended for 1  i  n , 1  j  n From the dynamical systems’ point of view, the vector case of the Kolmogorov backward equation can be reformulated as

1 ) ( ) , ( )

j i

t ij

i

t i

t

x x

x u t x GG x

x u t x f t

x u

where the mappings f and G are the system non-linearity and process noise coefficient matrix respectively and the Kolmogorov backward operator

, ) ( ) , ( )

a y

y v y

b s

leads to the Fokker-Planck equation,

), (

) ,

( 2

1 )

( )

, (

)

x

t x

g x

p t

x f

x x

t x

GG t

x f

2( ) ( , )(.) 2

1 )(.)

of the conditional moment (Jazwinski 1970) The Fokker-Planck equation

is also known as the Kolmogorov Forward equation

The second proof of this chapter begins with the Green function, the Kolmogorov forward

and backward equations involve the notion of the drift and diffusion coefficients as well as

mild regularity conditions (Feller 2000) The drift and diffusion coefficients are regarded as

the system non-linearity and the ‘stochastic perturbation in the variance evolution’

respectively in noisy dynamical system theory Here, we explain briefly about the formalism

associated with the proof of the Kolmogorov forward and backward equations Consider the

Green’s function

ut( x )   qt( x , y ) u0( y ) dy , (5)

where qt( y x , )is the transition probability density, ut(x )is a scalar function, xis the

initial point and y is the final point Equation (5) is modified at the time duration t  has

uth( x )   qth( x , y ) u0( y ) dy (6)

The Chapman-Kolmogorov equation can be stated as

)

, (

) ,

( )

, ( x y q x  q  y d 

x u x b t

, b (x )and a (x )are the drift and diffusion coefficients respectively (Feller 2000), and the detailed proof of equation (8) can be found in a celebrated book authored by Feller (2000) For the vector case, the Kolmogorov backward equation can

1 ) ( ) ( )

j i

t ij

i

t i

t

x x

x u x a x

x u x b t

x u

where the summation is extended for 1  i  n , 1  j  n From the dynamical systems’ point of view, the vector case of the Kolmogorov backward equation can be reformulated as

1 ) ( ) , ( )

j i

t ij

i

t i

t

x x

x u t x GG x

x u t x f t

x u

where the mappings f and G are the system non-linearity and process noise coefficient matrix respectively and the Kolmogorov backward operator

, ) ( ) , ( )

a y

y v y

b s

Stochastic Control8

, ) ( ) ( 2

1 ) ( ) ( )

s i s

y y

y v y a y

y v y

b s

y v

2

1 )(.) (

y a y

y b

For

i

b  anda j  ( GGT) j , the Kolmogorov forward operator assumes the structure

of the Fokker-Planck operator and is termed as the Kolmogorov-Fokker-Planck operator

The third proof of the chapter explains how the Fokker-Planck equation can be derived using

the definition of conditional expectation and Itô differential rule

E  ( xtdt)  E ( E (  ( xtdt) xt  x )). (10) The Taylor series expansion of the scalar function ( )

!

) ( )

m

m t dt

!

) , ( )

) (

(

2 0

x m

t x x

x x

m

m t

two for the Brownian motion process-driven stochastic differential equation that can be

explained via the Itô differential rule Equation (10) in conjunction with equation (11) leads

to



 ) ( xt dt

!

) , ( (2 0

x m

t x

dx t x p x m

, ) , ( ) ,

(

!

) 1 ( )

, (

x

t x p t

x m

t

t x

Finally, we derive the Fokker-Planck equation using the concept of the evolution of the

conditional moment and the conditional characteristic function Consider the state vectorxt  U ,  : U  R, i.e  ( xt)  R ,and the phase spaceU  Rn. The state vector xt satisfies the Itô SDE as well Suppose the function ( xt)is twice differentiable The evolution d   ( xt)

of the conditional moment is the standard formalism to analyse stochastic differential systems Further more, d   ( xt)

) , ) ( ( d x x0 t0

, ( ) ( 2

1 ) , ( ) ( ( )

(

i

t t

ii T i

t i

t

x t

x GG t

x f x

x x

x t

x GG

j i

t j

1 ,



dB t x G x

x

t i r

x x

x t x GG x

x t x G G x

x t x f x

d

q

t t

pq T

t t

pp T

t t

(),()(2

1),((

)

x

x t x G S e

t x f S e

dE

q

x S t pq T q p

t t

pp T p p

x S t p p x

The Kushner equation, the filtering density evolution equation for the Itô stochastic

differential equation, is a ‘generalization’ of the Fokker-Planck equation The Kushner equation is a partial-integro stochastic differential equation, i.e

, ) (

) ( ) ( p dt h h 1 dz h dt p L

, )

( )

( 2

1 )

( )

( )

s i

s

y y

y v

y a

y

y v

y

b s

y v

2

1 )(.)

y a

y

y b

For

i

b  anda j  ( GGT) j , the Kolmogorov forward operator assumes the structure

of the Fokker-Planck operator and is termed as the Kolmogorov-Fokker-Planck operator

The third proof of the chapter explains how the Fokker-Planck equation can be derived using

the definition of conditional expectation and Itô differential rule

E  ( xtdt)  E ( E (  ( xtdt) xt  x )). (10) The Taylor series expansion of the scalar function ( )

!

) (

)

m

m t

!

) ,

( )

) (

(

2 0

x m

t x

x x

x

m

m t

two for the Brownian motion process-driven stochastic differential equation that can be

explained via the Itô differential rule Equation (10) in conjunction with equation (11) leads

to



 ) ( xt dt

!

) ,

( (

2 0

x m

t x

(2

0

dx t

x p

x m

, )

, (

) ,

(

!

) 1

( )

, (

m

x

t x

p t

x m

t

t x

Finally, we derive the Fokker-Planck equation using the concept of the evolution of the

conditional moment and the conditional characteristic function Consider the state vectorxt  U ,  : U  R, i.e  ( xt)  R ,and the phase spaceU  Rn. The state vector xt satisfies the Itô SDE as well Suppose the function ( xt)is twice differentiable The evolution d   ( xt)

of the conditional moment is the standard formalism to analyse stochastic differential systems Further more, d   ( xt)

) , ) ( ( d x x0 t0

, ( ) ( 2

1 ) , ( ) ( ( )

(

i

t t

ii T i

t i

t

x t

x GG t

x f x

x x

x t

x GG

j i

t j

1 ,



dB t x G x

x

t i r

x x

x t x GG x

x t x G G x

x t x f x

d

q

t t

pq T

t t

pp T

t t

(),()(2

1),((

)

x

x t x G G S e

t x f S e

dE

q

x S t pq T q p

t t

pp T p p

x S t p p x

The Kushner equation, the filtering density evolution equation for the Itô stochastic

differential equation, is a ‘generalization’ of the Fokker-Planck equation The Kushner equation is a partial-integro stochastic differential equation, i.e

, ) (

) ( ) ( p dt h h 1 dz h dt p L

Stochastic Control10

where L (.)is the Fokker-Planck operator, p  p ( x , t z, t0    t ), the observation

, )

z    and h ( t xt, )is the measurement non-linearity Harald J

Kushner first derived the expression of the filtering density and subsequently, the filtering

density evolution equation using the stochastic differential rule (Jazwinski 1970)

Liptser-Shiryayev discovered an alternative proof of the filtering density evolution, equation (14),

involving the following steps: (i) derive the stochastic evolution 

) ( xt

d  of the conditional moment, where ( xt)  E ( ( xt) z , t0   t )

 (iii) the definition of the conditional expectation as well as integration by part formula lead to the filtering density evolution

equation, see Liptser and Shiryayev (1977) RL Stratonovich developed the filtering density

evolution for stochastic differential equation involving the

2

1-differential as well For this reason, the filtering density evolution equation is also termed as the Kushner-Stratonovich

equation

Consider the stochastic differential equation of the form

x t  f ( xt)  g ( xt) t, (15)

where tis the Ornstein-Uhlenbeck process and generates the process xt, a non-Markov

process The evolution of conditional probability density for the non-Markov process with

the input process with a non-zero, finite, smaller correlation timecor, i.e 0  cor  1,

reduces to the Fokker-Planck equation One of the approaches to arrive at the Fokker-Planck

equation for the OU process-driven stochastic differential equation with smaller correlation

time is function calculus The function calculus approach involves the notion of the

functional derivative The evolution of conditional probability density for the output

process xt, where the input processt is a zero mean, stationary and Gaussian process,

can be written (Hänggi 1995, p.85) as

), )

( ) ( )(

( )

( )

x x g x x

p x f x

p

s

t t

, cov(

) , (

s t

The functional derivative

s t

( )

 t   ) ,

s

d x

) (    

t t

x

g  ( )   ( ) 

), ) ( ) ( )(

( xt f xt g xt t

after some calculations, the integral counterpart of the above equation can be stated as

)).

) ) ( )

(

) ( ) ( ( ( exp(

) ( )

x f x g x

g x

s t