Báo cáo hóa học: " Research Article Some Shannon-McMillan Approximation Theorems for Markov Chain Field on the Generalized Bethe Tree" docx

Volume 2011, Article ID 470910, 18 pagesdoi:10.1155/2011/470910 Research Article Some Shannon-McMillan Approximation Theorems for Markov Chain Field on the Generalized Bethe Tree 1 Schoo

Volume 2011, Article ID 470910, 18 pages

doi:10.1155/2011/470910

Research Article

Some Shannon-McMillan Approximation

Theorems for Markov Chain Field on the

Generalized Bethe Tree

1 School of Mathematics and Physics, Jiangsu University of Science and Technology,

Zhenjiang 212003, China

2 College of Computer Science and Engineering, Changshu Institute of Technology,

Changshu 215500, China

Correspondence should be addressed to Wang Kangkang,wkk.cn@126.com

Received 26 September 2010; Accepted 7 January 2011

Academic Editor: J ´ozef Bana´s

Copyrightq 2011 W Kangkang and D Zong This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited

A class of small-deviation theorems for the relative entropy densities of arbitrary random field on

the generalized Bethe tree are discussed by comparing the arbitrary measure μ with the Markov measure μ Q on the generalized Bethe tree As corollaries, some Shannon-Mcmillan theorems for the arbitrary random field on the generalized Bethe tree, Markov chain field on the generalized Bethe tree are obtained

1 Introduction and Lemma

Let T be a tree which is infinite, connected and contains no circuits Given any two vertices

x / y ∈ T, there exists a unique path x  x1, x2, , xm  y from x to y with x1, x2, , xm

connecting x and y To index the vertices on T, we first assign a vertex as the “root” and label

it as O A vertex is said to be on the nth level if the path linking it to the root has n edges The root O is also said to be on the 0th level.

Definition 1.1 Let T be a tree with root O, and let {N n, n ≥ 1} be a sequence of positive

integers T is said to be a generalized Bethe tree or a generalized Cayley tree if each vertex

Root (0,1)

Level 2 Level 3

Figure 1: Bethe tree T B,2

which each vertex has N branches to the next level.

In the following, we always assume that T is a generalized Bethe tree and denote by



T n n

m0

N0· · · N m 1 n

m1

X  {Xt, t ∈ T} be the coordinate stochastic process defined on the measurable space Ω, F;

X T n





Xt, t ∈ T n

X T n

 x T n

 μx T n

Now we give a definition of Markov chain fields on the tree T by using the cylinder

distribution directly, which is a natural extension of the classical definition of Markov chains

see 1

Definition 1.2 Let Q  Qj | i One has a strictly positive stochastic matrix on S, q  qs0,

μ Q x 0,1   qx 0,1 ,

μ Q

x T n

 qx 0,1n−1

m0

N0···N m i1

Nm1i

jNm1 i−11

q

Then μ Q will be called a Markov chain field on the tree T determined by the stochastic matrix

Q and the distribution q.

fn ω  − T1n  logμX T n

fn ω  − T1n

⎡

⎣log qX 0,1 n−1

m0

N0···N m i1

Nm1 i jNm1 i−11

log q

X m1,j | X m,i

⎤

almost sure convergence is called the Shannon-McMillan theorem or the entropy theorem or

with the development of the information theory scholars get to study the Shannon-McMillan

drawn increasing interest from specialists in physics, probability and information theory

Shannon-McMillan theorem for PPG-invariant random fields on trees But their results only relate to

Shannon-McMillan theorems, the limit properties and the asymptotic equipartition property for Markov chains indexed by a homogeneous tree and the Cayley tree, respectively Shi and

second-order Markov chains indexed by a tree

In this paper, we study a class of Shannon-McMillan random approximation theorems for arbitrary random fields on the generalized Bethe tree by comparison between the arbitrary measure and Markov measure on the generalized Bethe tree As corollaries, a class of Shannon-McMillan theorems for arbitrary random fields and the Markov chains field on the generalized Bethe tree are obtained Finally, some limit properties for the expectation of the random conditional entropy are discussed

Lemma 1.3 Let μ1 and μ2 be two probability measures on Ω, F, D ∈ F, and let {τ n, n ≥ 0} be a positive-valued stochastic sequence such that

lim inf

n

τ n

then

lim sup

n → ∞

1

τnlog

μ2

X T n

μ1

In particular, let τn  |T n |, then

lim sup

n → ∞

1

T n logμ2



X T n

μ1

Proofsee 11 Let

ϕ

μ | μQ  lim sup

n → ∞

1

T n log μ



X T n

μQ

ϕμ | μQ  is called the sample relative entropy rate of μ relative to μ Q ϕμ | μ Q is also called

ϕ

μ | μQ ≥ lim inf

n → ∞

1

T n log μ



X T n

μ Q

random fields and the Markov chain fields on the generalized Bethe tree

2 Main Results

Theorem 2.1 Let X  {X t, t ∈ T} be an arbitrary random field on the generalized Bethe tree fn ω

and ϕμ | μQ  are, respectively, defined as 1.5 and 1.10 Denote α ≥ 0, H m Q X m1,j | X m,i  the

random conditional entropy of X m1,j relative to X m,i on the measure μ Q , that is,

H m Q

X m1,j | X m,i  − 

x m1,j ∈S

q

x m1,j | X m,i log q

Let

Dc ω : ϕ

n → ∞

1

T nm0 n−1N0i1 ···N m jNm1 Nm1 i−11 i E Q

log2q

X m1,j | X m,i · q X m1,j | X m,i −α | X m,i



< ∞,

2.3

when 0 ≤ c ≤ α2bα/2,

lim sup

n → ∞

⎧

⎨

1

T nm0 n−1N0i1 ···N m jNm1 Nm1ii−11 H m Q

X m1,j | X m,i

⎫

⎬

⎭

≤2cb α, μ-a.s ω ∈ Dc.

2.4

lim inf

n → ∞

⎧

⎨

1

T nm0 n−1N0i1 ···N m jNm1 Nm1ii−11 H m Q

X m1,j | X m,i

⎫

⎬

⎭

2.5

In particular,

lim

n → ∞

⎡

⎣f n ω − T1nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11

H m Q

X m1,j | X m,i

⎤

⎦

 0, μ-a.s ω ∈ D0,

2.6

where log is the natural logarithmic, E Q is expectation with respect to the measure μ Q

Proof Let Ω, F, μ be the probability space we consider, λ an arbitrary constant Define

E Q

qX m1,j | X m,i−λ | X m,i  x m,i



x m1,j ∈S

qx m1,j | x m,i1−λ; 2.7

denote

μQ

λ, x T n

 qx 0,1n−1

m0

N0···N m i1

Nm1i

jNm1 i−11

qxm1,j | x m,i1−λ

E Q

qX m1,j | X m,i−λ | X m,i  x m,i

We can obtain by2.7, 2.8 that in the case n ≥ 1,



x Ln ∈S

μQ

λ; x T n

x Ln ∈S

qx 0,1n−1

m0

N0···N m i1

Nm1i

jNm1 i−11

q

x m1,j | x m,i 1−λ

E Q

q

X m1,j | X m,i −λ | X m,i  x m,i





λ; x T n−1 

x Ln ∈S

N0···N n−1 i1

Nn i



jNn i−11

q

x n,j | x n−1,i 1−λ

EQ

q

Xn,j | X n−1,i −λ | X n−1,i  x n−1,i





λ; x T n−1N0···N n−1

i1

Nni



jNn i−11



xn,j ∈S

q

xn,j | x n−1,i 1−λ

E Q

q

x n,j | x n−1,i −λ | X n−1,i  x n−1,i



λ; x T n−1N0···N n−1

i1

Nni



jNn i−11

EQ

q

Xn,j | X n−1,i −λ | X n−1,i  x n−1,i



EQ

q

xn,j | x n−1,i −λ | X n−1,i  x n−1,i





λ; x T n−1

,

2.9



xL0 ∈S

μQ

λ; x T0

Therefore, μ Q λ, x T n

Let



λ, X T n

μ

lim

Hence by1.3, 1.9, 2.9, and 2.11 we get

lim sup

n → ∞

1

By1.4, 2.8, and 2.11, we have

1

T n  logU n λ, ω

 T1nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11





qX m1,j | X m,i−λ | X m,i



T1n logμ Q



X T n

μ

X T n

2.14

By1.10, 2.2, 2.13, and 2.14 we have

lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11





qX m1,j | X m,i−λ | X m,i



≤ ϕ μ | μQ ≤ c, μ-a.s ω ∈ Dc.

2.15

lim sup

n → ∞

1

T nm0n−1 N0i1 ···N m jNm1 Nm1ii−11 −λlog q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

≤ lim sup

n → ∞

1

T nm0n−1 N0···N m

i1

Nm1 i jNm1 i−11





q

X m1,j | X m,i −λ | X m,i



−E Q

2.16

By the inequality

 1 2

lim sup

n → ∞

1

T nm0n−1 N0i1 ···N m jNm1 Nm1ii−11 −λlog q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

≤ lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11



EQ

qXm1,j | X m,i−λ | X m,i



− 1



− λ log q X m1,j | X m,i | X m,i



 c

≤ lim sup

n → ∞

1 2Tnm0 n−1N0···N m

i1

Nm1i

jNm1 i−11

E Q

λ2log2

q

X m1,j | X m,i

·qX m1,j | X m,i−|λ| | X m,i



 c

≤ lim sup

n → ∞

λ2

2Tnm0 n−1N0i1 ···N m jNm1 Nm1ii−11 E Q

log2

q

X m1,j |X m,i · q X m1,j | X m,i −α | X m,i



 c 



1 2

2.18

When 0 < λ < α, we get by2.18

lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11

−log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

≤



1

2

λb αλ c , μ-a.s ω ∈ Dc.

2.19

lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m jNm1 Nm1ii−11−log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

≤2cb α, μ-a.s ω ∈ Dc.

2.20

When c  0, we select 0 < λ i < α such that λi → 0 i → ∞ Hence for all i, it follows from

2.19 that

lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m jNm1 Nm1 i−11 i −log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

≤ 0, μ-a.s ω ∈ D0.

2.21

lim inf

n → ∞

1

T nm0n−1 N0i1 ···N m Nm1 i

jNm1 i−11

−log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

≥ −2cb α, μ-a.s ω ∈ Dc.

2.22

lim sup

n → ∞

1

T n  logU n 0, ω  lim sup

n → ∞

1

T n logμ Q



X T n

μ

Noticing

H m Q

X m1,j | X m,i  E Q

− log q X m1,j | X m,i | X m,i

By1.4, 1.5, 2.20, and 2.23, we obtain

lim sup

n → ∞

⎡

⎣f n ω − T n1 m0 n−1N0i1 ···N m Nm1 i

jNm1 i−11

H m Q

X m1,j , Xm,i

⎤

⎦

≤ lim sup

n → ∞

1

T n  logμ Q



X T n

μ

X T n

 lim sup

n → ∞

1

T n m0 n−1N0i1 ···N m jNm1 Nm1ii−11

−log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

≤2cb α, μ-a.s ω ∈ Dc.

2.25

Hence2.4 follows from 2.25 By 1.4, 1.5, 1.10, 2.2, and 2.22, we have

lim inf

n → ∞

⎡

⎣f n ω − T1nm0n−1 N0i1 ···N m jNm1 Nm1ii−11 H m Q ω

⎤

⎦

≥ lim inf

n → ∞

1

T n log

⎡

⎢



X T n

μ

X T n

⎤

⎥

⎦

 lim inf

n → ∞

1

T nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11

−log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

≥ −ϕ μ | μQ −2cb α≥ −2cb α − c, μ-a.s ω ∈ Dc.

2.26

Corollary 2.2 Let X  {X t, t ∈ T} be the Markov chains field determined by the measure μQ on the generalized Bethe tree T ·fn ω, b α are, respectively, defined as1.6 and 2.3, and H m Q X m1,j | X m,i

is defined by2.1 Then

lim

n → ∞

⎧

⎨

1

T nm0 n−1N0i1 ···N m jNm1 Nm1ii−11 H m Q

X m1,j | X m,i

⎫

⎬

⎭ 0 μ Q -a.s. 2.27

Proof We take μ ≡ μ Q , then ϕμ | μ Q ≡ 0 It implies that 2.2 always holds when c  0.

3 Some Shannon-McMillan Approximation Theorems on

the Finite State Space

Corollary 3.1 Let X  {X t, t ∈ T} be an arbitrary random field which takes values in the alphabet

S  {s1, , sN } on the generalized Bethe tree f n ω, ϕμ | μ Q  and Dc are defined as 1.5,

1.10, and 2.2 Denote 0 ≤ α < 1, 0 ≤ c ≤ 2Nα2/1 − αe2 H m Q X m1,j | X m,i  is defined as

above Then

lim sup

n → ∞

⎡

⎣f n ω − T1nm0 n−1N0i1 ···N m jNm1 Nm1ii−11 H m Q

X m1,j | X m,i

⎤

⎦

3.1

lim inf

n → ∞

⎡

⎣f n ω − T n1 m0n−1 N0i1 ···N m Nm1 i

jNm1 i−11

H m Q

X m1,j | X m,i

⎤

⎦

√

3.2

Proof Set 0 < α < 1 we consider the function

φx  log x2x1−α, 0 < x ≤ 1, 0 < α < 1 Set φ0  0 3.3 Then

φ x  x −α

2

φx, 0 ≤ x ≤ 1 φe 2/α−1



 2

α − 1

2

n → ∞

1

T nm0 n−1N0i1 ···N m jNm1 Nm1ii−11 E Q

log2q

X m1,j | X m,i · qX m1,j | X m,i−α | X m,i



 lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m jNm1 Nm1ii−11 xm1,j∈Slog2q

x m1,j | X m,i · qx m1,j | X m,i1−α

≤ lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m jNm1 Nm1 i−11 i xsN

m1,j s1

 2

α − 1

2

e−2

 N



2

α − 1

2

n → ∞

T n − 1

T n   Nα − 12 2e−2< ∞.

3.6

lim sup

n → ∞

1

T nm0n−1 N0i1 ···N m jNm1 Nm1 i−11 i −λlog q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

3.7

In the case of 0 < λ < α, by3.7 we have

lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11

−log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

3.8

2cN/1 − α on the interval 0, α That is

lim sup

n → ∞

1

T nm0 n−1N0i1 ···N m jNm1 Nm1ii−11−log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

1 − α

√

2cN, μ-a.s ω ∈ Dc.

3.9

c  0 According to the methods of proving 2.4, 3.1 follows from 1.5, 2.23, and 3.9

lim inf

n → ∞

1

T nm0n−1 N0i1 ···N m jNm1 Nm1 i−11 i −log q

X m1,j | X m,i − E Q

log q

X m1,j | X m,i | X m,i

1 − α

√

2cN, μ-a.s ω ∈ Dc.

3.10

Imitating the proof of2.5, 3.2 follows from 1.5, 1.10, 2.2, and 3.10

Corollary 3.2 see 9 Let X  {X t, t ∈ T} be the Markov chains field determined by the measure

μ Q on the generalized Bethe tree T · fn ω is defined as 1.6, and H m Q X m1,j | X m,i  is defined as

2.1 Then

lim

n → ∞

⎧

⎨

⎩f n ω − T1nm0 n−1N0···N m

i1

Nm1 i jNm1 i−11

H m Q

X m1,j | X m,i

⎫

⎬

⎭ 0 μ Q -a.s. 3.11

Proof By3.1 and 3.2 in Corollary3.1, we obtain that when c 0,

lim

n → ∞

⎧

⎨

1

T nm0 n−1N0···N m

i1

Nm1i

jNm1 i−11

H m Q

X m1,j | X m,i

⎫

⎬

⎭ 0, μ-a.s ω ∈ D0.

3.12

Corollary 3.3 Under the assumption of Corollary 3.1 , if Q , then

lim

n → ∞

⎧

⎨

1

T nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11

H m Q

X m1,j | X m,i

⎫

⎬

μP

x T n

 px 0,1n−1

m0

N0···N m i1

Nm1i

jNm1 i−11

p

fn ω  − T1n

⎡

⎣log pX 0,1 n−1

m0

N0···N m i1

Nm1i

jNm1 i−11

X m1,j | X m,i

⎤

We have the following conclusion

Corollary 3.4 Let X  {X t, t ∈ T} be a Markov chains field on the generalized Bethe tree T whose initial distribution and joint distribution with respect to the measure μ P and μ Q are defined by3.14,

3.15 and 1.4, 1.5, respectively f n ω is defined as 3.16 If



h∈S



l∈S

pl | h − ql | h

lim sup

n → ∞

⎡

⎣f n ω − T1nm0 n−1N0i1 ···N m jNm1 Nm1ii−11 H m Q

X m1,j | X m,i

⎤

⎦

√

2cN, μ P -a.s.

3.18

lim inf

n → ∞

⎡

⎣f n ω − T1nm0 n−1N0i1 ···N m jNm1 Nm1 i−11 i H m Q

X m1,j | X m,i

⎤

⎦

√

3.19

Proof Let μ  μ P in Corollary3.1, and by1.5, 3.15 we get 3.16 By the inequalities log x ≤

x − 1 x > 0, a ≤ a,3.17, and 1.10, we obtain

ϕ

μ P | μ Q

 lim sup

n → ∞

1

T n logμ P



X T n

μQ

X T n

 lim sup

n → ∞

1

T n logpX 0,1

n−1 m0 N i10···N m Nm1i jNm1 i−11 p

X m1,j | X m,i qX 0,1 n−1 m0 N0···N m

i1 Nm1i jNm1 i−11 q

X m1,j | X m,i

≤ lim sup

n → ∞

1

T n logpX 0,1

qX 0,1 lim supn → ∞

1

T nm0 n−1N0i1 ···N m Nm1 i

jNm1 i−11

X m1,j | X m,i

q

X m1,j | X m,i

≤ lim sup

n → ∞

1

T nm0n−1 N0i1 ···N m jNm1 Nm1 i−11 i h∈Sl∈S δ h X m,i δ l

X m1,j logpl | h

ql | h

≤ lim sup

n → ∞

1

T nm0n−1 N0···N m

i1

Nm1i

jNm1 i−11



h∈S



l∈S

δ h X m,i δ l

X m1,j

ql | h− 1

"

h∈S



l∈S

lim sup

n → ∞

T n − 1

T n pl | h − ql | h ql | h

h∈S



l∈S

pl | h − ql | h

3.20

ϕ

3.2

4 Some Limit Properties for Expectation of Random Conditional Entropy on the Finite State Space

Lemma 4.1 see 8 Let X T n

 {X t, t ∈ T n } be a Markov chains field defined on a Bethe tree

 {X t, t ∈ T n } then for all

k ∈ S,

lim

n

S n k, ω

where π  π1, , πN is the stationary distribution determined by Q.

Theorem 4.2 Let X T n

 {X t, t ∈ T n } be a Markov chains field defined on a Bethe tree T B,N , and let H m Q X m1,j | X m,i  be defined as above Then

lim

n

1

T n

⎡

⎣N1

i1

H0Q X 1,i | X 0,1 n−1

m1

N1Nm−1 i1

Ni



jNi−11

H m Q

X m1,j | X m,i

⎤

⎦

k∈S



l∈S πkql | k log ql | k, μQ -a.s.

4.2

Proof Noticing now N1 N  1, for all n ≥ 2, N n  N, that therefore we have

N1

i1

H0Q X 1,i | X 0,1 n−1

m1

N1Nm−1 i1

Ni



jNi−11

H m Q

X m1,j | X m,i

N1

i1

− E Q

log qX 1,i | X 0,1  | X 0,1



n−1

m1

N1Nm−1 i1

Ni



jNi−11

− E Q

log q

X m1,j | X m,i | X m,i

 −N1

i1



x 1,i ∈S qx 1,i | X 0,1  log qx 1,i | X 0,1

−n−1

m1

N1Nm−1 i1

Ni



jNi−11



q

x m1,j | X m,i log q

x m1,j | X m,i