báo cáo hóa học:" Research Article Random Bit Flipping and EXIT Charts for Nonuniform Binary Sources and Joint Source-Channel Turbo Systems" docx

This short contribution clarifies very concisely several issues that arise with EXIT charts and nonuniform binary sources a nonuniform binary source can be the result of a nonbinary sour

Volume 2009, Article ID 354107, 6 pages

doi:10.1155/2009/354107

Research Article

Random Bit Flipping and EXIT Charts for Nonuniform Binary Sources and Joint Source-Channel Turbo Systems

Xavier Jaspar1and Luc Vandendorpe2

1 Laborelec, Rodestraat 125, B-1630 Linkebeek, Belgium

2 Communications and Remote Sensing Laboratory, Universit´e catholique de Louvain, Place du Levant 2,

B-1348 Louvain-la-Neuve, Belgium

Correspondence should be addressed to Xavier Jaspar,xavier.jaspar@uclouvain.be

Received 29 January 2009; Revised 7 June 2009; Accepted 7 August 2009

Recommended by Athanasios Rontogiannis

Joint source-channel turbo techniques have recently been explored a lot in literature as one promising possibility to lower the end-to-end distortion, with fixed length codes, variable length codes, and (quasi) arithmetic codes Still, many issues remain to

be clarified before production use This short contribution clarifies very concisely several issues that arise with EXIT charts and nonuniform binary sources (a nonuniform binary source can be the result of a nonbinary source followed by a binary source code)

We propose two histogram-based methods to estimate the charts and discuss their equivalence The first one is a mathematical generalization of the original EXIT charts to nonuniform bits The second one uses a random bit flipping to make the bits virtually uniform and has two interesting advantages: (1) it handles straightforwardly outer codes with an entropy varying with the bit position, and (2) it provides a chart for the inner code that is independent of the outer code

Copyright © 2009 X Jaspar and L Vandendorpe This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited

1 Introduction

Consider the generic system inFigure 1 It involves a serial

concatenation at the transmitter and a joint source-channel

serial turbo decoder [1,2] at the receiver and is sufficiently

general to describe several issues that arise with EXIT charts

[3] when the bitsU:are not uniform

Let us consider that the outer component inFigure 1is

a discrete source of symbols followed by a source code that

produces a sequence ofNbiased or nonuniform bits U1:N or

U: After random interleaving by Π, the inner component

is a channel code that produces a sequence of coded bits

R: which are sent across the channel We assume that the

channel code is linear and the channel is binary, symmetric,

memoryless, and time invariant At the receiver, an iterative

decoder is used, based on two decoders, one for each code

They exchange log-likelihood ratios (LLRs) iteratively, in a

typical joint source-channel serial configuration [1,2] In the

following, letL s O,:be the output LLRs of the (outer) source

decoder, let L c

O,: be the deinterleaved output LLRs of the

(inner) channel decoder, and letL c

I,: = L s O,:andL s

I,: = L c O,:

be the corresponding input LLRs

To assess the convergence of this iterative decoder, the EXIT charts introduced in [3] can be used when the bits are uniform Unfortunately, when the bits are not uniform,

a naive application of the EXIT charts that would neglect the bias might lead to inaccuracy issues A few contributions already paid attention to some of these issues, notably [2,4,

5] This paper attempts to clarify them all

This short paper presents concisely two simple histogram-based techniques in Section 2 to estimate the EXIT charts when the bits are not uniform and discusses

inSection 3the (dis)advantages of each The first technique

is based on the system in Figure 1 and is a generalization

of [5]; the second one is based on the system in Figure 2 where a random bit flipping is introduced These techniques are shown to lead to different but equivalent charts under some assumptions At last, we show that the well-known technique in [6], though historically developed for uniform bits, provides “correct” EXIT charts with nonuniform bits, which is an interesting conclusion for readers familiar with this technique Please note that, this paper relates only to histogram-based computations of EXIT charts but

coder

U: Π Channelcoder R:

Channel Channel

decoder

L s I,: = L c O,:

L s O,: = L c I,:

Π−1 Π

Y:

Source

decoder

Figure 1: Generic joint source-channel turbo system

Source

coder

U: U:

Π Channelcoder R



:

Channel Channel

decoder

Π−1

L c O,:

L s O,:

Π

Y:

Source

decoder

F:

Flipped source (de)coder

Figure 2: Equivalent system with a random bit flipping,

U k,U k ,F k ∈ {+1,−1} As inFigure 1, we defineL s I,: = L c O,: and

L s O,: = L c I,:

the presented ideas and concepts are compatible with the

analytical computation proposed in [7]

In the remainder, random variables are written with

capital letters and realizations with small letters.P(z) is the

abbreviation of the probabilityP(Z = z) The subsequence

(Z m,Z m+1, , Z n) is written Z m:n or Z: when m, n can be

omitted.I{ a }is the indicator function, that is,I{ a }equals 1

ifa is true, 0 otherwise E { Z }is the expectation ofZ I(Y ; Z)

is the mutual information betweenY and Z H(Z) = I(Z; Z)

is the entropy ofZ H b(p) is the binary entropy function, that

is,H b(p) = − p log2(p) −(1− p)log2(1− p).

2 Computation of the EXIT Charts

The main results of this section are summarized inTable 1

For clarity, the first method of computation, which is based

on biased bits, is given the name BEXIT while the second one,

which is based on flipped bits, is called FEXIT For the sake

of conciseness, some familiarity with [3] is assumed

2.1 Assumptions, Notations, and Consistency We assume

that the channel code is linear and the channel is binary,

symmetric, memoryless, and time invariant Besides, we

assume that the channel and source decoders, taken apart,

are optimal Specifically, letRc

k = (Y:,L c I,1:k −1,L c I,k+1:N) and

Rs

k =(L s I,1:k −1,L s I,k+1:N), and assume that the elements inRc

k

and inRs are independent, then the output LLRs onU kof

the channel and source decoders inFigure 1are considered

to be, respectively,

L c O,k =log p

Rc

k | U k =+1

p

Rc

L s O,k =log p

Rs

k,U k =+1

p

Rs

k,U k = −1

=log p

Rs

k | U k =+1

p

Rs

k | U k = −1+L U,k,

(2)

where the source biasL U,kis defined as

L U,k logP(U k =+1)

P(U k = −1), with 0< P(U k =+1)< 1 (3)

Note that in the flipped case inSection 2.3,L  U,k = 0 Note also generalizing the results below toP(U k = +1) ∈ {0, 1}

(e.g., in the case of pilot bits) is straightforward by taking the limit of| L U,k |toward infinity where appropriate

To avoid any confusion, here are some further consider-ations Firstly, we prefer not to use the term “extrinsic” for the LLRsL c O,k andL s O,k because this term is used differently

by authors in literature Some authors consider theseL c O,k

andL s O,k in (1)-(2) as extrinsic; others considerL c O,k (with

Y k excluded in the case of a systematic channel code) and

L s O,k − L U,k as extrinsic Secondly, we consider a typical serial concatenation where only the (inner) channel decoder has access to the channel values It must therefore share this piece of information with the source decoder through

L c O,k This is whyL c

O,k in (1) depends on all channel values

Y: through Rc

k (even if the channel code is systematic) Similarly, only the (outer) source decoder “knows” the source

a priori probabilities This is why, to share it with the channel decoder, the source biasL U,kis included inL s

O,kin (2)

Definition 1 (P-consistency and L-consistency) L p is

posterior- consistent or P-consistent with U if

p(L P = l, U =+1)= e l p(L P = l, U = −1). (4)

L L is likelihood-consistent or L-consistent with U if p(L L = l | U =+1)= e l p(L L = l | U = −1). (5) Note that ifL Lis L-consistent, thenL L+L Uis P-consistent

in (1), is L-consistent with U k The output LLR of the source decoder, L s O,k in (2), is P-consistent with U k

Proof It follows from (1)-(2) and integration ofp(Rc

k | u k) andp(Rs

k | u k)

Note that when the symmetry condition p(L = − l | U =

+1) = p(L = l | U = −1) is satisfied, the consistency in [4,6] and the L-consistency (5) are equivalent

Table 1: Summary of the two Monte-Carlo methods.

BEXIT charts: with biased bits,Figure 1 FEXIT charts: with flipped bits,Figure 2

Generation of the

input LLRs

Given a value of I I, μ L = J L −1 U(I) and N L ∼

N (0, 2μ L),

For both channel and source codes, given

a value ofI I ,μ  L = J0−1(I ) and

N L  ∼ N (0, 2μ 

L),

whereL sis L-consistent andL cis P-consistent

Measurement of

the output

information for

consistent LLRs

If L s

O is P-consistent andL c

O is L-consistent, we have

IfL  Ois consistent, we have for both channel and source codes

(IV) I s

O = H U −E{log2(1 +e −UL s O)}, (VIII) I 

O =1−E{log2(1 +e −U  L 

O)}, (V) = H U −E{ H b(1/1 + e L s

(VI) I c

O = H U −E{log2(1 +e −U(L U+L c O))}, Under some ergodic assumption, these (VII) = H U −E{ H b(1/1 + e L U+L c O)} expectations can be approached by time

averages as in [6]

Under some ergodic assumption, these expecta-tions can be approached by time averages as in [6]

Note,L  U =0 andH U  =1

Link between

BEXIT charts and

FEXIT charts

LetI O,c I,s beI s = I c

O,I O,c I,sbeI I s = I O c,I O,s I,cbeI c = I s

O,I O,s I,cbeI I c = I O s, then:

(X)I O,c I,s ≈ J L U(J0−1(O,c I,s)) ←→ I O,c I,s ≈ J0(J L −1 U(O,c I,s)).

(XI)I O,s I,c = I O,s I,c −(1− H U) ←→ I O,s I,c = I O,s I,c+ (1− H U).

Let us assume thatL U = L U,kis independent ofk; that is,

the bitsU khave the same entropyH U = H U,k  H b(P(U k =

+1)) independently ofk Let us then measure the input and

output levels of information as

I = lim

N →+∞

1

N

N



k =1

I(U k;L k)∈[0,H U], (6)

where (I, L k) is (I c,L c I,k), (I s,L s I,k), (I O c,L c O,k), or (I O s,L s O,k)

As in [3], we will compute the charts I O c = T c(I c) and

I O s = T s(I I s) by feeding the decoders with input LLRs and

by measuring the level of output information

Remark 1 The channel (resp., source) decoder will be fed

with input LLRs that are independent, P-consistent (resp.,

L-consistent, in agreement withProposition 1), and Gaussian

and let it be equal to L U if L is L-consistent Then

P(U = u | L = l) = 1

1 +e − u(L  U+l) (7)

Proof It follows from (4)-(5) and from P(U = +1 | L =

l) + P(U = −1| L = l) =1

2.2 BEXIT Charts, Figure 1 : Biased Bits We can compute

the BEXIT charts of the system in Figure 1 by analytical

generalization of the original EXIT charts [3]

(1) Generating the input LLRs Let us consider a sequence of

bitsU:generated by the source coder and let us focus on one

of these bits, namely, U Given a value I I = I s, the input

LLRL s I onU is generated as in (I) in Table 1, whereN Lis

a centered Gaussian random variable of variance 2μ Land the invertible functionJ L U(·) is given by

J L U



μ

u ∈{+1,−1} P(U = u)

·

+∞

−∞

e −( ξ − μ u)2

/(4μ)



4πμ log2



1 +e − u(L U+ξ)

dξ.

(8)

This L s I in (I), Table 1, is L-consistent, in agreement with Remark 1

For the channel decoder, given a valueI I = I c, the input LLRL c onU is generated as in (II),Table 1 Compared to (I),Table 1, the constant termL U is necessary to makeL c P-consistent

(2) Measuring the output information Let us consider the

whole sequence of bits U: and the corresponding output LLRs L O,: For both decoders, we can measure the output information by (6) with

I

U k;L O,k



log2 1

P

U k | L O,k

 . (9)

This expression can be evaluated by approaching P(U k |

L O,k) with histogram measurements as in [3]

Assuming consistent LLRs makes things simpler We can indeed simplify (6) and (9) into (IV) and (VI) ,Table 1, by

Proposition 2and by (9) In addition, we can simplify (IV)

into (V) since

E

log2

1 +e − UL s

O



| L s O

u ∈{+1,−1}

P

U = u | L s O



log2

1 +e − uL s O



= H b

1



1 +e L s



= H b

1



1 +e − L s



, (10)

where P(U = u | L s O) is given in (7) Similarly, we can

simplify (VI) into (VII),Table 1 Note that (IV),Table 1, is

equivalent to [5, equation (4)] and is an extension of [6,

equation (4)]

2.3 FEXIT Charts, Figure 2 : Flipped Bits Let us now consider

the system inFigure 2 To make the bit stream uniform, we

have introduced a random bit flipping before the interleaver

Π, that is, U k  = U k F k for allk, where the F k ∈ {+1,−1}

are independent and uniformly distributed At the receiver,

the corresponding LLRs are flipped accordingly By linearity

of the channel code and symmetry of the channel, the

flipped system inFigure 2is equivalent to the original system

in Figure 1 Consequently, the EXIT charts of the flipped

system, namely, FEXIT charts, can be used to characterize the

original system For clarity, all symbols related to the flipped

system use a prime () notation

With FEXIT charts, we are interested in the exchange

of information aboutU: between the channel decoder and

the flipped source decoder in Figure 2 Since the bits U k 

are uniform, we can use the results obtained so far with

L  U =0 andH U  =1 (seeTable 1) This is equivalent to [3];

in particular, the functionJ0(·) = J L U =0(·) is related to the

functionJ( ·) in [3] withJ0(μ )= J(

2μ )

3 Transformations, Equivalence,

and Discussion

3.1 Transformations and Equivalence The BEXIT and

FEXIT charts are equivalent under the assumptions of

Section 2.1, up to the approximation (X), Table 1 Indeed,

under these assumptions, transformations to obtain the

FEXIT chart from the BEXIT chart, and vice versa, are given

in (X) and (XI) andTable 1 To prove (X) andTable 1, let

us considerL sgiven in (I),Table 1, in the biased case If we

apply the flippingF on L s, we getL  s = L s F = μ L UF + N L F =

μ L U +N L F, and it is self-evident that this L  sis equivalent

to (III),Table 1, ifμ L = μ  L, that is, ifJ L − U1(I s) = J −1(I  s),

which proves (XI),Table 1, for consistent Gaussian LLRs For

consistent non-Gaussian LLRs, we can invoke the empirical

robustness of EXIT charts with respect to the statistical

model of the LLRs and assume that (X),Table 1, is a sufficient

approximation, hence the approximation symbol “≈” To prove (XI),Table 1, we use the following equalities:

I s

O − H U = −Elog2

1 +e − UL s O



, by (IV), (11)

= −Elog2

1 +e − U  L  O s



= I O  s −1, by (VIII) (13) where (12) comes fromU  L  O s =(UF)(L s O F) = UL s O

3.2 Simulation Results To illustrate the equivalence, let us

compute the charts of the following (nonoptimized) system The outer component inFigure 1is a memoryless source of

3 symbols with probabilities 0.85, 0.14, and 0.01, transcoded, respectively, by the variable length codewords (+1), (−1,−1), (−1, +1,−1), leading to P(U = +1) = 0.741 and H U =

0.825 The channel code is a rate −(1/2) recursive systematic

convolutional code with forward generator 358(in octal) and feedback generator 238 The channel is an additive white Gaussian noise channel with binary phase-shift keying and

E b /N0=1.4 dB—E bis the energy per bit of entropy andN0/2

the double-sided noise spectral density Note that the source decoder is based on the BCJR algorithm [8] on Balakirsky’s bit-trellis [9]

The BEXIT and FEXIT charts of the system are given in Figures 3 and 4 The solid lines show the charts obtained with the methods described in Sections 2.2 and 2.3 The data points show the BEXIT and FEXIT charts obtained by applying the transformations (X)-(XI),Table 1, respectively,

on the FEXIT and BEXIT charts given in solid lines The good match between the data points and the solid lines illustrates the equivalence between FEXIT and BEXIT charts All system configurations we have tested confirm this equivalence, at least in the tested rangeP(U =+1)∈[0.05, 0.95].

Finally, when we neglect the bias and apply blindly the original method of [3]—generating the input LLRs with [3, equation (9)] and [3, equation (18)] and measuring the output information with [3, equation (19)]—, we obtain actually the FEXIT chart of the channel decoder for the channel decoder and the dashed line in Figure 4 for the source decoder As we can see, these charts intersect each other and give a prediction of convergence that is too pessimistic

3.3 Discussion In terms of (dis)advantages of each method,

FEXIT charts, unlike BEXIT charts, are limited to linear channel codes and symmetric channels Nevertheless, when the channel code is linear and the channel symmetric, FEXIT charts have at least two benefits Firstly, the FEXIT chart of the (inner) channel code is independent of the (outer) source code; so we do not need to recompute it when the source code changes By contrast, we need to recompute the BEXIT chart

of the channel code whenL U changes since it depends onL U

(see (II), (VI), and (VII) inTable 1) Secondly, FEXIT charts can handle very easily (outer) source codes with an entropy

H U,k that depends on k—recall that we have assumed that

L U,k, therefore H U,k, is independent ofk in Section 2.1—, simply because the random bit flipping makes the entropy

0 0.2 0.4 0.6 0.8 1

I c · · · I s O

0

0.2

0.4

0.6

0.8

1

c O ···

s I

BEXIT charts

FEXIT charts, transformed

Figure 3: BEXIT charts of the system described inSection 3.1

I I c · · · I O s

0

0.2

0.4

0.6

0.8

1

c O ···

s I

BEXIT charts

FEXIT charts, transformed

EXIT chart VLC, bias neglected

Figure 4: FEXIT charts of the system described inSection 3.1

equal to 1 for allk On the contrary, no method to handle

them with BEXIT charts is known to the authors Note that

a varying H U,k is not uncommon in practice: fixed length

codes, variable length codes with periodic bit-clock trellises,

mixture of such codes, and so forth

At last, among related contributions in literature, the well-known technique in [6] is of particular interest Though historically developed for uniform bits, this technique gives without bit flipping a correct prediction of convergence when the channel code is linear and the channel is symmetric It computes indeed [6, equation (5)] the output information

as (with some mathematical rewriting)

I O[6]=1−E

log2

1 +e − UL

Since U  L  = (UF)(LF) = UL, it is equivalent to (VIII),

Table 1, and thus to the FEXIT charts presented in this paper

4 Conclusion

Two methods have been presented to handle nonuniform bits in the computation of EXIT charts Though proved

to be equivalent for the prediction of convergence under certain assumptions, they have different pros and cons For example, the FEXIT method is restricted to linear inner codes and symmetric channels while the BEXIT method is not But the FEXIT method handles very easily a mixture

of bits having different entropies and offers a chart for the inner channel decoder (of a serial concatenation) that is independent of the outer source code, unlike the BEXIT method, which simplifies greatly subsequent optimizations

of the concatenated code In practice, both methods are therefore complementary and help to analyze joint source-channel turbo systems via EXIT charts

Acknowledgment

The authors greatly thank the reviewers for their constructive comments The work of X Jaspar is supported by the F.R.S.-FNRS, Belgium

References

[1] A Guyader, E Fabre, C Guillemot, and M Robert, “Joint source-channel turbo decoding of entropy-coded sources,”

IEEE Journal on Selected Areas in Communications, vol 19, no.

9, pp 1680–1696, 2001

[2] M Adrat and P Vary, “Iterative source-channel decoding:

Improved system design using EXIT charts,” EURASIP Journal

on Applied Signal Processing, vol 2005, no 6, pp 928–947, 2005.

[3] S ten Brink, “Convergence behavior of iteratively decoded

parallel concatenated codes,” IEEE Transactions on

Communi-cations, vol 49, no 10, pp 1727–1737, 2001.

[4] J Hagenauer, “The EXIT chart—introduction to extrinsic

information transfer in iterative processing,” in Proceedings of

the 12th European Signal Processing Conference (EUSIPCO ’04),

Vienna, Austria, September 2004

[5] N D¨utsch, “Code optimisation for lossless compression of

binary memoryless sources based on FEC codes,” European

Transactions on Telecommunications, vol 17, no 2, pp 219–229,

2006

[6] M T¨uchler and J Hagenauer, “EXIT charts of irregular codes,”

in Proceedings of the 36 th Annual Conference on Information

Sciences and Systems (CISS ’02), Princeton, NJ, USA, March

2002

[7] M Adrat, J Brauers, T Clevorn, and P Vary, “The

EXIT-characteristic of softbit-source decoders,” IEEE

Communica-tions Letters, vol 9, no 6, pp 540–542, 2005.

[8] L R Bahl, J Cocke, F Jelinek, and J Raviv, “Optimal decoding

of linear codes for minimizing symbol error rate,” IEEE

Transactions on Information Theory, vol 20, no 2, pp 284–287,

1974

[9] V B Balakirsky, “Joint source-channel coding with variable

length codes,” in Proceedings of the IEEE International

Sympo-sium on Information Theory (ISIT ’97), p 419, Ulm, Germany,

July 1997

on the FEXIT and BEXIT charts given in solid lines The good...

Table 1, and thus to the FEXIT charts presented in this paper

4 Conclusion

Two methods have been presented to handle nonuniform bits in the computation of EXIT charts. ..

3 Transformations, Equivalence,

and Discussion

3.1 Transformations and Equivalence The BEXIT and< /i>

FEXIT charts are equivalent