Linear Block Code ppsx

Today, we are going to talk about: The error detection and correction capability  Encoding and decoding  Hamming codes  Cyclic codes... Error control techniques Full-duplex connecti

Linear Block Codes

Last time we talked about:

 Evaluating the average probability of

symbol error for different bandpass

modulation schemes

 Comparing different modulation

schemes based on their error

performances.

Today, we are going to talk about:

 The error detection and correction capability

 Encoding and decoding

 Hamming codes

 Cyclic codes

Block diagram of a DCS

Format encodeSource

Format decodeSource

Channelencode

Pulsemodulate

Bandpassmodulate

Channeldecode

Demod SampleDetect

 Channel coding:

 Transforming signals to improve communications performance by increasing the robustness against channel impairments (noise, interference, fading, )

 Waveform coding: Transforming waveforms to

better waveforms

 Structured sequences: Transforming data

sequences into better sequences, having structured redundancy

 “Better” in the sense of making the decision process less subject

to errors.

What is channel coding?

Error control techniques

 Full-duplex connection, error detection codes

 The receiver sends a feedback to the transmitter, saying that if any error is detected in the received packet or not (Not-Acknowledgement (NACK) and Acknowledgement (ACK), respectively)

 The transmitter retransmits the previously sent packet if it receives NACK

 Simplex connection, error correction codes

 The receiver tries to correct some errors

 Full-duplex, error detection and correction codes

Why using error correction coding?

 Error performance vs bandwidth

 Power vs bandwidth

 Data rate vs bandwidth

 Capacity vs bandwidth

(dB) /N0

E b

B

P

A F B D

C

Coded

Coding gain:

For a given bit-error probability,

the reduction in the Eb/N0 that can be

realized through the use of code:

[dB]

[dB]

[dB]

c 0 u

E

Channel models

 Discrete input, discrete output

 Binary input, binary output

 Discrete input, continuous output

Linear block codes

are useful in understanding Linear block codes.

Some definitions

 The set {0,1}, under modulo 2 binary addition and

multiplication forms a field

 Binary field is also called Galois field, GF(2).1 1 0

1 0 1

1 1 0

0 0 0

0 0 1

0 1 0

0 0 0

Some definitions…

 Fields :

 Let F be a set of objects on which two operations

‘+’ and ‘.’ are defined

 F is said to be a field if and only if

1 F forms a commutative group under + operation The additive

identity element is labeled “0”.

2 F-{0} forms a commutative group under Operation The

multiplicative identity element is labeled “1”.

3 The operations “+” and “.” distribute:

F a

b b

a F

b

∀ ,

F a

b b

a F

b

∀ ,

)(

)(

)

Some definitions…

 Let V be a set of vectors and F a fields of

elements called scalars V forms a vector space

v V

v u

v v

⋅

a ) and ( )(

)(

,

v v

V

∀ , 1

Some definitions…

 Examples of vector spaces

 The set of binary n-tuples, denoted by

 A subset S of the vector space is called a

subspace if:

 The all-zero vector is in S.

 The sum of any two vectors in S is also in S.

 Example:

ofsubspacea

is )}

1111(

),1010(

),0101(

),0000

),1101(

),1100(

),1011(

),1010(

),1001(

),1000(

),0111(

),0101(

),0100(

),0011(

),0010(

),0001(

),0000

{(

4 =

V

Some definitions…

 A collection of vectors ,

the linear combinations of which include all vectors in

a vector space V, is said to be a spanning set for V

Linear block codes

 A set with cardinality is called a linear block code if, and only if, it is a subspace of the vector space

 Members of C are called code-words.

 The all-zero codeword is a codeword.

 Any linear combination of code-words is a codeword.

Linear block codes – cont’d

Linear block codes – cont’d

rateCode

bits

Redundant

n

k R

n-k

c =

Linear block codes – cont’d

w(U), is the number of non-zero elements in U.

and V, is the number of elements in which they differ

 The minimum distance of a block code is d ( U, V ) = w ( U ⊕ V )

)(

min)

,(

min

min i j d i j i w i

≠

Linear block codes – cont’d

defined as the maximum number of guaranteed correctable errors per codeword, is

d t

1

min −

= d e

Linear block codes – cont’d

the decoder commits an erroneous decoding is

 is the transition probability or bit error probability over channel

j n j

n

t j

n

t j

j

n j n

Linear block codes – cont’d

 Note that for coded systems, the coded bits are

modulated and transmitted over channel For

example, for M-PSK modulation on AWGN channels (M>2):

where is energy per coded bit, given by

R E M Q

M M

N

E M Q

M

log

2 sin

log

2 log

2

0

2 2

0

2 2

Linear block codes –cont’d

 A matrix G is constructed by taking as its rows the

vectors on the basis,

, , {V1 V2  Vk

k

n n

k

v v

v

v v

v

v v

2 22

21

1 12

11 1

V V G

Linear block codes – cont’d

 The rows of G, are linearly independent

mG

k n

k

k n

m m

m u

u u

m m

m u

u u

V V

V

V

V V

⋅ +

+

⋅ +

2 1

1 2

1

2 1

2 1 2

1

) , , , (

) ,

, ,

( ) , , , (

Linear block codes – cont’d

0 0 1

0 0 0

1 1 1

0 0 1

1 1 0 1

3 2 1

V V

V G

1

1 1

1 1

0 0

0 0

1 0

1 1

1 1

1 1 0

1 1

0 0 0

1 1 0

1 1 1

1 1 0

0 0 1

1 1

1

0 0 0

1 1 0 1

0 0

0 1 0

0 0 1

0 1 0

0 1 1

0 0 1

0 0

0

0 1 0

0 0 1 0 Message vector Codeword

Linear block codes – cont’d

 For a systematic code, the first (or last) k elements in the codeword are information bits

matrix

) (

matrix identity

] [

k n

k

k k

k k

I P

G

) , ,

, ,

, , ,

( )

, , ,

(

bits message

2 1

bits parity

2 1

Linear block codes – cont’d

, which its rows are orthogonal to rows of :

are linearly independent.

n k

Linear block codes – cont’d

error

) , , ,

(

or vector codeword

received

) , , ,

(

2 1

2 1

n

n

e e

e

r r

Linear block codes – cont’d

1 For row , find a vector in of minimum

weight which is not already listed in the array.

2 Call this pattern and form the row as the

corresponding coset

k k

n k

n k

n

k k

2 2

2 2

2

2 2

2 2

2

2 2

1

U e

U e

e

U e

U e

e

U U

i = 2,3, ,2 − V n

i

Linear block codes – cont’d

1. Calculate

2. Find the coset leader, , corresponding to

3. Calculate and corresponding

) ˆ ˆ

( ˆ

Linear block codes – cont’d

010110 100101

010001

010100 100000

100100 010000

111100 001000

000110 110111

011010 101101

101010 011100

110011 000100

000101 110001

011111 101011

101100 011000

110111 000010

000110 110010

011100 101000

101111 011011

110101 000001

000111 110011

011101 101001

101110 011010

110100 000000

Linear block codes – cont’d

111 010001

100 100000

010 010000

001 001000

110 000100

011 000010

101 000001

000 000000

(101110) (100000)

(001110) ˆ

ˆ

estimated is

vector corrected

The

(100000) ˆ

is syndrome this

to ing correspond pattern

Error

(100) (001110)

: computed is

of syndrome The

received.

is (001110)

ted.

transmit (101110)

= +

= +

H rH

S

r r

U

T T

Error pattern Syndrome

 The columns of the parity-check matrix, H, consist of all

non-zero binary m-tuples.

m k

n

m m

1

: capability correction

Error

:

bits parity of

Number

1 2

: bits

n informatio of

Number

1 2

:

length Code

Hamming codes

]

[ 1

0 1

1 1

0 0

1 1

0 1

0 1

0

1 1

1 0

0 0

1

3 3

T

P I

1 0

0 0

1 1

1

0 1

0 0

0 1

1

0 0

1 0

1 0

1

0 0

0 1

1 1

Cyclic block codes

performed using feedback shift-registers.

 Hence, relatively long block codes can be

implemented with a reasonable complexity

Cyclic block codes

cyclic shifts of a codeword are also a codeword.

 Example:

) , ,

, ,

, , ,

, (

) , ,

, ,

(

1 2

1 0

1 1

) (

1 2

1 0

−

−

− +

i n i

n i

n

u u

u u

u u

u

u u

U U

) 1011 (

) 0111 (

) 1110 (

) 1101

(

) 4 ( )

3 ( )

2 ( )

1 (

Cyclic block codes

codewords in polynomial form

 Hence:

) 1 (

degree

) ( X = u0 + u1X + u2X 2 + + un−1X n−1 n-

U

) 1 (

) (

, )

(

1

) 1 (

) 1 (

1 1

) (

1 2

2 1 0

1

1

1 2

2 1 0

1 )

1 (

+ +

=

+ +

+ + +

+

=

+ +

X u

n

n n

X

n n

n

n n

n n

X u

X

u X

u X

u X

u X u u

X u

X u

X u X u X

X

n n

modulo )

( )

modulo )

( )

(

) 1

U

Cyclic block codes

 Let C be a binary (n,k) linear cyclic code

1 Within the set of code polynomials in C, there is a unique

monic polynomial with minimal degree is called the generator polynomials.

2 Every code polynomial in C, can be expressed uniquely

g g

Cyclic block codes

4. The orthogonality of G and H in polynomial form is

expressed as This means is also a factor of

5 The row , of generator matrix is formed by the

coefficients of the cyclic shift of the generator polynomial.

r r

k

g g

g

g g

g

g g

g

g g

g

X X

X X X

1 0

1 0

1 0

1 ( )

) (

) (

1)

()

)

(X

h

k i

i, =1, ,

"

1

"i −

Cyclic block codes

Cyclic code:

1. Multiply the message polynomial by

2. Divide the result of Step 1 by the generator

polynomial Let be the reminder

3. Add to to form the codeword

Cyclic block codes

generator polynomial

1 Find the codeword for the message

) 1 1 0 1 0 0 1 (

1 ) ( )

( )

(

: polynomial codeword

the Form

1 )

1 ( 1

(

: (

by ) ( Divide

) 1

( )

( )

(

1 ) ( )

1011 (

3

, 4

, 7

bits message bits

parity

6 5

3 3

) ( remainder generator

3 quotient

3 2

6 5

3

6 5

3 3

2 3

3

3 2

+

= +

=

+ +

+ +

+ +

= +

+

+ +

= +

+

=

=

+ +

U

g m

m m

m m

p g

q

X X

X X

X X

X

X X X

X X X

X X

X) X

X

X X

X X

X X

X X

X X

X X

X

k n k

n

X (X)

(X)

k n

k n

)1011(

=

m

3

1)(X = + X + X

g

Cyclic block codes

2. Find the generator and parity check matrices, G and H,

⋅ +

⋅ +

=

1 0 1 1 0 0 0

0 1 0 1 1 0 0

0 0 1 0 1 1 0

0 0 0 1 0 1 1

) 1101 (

) , , , ( 1

0 1

1 )

row(2) row(1)

row(3) row(3)

row(1)

→ +

+

→ +

0 1 0 0 1 1 1

0 0 1 0 1 1 0

0 0 0 1 0 1 1

0 1 1 1 0 1 0

1 1 0 1 0 0 1

Cyclic block codes

 Received codeword in polynomial form is given by

 The syndrome is the reminder obtained by dividing the

received polynomial by the generator polynomial

 With syndrome and Standard array, error is estimated

 In Cyclic codes, the size of standard array is considerably reduced

)()

()

Received codeword

Error pattern

)()

()()

Example of the block codes

Linear Block Codes

Basic Definitions

v be the corresponding n-bit codeword.

A total of 2k n-bit codewords constitute a (n,k) code.

codeword.

codeword in every linear block code.

, 1 0

, 1

1 , 0 01

00

n k k

k

n

g g

g

g g

0

g g G

corresponding to the information sequences that have a single nonzero element Clearly this set in linearly independent.

n-k

check bits

k

information bits

Generator Matrix (cont’d)

 EX: The generating set for the (7,4) code:

n k

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1 1 0 1

1 1 1

1 1 0

0 1 1

) (

Parity-Check Matrix

 For G = [ P | Ik ], define the matrix H = [In-k | PT]

 (The size of H is (n-k)xn).

 It follows that GHT = 0.

 Since v = u•G, then v•HT = u•GHT = 0.

 The parity check matrix of code C is the

generator matrix of another code Cd, called the dual of C.

0 1

0 0

1

H

Encoding Using H Matrix

(Parity Check Equations)

7 6

5 3

6 5

4 2

7 6

4 1

7 6

5

3

6 5

4

2

7 6

4

1

7 6

5 4

3 2

1

0 0 0

1 0

=v v

v v

=v v

v v

=v

v

v

v +v

v

v v

+v

v

v v

+v

v

v v

v v

v v

= +

Encoding Circuit

Minimum Distance

 DF: The Hamming weight of a codeword v , denoted

by w(v), is the number of nonzero elements in the

codeword.

 DF: The minimum weight of a code, wmin, is the

smallest weight of the nonzero codewords in the

code wmin = min {w(v): v ∈ C; v ≠0}.

 DF: Hamming distance between v and w, denoted by d(v,w), is the number of locations where they differ.

Minimum Distance (cont’d)

there exists l columns of H such that the vector

sum of these columns is zero Conversely, if there

exist l columns of H whose vector sum is zero,

there exists a codeword of weight l.

 COL 3.2.2 The dmin of C is equal

to the minimum numbers of

columns in H that sum to zero

H

Decoding Linear Codes

r = v + e

e ≡ error pattern = e1e2 en, where

The weight of e determines the number of

errors.

and error correction.

e i =  0 otherwise 1 if the error has occured in the locationi th

+

v

e

r

Error Detection

s = rHT = (s0, s1, …, sk-1)

then s = 0 as well, and the error is undetectable.

10 1 1

0 1

10 0 1

0 1

1

r r r r r r

2 1

0 s s

s

6 5

4 2

2

5 4

3 1

1

6 5

3 0

0

r r

r r

= s

r r

r r

= s

r r

r r

=

s

+ +

++ + +

Syndrome Circuit for 7,4) Hamming Code

Error Correction

 s = rHT = (v + e) HT = vHT + eHT = eHT

 The syndrome depends only on the error pattern.

 Can we use the syndrome to find e, hence do the correction?

 Syndrome digits are linear combination of error digits They provide information about error

location.

 Unfortunately, for n-k equations and n unknowns

there are 2k solutions Which one to use?

above equations, some of them are

Standard Array Decoding

v1, v2, …, v2k

 Partition the 2n words into 2k disjoint subsets D1,

D2,…, D2k such that the words in subset Di are

Standard Array Construction

1 List the 2k codewords in a row, starting with the all-zero codeword v 1 .

2 Select an error pattern e 2 and place it below v 1 This error pattern will be

a correctable error pattern, therefore it should be selected such that:

(i) it has the smallest weight possible (most probable error)

(ii) it has not appeared before in the array.

3 Add e 2 to each codeword and place the sum below that codeword.

4 Repeat Steps 2 and 3 until all the possible error patterns have been

accounted for There will always be 2n / 2k = 2 n-k rows in the array Each

row is called a coset The leading error pattern is the coset leader.

 Note that choosing any element in the coset as coset leader does not

change the elements in the coset; it simply permutes them.

Standard Array

 TH 3.3

No two n-tuples in the same row are identical

Every n-tuple appears in one and only one row.

k k

n k

n k

n k

n

k k k

2 2

3 2

2 2

2

2 3

3 3

2 3

3

2 2

3 2

2 2

2

2 3

2 1

v e

v e

v e

e

v e

v + e v

+ e e

v e

v + e v

+ e e

v v

v 0

v

-

+ +

Standard Array Decoding is Minimum

Distance Decoding

 Then r = el + vi

 d(r,vi) = w(r + vi) = w(el + vi + vi) = w(el)

 d(r,vj) = w(r + vj) = w(el + vi + vj) = w(el + vs)

 As el and el + vs are in the same coset, and el is

selected to be the minimum weight that did not

appear before, then

w(el) ≤ w(el + vs)

 Therefore d(r,vi) ≤ d(r,vj)

Standard Array Decoding (cont’d)

 TH 3.4

Every (n,k) linear code is capable of correcting exactly

2n-k error patterns, including the all-zero error

pattern.

 EX: The (7,4) Hamming code

# of correctable error patterns = 2 3 = 8

# of single-error patterns = 7

Therefore, all error patterns, and only error patterns can be corrected (Recall the Hamming Bound, and the fact that Hamming codes are perfect

single-Standard Array Decoding (cont’d)

EX 3.6: The (6,3) code defined by the H matrix:

3

6 4

2

6 5

1

v

=v v

v

=v v

v

=v v

+ + +

Codewords000000

110001101010011011011100101101110110000111

3

Standard Array Decoding (cont’d)

The Syndrome

required by standard array decoding.

s = rHT = (v + e) HT = eHT

and not on the transmitted codeword.

The Syndrome (cont’d)

All the 2k n-tuples of a coset have the same

syndrome The syndromes of different cosets are different

(el + vi )HT = elHT (1st Part)

Let ej and el be leaders of two cosets, j<l Assume

they have the same syndrome.

ejHT = elHT (ej +el)HT = 0.

This implies ej +el = vi, or el = ej +vi

This means that el is in the jth coset Contradiction.