Lecture 3 Block

Lecture 3 Block

Data Compression

Lecture 3

Block Coding

Alexander Kolesnikov

Shortcoming of Huffman codes

Alphabet: a, b

pa=p=0.99, pb=q=0.01

1) Entropy

H1=-(p*log2(p)+q*log2(q))=0.081 bits/pel

2) Huffman code: pa=’0’, pb=’1’

Bitrate R1 = 1*p+1*q = p+q = 1 bit/pel!

Make a new alphabet blocking symbols!

Block coding: n=2

New alphabet: ’A’=’aa’, ’B’=’ab’, ’C’=’ba’, ’D’=’bb’

pA=p 2 =0.9801, pB=pq=0.0099, pC=pq=0.0099, pD= q 2 =0.0001

+ 0.0099*log2(0.0099) + 0.0001*log2(0.0001))=

=(0.0284+0.0659+0.0659+0.0013)/2= 0.081 bits/pel

Block coding: n=3

Huffman code:

Bitrate:

= 0.353 bits/pel

Block coding: n 

Bitrate for Hufman coder:

Problem - alphabet size and Huffman table size grows exponentially with number of symbols n blocked















N i

n n

N i

n n

n

n

B p B

p n

R B

p B

p

* 2

1 ) ( log ) (

1 )

( log

) (











N i

n n

N i

n n

n

n

B p B

p n

R B

p B

p

* 2

1 ) ( log ) (

1 )

( log

) ( 1

Block coding: Example 2, n=1

1) Entropy

2) Huffman code: a= ’0’; b=’1’

Block coding: Example 2, n=4

1) Entropy

H=-((12/16)*log2(12/16)+(4/16)*log2(4/16))/4=0.203 bits/pel

2) Huffman code: A=’0’, B=’1’

Bitrate R = (1*pA+1*pB)/4=0.250 bits/pel

Thực hiện mã khối với n=2;