Lecture 4 DictionaryMethods

Lecture 4 DictionaryMethods

Dictionary based methods

• Statistical compression methods use a statistical model

of the data, and the quality of compression they achieve depends on how good that model is

• Dictionary-based method do not use statistical model

of data Instead they select strings of symbols and

encode each string as a token using a dictionary

• The dictionary holds strings of symbols and it may be static or dynamic (adaptive)

• To encode text one can use a (static) dictionary of the English (Finnish, etc.) language

Dictionary-based methods: Main idea

• Encoder: As the input is processed, develop a dictionary

and transmit the index of strings found in the dictionary

• Decoder: As the code is processed, reconstruct the

dictionary to invert the process of encoding

Dictionary-based methods: History

Sử dụng một con trỏ dịch chuyển trên xâu kí tự

đầu vào Ban đầu con trỏ trỏ vào vị trí 1

Qui ước: từ con trỏ trở về trước được gọi là quá

khứ, còn từ con trỏ trở về sau được gọi là tương lai Tránh trường hợp quá khứ dài vô tận người ta

đặt kích thước của cửa sổ quá khứ là w bằng số kí

tự dài nhất có thể trùng khớp với các kí tự trong tương lai

Output - thể hiện trong dạng (i, j) C:

 (i,j) thể hiện vị trí so khớp và độ dài xâu tương

ứng

 C kí tự rõ trong buffer

sir_sid_eastman_ea  (0,0,’s’)r’s’)) sir_sid_eastman_eas  (0,0,’s’)_’s’)) sir_sid_eastman_easi  (4,2,’s’)d’s’))

LZ77: Example 2

LZ77: Example 3

LZ77: Example 4

 Cặp (i,j) vị trí so khớp và độ dài xâu so khớp

 Kí tự vừa đọc được hoặc xâu so khớp nhưng có độ dài nhỏ hơn MIN_LENGTH

LZSS: Example 1

• This Dictionary starts (almost) empty

• The encoder add new entries to the Dictionary during

the message encoding

• The decoder decode codes using Dictionary and

add new entries to the Dictionary during decoding

15LZ78- Thuật toán nén

LZ78-Thuật toán giải nén

17LZ78-Example 1

LZW: Encoding algorithm

Initialize Dictionary with alphabet

STRING = get input character

WHILE there are still input characters DO

CHAR = get input character

IF STRING+CHAR is in the Dictionary THEN

STRING = STRING+CHAR ELSE

output the code for STRING add STRING+CHAR to Dictionary STRING=CHAR

ENDIF END of WHILE

output the code for STRING: 0

add STRING+CHAR=ab to Dictionary STRING=CHAR=b

ENDIF

IF STRING+CHAR=ab is in the Dictionary: No

THEN STRING = STRING+CHAR ELSE

output the code for STRING: 0 add STRING+CHAR=ab to Dictionary STRING=CHAR=b

ENDIF

output the code for STRING: 1

IF STRING+CHAR=ba is in the Dictionary: No

THEN STRING = STRING+CHAR ELSE

output the code for STRING: 1 add STRING+CHAR=ba to Dictionary

IF STRING+CHAR=ab is in the Dictionary: Yes

THEN STRING = STRING+CHAR=ab

ELSE output the code for STRING add STRING+CHAR to Dictionary STRING=CHAR

ENDIF

output the code for STRING: 2

IF STRING+CHAR=aba is in the Dictionary: No

THEN STRING = STRING+CHAR ELSE

output the code for STRING: 2 add STRING+CHAR=aba to Dictionary

IF STRING+CHAR=ab is in the Dictionary: Yes

THEN STRING = STRING+CHAR: ab

ELSE output the code for STRING add STRING+CHAR to Dictionary STRING=CHAR

ENDIF

IF STRING+CHAR=aba is in the Dictionary: Yes THEN

STRING = STRING+CHAR: aba

ELSE output the code for STRING add STRING+CHAR to Dictionary

output the code for STRING: 4

add STRING+CHAR=abab to Dictionary STRING=CHAR= b

ENDIF

IF STRING+CHAR=abab is in the Dictionary: No THEN

STRING = STRING+CHAR ELSE

output the code for STRING: 4 add STRING+CHAR=abab to Dictionary STRING=CHAR= b

ENDIF

IF STRING+CHAR=ba is in the Dictionary: Yes THEN

STRING = STRING+CHAR: ba

ELSE output the code for STRING add STRING+CHAR to Dictionary

LZW: Decoding

Initialize Dictionary

Input code c

Decode code c (index) to w

Output decoded string w

Put w? in Dictionary

REPEAT

a) Input code c

Decode the 1st symbol s1 of the code c

Complete the previous Dictionary entry with s1

b) Finish decoding the remainder of the code c

Output decoded string w

Input code c=0

Decode code c to w=a

Output decoded string w=a

Put w?=a? in Dictionary

a) Input code c=1

Decode the 1st symbol s1=b of the code c=1

Complete the previous Dictionary entry with s1=b

Put put w?=b? in Dictionary

b) Finish decoding the remainder of the code c=1

Output decoded string w=b

Put put w?=b? in Dictionary

a) Input code c=2

Decode the 1st symbol s1=a of the code c=2

Complete the previous Dictionary entry with s1=a

Put put w?=ab? in Dictionary

a) Finish decoding the remainder of the code c=2

Output decoded string w=ab

Put put w?=ab? in Dictionary

 Decode the 1st symbol s 1 =a of the code c=4

Complete the previous Dictionary entry with s1=a

a) Input code c=4

 Decode the 1st symbol s 1 =a of the code c=4

Complete the previous Dictionary entry with s1=a

LZW: Example (4a) step 2

 Complete the previous Dictionary entry with s 1 =a

a) Input code c=4

Decode the 1st symbol s1=a of the code c=4

 Complete the previous Dictionary entry with s 1 =a

Put put w?=aba? in Dictionary

b) Finish decoding the remainder of the code c=4

Output decoded string w=aba

Put put w?=aba? in Dictionary

a) Input code c=3

Decode the 1st symbol s1=b of the code c=3

Complete the previous Dictionary entry with s1=b

Put put w?=ba? in Dictionary

b) Finish decoding the remainder of the code c=3

Output decoded string w=ba

Put put w?=ba? in Dictionary

LZW: Example (6a) step 1

 Decode the 1st symbol s 1 =b of the code c=6

Complete the previous Dictionary entry with s =b

 Complete the previous Dictionary entry with s 1 =b

a) Input code c=6

Decode the 1st symbol s1=b of the code c=6

 Complete the previous Dictionary entry with s 1 =b

a) Finish decoding the remainder of the code c=6

Output decoded string w=bab