Analysis, improvement and extensions of a lee metric list decoding algorithm for alternant codes

... Tal’s algorithm: A hard or a soft -decoding algorithm? Ido Tal as a Lee metric hard -decoding algorithm Soft -decoding: independent of the distance used Koetter and Vardy’s soft -decoding. .. (See Chapters and 5) Therefore, this first chapter contains a classical description of a communication system The concepts of Lee and Hamming distance, alternant codes and list decoding are properly... [14], a Hamming and Lee metric a- stage decoding algorithm for alternant codes over GR(pa , m) can be designed In conclusion in Chapter 9, we show that Ido Tal’s algorithm can be considered as a simplified

ANALYSIS, IMPROVEMENT AND EXTENSIONS OF A

LEE METRIC LIST DECODING ALGORITHM FOR

ALTERNANT CODES

Olivier de Taisne

A THESIS SUBMITTEDFOR THE DEGREE OF MASTER OF ENGINEERING

DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING

NATIONAL UNIVERSITY OF SINGAPORE

2004

A Bo, Seb, David, Phil, Hsieng, Robin, Yipin, Jerry, Joan, Junjun et tous les

autres qui ont rendu merveilleux mon s´ejour `a Singapour

“Dieu, tu es mon Dieu, je te cherche d`es l’aube, mon ˆame a soif de toi, apr`es toi,

languit ma chair, terre aride, alt´er´ee sans eau (Ps 63)”

I would like to thank Marc Andre Armand for his kindness, patience, optimismand trust and at the first place for having accepted me as his student I thank alsoIdo Tal for having kindly sent me his thesis and kindly answered to my questions.All my friends should also be thanked for their support, especially Hse Sieng forall her priceless advices, Seb for all the challenging discussions we had, Junjun forhaving corrected some mistakes in my thesis, David for his constant and generoushelp

In 1999, Guruswami and Sudan revolutionized the decoding of Hocquenghem(BCH) and Reed-Solomon codes as for the first time, decoding abovehalf the minimum distance of the code has been made possible in polynomialtimes and at all code rates Their algorithm involves a hard-decision list decodingapproach in the Hamming metric, for which the output of the decoder is no longer asingleton but a set of codewords This metric is the most commonly used measure

Bose-Chaudhuri-of the distance between codewords in the theory Bose-Chaudhuri-of error correcting codes A wellknown alternative distance measure is the Lee metric The Guruswami-Sudan(GS) algorithm was recently adapted by Ido Tal when the Lee distance measure isused However, various questions concerning this algorithm remained unanswered

We have thus studied this algorithm and proposed an easy way of evaluating itsperformance We have also :

(i) Improved this algorithm for some applications (specifically, for large codesand for Reed-Solomon codes over fields of characteristic 2),

(ii)Extended it to operate over a Galois ring through multistage decoding, anapproach we also apply to the Hamming metric GS decoder

(iii)Compared the performances between Lee and Hamming metric decodersunder Phase Shift Keying(PSK) and Quadrature Amplitude Modulation(QAM)modulations

Further, some links between this algorithm and a soft-decision decoder proposed

by Koetter and Vardy have been identified leading to a new interpretation of ourLee metric decoder, as a simplified soft-decision decoder

LIST OF SYMBOLS

[a, b] The ensemble of all integers included between a and b

p A prime number

q The power of a prime number

k The dimension of the code

n The length of the code

F A field of finite cardinality

R A ring of finite cardinality

y The received signal at the output of the demodulater

c The sent codeword

f The uncoded information vector of length k

Ps The symbol error probability

Pw,n The word error probability of a word of length n

l The size of the list output of the list decoding algorithm

R[X] The ensemble of the polynomials over the ensemble R

Rk[X] The ensemble of the elements of R[X] of degree less than k

GF (q) The Galois field of cardinality q

GR(pa, m) The Galois ring of cardinality pa×m, whose residue field is GF (pm)

N o Noise density

xi The ith coordinate of the vector x

x(i) The ith element of the p-adic expansion of the vector x

wLee,F The Lee weight and distance extended over the field F

dLee,F The Lee distance extended over the field F

wLee,R The Lee weight extended over the ring R

dLee,R The Lee distance extended over the ring R

dLee,R The Lee distance extended over the ring R

wLee,p,i The ith p-adic Lee weight over a ring

wHamming,p,i The ith p-adic Hamming weight over a ring

wLee,p The p-adic Lee weight over a ring

wHamming,i The p-adic Hamming weight over a ring

wLee,p,i The ith p-adic Lee weight over a ring

wHamming,p,i The ith p-adic Hamming weight over a ring

φi The function modulo pi over a ring

ˆ

X An estimater of the random variable X

~vt The transposed vector of ~v

LIST OF TABLES5.1 Number of errors corrected by Ido Tal’s algorithm for a BCH codeover GF (4) of length 15 and various dimensions 457.1 wT For BCH codes over Z2l of length 1024, rate 1/2, designeddistance 116, and tGS = 58 737.2 wT For BCH codes over Z2l of length 1024, rate 1/4, designeddistance 214, and tGS = 112 747.3 wT For BCH codes over Z2l of length 1024, rate 3/4, designeddistance 53, and tGS = 25 74

LIST OF FIGURES

1.1 Basic elements of a digital communication system 7

1.2 A Lee distance of 2 11

4.1 Lee distance on an example 39

4.2 42

5.1 Comparison between different Lee and Hamming metric BCH de-coders (the code has length 15, dimension 4 and designed distance 5) 47

5.2 Comparison between different Lee and Hamming metric BCH de-coders (the code has length 15, dimension 6 and designed distance 8) 48

5.3 Results for the decoding of a RS code of length 7 and dimension 2 over GF (8), with t=2 49

6.1 Example of the encoding and decoding processes 54

6.2 Word error probability for Ido Tal’s decoder using Ψ to decode a Reed-Solomon code of length 7 and dimension 2 over GF (8) 58

6.3 Word error probability for an Ido Tal’s decoder using Ψ to decode a Reed-Solomon of length 15 and dimension 4 over GF (16) 58

7.1 Comparison of the different algorithms to decode an alternant code over GR(4, 2) (subring of GR(4, 4)) of length 15, dimension 4 and designed distance 10) 67

7.2 Comparison of the different algorithms to decode an alternant code over GR(4, 2) (subring of GR(4, 4)) of length 15, dimension 8 and designed distance 6) 68

TABLE OF CONTENTS

1 Channel Model, Lee distance and definition of alternant codes 4

1.1 Model of the digital communication system 6

1.2 Presentation of the list decoding approach 7

1.3 Definition of a Galois field 8

1.4 Definition of a Galois ring 9

1.5 Representation of an element of a Galois ring 9

1.6 The Hamming distance 10

1.7 The Lee distance 11

1.7.1 The Lee distance over Zq 11

1.7.2 Over a Galois field F and a Galois ring R of cardinality q 12

1.8 Hamming and Lee metric p-adic weights 13

1.9 The t-Lee weight 14

1.10 Alternant codes defined over a field and a ring 14

1.11 Cyclic GRS codes: BCH and RS codes 15

1.12 Codewords, messages and some other definitions 16

2 Presentation of Guruswami-Sudan and Ido Tal’s algorithms 17 2.1 Some useful definitions 17

2.2 Presentation of Guruswami-Sudan (GS) algorithm 18

2.2.1 Guruswami-Sudan algorithm 18

2.2.2 Description of the GS algorithm 19

2.3 Presentation of Ido Tal’s algorithm 20

2.3.1 Ido Tal’s algorithm 20

2.3.2 Correctness of the algorithm 22

2.4 Complexity of Ido Tal’s algorithm 25

3 Improvement of Ido Tal’s algorithm for Reed-Solomon codes over fields of characteristic 2 27 3.1 Improvement of Ido Tal’s algorithm 27

3.2 The algorithm 31

3.3 Comparison of the performances of Ido Tal’s algorithm and of the improved algorithm 32

4 Computation of the word error probability for Lee decoder and its application to Ido Tal’s decoding of a PSK modulated signal and to ring decodings of a QAM modulated signal 34 4.1 Calculation of the word error probability 35

4.1.1 Computation of the word error probability in the case of a Hamming metric decoder 35

4.1.2 Computation of the word error probability in the case of a Lee metric decoder 36

4.1.3 Special case for which t <jq2k 37

4.2 Computation of the symbol error probability for a P SK modulated

signal 38

4.3 Calculation of symbol and word error probabilities in the case of the Lee distance over a ring for a QAM modulated signal 40

4.3.1 Calculation of the symbol error probability in the case of a QAM constellation 41

4.3.2 Calculation of the Lee metric word error probability for a QAM modulation 43

5 Results of Ido Tal’s algorithm 44 5.1 Correction capability of Ido Tal’s algorithm for some BCH codes 44

5.2 Lee metric BCH codes: Roth and Siegel’s Euclidian algorithm 45

5.3 Over a channel with a PSK modulation 47

5.3.1 Results for QPSK modulation 47

5.3.2 Analysis with t < q2 48

6 Use of a bijection between ((Zqa)n)m and ((Zqm)n)a to increase Ido Tal’s error correcting capability for large alphabets 50 6.1 Definition of the function Ψq,m→a 51

6.2 Some comments about the non-linearity of Ψ 53

6.3 Encoding and decoding of the message over GF (qa) but transmission over [0, qm− 1] 53

6.4 Results 55

6.4.1 Evaluation of the performance of our algorithm based on a statistical approach 55

6.4.2 Study of the fraction of errors corrected by our algorithm 55

6.4.3 Results obtained over an AWGN channel with PSK modulation 57 7 Presentation of a multi-stage list decoder over a ring 59 7.1 Ido Tal’s algorithm over a ring 59

7.1.1 Definition of the alternant code over a ring 59

7.1.2 The algorithm over a ring 60

7.2 Improved version of GS’s and Ido Tal’s algorithms over Galois rings with an a-stage decoder 61

7.2.1 The a stage decoder 62

7.2.2 Correctness and performance of the algorithm 63

7.3 Discussion about the selection of the message 64

7.4 Number of codewords corrected by fixed length multi-stage algorithm 65 7.4.1 In the case of the Lee distance 65

7.4.2 In the case of the Hamming distance 65

7.5 Analysis of the performance of the algorithm for the decoding of a QAM modulated signal 66

7.6 Presentation of the Hamming metric multistage algorithm with a decreasing input length 69

7.6.1 Presentation of the algorithm 69

7.6.2 Number of errors corrected 71

7.6.3 Results 72

8 A recursive approach to evaluate the number of sequences cor-rected by our algorithm 76 8.1 Evaluation of the number NLee,q,n(L) of words of length n of Lee weight equal to L over an alphabet of size q 77

8.2 Evaluation of the naive Lee metric decoder 79

8.3 Evaluation of the number of errors of Lee weight equal to N over the alphabet of size q which can be corrected by the Lee metric multi-stage algorithm 81

8.4 Evaluation of the number of errors of Hamming weight equal to N over the alphabet of size q which can be corrected by the multi-stage algorithm with fixed length 82

8.5 A statistical approach 83

9 Ido Tal’s algorithm: A hard or a soft-decoding algorithm? 86 9.1 Ido Tal as a Lee metric hard-decoding algorithm 86

9.2 Soft-decoding: independent of the distance used 87

9.3 Koetter and Vardy’s soft-decoding algorithm 87

9.4 Ido Tal: A Koetter-Vardy decoder over an AWGN PSK channel 88

10 Conclusion 90 10.1 Theoritical results about the Lee metric 90

10.1.1 Importation of some theoritical results of the well known Hamming metric coding theory 90

10.2 The different algorithms presented in the thesis 91

10.3 Conclusion 92

Bibliography 96 APPENDIX 90

ACKNOWLEDGEMENTS ii

ABSTRACT iii

LIST OF SYMBOLS iv

LIST OF TABLES vi

LIST OF FIGURES vii

Chapter 1

Channel Model, Lee distance and

definition of alternant codes

Introduction

The Lee distance is an alternative distance for error correcting codes introduced byLee in [1] Many error correcting codes have thus been designed with this metric(See for example [2], [3] and [4]) Since the discovery that the non-linear binaryKerdock, Preparata and Goethals codes (See [5]), which are known to containmore codewords than any other codes with the same minimal Hamming distance,can be considered as the binary images of Lee metric linear codes over the inte-ger residue ring Z4, the interest in the Lee distance has been growing From thesame time, list decoding, originally introduced by [6], received renewed interest,due to the discovery of a new list decoding algorithm by Guruswami and Sudan(See [7], [8], [9], [10], [11], [12]) This decoding algorithm enabled for the firsttime Reed-Solomon codes to correct beyond half the minimal distance at any rateand in polynomial time Originally developed as a Hamming metric hard-decisiondecoder, it has since been adapted to the Lee metric by Ido Tal ([4]) This disser-tation builds on Ido Tal’s decoder In Chapter 2, the algorithm is presented andits error correcting capability is demonstrated The algorithm is then improved todecode Reed-Solomon codes over field of characteristic 2 (Chapter 3) and to decode

codes over large alphabets (Chapter 6) It is difficult to make a fair performancecomparison between our improved version of Ido Tal’s decoder and other Lee met-ric decoders in the literature, as Ido Tal’s algorithm and the improved versions wepropose are efficient for low rate codes whereas the other algorithms perform wellfor high rates (See [2] and [3] and Chapter 5) We therefore choose to compare it

to the Hamming-metric based Guruswami-Sudan (GS) algorithm This has almostnever been done before, as the characteristics of Lee metric codes (e.g.: minimalLee distance, number of Lee errors that can be corrected) are specific to the case

of the Lee metric We study the performance of our Lee decoder on an AdditiveWhite Gaussian Noise (AWGN) channel with Phase Sifted Keying (PSK) modu-lation This is in line with the original definition of the Lee metric, which involvedPSK signalling (See [1]) Therefore, it seems natural to compare the performances

of Hamming and Lee metrics based decoding under PSK modulation The word(or bit) error probablity is one common criterion to evaluate the performance of acode under Hamming metric based decoding with regard to a particular modula-tion For the first time, the computation of the word error probability in the case ofLee metric based decoding is presented in Chapter 4 Chapter 5 is then devoted tothe performance comparison between the GS decoder and our lee metric decoder

In addition, we study Ido Tal’s algorithm over a Galois ring: building on the work

of Armand, who has proved in [9] that most of the results in [7] can be extendedover a finite commutative ring, we first extend Ido Tal’s algorithm to the ring caseusing a finite chain ring as in [13] and [14] One advantage of decoding over a ring

is that it decreases the probability of picking up the wrong codeword from the listgenerated by a list decoder as shown in [9] We also show in Chapter 7 that byusing the presence of zero divisors in the ring as in [15], and the p-adic expansion

of the elements of the ring as in [14], a Hamming and Lee metric a-stage decodingalgorithm for alternant codes over GR(pa, m) can be designed In conclusion inChapter 9, we show that Ido Tal’s algorithm can be considered as a simplified softdecoding algorithm by building on Koetter and Vardy’s work (See [16]).

Error correcting codes are one of the basic building blocks of a communicationsystem Their purpose is to help to transmit reliably a signal over a channel It

is interesting to study the performance of the Lee metric codes over a channelespecially because it gives a way to compare them to their Hamming counterparts(See Chapters 4 and 5) Therefore, this first chapter contains a classical description

of a communication system The concepts of Lee and Hamming distance, alternantcodes and list decoding are properly defined

The purpose of a digital communication system is to transmit an information signalthrough a noisy channel by minimizing the loss of information (See [17] and Fig.1.1) An information sequence of length k over an alphabet of size q is passed to

a channel encoder of code rate nk, which converts it into a sequence of length nover a second alphabet of size q0 called a codeword Each symbol of this sequence

is mapped by a digital modulator into a waveform s(t) and transmitted throughthe communication channel, which is the physical medium that is used to send thesignal from the transmitter to the receiver The signal is corrupted in a randommanner At the receiving end, the digital demodulator processes the channel-corrupted transmitted waveform and reduces it to a sequence of n symbols Thissequence is passed through the channel decoder, which attempts to reconstruct theoriginal information A measure of how well the demodulator and decoder perform

Figure 1.1: Basic elements of a digital communication system.

is the frequency with which errors occur in the decoded sequence We call thesymbol error probability Ps, the probability that there is a difference between thesymbol modulated by the transmitter and the symbol demodulated by the receiver

Pw,n, the word error probability, is the probability that the sequence at the input ofthe channel encoder is different from the one at the output of the channel decoder

of the receiver In this thesis, we take as channel decoders Ido Tal’s ([4]) andGuruswami-Sudan’s ([7]) algorithms, as modulators M -ary PSK (Chapter 5) and

M -ary QAM modulators (Chapter 9), with M denoting the number of points inthe constellation, and an additive white Gaussian noise of density N o is added tothe signal to modelize the noise

Traditionally, a t-error correcting code receives a corrupted vector y of length nand if the codeword is located at a distance d less than t of the received vector, itrecovers the sent codeword c from y Instead of a singleton, Elias has suggested in[6] that the algorithm provides a list of size less than l of possible codewords located

at a distance less than t Adopting this approach, the Gurusami-Sudan algorithm(See [7]) can correct more than the classical bound of half the minimum distance

of the code A step is added to the decoding process, which is to pick up the rightcodewords in the list output: several methods exist, such as to choose the closestone to the received vector with regards to a distance (Nearest Codeword Decoding)

or to choose the most probable codeword c by computing the probabilities of c sent

if y received (Maximum Likehood Decoding)

The definitions of a group, a ring and a field are given in the Appendix Our rithms operate over fields, usually noted F , or rings, noted R, of finite dimensions,which are Galois fields or rings, defined as follows The simplest field of finitedimension known is Zp when p is prime, where Zp stands for the ensemble of theinteger modulo p and F [X] for the polynomials over F To construct a field ofdimension pm, we use an irreducible primitive polynomial P of degree m over Zp

algo-By irreducible, we mean that it can not be expressed as product of polynomialsover F [X], and primitive that P has a root α of order pm − 1 The finite field isthen obtained by considering F [X] modulo P (X), which can be proven to be afinite field of cardinality pm As all the finite fields of characteristic pmare identical

up to isomorphism, we can adopt an unique notation GF (pm) to design any field

of cardinality (i.e the number of distinct elements) pm

1.4 Definition of a Galois ring

We build on the definition of the Galois field GF (pm) The Galois ring GR(pa, m)

is constructed such that an element of the ring modulo p is an element of theresidue field GF (pm) Therefore, a polynomial P irreducible both on the residuefield and on the ring is needed: it should be a monic (of leading coefficient equal tothe identity element of the ring) irreducible (primitive) polynomial P (X) over Zp a

with also P (X) modulo p irreducible (primitive) over Zp (P is said to be basic).Then, the ensemble Zpa[X] modulo P (X), with P a basic irreducible primitivepolynomial over Zpa is a ring of cardinality pa×m As all the rings with a residualfield of cardinality pm and cardinality pa×m are identical up to isomorphism, weadopt the notation GR(pa, m) for any of these rings An unit of R is an element,whose mapping over the residue field is not zero

A polynomial representation of an element of R = GR(pa, m) is derived as follows.Given ξ a primitive root of P (X) modulo p, each element y of R can be repre-sented as a polynomial of degree strictly less than m over Zpa in ξ:

Definition of the p-adic Lee and Hamming weights wLee,p of an element

Remark: Another proof is presented in [14] The two proofs are equivalent if for x

an element of the field, we define the element pix, as the element of the ring whichbelongs to piR/(pi+1R) and which is isomorphic to the element of the residue field

x We recall that piR contains the elements of R which taken modulo pi are equal

to zero and that by piR/(pi+1R), we denote that we take these elements of the ringmodulo pi+1

The definition of a distance is given in the Appendix The Hamming metric isdefined independently of the representation of the elements: the Hamming distancebetween two elements x, y is given by the number of coordinates by which the two

Figure 1.2: A Lee distance of 2vectors differ In what follows, we denote by xi the ith coordinate of a vector x.Formally, dHamming(x, y) = X

x i 6=yi

1

Originally in [1], the Lee distance has been defined over Zp, which explains whySigel and Roth in [2] have focused their analysis on BCH codes over Zp Followingthis way, in [3], Byrnes has extended directly the definition over Zpn to study thealternant codes over Zpn over a ring Ido Tal in [4] was the first one to define theLee metric properly over a Galois field by introducing a bijection between Zq and

GF (q) Over the many possible extensions of the Lee distance over the ring, wehave chosen the most relevant to our ring algorithm which will be developed inChapter 7

The first definition of the Lee distance in [1] is based on a circular representation

of the signal (See fig.1.2)as in a PSK modulated signal Given an element x of Zq,

we denote by ¯x the number of [0, q − 1] with x = ¯x modulo q The Lee weight wZq

Lee

of x is then defined as the minimum between ¯x and q − ¯x:

m−1

X

i=0

yiξiwith yi element of [0, p − 1] (respectively of [0, pa− 1]) and ξ a primitive element ofthe field GF (pm) (respectively of the ring) Therefore we define the absolute value

by wLee,F (respectively wLee,R) and dLee (respectively dLee,R) or even wLee and dLee

if it is clear in the context which distance is used The Lee weight wLee of y is thengiven by

wLee(y) = min(¯y, pm− ¯y)

The Lee weight of a vector y is the sum of the Lee weight of its coordinates:

With the representation of y in R derived from lemma 1, we define the Lee andHamming ith p-adic weight of y as:

wLee,p,i(y) = maxk∈[0,i](wLee,F(y(k)))and

wHamming,p,i(y) = maxk∈[0,i](wHamming,F(y(k)))and the Lee and Hamming p-adic weight as

wLee,p = max0≤i≤a−1(wLee,p,i(y0)))and

wHamming,p(y) = max0≤i≤a−1(wHamming,p,i(y(i)))

We stress that mathematically speaking, wLee,p does not define a weight as itdoes not satisfy the triangular property (for example, over the ring Z49, wLee,p(13) =

1, wLee,p(8) = 1 but wLee,p(21) = 3 > wLee,p(8) + wLee,p(13))

We also introduce φi the function mod pi:

φ1(y) = y(0)andφi(y) =

i−1

X

j=0

y(j)pj

As stated in [14], φi is a ring epimorphism:

For x, y in R, φi(x + y) = φi(x) +Rφi(y) and φi(x × y) = φi(xR) ×Rφi(y), with+R and ×R the addition and multipication laws over the ring

1.9 The t-Lee weight

For the sake of simplicity, we sometimes use the t-Lee weight, which is simplydefined for an element y as wLee(y) if wLee(y) ≤ t and else zero Mathematicallyspeaking, it does not define a proper distance but only a truncated one It maynevertheless be useful to evaluate the performances of our algorithm, when onlyerrors over the symbols of amplitude less than t are to be corrected

We keep the same definition as in [7] and [9] (the definition of the units of a ringand of the elements ξ are to be found in Chapter 7)

Let F be a field (and a ring R, respectively), ~v a vector with nonzero elementsover F (and units of R, respectively) and ~α a vector of distinct elements in F (andsuch that for j 6= i, αi − αj is never a zero divisor of R, respectively) We thendefine a Generalized Reed-Solomon (GRS) code as a mapping from Fk to Fn (andfrom Rk to Rn, respectively):

If K is a subfield of F (and a subring of R, respectively), then the subfield (andthe subring, respectively) subcode GRSn,k(~α, ~αa T

Kn is an alternant code

As sometimes codes are introduced with their parity check matrix or with

their generator matrix, we provide the canonical generator matrix of GRSn,k(~α, ~v)

obtained as the image by the code of the polynomial basis (1, X, , Xi, , Xk−1):

The main parameters of these codes are given by the following theorem([17])

Theorem 1 Given GRSn,k(~α, ~v) a [n,k,d] linear code, the alternant code GRSn,k(~α, ~v)T

Kn

is a [n, k0, d0] linear code, with k0 ≥ n − (n − k)m and d0 ≥ n − k + 1

More specifically, we are going to deal more with some cyclic GRS subcodes A

cyclic code is such that any cyclic shifted codeword of the code remains a codeword

Under certain conditions, a GRS code could be made cyclic: for example, if α is

a primitive nth root of unity in the field F , (~αa)i = αa×i for i in [0, n − 1], then,

GRSn,k(~α, αa) is cyclic: it is indeed the classical RS code, and is said to be primitive

if n = |F | − 1, and narrow sense, if ~vi = 1 Its generator polynomial is given by

Q t

j=1(X − αj+b), where t = n − k and the integer b is a fixed integer

For K a subfield of F , the cyclic alternant code GRSn,k(~α, αa)T

Kn defines aBose-Chaudhuri-Hocquenghem (BCH) code of designed distance t + 1, with t =

n − k Our simulations are done with BCH and RS codes

1.12 Codewords, messages and some other definitions

We fix here the terminologies The information polynomial f of degree strictly lessthan k is the message, which is encoded in a vector of length n over K, called thecodeword and noted cf Conversely fc stands for the polynomial such that cf isthe encoding of fc Sent through a channel, cf is corrupted and an error vector

e is added to it The received vector y can therefore be written as y = cf + e.The coordinates of y are the symbols when they are modulated The list decodingproblem as introduced by Elias in [6] for GRS and alternant codes is as follows.Given a vector y and the parameters of the code, F , K, ~v, ~α, and nerrors, find alist of polynomials f of size less than l, such that d(cf, y) ≤ nerrors, where d is adistance defined over F and nerrors is a parameter of the code The decoding isconsidered a success, if given a sent codeword cf and a received vector y = cf + e,

cf is included in the list output Guruswami-Sudan’s algorithm and Ido Tal’salgorithm solve this problem for the Hamming distance and the Lee distance.Now that we have set up definitions and notations, we can move forward tostudy these algorithms

f (xi) = yi in more than nHamming points, with nHamming < n Imposing certainconditions on a constructed bivariate polynomial Q(X, Y ), they make sure that if

f is a solution of this problem, Y − f (X) is a factor of Q(X, Y ) Guruswami andSudan originally proposed a Hamming metric GRS list decoder, but by modifyingthe constraints put on Q, this decoder can become a soft decoding RS code (see[16] and Chapter 10) or a Lee metric list decoder, as shown by Ido Tal in [4] Wepresent here the GS decoder and then Ido Tal’s version We give clear means ofevaluating Ido Tal’s performances, together with proofs of them, which were notprovided in Ido Tal’s thesis

Guruswami, Sudan and Ido Tal have introduced useful notations, which help toformalize the problem

A bivariate polynomial Q(X, Y ) has a singularity of order r in (αi, u) if all

coefficients of Q(X + αi, Y + u) of degree strictly less than r are equal to zero.Roughly speaking, a singularity of order r is a point where the curve given byQ(x, y) = 0 intersects itself r times.

The (a, b)-weighted degree of a bipolynomial Q(X, Y ) =

de-fb(X) being two polynomials of degree respectively a and b

The t-score of a vector u respectively to a vector y over Fnis defined as St,y(u) =

n

X

i=1

r(t − dLee(yi, ui)) = r(nt − dLee(y, u))

β such that, for all (αi,yi

v i), all coefficients of total degree less than r of Q(X +

αi, Y +yi

v i) are equal to zero

Step 2: Find all the polynomials f ∈ F(k−1)[X] with Y − f (X) a factor of Qand, if dHamming(cf, y) ≤ nHamming, include f in the list output

The reader can find in [7] the proof of the correctness of this algorithm We onlyintend to give here a short description, which should ease the understanding of IdoTal’s algorithm The algorithm has two steps: the first one is the computation

of a bivariate polynomial Q with singularities at chosen interpolation points Thesecond is the research of the factor of Q of the shape Y − f (X), which are thesolutions of the list decoding problem The crucial point in GS algorithm is tograsp how imposing Q to have singularities of certain order on certain points leads

to Y − f (X) being a factor of Q The next lemma (Lemma 4 in [7]) gives us theexplanation

Lemma 2 If Q is of singularity r at the point (αi,yi

v i) and f (αi) = yi

v i then (X −αi)rdivides g(X) = Q(X, f (X))

With this lemma, we see that if f (αi) = yi

v i in more than nHamming different points,

we then obtain different nHamming factors of g(X) of degree r, and as the αi aredistinct, we found one factor of g of degree nHammingr, more than the degree of

g, as the degree of g is upper bounded (see definition of the (1, k − 1)-weighteddegree) by β = rnHamming − 1 Consequently, Y − f (X) must be a factor ofQ(X, Y ) Thereafter, the reader may ask if such a bivariate polynomial with theseconstraints can be computed The answer is yes Guruswami and Sudan list nr(r+1)2equations imposed on the coefficients of Q and there are at least β(β+2)2(k−1) coefficients

As we have ensured nr(r+1)2 ≤ β(β+2)2(k−1), the solution exists and an optimum one isprovided by the values of the parameters r and β given.

Remark: Before going through Ido Tal’s algorithm, we would like to late GS algorithm differently to understand the role of the Hamming distance inthis algorithm As (αi,yi

formu-v i) corresponds indeed to the only point (αi, u) such that

i), Qhas a singularity of order r(t − dLee(u, yi)), with t being the maximum Lee weighterror over a symbol that our algorithm can correct And the degree of the factorderived from Lemma 1 is r(nt − dLee(u, yi))

All what follows is done in the case t =jq2kin which Ido Tal’s algorithm can correctall errors of Lee weight less than nLee But we can also choose t <jq2k Then, anerror over a symbol of amplitude more than t goes undetected Nevertheless, thereader may check that all the following proofs remain true with the t-truncateddefinition of the Lee distance (See in Chapter 1) and a choice of singularity replaced

by min(r(t − (wLee(u, yi)), 0) at all the points (αi,vu

i)

Algorithm 2

Step 0: Given l the maximum output list size, t, n and k (for an alternant code,

k = n − d, with d the designed distance of the code), compute parameters r, β, nLeeSet β = 0 and r = 0

of Q(X + αi, Y +vu

i) are equal to zero

Step 2: Find all the polynomials f ∈ Fk−1[X] such that Y − f (X) is a factor

of Q and, if dLee(cf, y) ≤ nLee, include f in the list output

Example 1 For example, we consider the Generalised Reed Solomon code over

GF (8) of length n = 7 and dimension k = 2, with ~α = [0, 1, 2, 3, 4, 5, 6] and

~v = [1, 3, 5, 1, 6, 5, 1] The parameters for the decoding are l = 10, t = 2 (we do notcorrect the symbol errors of Lee weight more than 2), β = 8, r = 1 and nLee = 6

We choose to send the message [1, 5] corresponding to the polynomial 1 + 5X Thecorresponding codeword is c = [1, 7, 0, 5, 1, 3, 2] We assume that the error vector

e = [7, 7, 1, 1, 2, 0, 0] of Lee weight 6 is added to c With the interpolation points asdefined in the algorithm, we find the bipolynomial Q given by

Q(X, Y ) = 7Y2+ 3Y3+ 6Y4+ 2Y5+ X(5Y + 5Y3+ Y5+ 4Y6) + X2(Y2+ 4Y3+ Y4)+X3(5 + 7Y + 7Y2+ 5Y3+ 4Y4) + X4(7 + 5Y + 3Y2+ 2Y3) + X5(2 + 4Y + 7Y2) + X6

Q has only two factors of the form Y − f (X): Y − 1 + 5X and Y − 2X, whoseimages by the code are at a Lee distance of y of 6 and 11 respectively So we select[1, 5]

The choice of the parameters of Ido Tal’s algorithm derives from the previousremark: we choose the singularity at a point (αi,vu

i) to be inversely linear with itsLee distance to the received letter yi By doing so, for a given polynomial f , thefactor derived from Lemma 1 of g(X, f (X)) has a degree also inversely linear toits Lee distance with y, which ensures that if the Lee distance between cf and y issmall enough, Y − f (X) is factor of Q(X, Y ) But let us demonstrate it formally.All what follows are new materials not presented in Ido Tal’s thesis, especially thecomputation of nLee We assume in the first part that a bivariate polynomial Qhas been found with a (1, k − 1) weighted degree β and we prove that for a givencodeword cf, if dLee(cf, y) < nLee, then Y − f (X) is a factor of Q (Lemma 2)

In the second part, we determine the parameters of Q (Lemma 3) The followinglemma is a mirror lemma of the Lemma 5 in [7]

Lemma 3 Given a codeword cf, if St,y(c) > β then Y − fc(X) is a factor ofQ(X, Y )

Proof On one hand, as Q is of singularity of order r(t − dLee(yi, ci)) at thepoints (αi,ci

v i) and fc(αi) = ci

v i, from Lemma 1, we derive that (X − αi)r(t−(d Lee (y i ,c i ))

divides Q(X, fc(X)) As the αi are distinct, these factors are prime two by twoand therefore, their multiple is a factor of Q(X, fc(X)) of degree St,y(u) as

n

X

i=1

r(t − dLee(yi, ui)) = r(nt − dLee(y, u)) = St,y(u)

On the other hand, as the degree of fc is less than k − 1, the degree of Q(X, fc(X))

is upper bounded by β Therefore, as St,y(c) > β, the degree of Q(X, fc(X)) isless than the degree of one of its factor, which leads to Q(X, fc(X)) = 0 and to

Assuming that Q can be computed, the next lemma gives the maximum errorvector Lee weight corrected by Ido Tal’s algorithm

Lemma 4 With nLee defined as follows, nLee = (nt −βr), if a codeword cf over K

is located at a Lee distance of y less than nLee, then its score St,y(c) is greater than

β and hence, Y − fc(X) is a factor of Q(X, Y )

Proof With the definition of St,y, it is easy to check that St,y(c) > β is equivalent

to dLee(cf, y) > nLee With Lemma 2, it is then easy to see that Y − fc(X) is factor

As in GS’s paper, we now compare the number of constraints with the number

of coefficients of Q(X, Y ) to choose adequately the parameters r and β

The Lee weight distribution of the elements of K is as follows: the Lee weights

of the elements belong to the interval [0, t], with t = jq2k being the maximimumLee weight One element has a Lee weight of zero (zero), two elements have a Leeweight of e for e in [1, t − 1], and one or two of Lee weight t, depending on wether

q is odd or even For each element u of K of Lee weight e and each αi, Q has

a singularity of order r(t − wLee(u)) in (αi,yi +u

equa-(β + 2)β2(k − 1) ≥ nNt(r) As seen in Lemma 3, the error capacity nLee decreases when β increases, so

we aim to minimize β This minimization leads to β(r) =jq1 + 2n(k − 1)Nt(r)k(i.e the integer part of the value of the bigger positive real root of the polynomial

β2+ 2β − 2n(k − 1)Nt(r)) To limit the list size at less than l elements, as the listsize is upper bounded by j(k−1)β(r) k, we add the condition l ≥j(k−1)β(r) k Now, how do

we choose r? nLee and j(k−1)β(r) k are both growing functions of r, so a greater value

of r increases both the code error capability and the output list size Therefore,given the parameter l, we choose the maximum r such that j(k−1)β(r) k≤ l, and then,β(r) and nLee(r) are computed The minimal list output size achievable with thisalgorithm is provided by the smallest value of r, r = 1 Nevertheless, the listoutput size is usually less than l

Remark 1 The simplification we did to Ido Tal’s algorithm has been to introducethe parameter t and to make the order of the singularity linear with t − dLee(yi, u)

We could also have introduced another parameter t0 such the order would havebeen of the form r(t − dLee(yi, u)) + t0 But it is obvious that this contributes to

an increase in the number of equations (Nt(r) ≤ Nt,t0(r)) independent of r, which

in turn decreases the error correcting capability of the code Therefore, we havefixed t0 = 0 An immediate consequence of this choice is that at the point located

at a Lee distance t from (αi, yi), Q has a singularity of order 0: we don’t have tointroduce the parameter λ as in Ido Tal’s thesis (whose value depends on whether

q is odd or even) and the number of equations is minimized

We construct on Mc Eliece’s work ([8]) to evaluate the complexity of Ido Tal’salgorithm In [8], the author explains that the interpolation step (the choice of theinterpolation points and the search for the bivariate polynomial Q) of the algorithmcan been computed in O(C2), (with f (t) = O(g(t)) means that there exist positiveconstants c and t0 such that f (t) ≤ c × g(t) for all t ≥ t0) by Kotter algorithm(See [8] pp.23-25), where C is the number of equations over the coefficients In thecase of Guruswami-Sudan algorithm, the complexity is therefore O(n2r4) In thecase of Ido Tal, we refer to Chapter 3, the complexity is O((St(r)n)2), and St(r) isequivalent to t33r2, so in total, the complexity is O(t6r94n2) As shown also in [8], thefactorization step can be solved with a complexity of O(L.log2(L)k(n + L.log(q))),with L being the maximum length of the output list and q the size of the alphabet

As seen in this chapter, constructing on Ido Tal’s work, we have been able

to introduce a Lee metric Guruswami-Sudan based algorithm to decode alternantcodes over Galois rings We have given the error correction capability in terms

of the number of Lee metric errors, which can be corrected The use of the Lee

metric creates some opportunities to improve the algorithm, in the case of fields

of characteristic 2 and over large alphabets as we will see in Chapers 3 and 6

Chapter 3

Improvement of Ido Tal’s algorithm for Reed-Solomon codes over fields of

characteristic 2

In Ido Tal’s algorithm, for each point (αi, yi), each Lee weight in [1,q2−1] results

in two additional interpolation points of Q In the case of Reed-Solomon codes(vi = 1) and fields of characteristic 2, we suggest here a method to reduce thisnumber to one For one of the two elements e of Lee weight in [1,q2− 1], we choosethe interpolation points to be (αi, (yi+ e)(u − e − yi)) rather than merely (αi, yi+ e)with a well chosen element u in F Therefore, if f is a solution, Y −(u−f (X))f (X)(instead of Y − f (X)) is a factor of the bivariate polynomial Q, which is now a(1, 2k −2)-weighted degree polynomial The advantage of these changes is to dividethe number of interpolation points and the list output size by two, the complexity

of the algorithm by 4 and to increase the correction error capability of the code insome cases

We first demonstrate a useful property of the Lee distance, which is used to reducethe complexity of Ido Tal’s algorithm

Proposition 1 Given a field F of cardinality q = pm, with p a prime number, we

denote u as the element u =

m−1

X

k=0

(p − 1)ξi, of Lee weight 1 Given any element

e of F , the Lee weight of u − e is equal to either wLee(e) + 1 (if ¯e ∈ [0,q−12 ]) or

wLee(e) − 1 (if ¯e ∈ [q2, q − 1])

Proof

1)If ¯e ∈ [0,q2 − 1], then wLee(e0 = u − e) = q − ¯e0 = ¯e + 1 = wLee(e) + 1

2)If ¯e ∈ [q2, q − 1], then wLee(e0 = u − e) = ¯e0 = q − 1 − ¯e = wLee(e) − 1 ‡

In this chapter, u denotes the element defined by Proposition 1

Example 2 For example, over Z7, u=6, the Lee weight of 2 is 2 and the Leeweight of 6-2=4 is 3 and conversely

We then classify the elements of the alphabet GF (q) in two ensembles: thosesuch that their Lee weight is equal to their absolute value, which are the elements ofabsolute value strictly less than 2q, and those such that their Lee weight is equal to

q minus their absolute value, which are the elements of absolute value greater than

q

2 Proposition 1 provides us a means to generate the elements of the second groupfrom the elements of the first We derive from it a way to define the interpolationpoints of Q justified by the next lemma

Lemma 5 Over F of characteristic 2 and given f a polynomial over F , if a variate polynomial Q(X, Y ) is of singularity r at the point (αi, (u + yi+ e)(yi+ e))then:

bi-1) If f (αi) = yi+ e then (X − αi)r divides Q(X, (u + f (X))f (X))

2) If f (αi) = yi+ u + e then (X − αi)r divides Q(X, (u + f (X))f (X))

Proof If we set f0(X) = (u + f (X))f (X), then it is clear that in both cases, as2u = 0 (F is a field of characteritic 2), f0(αi) = (u + yi+ e)(yi+ e) which leads to(X − αi)r dividing Q(X, f0(X)), by using Lemma 2 in Chapter 2 ‡

In Ido Tal’s algorithm, we had to put one constraint at every point f (αi) = yi+ efor e in F By using the previous lemma, we see that by putting one constraint

at the point (αi, (u + yi + e)(yi + e)), we cater to the two cases f (αi) = yi + eand f (αi) = yi + e + u In addition, by using Proposition 1, it is clear that if

we take for e all the elements of absolute values in [0, ,q2 − 1], u + e generatesall the elements of absolute values in [q2, q − 1] Therefore, we choose Q to be ofsingularity of order r(t − wLee(e)) at the point (αi, (u + yi+ e)(yi+ e)) with e suchthat its absolute value belongs to [0, ,q−12 ] With e0 = u + e, Q is then also byconstruction of singularity r(t + 1 − wLee(e0)) at the point (αi, (u + yi+ e0)(yi+ e0))(and of singularity r(t − wLee(e0)) which belong to the first group) We define themodified t-score of a vector x with respect to a vector y over Fn as

SM od,t,y(x) =

n

X

i=1

r(t − 1 − min(dLee(yi, xi), dLee(yi, u + xi)))

By using Lemmas 3 and 4 in Chapter 2, we have:

Lemma 6 If the t-score of a codeword cf with respect to the input y over K isgreater than β, then Y − fc(X) is a factor of Q(X, Y )

Now, we can study under what conditions we have SM od,t,y(x) > β

Lemma 7 If dLee(y, cf) < nt − n+ − β

r, with n+ being the number of “positive”coordinates of cf, then the score SM od,t,y(cf) of cf is strictly greater than β and

So if we note I+ the elements of F such that ¯e ∈ [0, q2 − 1] and I− = F − I+, thescore of a vector x can then be expressed as

equa-(β + 2)β4(k − 1) ≥ nNt(r)

As seen in Lemma 3, the error capacity nLee decreases when β increases, so weaim to minimize β This minimization leads to β(r) = jq1 + 4n(k − 1)Nt(r)k(i.e the integer part of the value of the bigger positive real root of the polynomial

β2+ 2β − 4n(k − 1)Nt(r)) To limit the list size to less than l elements, as the listsize is upper bounded by j4(k−1)β(r) k, we add the condition l ≥j4(k−1)β(r) k

Algorithm 3

Step 0: Given l the maximum output list size, n and k (for an alternant code,

k = n − d, with d the designed distance of the code), t = 2q and n+, compute theparameters r, β and nLee

Step 1: Find a bivariate polynomial Q(X, Y ) of (1, k − 1)-degree less than βsuch that, for all e in K with absolute value in [0,2q], all coefficients of total degreeless than r(t − wLee(e)) of Q(X + αi, (Y + u + yi+ e)(Y + yi+ e)) are equal to zero.Step 2: Find all the polynomials f ∈ Fk−1[X] such that Y − f (X)(u + f (X))

is a factor of Q and, if dLee(cf, y) ≤ nLee, include f in the list output

Remark: Step 2 of the algorithm can still be performed by the Roth andRuckenstein algorithm([12]) to find the factor of Q of the shape Y − g(X), andthen among the g, we choose those of the form (u + f (X))f (X)

and of the improved algorithm

The reduction in complexity is obvious The gain in error correction capabilitydepends on the number of “positive” (with absolute value in [0,q2 − 1]) and “neg-ative” (with absolute values in [2q, q − 1]) coordinates of the error vector With

nLee,max defined as nLee,max = nt − βr with β(r) = jq1 + 4n(k − 1)Nt(r)k and rchosen such that l ≥j4(k−1)β(r) k, our algorithm corrects all errors of Lee weight amongthe sequences of error vectors with n+ “positive” coordinates, i.e our algorithmcan correct up to nLee,max− n+

Example 3 For n=15, k=6 and l=10 (list output size), Ido Tal’s algorithm cancorrect 5 errors With our algorithm, we find nmax,Lee = 9 for a list size of 4(and 17 respectively with a list size of 10) That means that our algorithm cancorrect 9 “negative” errors if there are no positive errors at all, and all errorswith n+ positive coordinate of Lee weight less than 9 − n+ For example, with

n+ = 10, we can correct all error vectors with 10 positive coordinates and total Lee