combining discrete orthogonal moments and dhmms for off-line handwritten chinese character recognition

These properties make recognition performance of two discrete orthogonal discrete orthogonalmomentsmuchmoresuitable for2-D moments is higher than that of the continuous discrete images a

COMBINING DISCRETE ORTHOGONAL MOMENTS AND DHMMS FOR OFF-LINE

Xianmei Wang, Yang Yang, and Kang Huang

School ofInformation and Engineering

University of Science and Technology Beijing

No 30, Xueyuan Road, Beijing, China, 100083

plum-wanggtom.com

Abstract momentsrequiresacoordinatetransformationwhichmay Discrete orthogonal moment set is one of the novel alsocauseprecisionloss

feature moment-based descriptors for image analysis Some researchers such as MuKundan et al have The Tchebichef moments and Krawtchouk moments are suggested the use of discrete orthogonal moments to the two representatives in this class This paper studies overcome the problems associated with the continuous theperformance of the two discrete orthogonal moments orthogonal moments In the past few years, some

intherecognition ofoff-linehandwrittenChinese amount different discrete orthogonalmoments areproposed such

in words under Discrete-time Hidden Markov Models as Tchebichefmoments [5-7] and Krawtchoukmoments (DHMMs) framework The lower order moments are [8] [9]. These discrete orthogonalmoments are directly

employed asfeatures A serial ofexperimentsare carried defined in the image coordinate space [(O, 0), (N-1,

M-out to compare their performance with that of the 1)]. The implementation of the discrete orthogonal

continuous orthogonal movements such as Zernike and moments doesn't involve any numerical approximation

Legendre Experimental results suggest that the and coordinate transformation These properties make recognition performance of two discrete orthogonal discrete orthogonalmomentsmuchmoresuitable for2-D

moments is higher than that of the continuous discrete images as pattern features It has been shown that these moments In additional, differentvalues of the number of discrete orthogonal moments have better performance

zones, observation symbols and states are also used to than the conventional continuous orthogonal moments

findthe better model structurefor the new approach forimagereconstruction [5] [8] [9].

As we know, for off-line handwritten character Keywords: Discreteorthogonal moments; DHMMs; Off recognition, most of obstacles remain in the strong line handwritten characterrecognition variability of the handwriting styles. Hidden Markov

models (HMMs) are stochastic models which can deal with dynamic properties and variations among human

1 INTRODUCTION widely handwriting Since the last decade, HMMs have beenused for off-line handwritten character Momentswith orthogonal basis functions, introduced recognition There are basically two classes of HMMs

by Teague [1], have minimal informationredundancy in depending of the type of observation sequence, i.e

a moment set. In this class, the two most important discrete-time HMMs (DHMMs) and continuous-time orthogonal moments which have been extensively HMMs(CHMMs) Both of them have been successfully researched in pattern recognition field are Zernike applied to the recognition of off-line handwritten moments and Legendre moments [2-4] Now they have character However,DHMMsaremoreattractive because been widely used as fundamental features for character oftheirlowcomputationalcost

recognition But both Zemike and Legendre moments In this paper, we proposed a method by extracting belong to the class of continuous moments For digital discrete orthogonal moments feature for DHMMs-based images, their computation requires numerical character recognition Then we study and compare the

approximation of continuous integral. This process can discrete orthogonal moments for unconstrained off-line cause error, especially when the order of the moments handwritten Chinese amount in words recognition with increases Additionally, the use of Zernike and Legendre 12 Chinese words In the literature of image processing,

' ~~~~~~~~~itis wellestablished that the important and perceptually

significant information of an image lies in the lower ofcontinuous integrals, and doesnot require coordinate order coefficients The higher order coefficients can be spacetransformations

discarded without significantly affecting the quality of

the image So moments of lower order are extracted in 2.2 Tchebichef Moments

ourmethod as the featuredescriptors

The organization ofthis paper is as follows Section 2 Tchebichef moments are a set of moments formed by presents the definition of discrete orthogonal moments using Tchebichef polynomials as the basis function set including Tchebichef moments and Krawtchouk [5].

moments Our HMMs-based recognition approach The definition of the nth order Tchebichef combining discrete orthogonal moments for off-line polynomial is

handwritten character recognition is provided in section

3 Section4presents experimental results of our method -n, -X, 1+n

Section 5 then concludes this paper t,(x)=(I-N),, 3F2 1 i-N 1 (4)

MOMENTS andthe functionpFq(.)isdefinedas The discrete orthogonal moment functions are based r _ _ _k

on discrete polynomials set which is orthogonal in the ap (a1,a2, .)k zk(5)

discrete domain of the image coordinate space In this q k=., bq)k k!

section, we first introduce the theorem on the discrete

orthogonal polynomials and discrete orthogonal wheresymbol (a)k isgiven by

moments, followed by Tchebichef moments and

2.1 The Generalized Discrete Orthogonal For Tchebichef polynomials, the weight function

Polynomials and Moments cw(x) andsquarednormp(n)aregiven by

Suppose {f,(x)} is a set of discrete orthogonal N +n7

polynomials,then it satisfiesfollowingcondition[5] p(n, N)=(2n)! 2n 0°<mln<N-1 (8)

N-1

Z)(x)f, (x)fm (x)= p(n;a,b)dmn a < x <b (1) Because the value of Tchebichef polynomial grows

where c(x)is called the weight function and p(n) the polynomials is introduced by R Mukundan, S H Ong

Thegeneralized discrete orthogonalmomentsof order polynomials.

(n±m) in term of f, (x) for an image with intensity The definition of weighted Tchebichef polynomial

function I(x,y) ( O<x<N-1 , O<y<M-1 ) are It,(x)} is

F = E1 o°(x)fn (x)fm(y)I(x, y) (2)

Fnm-p(n,N)p(m,M) x=O y=o where,8(n, N) is a suitable constant to maintain the

n=0,1,2, N-1,m=0,1,2, ,M-1 value of weighted Tchebichef polynomial within the For a given moment function Fnmnm, p(n, N) is rangeof[-1,1] ~~~~Under (9), p(n,N) also getsmodifiedby

independent ofx.So(2)canalso be writtenas 9 '

We can see that, the momentdefinitiongiven in (2) or There are some choices for the function,6(n,N) [8-9] (3) completely eliminates the need for any approximation Tesmls omi

The Tchebichefmoments of order (n+m) in term of 3 HMMS FOR CHARACTER

tn(x) for an image with intensity function I(x,y) RECOGNITION

(0.x<N-1, 0.y.M-1),aredefinedas

HMMs have been widelyused inthe field of pattern

1 N-IM-1 recognition They are stochastic models for time

Tn = E tn (X)t,,,(y)I(x,Y) sequences, which were introduced and studied in the late

5(n,N)5(m,M) x=O y=O 1960s and early 1970s [10] [11] Their original

NIM1 tl(x)t,(Y) I(x, ) (12) application was in speech recognition During the last x=Oy=op(n,N)p(m,M) decade, HMMs have become popular in off-line character N-IM-1t (X)t(X)trn(y)f8(n~N)U8(mN)13(m, Y ^I(x, y)M) recognitionFeature extraction issystem.Like allakey forotheraHMMs-based classifier.classification methods,

x=Oy=0 p(n, N)p(m, M) In thispaper, we studied discrete orthogonal moments as

n 0,1,2, N-1,m =0,1,2, ,M-1 features under HMMs framework for off-line

handwritten Chinese characterrecognitionwitha limited

The set of Krawtchouk moments was first introduced 3.1 The Introduction ofDHMMs

byYap etal [8] [9] The kernel of Krawtchoukmoments

consists ofthesetof Krawtchoukpolynomials Here the HMMs are referenced as DHMMs A The definition of the nth order Krawtchouk DHMM is a probabilistic model that describes a random polynomial is [8] sequenceO= 1,02, , T as the indirect observation of

a hidden random sequence Q Q12I , QT, where this

Kn(x;p,N) pak,n,p, F fl,=x (13) hidden process is Markovian A DHMM /2 {A, B,2T}is

k=O -N P) characterizedby followingelements:

wherex,n =0,1,2, N,N>0,pE(0,1). N: the number ofstates inamodel;

The functions Fq and symbol are given by (5) M the totalnumber of observationsymbols;

The functionsandsym lak A (a,)NN :thestatetransition probability matrix.

and (6) respectively.and(6)respectively ~~B=(b )Am : the observation symbol probability

functionc(x) andsquarednormp(n) aregivenby matrix in each state

;T =(;Ti )N the initialstateprobabilitymatrix

NNx

c(x;p,N)=K x (1-p)Nx- (14) 3.2 System Overview

p(n;p,N)= (_P)n rl , 0< n <N-1 (15) At thetoplevel,atraditional DHMMs-based character

functional components: training and recognition Both Same to the Tchebichefpolynomials, instability also training and recognition share a common pre-processing, exists among Krawtchouk polynomials Thedefinition of framesgeneration and feature extraction stage

weighted Krawtchoukpolynomial {Kn(x;p,N)} is In the training phase, after applying pre-processing

steps including binarization, noise removal, boundary

Kn (x; ~, N) x;p,N) (16) obtainment, and size normalization, an image

p(n; p, N) I(x,y) (0<x, y<L) is segmented into T frames The Krawtchoukmoments of order (n±m) interm of frame(i) (1<i<T) using sliding window technique to

suit for DHMMs recognition engine The feature weighted Krawtchouk polynomials for an image with extraction module then transforms a frame intensity function I(x, y) and NxM pixelsaredefinedas imageframe(i) into a feature vector fv(i),which isthen

Krnm =E Kn(x;p1 ,N- )Krn(y;p2,M- )I(x,y) (17) translated into a symbol O(i)(l <i<.T) by clustering

outputby VQ is kept for further use to quantized feature

vectorsinto symbols By abovesteps, acharacterpattern

symbols0=0(1), 0(2) , O(T). For each pattern class, Z n 1

parameters{A, B,;T}, so that the probability of the train .ZoneZ

observation sequencesP(O/2) is maximized

Inthe recognitionphase, after pre-processing, frames

generation and feature extraction, T sequential feature According to (12) and (17), it's obvious that if the vectors are obtained Then these feature vectors are highest order of moments is d, the dimension of a local quantized into thepossible codebookvectors Thus, each featurevector for a zone is

word image is represented by a sequence of

corresponding observation symbols The resulting D'=(d+1)x(d+2)72 (19)

observation sequence is then used to calculate the

log-likelihoods for the model The word associated with an Then the total dimension of a feature vector for a

HMM of thehighest log-likelihood is declared tobe the frame is

In our classifier, the size ofanormalized image was

64x64 The length ofan observation symbol sequence In ourmethod, onlydiscreteorthogonalmomentswith was set to T=8.The K-Meansalgorithmwasemployed lower orders from 0 to 3 are extracted in a zone. For a

forVQbecause of its simpleness The model estimation given zone Zone(j) ( 1<j<Z ) of a frame and matching module were performed by Baum-Welch frame(i) (1<i<8 ),(12) and (17) can be written as algorithm and Viterbialgorithmrespectively [12]

therecognition system anda suitabletopology canmake Knm(i, j) Z ZKn(x'; p1,W-I)Km(Y;P2,H-I)I(x,y)

a significant increase in recognition accuracy In x=O y=O

because they well suit people's writing habit In this

paper, we choose the left-right topology with 1 skip where, x'=x-(i-l)xW, y'=y-(j-1)xH.

(showninFig 1)to model theimage And the number of Because the maximum order(n+m) is 3, a local states and observation symbols is set to different values Tchebiechef feature vector Tz(i j) or a local

tofind the betteroneinlaterexperiments (seeSection3). Krawtchouk feature vector Fz (i, j) extracting from a

zone canbewrittenas

Qe ~~~~~~~~~~~~~~~~TZ(j)={TOO(i j),TO(I 0 j), FIO0, j), TII0 j), T02(i, j),

T20(,1 {,T30(i,j), T03 (i,2j),'7 (i,j), T21 (i),j) (23) Fig 1 The left-righttopology with 1 skip

or

3.4 Extracting Discrete Orthogonal Moments Fz (i, j)={FOO (i, j),FO (i,j), FO(i,j), Fl 1 (j,j),F02 (,1 j),

Feature under DHMMs Framework

In order to detail the image, zoning sliding window The feature vector fora frame frame(i)is formed by

techniqueisusedto extractthesequentialfeaturevectors.

Each frame is divided into Z equal-sizedzones fromup lirking the Z local feature vectors That is to say, a

todownasshowninFig 2 frame vector fi - T(i) or fi- K(i) can be written as Then the width W and the height H of a zone are

3.5 The Acceleration of Feature Extraction tested The results are shown in Table 1 and Table 2 It

should be pointed out that the recognition speed in our According to (12), the computation of Trnminvolves paper doesn't include the time used for pro-processing

and thesaving of the finalrecognitionresults

the computation of the function t (x) and p(n,N) And

each function consists of several multinomials for Table 1 Recognition accuracyofusing different multiplication operation, which will require much orthogonal moments with M=64, Z=4

computation time In order to speed up the process of ateNumber

feature extraction, the coefficients t (x) and p(n,N) for Mome > 6 9 12

Tchebichef moments are computed and saved in advance Tchebichef 91.42 92.69 92.36 Then in the feature extracting stage, all coefficients are

loaded to memory from hard disk and used as constants Krawtchouk 89.89 90.89 90.81

tojoin the real-time calculation ofTchebichef moments Zernike 86.47 89.55 89.39 The same acceleration method is also used in the Legendre 86.19 88.72 88.03 extractionprocessof Krawtchoukmoments

Table 1 indicates that the recognition accuracy of

4 EXPERIMENTS AND ANALYSIS Tchebief and Krawtchouk momentsis higher than that of

the othertwocontinuousmoments, no matterthenumber Our work detailed in this paper deals with the of states is set to 6, 9 or 12 It also canbe seen thatthe recognition of the isolated unconstrained off-line highest recognition accuracy can be achieved by setting handwritten Chinese amount inwords including Chinese Nto 9 However, the differencesin accuracy rate for the characters from V to X and JU All the handwritten values ofN with9and 12 aresmall Additionally, Table character samples used in the following experiments 1 also indicates that the recognition accuracy of Legendre were collected by our laboratory They are written by moments isthe lowest

numerouswriters and in various writing styles There are

totally 11,966 binary digital images We used 8,366 Table 2 Recognition speedofusing different images for training and 3,600images for testing orthogonal momentswith M=64,Z=4

All experiments wereperformed on a COMPAQ Evo tateNumber

CPU All programs arewritten with Matlab6.0language E

Becauseofits simplestform, (11)wasselected for the Tchebichef 513 477 442

function 8'(n,N) to maintain the equal weight of Krawtchouk 515 478 450

different Tchebichefmoments In (17), the condition of Zernike 239 235 223

p= q=0.5wasusedto extractKrawtchoukmoments. Legendre 511 476 439

4.1 Comparison of Recognition Performance Table 2 shows the recognition speed of different

moments Note that all coefficients were computed in

In order to compare the recognition performance of advance (see Section3).From thistable, we can seethat differentmoments, following conditions areusedtocarry the recognition speed of Tchebichef and Krawtchouk

outtheexperiments inthis sub-section: (1) each frame is moments corresponds to that ofLegendre moments, but divided into Z=4 zones, (2) the number of observation is much faster than that ofZernikemoments

symbols isset to M =64 Considering both the recognition accuracy and Therecognitionaccuracy q and recognition speedper recognition speed, we can conclude that the recognition minutes aredefinedas performance of discrete orthogonal moments is much

better than that ofcontinuous orthogonalmoments Number ofcorrectlyclassifiedimages 100

The total number of images used in the test 0004.2 Effect of the Number of Zones Z

The total number ofimagesusedinthetest

s The total number ofimagesused inthetestx60 The number ofzones determines the size ofa feature The total time used inrecognition test vector In this paper, we tested the effect of thenumber

(28) of zones ontherecognition accuracywith 9 statesand 64

In HMMs-based recognition system, the recognition observation symbols Table 3 gives the results

rate is affected by the number of states Theperformance From Table 3, we can find that, the recognition

of different moments with state number 6, 9 and 12 was accuracy increases when increasing the number of zones

With the number ofzones given in Table 3, the highest discreteorthogonal moments.

accuracy canbe obtainedby setting Z to 8 In next

sub-section, Z =8 is used to evaluate the number of References

observation symbols M [1] M M R Teague, "Image analysis via thegeneral theory of

moments,"J Opt Soc Amer., vol 70, no 8, pp 920-930,

Table3 Recognition accuracy of using different 1980

numberof zones with M=64 and N=9

1 2 4 8 [2] Khotanzad, "Invariant image recognition by Zernike

moments," IEEE Trans Pattern Anal. Mach.Intell., vol. 12,

Tchebichef 90.88 92.36 92.69 93.19 no 5, pp 489-497, 1990

[3] Rutvik Desai and H D Cheng, "Pattern recognition by

4 vautifObservation local radial moments," Proceedings of the 12th

4.3 Evaluation of the Number ofObservahon International Conference on Pattern Recognition, vol 2,

The number of observation symbols (namely the [4] Mehdi Dehghan and Karim Faez, "Farsi Handwritten codebook size in the stage ofVQ) was experimentally Character Recognition with Moment invariants," optimized We had evaluated the number of observation Proceedings of13th International Conference on Digital

symbols with 64, 96, 128 and 256 Table 4 gives the Signal Processing, vol 2,pp.507-510, 1997

recognition accuracy of the two discrete orthogonal [5] Mukundan R., ong S.H., and Lee P.A "Image Analysis by moments with 9 states and 8 zones The best result is Tchebichef Moments," IEEE Transactions on Image obtainedby usingM =256 Processing, vol 10, no 9,pp.1357-1364,2001

Table 4 The recognition accuracyof discrete [6] R Mukundan, "Some Computational Aspects of Discrete orthogonalmomentswithdifferent values of M Orthonormal Moments," IEEE Transactions on Image

64 96 128 256 Processing, vol.13,no.8,pp. 1055-1059,2004.

Tchebichef 93.19 94.11 94.78 95.58 [7] P T Yap and P Raveendran "Image focus measure based

Krawtchouk 93.00 93.97 94.58 95.36 on Chebyshev moments," IEE Proc -Vis Image Signal

Process,vol. 151,no.2,pp. 128-136,2004.

5 CONCLUSIONS [8] P.T.Yap, P.Raveendran and S.H.Ong "Krawtchouk

Moments as a new set of discreteorthogonalmoments for

In this paper, we have studied discrete-orthogonal- image reconstruction," International joint conference on moment-based feature descriptors in the recognition of Neural Network, WashingDC, pp 908-912,2001

unconstrained off-line handwritten Chinese amount in

words under DHMMs framework Two main works are [9] Pew-ThianYap, Raveendram Paramestran andSeng-Hunt done in this paper First, we discussed the theory of Ong "Image Analysis by Krawtchouk Moments," IEEE Tchebichef and Krawtchouk moments Second, we Transactions on image processing, vol 12, no 11, pp combine the discrete orthogonal moments and DHMMs 1367-1377, 2003

for character recognition A serial of experiments are [10] L E. Baum and T Petrie "Statistical Inference for

carriedout to testtheperformance ofourapproachinthe Probabilistic Functions of Finite State Markov Chains,"

recognition of off-line handwritten Chinese amount in AnnalsofAMath. Statistics, vol 37, pp.1554-1563, 1966 words Within the parametric range discussed in our

paper,thepreliminary resultsseem toindicate that: [11] L. E Baum and T. Petrie, G Soules and N. Weiss "A

(1) The performance of discrete orthogonal moments MaximizationTechnique Occurring in StatisticalAnalysis

is better than that of continuous orthogonal moments of Probabilistic Functions of Markov Chains," Annals of because discrete orthogonalmoments aredirectly defined Math.Statistics, vol 41, no 1,pp.164-171, 1970

intheimage coordinatespace [12] L Rabiner, "A Tutorial on Hidden Markov Models and (2) The recognition accuracy of Tchebichefmoments Selected Applications in Speech Recognition," In Proc.

isalittle better than that of Krawtchoukmoments IEEE, vol 77, no 2, pp 257-286, 1989

(3) Increasing the number ofzones and observation

symbols can results in better recognition accuracy for