Proceedings VCM 2012 53 phát hiện và nhận dạng mã vạch một chiều từ hình ảnh

A vision based method for 1D barcode detection and recognition Phát hiện và nhận dạng mã vạch một chiều từ hình ảnh Trần Thị Thanh Hải Viện nghiên cứu quốc tế MICA, HUST - CNRS/UMI - 2

A vision based method for 1D barcode detection and recognition Phát hiện và nhận dạng mã vạch một chiều từ hình ảnh

Trần Thị Thanh Hải

Viện nghiên cứu quốc tế MICA, HUST - CNRS/UMI - 2954 - INP Grenoble e-Mail: thanh-hai.tran@mica.edu.vn

Abstract:

While traditional methods for barcode reading use specific devices (e.g laser scanners) which are very disadvantageous because of their lack of mobility, reading barcodes from a camera phone is becoming an interesting and low-cost solution in this ubiquitous computing era This paper presents a method for 1D barcode recognition from images This method is composed of 2 main phases: barcode location and barcode decoding Our contribution found in the barcode location phase where we combine Discrete Cosine Transform (DCT) based technique and scan-line based techniques to improve the location rate while reducing the computational times For the barcode decoding, a statistical recognition is used The experimental results show the good performance of our method in comparison with state of the art methods

Tóm tắt

Hướng tiếp cận truyền thống để nhận dạng mã vạch thường sử dụng các thiết bị chuyên dụng như máy quét laser Các thiết bị này thường gắn ở một vị trí cố định, khó di chuyển Trong một số ứng dụng như tra cứu sản phẩm tại chỗ (ví dụ trong siêu thị, trên quảng cáo, v.v), hướng tiếp cận đọc mã vạch từ điện thoại di động là một giải pháp thú vị và ít tốn kém Bài báo này trình bày một phương pháp nhận dạng mã vạch từ ảnh thu nhận

từ camera của điện thoại di động Phương pháp đề xuất gồm hai pha chính: định vị mã vạch và giải mã Các đóng góp chính của chúng tôi là sử dụng kết hợp hai kỹ thuật quét dòng và phép biến đổi cosin rời rạc cho phép nâng cao độ chính xác định vị, đồng thời giảm thời gian tính toán Các thử nghiệm cho thấy phương pháp

đề xuất cho hiệu quả nhận dạng cao, hoàn toàn có khả năng tích hợp trên điện thoại di động cho các ứng dụng khác nhau như tra cứu sản phẩm trong siêu thị

1 Introduction

Nowadays, barcodes, 1D barcodes in particular,

play an important role in the modern life The need

to use barcodes in different applications requires

researches on barcode symbologies (codings) as

well as barcode readers In reality, there are a lot of

devices for reading barcodes such as barcode pens,

hand-held laser scanners, etc Laser scanner is the

most popular device that can be found in any marts

/ stores or documentation centers However, the

biggest disadvantage of this device is its lack of

mobility because most of the time, it can only be

used together with the corresponding Point of Sale

machine

A current trend for barcode reading is to recognize

1D barcode from images captured by a camera

because of its low-cost and mobile properties

However, the problem of visual barcode

recognition is not simple The difficulties come not

only from mistakes during printing barcodes but

also from imaging condition (e.g shadow,

reflection, dirty surface) that produces distorted,

skew, dirty, too dark/bright barcode images

This paper presents a method for visual recognition

of 1D barcodes while overcoming some of the above difficulties Our framework consists of two main components: barcode location and barcode decoding Our main contribution in this paper found in the barcode location phase where we combine Discrete Cosine Transform (DCT) based technique and scan-line based technique to improve the location rate while reducing the computational times

The paper is organized as follows In the section II,

we present and evaluate some related works In the section III, we propose a framework for 1D barcode recognition and explain in more detail each component of the framework Some experimental results will be presented in section

IV

2 Related works

All methods for barcode recognition from images must deal with two main problems: barcode location and barcode decoding In the literature,

there exists numerous approaches for barcode

localization using analysis of spatial domain [1],

Gabor filtering [2], analysis of Wavelet domain [3],

or DCT domain [4]

Based on the observation that a barcode is

composed of parallel and adjacent bars and spaces,

which are usually aligned horizontally, a barcode

region should be dominated with vertical texture

then a wavelet based method has been used to

locate barcode regions in [3] With wavelet

transform, an image is divided into four subbands

(one low-frequency subband and three

high-frequency subbands) Using the characteristics of

high-frequency wavelet subbands, some criteria

based on energy of coefficients on the levels of

subbands have been defined to locate barcode

regions Once a barcode region is located, edges of

bars will be determined by finding the

zero-crossings of the average intensity curve built from

8 parallel scan-lines For barcode decoding, the

nearest neighbor classification method is used to

find out the most similar reference vector This

method has been tested with 292 EAN-13 barcode

images taken by a NOKIA 7650 camera phone,

shows a good performance of the method: the

correct barcode location rate is 94.18% and the

correct barcode recognition rate is 85.62%

However, this method works only under the

following assumptions: barcodes must be placed as

in front of the camera as possible, in the center of

the camera view and the physical barcode’s size

should be bigger than 3cm in length

DCT expresses a sequence of finitely many data

points in terms of a sum of cosine functions

oscillating at different frequencies A barcode is

composed of black and white stripes alternative

which are aligned in i-direction, AC-coefficients of

the DCT in i-direction are of high magnitude As

DCT coefficients of a barcode region not only can

be distinguished from non-barcode region, they

give also information about orientation of the

barcode These properties are explored in [4] to

locate barcode regions in image Following the

authors, this method is very fast and gives good

results in case where there are no high textured

regions in the image than barcode regions

However, the paper did not resolve the problem of

barcode recognition

Wavelet-based and DCT-based approaches allow

locating barcode regions in images However, these

methods are sometime quite time-consuming A

simple but quite efficient technique that is widely

used in the literature is scan-line-based [5], [6]

First, a scan-line passing the center of the image

will be extracted Then the curve built from the

intensity of points lying on this scan-line will be

smoothed using a Gaussian filter The maxima / minima of this smoothed curve will be determined that help for finding dynamic thresholds for binarizing the curve With the binary curve, the width of spaces and bars will be calculated Barcode recognition is done by digit classification using similarity measure between the candidate and the references About 1000 images of barcodes have been taken by a N95 camera phone The recognition rate of 90.5% at 640x480 image resolution was achieved with assumptions that there is only one horizontal barcode at the center of each image

In summary, most of methods for barcode location are based on specific properties of barcode, which are the parallel and dense distribution of bars and spaces DCT can locate barcode area in all directions while current scan-line based approach can only deal with horizontal barcode The barcode recognition rate depends strongly of barcode location and binarization

Nowadays, barcode recognition has been developed as a commercial product and integrated

in several mobile phone platforms such as Red laser [7], ShopSavvy [8], XZing [9] The problems with these products are: 1) The algorithm developed in each product is not public so we cannot understand why it works and why not in a certain case; 2) As commented by some users, the algorithm does not work well in poor lighting In this paper, we would like to present a public method that provides comparable performance

3 Vision based 1D barcode recognition 3.1 Brief description of 1D barcodes

In this paper, we are interested into 1D barcodes: UPC-A, EAN-13 or ISBN-13 because they are arguably the most widely used throughout the world to mark retail products that are scanned at points of sale However, the framework that we propose can deal with all kinds of barcodes with a little modification in the algorithm because both barcode location and barcode recognition algorithms are based on the knowledge of barcode structures and their appearances

The Figure 1 shows an image of a EAN-13 barcode The human readable data is written under the barcode i.e the machine readable data This barcode consists of 13 digits The last digit is a checksum computed from the first 12 digits The barcode starts with a left-hand guard bar (black-white-black) and ends with a right-hand guard bar (black-white-black) Between the guard bars, there are two blocks, each composed of 6 digits, separated by a center guard bar (white-black-white-black-white) Two different EAN-13 barcodes will

have the same amount of bars and spaces but the

width of these bars and spaces will be different

because they encode different numbers These

characteristics of barcode will be used in barcode

location and recognition presented in the next

sections

Figure 1 Structure of an EAN-13 barcode

3.2 Proposed framework for 1D barcode

recognition

Based on analysis on barcode structures and their

appearances, we found that both DCT and

scan-line based techniques are good for barcode

location For barcode decoding, a statistical method

seems to be convenient We propose therefore a

framework for barcode location and decoding as in

the Figure 2 The framework is composed of 3

principal modules: 1) localization of 1D barcode

region in the image; 2) once the barcode region is

located, it is extracted from the image and

binarized; 3) the binarized barcode region will be

decoded to output a sequence of digits (final

result) In the following, we will explain in more

detail each component of the whole 1D barcode

recognition system

Figure 2 Framework of 1D barcode recognition

system 1) Localization of Barcode region: This

module consists of 2 components First, we locate

1D barcode regions using DCT technique As this

localization is always inaccurate, we then apply the

scanline based technique to re-locate it in order to

find more accurate barcode boundaries (guard

bars)

a) DCT-based method for 1D barcode region

localization:

The algorithm of DCT based barcode location consists of 10 steps:

1 Divide the input image into 8x8 pixels blocks

2 Apply DCT on each 8x8 pixels block This step will produce for each block 64 coefficients in which the first coefficient DC-value represents the average DC-value of the image block, 63 remaining AC-values represent spatial frequencies of image block in the ascending order We set all DC-values of all image blocks to 0

3 Calculate the average DCT-block from all blocks of 8x8 DCT-coefficients

4 Group all DCT-coefficients c ijof one

frequency range f of the average DTC block

into an array G f , so that

}) 1 , , 1 ) ( } , , 1 ) (

For example, G 3 { c31, c32, c33, c13, c23} Then calculate the largest DCT-coefficient

) max(

c  from each frequency range f in

the average block The coefficients

max

f

c indicate the coefficients in the barcode

area that are strongest

5 Compute a weight matrix W of dimension 8x8 where each elements w ij is defined as follow:













else k

c c if k w

d

f ij e ij

,

Where k d and k e are the emphasis and desemphasis factors, respectively

6 Perform an element multiplication of each 8x8 DCT block with W Then calculate the sum of each DCT block The higher the sum of the

DCT-block, the higher is the likelihood that it belongs to barcode regions The DCT-sums make up a subsampled DCT image by a factor

8 in each dimension We set 0 to negative values and scale positive values to the range

[0, 255] to create a gray-scale image

7 Perform morphological closing on the

gray-scale image obtained from step 6 to smooth

the barcode region

8 Convert gray-scale data into binary using Otsu thresholding technique

9 Look for 8-connected components

10 Choose rectangular connected regions as candidates of barcode region

b) Scanline-based method

The DCT based algorithm gives an approximate location of the barcode region We consider it as the region of interest for which we will apply the

scan-line based technique to re-localize in a more

accurate manner The output of this phase is a

region of barcode well localized with boundaries

which are correctly determined to pass to the

barcode decoding

2) Barcode Decoding: Barcode decoding is

carried out in the following steps as illustrated in

the Figure 3 We can see in this figure two main

phases: 1) learning digit classifiers and 2) barcode

decoding given a new barcode area

a) Digit and barcode representation

As analyzed above, bars and spaces can cover one

to four modules of the same color Each digit is

composed of two bars and two spaces with a total

width of 7 modules We represent each digit by a

vector of 4 elements corresponding to the widths of

ordered bars and spaces This vector will be

normalized to unit magnitude so that it is invariant

in case of scale change of barcode

Given a binary barcode region, we compute the

widths of bars and spaces by counting the sum of

black /white pixels To reduce errors produced

during binarization, first we compute average

width of single module then the double, triples;

quadruple modules widths will be calculated Now,

each barcode region is represented by a 12

elements-vector representing 12 digits Each

element, corresponding to a digit, is again a 4

elements-vector

Figure 3 Barcode decoding schema

b) Statistical learning of digit classifiers

The learning of digit classifiers is simple To build

a digit classifier, we take normalized samples of

this digit from training dataset, compute the

average one and considered it as the reference

vector for this digit class We do the same for 30

encoded digits (because left-hand digits are

encoded into 2 sets A, B, and right hand digits are

encoded in set C) to obtain 30 reference vectors

c) Barcode decoding

For recognition, each digit will be compared to digits learnt during training phase Left-hand digit will compared with 20 digits in A, B sets to choose the most similar one (Euclidian distance based measure) Similarly, right-hand digit will be compared with 10 digits in C set to choose the most similar one This makes a hypothesis of the barcode This hypothesis will be passed to the verification step of the checksum number If it is correct, the hypothesis is confirmed If not, we generate other hypothesis (by changing the meta-number) and verify until we obtain a valid code

4 Preliminary results and discussions 4.1 Dataset preparation

For learning and testing, a dataset needs to be prepared Until now, the number of samples for training each digit class remains quite modest (we use only 10 images of barcode) For testing we use

100 images taken from different sources (http://cvpr.uni-muenster.de/research/barcode) or

by ourselves These images contain barcodes with different status: change in size, orientation, noise, dirty, reflections, etc The image resolution is 640x480

4.2 Experimental results

The proposed method for barcode recognition will

be evaluated based on 3 performance measures: 1) barcode localization rate; 2) barcode recognition rate; and 3) computational time

Barcode localization rate is the ratio between the number of located barcode regions and the total number of barcode regions in the ground truth A barcode region is considered as located if the located region covers more than 80% the ground truth region Barcode recognition rate is the ratio between the number of barcodes which are successfully decoded and the total number of barcodes in the ground truth The computational time is the total time (in second) that the recognition system takes to read an input image; process it then output a localized barcode region and a digit sequence

The TABLE I shows performance evaluation of barcode recognition system using the combined technique for barcode localization In all cases, we use the same recognition technique for barcode decoding

TABLE I PERFORMANCE EVALUATION OF

BARCODE RECOGNITION METHODS

Method Localizatio

n rate (%)

Recogniti

on rate (%)

Time (s)

One scan-line 75 70 1.38

Multiple (6)

scan-lines

scan-line

We found that the technique based on one scan-line

gives worst results in term of barcode localization

and recognition rate When using multiple

scan-lines, the noisy influence will be decreased, that

improves the performance in both localization and

recognition rate However, the computational time

will increase Our proposed method (DCT

combined with one scan-line technique) gives a

better performance in localization rate than the case

using multiple scan-lines An additional advantage

of this combined technique is that it is quite

efficient in term of computational time

In the below figures, we show some examples of

barcode localization using our combined method

We can see this method can deal with difficult

situations such as barcode is un-planar (Figure 4),

barcode is rumpled (Figure 5), or shadowed

(Figure 6) In all of these figures, the green

rectangles are the barcode regions located by the

only DCT technique; the red ones are the barcode

regions localized using the combined technique

Figure 4 Localization of unplanar barcode

Figure 5 Localization of rumpled barcode

Figure 6 Localization of shadowed barcode

5 Conclusion

In this paper, we presented a framework for barcode recognition from images The main contribution of our framework in comparison with the literature ones is we used DTC technique to locate barcode regions that is robust to orientation and size The scan-line technique is used next to re-localize the barcode region so we obtain a more accurate location of barcode boundaries This combined technique is better in barcode location and computational time for recognition In the future, we would like to deploy this method on mobile phone platform to build an online assistance system for customers in stores

Acknowledgments

This study was done in the framework of the International cooperation project 10/2011/HĐ-NĐT

References

[1] D Chai and F Hock, Locating and Decoding EAN-13 Barcodes from Images Captured by Digital Cameras, in 2005 Fifth International Conference on Communications and Signal Processing, 2005: Bangkok, Thailand p 1595

- 1599 [2] A.K Jain and Y Chen Bar code localization using texture analysis in Proceedings of the Second International Conference on Document Analysis and Recognition 1993

[3] R Oktem, Barcode localization in wavelet domain by using binary morphology, in Proc

of IEEE SIU'04 2004 p 499-501

[4] A Tropf and D Chai, Locating 1-D Bar Codes

in Dct-Domain, in Proceedings 2006 IEEE International Conference on Acoustics, Speech and Signal Processing 2006: Toulouse, France [5] O Gallo and R Manduchi, Reading Challenging Barcodes with Cameras, in Proc

of IEEE Workshop on Applications of Computer Vision 2009, 7 p 1-6

[6] S Wachenfeld, S Terlunen, and X Jiang, Robust 1-D Barcode Recognitionon Camera Phones and Mobile Product Information Display, in Lecture Notes in Computer Science, Springer-Verglas Berlin Heidelberg p 53-69 [7] http://redlaser.com/

[8] http://shopsavvy.com/

[9] http://code.google.com/p/zxing/

Bibliography

Thi Thanh Hai TRAN

graduated in Information Technology from Hanoi

University of Science and Technology in 2001

She has followed MS degree in Imagery Vision

and Robotic at Grenoble Institute of Technology in

2002 She received her Ph.D degree from

Grenoble Institute of Technology, France in

2006 She is currently lecturer/researcher at

Computer Vision group, International Institute

MICA, Hanoi University of Science and

Technology Her main research interests are

visual object recognition, video understanding,

and human-robot interaction