Unconstrained Face Recognition from a Single Image 1Siemens Corporate Research, Princeton; 2University of Maryland In most situations, identifying humans using faces is an effortless tas
Trang 1670 CHAPTER 23 Fingerprint Recognition
FIGURE 23.17
Aligned ridge structures of mated pairs Note that the best alignment in one part (top left) of theimage results in large displacements between the corresponding minutiae in the other regions(bottom right)[22] © IEEE
(template) minutiae string The string representation is obtained by imposing a linearordering based on radial angles and radii The resulting input and template minutiaestrings are matched using an inexact string matching algorithm to establish the corres-pondence
The inexact string matching algorithm essentially transforms (edits) the input string
to template string, and the number of edit operations is considered as a metric of the(dis)similarity between the strings While permitted edit operators model the impressionvariations in a representation of a finger (deletion of the genuine minutiae, insertion ofspurious minutiae, and perturbation of the minutiae), the penalty associated with each
edit operator models the likelihood of that edit The sum of penalties of all the edits (edit distance) defines the similarity between the input and template minutiae strings Among
several possible sets of edits that permit the transformation of the input minutiae stringinto the reference minutiae string, the string matching algorithm chooses the transformassociated with the minimum cost based on dynamic programming
The algorithm tentatively considers a candidate (aligned) input and a candidate plate minutiae in the input and template minutiae string to be a mismatch if theirattributes are not within a tolerance window (seeFig 23.18) and penalizes them for
Trang 2Bounding box and its adjustment[22] ©IEEE.
deletion/insertion edits If the attributes are within the tolerance window, the amount
of penalty associated with the tentative match is proportional to the disparity in the
values of the attributes in the minutiae The algorithm accommodates for the elastic
distortion by adaptively adjusting the parameters of the tolerance window based on the
most recent successful tentative match The tentative matches (and correspondences) are
accepted if the edit distance for those correspondences is smaller than any other
corres-pondences
Figure 23.19shows the results of applying the matching algorithm to an input and
a template minutiae set pair The outcome of the matching process is defined by a
matching score Matching score is determined from the number of mated minutia from
the correspondences associated with the minimum cost of matching input and
tem-plate minutiae strings The raw matching score is normalized by the total number of
minutia in the input and template fingerprint representations and is used for deciding
whether input and template fingerprints are mates The higher the normalized score, the
larger the likelihood that the test and template fingerprints are the scans of the same
finger
The results of performance evaluation of the fingerprint matching algorithm are
illustrated inFig 23.20for 1,698 fingerprint images in the NIST 9 database[41]and in
Fig 23.13for 490 images of 70 individuals in the MSU database Some sample points
Trang 3672 CHAPTER 23 Fingerprint Recognition
on the receiver operating characteristics curve are tabulated inTable 23.2 The accuracy
of fingerprint matching alogirthms heavily depends on the testing samples For instance,the best matcher in FpVTE2003[43]achieved 99.9% true accept rate (TAR) at 1% falseaccept rate (FAR), while the best matcher in FVC2006[12]achieved only 91.8% TAR at1% FAR on the first database in the test (DB1) Commercial fingerprint matchers are veryefficient For instance, it takes about 32 ms for the best matcher in FVC2006 to extractfeatures and perform matching on a PC with an Intel Pentium IV 3.20 GHz
Trang 4TABLE 23.2 False acceptance and false reject rates on two data sets with different
threshold values[22] © IEEE
Threshold False acceptance False reject False acceptance False reject
False acceptance rate (%)
FIGURE 23.20
Receiver operating characteristic curve for NIST 9 (CD No 1)[22] © IEEE
With recent advances in fingerprint sensing technology and improvements in the
accu-racy and matching speed of the fingerprint matching algorithms, automatic personal
identification based on a fingerprint is becoming an attractive
alternative/comple-ment to the traditional methods of identification We have provided an overview of
fingerprint-based identification and summarized algorithms for fingerprint feature
Trang 5674 CHAPTER 23 Fingerprint Recognition
extraction, enhancement, matching, and classification We have also presented a formance evaluation of these algorithms
per-The critical factor for the widespread use of fingerprints is in meeting the mance (e.g., matching speed and accuracy) standards demanded by emerging civilianidentification applications Unlike an identification based on passwords or tokens, theaccuracy of the fingerprint-based identification is not perfect There is a growing demandfor faster and more accurate fingerprint matching algorithms which can (particularly)handle poor-quality images Some of the emerging applications (e.g., fingerprint-basedsmartcards) will also benefit from a compact representation of a fingerprint and more effi-cient algorithms The design of highly reliable, accurate, and foolproof biometric-basedidentification systems may warrant effective integration of discriminatory informa-tion contained in several different biometrics and/or technologies The issues involved
perfor-in perfor-integratperfor-ing fingerprperfor-int-based identification with other biometric or nonbiometrictechnologies constitute an important research topic[24, 37]
As biometric technology matures, there will be an increasing interaction among the(biometric) market, (biometric) technology, and the (identification) applications Theemerging interaction is expected to be influenced by the added value of the technology,the sensitivities of the population, and the credibility of the service provider It is tooearly to predict where, how, and which biometric technology will evolve and be matedwith which applications But it is certain that biometrics-based identification will have
a profound influence on the way we conduct our daily business It is also certain that,
as the most mature and well-understood biometric, fingerprints will remain an integralpart of the preferred biometrics-based identification solutions in the years to come
REFERENCES
[1] A K Jain, R Bolle, and S Pankanti, editors Biometrics: Personal Identification in Networked Society.
Springer-Verlag, New York, 2005.
[2] R Bahuguna Fingerprint verification using hologram matched filterings In Proc Biometric
Consortium Eighth Meeting, San Jose, CA, June 1996.
[3] G T Candela, P J Grother, C I Watson, R A Wilkinson, and C L Wilson PCASYS: a pattern-level
classification automation system for fingerprints NIST Tech Report NISTIR 5647, August 1995 [4] J Canny A computational approach to edge detection IEEE Trans PAMI, 8(6):679–698, 1986.
[5] R Cappelli, D Maio, D Maltoni, and L Nanni A two-stage fingerprint classification system In
Proc 2003 ACM SIGMM Workshop on Biometrics Methods and Applications, 95–99, 2003.
[6] CDEFFS: the ANIS/NIST committee to define an extended fingerprint feature set http://
Trang 6http://www.boingboing.net/2006/09/01/walt-disneyworld-[9] L Lange and G Leopold Digital identification: it’s now at our fingertips Electronic Engineering
Times, No 946, March 24, 1997.
[10] Federal Bureau of Investigation The Science of Fingerprints: Classification and Uses U S
Govern-ment Printing Office, Washington, DC, 1984.
[11] J Feng Combining minutiae descriptors for fingerprint matching Pattern Recognit., 41(1):342–352,
2008.
[12] FVC2006: the fourth international fingerprint verification competition http://bias.csr.unibo.
it/fvc2006/
[13] R Germain, A Califano, and S Colville Fingerprint matching using transformation parameter
clustering IEEE Comput Sci Eng., 4(4):42–49, 1997.
[14] L O’Gorman and J V Nickerson An approach to fingerprint filter design Pattern Recognit.,
22(1):29–38, 1989.
[15] L Hong, A K Jain, S Pankanti, and R Bolle Fingerprint enhancement In Proc IEEE Workshop
on Applications of Computer Vision, Sarasota, FL, 202–207, 1996.
[16] L Hong Automatic personal identification using fingerprints PhD Thesis, Michigan State
University, East Lansing, MI, 1998.
[17] L Hong and A K Jain Classification of fingerprint images MSU Technical Report, MSU Technical
Report MSUCPS:TR98–18, June 1998.
[18] D C D Hung Enhancement and feature purification of fingerprint images Pattern Recognit.,
[22] A K Jain, L Hong, S Pankanti, and R Bolle On-line identity-authentication system using
fingerprints Proc IEEE (Special Issue on Automated Biometrics), 85:1365–1388, 1997.
[23] A K Jain, S Prabhakar, and L Hong A multichannel approach to fingerprint classification In Proc.
Indian Conf Comput Vis., Graphics, and Image Process (ICVGIP’98), New Delhi, India, December
21–23, 1998.
[24] A K Jain, S C Dass, and K Nandakumar Soft biometric traits for personal recognition systems.
In Proc Int Conf Biometric Authentication (ICBA), Hong Kong, LNCS 3072, 731–738, July 2004.
[25] A K Jain, Y Chen, and M Demirkus Pores and ridges: high resolution fingerprint matching using
level 3 features IEEE Trans PAMI, 29(1):15–27, 2007.
[26] T Kamei and M Mizoguchi Image filter design for fingerprint enhancement In Proc ISCV’ 95,
Coral Gables, FL, 109–114, 1995.
[27] K Karu and A K Jain Fingerprint classification Pattern Recognit., 29(3):389–404, 1996.
[28] M Kawagoe and A Tojo Fingerprint pattern classification Pattern Recognit., 17(3):295–303, 1984.
[29] H C Lee and R E Gaensslen Advances in Fingerprint Technology CRC Press, Boca Raton, FL,
2001.
[30] D Maltoni, D Maio, A K Jain, and S Prabhakar Handbook of Fingerprint Recognition Springer
Verlag, New York, 2003.
Trang 7676 CHAPTER 23 Fingerprint Recognition
[31] B M Mehtre and B Chatterjee Segmentation of fingerprint images—a composite method Pattern
[34] T Pavlidis Algorithms for Graphics and Image Processing Computer Science Press, New York, 1982.
[35] N Ratha, K Karu, S Chen, and A K Jain A real-time matching system for large fingerprint
database IEEE Trans PAMI, 18(8):799–813, 1996.
[36] H T F Rhodes Alphonse Bertillon: Father of Scientific Detection Abelard-Schuman, New York,
1956.
[37] A Ross, K Nandakumar, and A K Jain Handbook of Multibiometrics Springer Verlag, New York,
2006.
[38] R K Rowe, U Uludag, M Demirkus, S Parthasaradhi, and A K Jain A multispectral whole-hand
biometric authentication system In Proc Biometric Symp., Biometric Consortium Conf., Baltimore,
September 2007.
[39] M K Sparrow and P J Sparrow A topological approach to the matching of single
finger-prints: development of algorithms for use of rolled impressions Tech Report, National Bureau
of Standards, Gaithersburg, MD, May 1985.
[40] US-VISIT http://www.dhs.gov
[41] C I Watson NIST Special Database 9, Mated Fingerprint Card Pairs National Institute of Standards
and Technology, Gaithersburg, MD, 1993.
[42] C L Wilson, G T Candela, and C I Watson Neural-network fingerprint classification J Artif.
[45] N D Young, G Harkin, R M Bunn, D J McCulloch, R W Wilks, and A G Knapp Novel
fingerprint scanning arrays using polysilicon TFT’s on glass and polymer substrates IEEE Electron
Device Lett., 18(1):19–20, 1997.
Trang 8Unconstrained Face
Recognition from a Single
Image
1Siemens Corporate Research, Princeton; 2University of Maryland
In most situations, identifying humans using faces is an effortless task for humans Is this
true for computers? This very question defines the field of automatic face recognition
[1–3], one of the most active research areas in computer vision, pattern recognition, and
image understanding Over the past decade, the problem of face recognition has attracted
substantial attention from various disciplines and has witnessed a skyrocketing growth of
the literature Below, we mainly emphasize some key perspectives of the face recognition
problem
24.1.1 Biometric Perspective
Face is a biometric As a consequence, face recognition finds wide applications in
authen-tication, security, and so on One recent application is the US-VISIT system by the
Department of Homeland Security (DHS), collecting foreign passengers’ fingerprints
and face images
Biometric signatures of a person characterize their physiological or behavioral
char-acteristics Physiological biometrics are innate or naturally occuring, while behavioral
biometrics arise from mannerisms or traits that are learned or acquired.Table 24.1lists
commonly used biometrics Biometric technologies provide the foundation for an
exten-sive array of highly secure identification and personal verification solutions Compared
with conventional identification and verification methods based on personal
identifica-tion numbers (PINs) or passwords, biometric technologies offer many advantages First,
biometrics are individualized traits while passwords may be used or stolen by someone
other than the authorized user Also, biometrics are very convenient since there is nothing
677
Trang 9678 CHAPTER 24 Unconstrained Face Recognition from a Single Image
TABLE 24.1 A list of physiological and behavioral biometrics
Physiological biometrics DNA, face, fingerprint, hand geometry,
iris, pulse, retinal, and body odorBehavioral biometrics Face, gait, handwriting, signature, and voice
to carry or remember In addition, biometric technologies are becoming more accurateand less expensive
Among all biometrics listed inTable 24.1, face is a very unique one because it is theonly biometric belonging to both the physiological and behavioral categories While thephysiological part of the face has been widely exploited for face recognition, the behavioralpart has not yet been fully investigated In addition, as reported in[4, 5], face enjoys manyadvantages over other biometrics because it is a natural, nonintrusive, and easy-to-usebiometric For example[4], among the six biometrics of face, finger, hand, voice, eye,and signature, face biometric ranks the first in the compatibility evaluation of a machinereadable travel document (MRTD) system in terms of six criteria: enrollment, renewal,machine-assisted identity verification requirements, redundancy, public perception, andstorage requirements and performance Probably the most important feature of acquiringthe face biometric signature is that no cooperation is required during data acquisition.Besides applications related to identification and verification such as access control,law enforcement, ID and licensing, surveillance, etc., face recognition is also useful inhuman-computer interaction, virtual reality, database retrieval, multimedia, computerentertainment, etc See[2, 3]for recent summaries on face recognition applications
24.1.2 Experimental Perspective
Face recognition mainly involves the following three tasks[6]:
■ Verification: The recognition system determines if the query face image and theclaimed identity match
■ Identification: The recognition system determines the identity of the query faceimage
■ Watch list: The recognition system first determines if the identity of the query faceimage is in the watch list and, if yes, then identifies the individual
Figure 24.1 illustrates the above three tasks and corresponding metrics used forevaluation Among these tasks, the watch list task is the most difficult one
This chapter focuses only on the identification task We follow the face recognitiontest protocol FERET[7]widely used in the face recognition literature FERET standsfor “facial recognition technology.” FERET assumes the availability of the following threesets, namely a training set, a gallery set, and a probe set The training set is provided for therecognition algorithm to learn the features that are capable of characterizing the whole
Trang 10Identification:
Watch List:
Identity unknown
Unknown individual
Identification algorithm
WL algorithm
Claimed identity
Verification algorithm Accept or
reject
Estimate identity
On list?
Verification rate
ID rate
Cumulative match characteristic
Receiver operator characteristic
Receiver operator characteristic
False accept
ID rate
FIGURE 24.1
Three face recognition tasks: verification, identification, and watch list (courtesy of P J Phillips)
human face space The gallery and probe sets are used in the testing stage The gallery set
contains images with known identities and the probe set with unknown identities The
algorithm associates descriptive features with images in the gallery and probe sets and
determines the identities of the probe images by comparing their associated features with
features associated with gallery images
24.1.3 Theoretical Perspective
Face recognition is by nature an interdisciplinary research area, involving researchers from
pattern recognition, computer vision and graphics, image processing/understanding,
statistical computing, and machine learning In addition, automatic face recognition
algorithms/systems are often guided by psychophysics and neural studies on how humans
perceive faces A good summary of research on face perception is presented in[8] We now
focus on the theoretical implication of pattern recognition for the task of face recognition
We present a hierarchical study of face pattern There are three levels forming the
hierarchy: pattern, visual pattern, and face pattern, each associated with a corresponding
theory of recognition Accordingly, face recognition approaches can be grouped into
three categories
Pattern and pattern recognition: Because face is first a pattern, any pattern recognition
theory [9]can be directly applied to a face recognition problem In general, a vector
Trang 11680 CHAPTER 24 Unconstrained Face Recognition from a Single Image
representation is used in pattern recognition A common way of deriving such a vector
representation from a 2D face image, say of size M ⫻ N , is through a “vectorization”
operator that stacks all pixels in a particular order, say a raster-scanning order, to an
MN ⫻ 1 vector Obviously, given an arbitrary MN ⫻ 1 vector, it can be decoded into
an M ⫻ N image by an inverse-“vectorization” operator Such a vector representation
corresponds to a holistic perception viewpoint in the psychophysics literature[10].Subspace methods are pattern recognition techniques widely invoked in various facerecognition approaches Two well-known appearance-based recognition schemes uti-lize principal component analysis (PCA) and linear discriminant analysis (LDA) PCAperforms[11]an eigen-decomposition of the covariance matrix and consequently mini-mizes the reconstruction error in the mean square sense LDA minimizes the within-classscatter while maximizing the between-class scatter The PCA approach used in face recog-nition is also known as the “Eigenface” approach[12] The LDA approach[13] used
in face recognition is referred to as the “Fisherface” approach[14] since LDA is alsoknown as Fisher discriminant analysis Further, PCA and LDA have been combined(LDA after PCA) as in[14, 15]to obtain improved recognition Other subspace meth-ods such as independent component analysis[16], local feature analysis (LFA)[17],probabilistic subspace[18, 19], and multiexemplar discriminant analysis[20]have alsobeen used A comparison of these subspace methods is reported in[19, 21] Otherthan subspace methods, classical pattern recognition tools such as neural networks[22],learning methods[23], and evolutionary pursuit/genetic algorithms[24]have also beenapplied
One concern in a regular pattern recognition problem is the “curse of dimensionality”
since usually M and N themselves are quite large numbers In face recognition, because
of limitations in image acquisition, practical face recognition systems store only a smallnumber of samples per subject This further aggravates the curse of dimensionalityproblem
Visual pattern and visual recognition: In the middle of the hierarchy sits the visualpattern Face is a visual pattern in the sense that it is a 2D appearance of a 3D object cap-tured by an imaging system Certainly, visual appearance is affected by the configuration
of the imaging system An illustration of the imaging system is presented inFig 24.2.There are two distinct characteristics of the imaging system: photometric andgeometric
■ The photometric characteristics are related to the lighting source distribution inthe scene.Figure 24.3shows the face images of a person captured under varyingillumination conditions Numerous models have been proposed to describe theillumination phenomenon, i.e., how the light travels when it hits the object Inaddition to its relationship with the light distribution such as light direction andintensity, an illumination model is in general also relevant to surface materialproperties of the illuminated object
■ The geometric characteristic is about the camera properties and the relative tioning of the camera and the object Camera properties include camera intrinsic
Trang 12posi-Light source emits light.
Light travels in a
straight line.
Some light
reaches the camera.
Light bounces off in a new direction, is absorbed, and so on.
Camera admits light and forms image.
FIGURE 24.2
An illustration of the imaging system
parameters and camera imaging models The imaging models widely studied in the
computer vision literature are orthographic, scale orthographic, and perspective
models Due to the projective nature of the perspective model, the orthographic or
scale-orthograhic models are used in the face recognition community The relative
positioning of the camera and the object results in pose variation, a key factor in
determining how the 2D appearances are produced.Figure 24.3 shows the face
images of a person captured at varying poses and illuminating conditions
Studying photometric and geometric characteristics is one of the key problems in
the object recognition literature, and consequently visual recognition under illumination
and pose variations is the main challenge for object recognition A full review of the
visual recognition literature is beyond the scope of the chapter However, face recognition
addressing the photometric and geometric challenges is still an open question
Approaches to face recognition under illumination variation are usually treated as
extensions of research efforts on illumination models For example, if a simplified
Lambertian reflectance model ignoring the shadow pixels[26, 27]is used, rank-3 subspace
can be constructed to cover appearances arbitrarily illuminated by a distant point source
A low-dimensional subspace[28]can be found in the Lambertian model that includes
attached shadows Face recognition is conducted by checking if a query face image lies
in the object-specific illumination subspace To generalize from the object-specific
illu-mination subspace to class-specific illuillu-mination subspace, bilinear models are used in
[29–31] Most face recognition approaches addressing pose variations use view-based
Trang 13682 CHAPTER 24 Unconstrained Face Recognition from a Single Image
is reported to be partially robust to illumination and pose variations.Sections 24.2and24.3present detailed reviews of the related literature
Another important extension of visual pattern recognition is in exploiting video Theubiquitousness of video sequences calls upon novel recognition algorithms based onvideos Because a video sequence is a collection of still images, face recognition fromstill images certainly applies to video sequences However, an important property of avideo sequence is its temporal dimension or dynamics Recent psychophysical and neuralstudies[41] demonstrate the role of movement in face recognition: Famous faces areeasier to recognize when presented in moving sequences than in still photographs, evenunder a range of different types of degradations Computational approaches utilizingsuch temporal information include[42–46] Clearly, due to the free movement of human
Trang 14faces and uncontrolled environments, issues like illumination and pose variations still
exist Besides these issues, localizing faces or face segmentation in a cluttered environment
in video sequences is very challenging too
In surveillance scenarios, further challenges include poor video quality and lower
resolution For example, the face region can be as small as 15⫻ 15 Most feature-based
approaches [35, 40]need face images of size as large as 128⫻ 128 The attractiveness
of the video sequence is that the video provides multiple observations with temporal
continuity
Face pattern and face recognition: At the top of the hierarchy lies the face pattern The
face pattern specializes the visual pattern by specializing the object to be a human face
Therefore, face-specific properties or characteristics should be taken into account when
performing face recognition
■ Expression and deformation Humans exhibit emotions The natural way to express
the emotions is through facial expressions, yielding patterns under nonrigid
defor-mations The nonrigidity introduces very high degrees of freedom and perplexes
the recognition task.Figure 24.4(a)shows the face images of a person exhibiting
different expressions While face expression analysis has attracted a lot of
atten-tion [49, 50], recognition under facial expression variation has not been fully
explored
■ Aging Face appearances vary significantly with age and such variations are
spe-cific to an individual Theoretical modeling of aging[48]is very difficult due to
the individualized variation.Figure 24.4(b)shows the face images of a person at
different ages
■ Face surface One speciality of the face surface is its bilateral symmetry The
sym-metry constraint has been widely exploited in[31, 51, 52] In addition, surface
integrability is an inherent property of any surface, which has also been used in
[27, 31, 53]
(a)
(b)
FIGURE 24.4
(a) Appearances of a person with different facial expressions (from[47]) (b) Appearances of a
person at different ages (from[48])
Trang 15684 CHAPTER 24 Unconstrained Face Recognition from a Single Image
■ Self-similarity There is a strong visual similarity among face images of different
individuals Geometric positioning of facial features such as eyes, noses, andmouths is similar across individuals Early face recognition approaches in the 1970s[54, 55]used the distances between feature points to describe the face and achievedsome success Also, the properties of face surface materials are similar within thesame race As a consequence of visual similarity, the “shapes” of the face appear-ance manifolds belonging to different subjects are similar This is the foundation
of approaches[19, 20]that attempt to capture the “shape” characteristics using theso-called intraperson space
■ Makeup, cosmetic, etc These factors are very individualized and unpredictable.
Other than the effect of glasses which has been studied in[14], effects induced
by other factors are not widely understood in the recognition literature However,modeling these factors can be useful for face animation in the computer graphicsliterature
24.1.4 Unconstrained Face Recognition
A wide array of face recognition approaches has been proposed in the literature Earlyface recognizers[11–14, 16–19]yielded unsatisfactory results especially when confrontedwith unconstrained scenarios such as varying illumination, varying poses, expression,and aging In addition, the recognizers have been further hampered by the registrationrequirement as the images that the recognizers process contain transformed appearances
of the object Recent advances in face recognition have focused on face recognition underillumination and pose variations[28, 30–33, 35, 39, 56] Face recognition under variations
in expression[57]and aging[58–61]have been investigated too
In this chapter, we attempt to present some representative face recognition worksthat deal with illumination, pose, and aging variations.Section 24.2describes the linearLambertian object approach[31, 62]for face recognition under illumination variationandSection 24.3the illuminating light field approach[39, 63]for face recognition underillumination and pose variations Finally,Section 24.4 elaborates face modeling andverification across age progression[59–61]
ILLUMINATION VARIATION 24.2.1 Linear Lambertian Objects
Definition: A linear Lambertian object is defined as a visual object simultaneously obeying
the following two properties:
■ It is linearly spanned by basis objects
■ It follows the Lambertian reflectance model with a varying albedo field