Luận văn automatic discovery of connections between vietnamese's anthropometric features

It is a work of recreating the face of a person from hís skelctal rernning, ÂL the first days, lueiul reconstruction ncture af is dane using clay, where a skillfal experts who 1mderst

VIRTNAM NATIONAL UNTVERSITY, HANOT UNIVERSITY OF ENGINEERING AND TECHNOLOGY

DINH QUANG IIUY

AUTOMATIC DISCOVERY OF CONNECTIONS

BETWEEN VIETNAMESE’S ANTHROPOMETRIC

FEATURES

MASTER’S TIIESIS

Hanoi — 2010

VIETNAM NATIONAL UNTVERSITY, HANOT UNIVERSITY OF ENGINEERING AND TECHNOLOGY

DINH QUANG IIUY

AUTOMATIC DISCOVERY OF CONNECTIONS

BETWEEN VIETNAMESE’S ANTHROPOMETRIC

Abstract

Long time ago, when people found a skeleton, it wee hard to determine who the victim was However, people are trying to find a way to solve this problem due to its demands and importance Several methods have been introduced for identifying deccased persons, some more cffective than others Facial reconstruction is onc of them It is a work of recreating the face of

a person from hís skelctal rernning, ÂL the first days, lueiul reconstruction

ncture af

is dane using clay, where a skillfal experts who 1mderstand the s1

skull and skin very well to use clay to build up the depth of tissue on the skull to that of a living individnal Later, this method is eamputerized and

people tend to develop 3L facial reconstruction systems

In the facial reconstruction systems, the most important issue is to predict

the soft tissue depths at every location or some locations Most researches try to obtain a database of soft tissuc thicknesses at facial landmarks, and store the average thickness for cvery landmark When performing the re- construction, these thicknesses are referenced, and the face is built based on the skull model Their approaches have some problems in data collecting,

and they do not make use of the discovered skull to predict the thicknesses

Therefore, the accuracy is very low and most: of the time, they need to man- ually modify the model generated from the system a lot in order to receive

a suitable face

Realizing thot the soft tissuc thickness and some other anthropometric features may have some relationships with the skull shape, we propose a method for automatic discovery of these connections We first collect data

using the CT technique which is the most accurate method al the moment After that, we try some machine iearning techniques on the data to see the

performance The evaluations and comparison with other approaches are

also given in the thesis

2.2 Maciel Teconatrnetion Systems - ` ` và và

2.2.1 8ystem developed by Thjom Anderson, Martin Valfridseon in

WW co 2.2.2 System developed by Kolja Kahler and Jérg Haber

bộ 6 BoÏl tissue thickness sudies

§ Available Soft Tissue Thickness Data

iv ‘TABLE OF CONTENTS

33 Discovery of anthropometric relationships using linear regression 28

3 Discovery of anthropometric relationships nsing nenral networks 25

34.1 Selech network struchure See 25

$4.2 Initiolive and train the mework 6

4 Evaluation and Result 20

5 Conclusions and Future Work 35

Matching skull iuto drawing pOTAÌ các ee 5

Suceœs[ul cluy rocoustriou by L6U Euce Lab 6 Process of Reeonstrniction using vahimetric data 8 Result of Bjorn Anderson and Martin Valfridsson’s reconstruction 9 Facial Recoustruction Diagram by FACES 02 ee H

Iead CT image taken with sagittal plene ` 20 Head CT image taken with vertivle plane that goes through the mid

dic of the left eye socket - TH nh va 21

Head CT image taken wilh vertical gian hai woes thongh the forehead 22 Head C' image taken with horizontal plane that goes through the

zygion landmarks 39 Head C'l' image taken with horizontal plane that goes 5 through the

Neural network structure used in the stady

Regression resulls obtained by ten-fold cross validation fer pronasale

thickness using (a) ueural uetwork model and (b) linear regression

v

4.2 Regression results obtained by ten-fold cross validation for nose length

using (a) nenral network model and (b) Tinear regression model 3 4.5 Regression results oblained by ten-fold cross validation for upper lip

border using (a) neural uetwork anedcl and (b) lincar regrevsien model 31 4.3 Regression results obtained by ten-fold cross validation for nose height

using (a) nenral network model and (b) Tinear regression model 32 4.4 Regression revulis obtained by ten-fold cross validation lor pupil-

pupil distance using (a) neural nevwork model aud (b) linear re-

4-6 Regressicn results obtained by ten-fold cross validation for lower lip

border nsing (a} nenral network model and (b) linear regression model 32 4.7 Facial Recoustruction Result Using Linear Regrevsion Equations 33 4.8 Matching the fuec und the skull 0 ee eee 33

List of Tables

4.1 MSR values for ‘average method’ (AVG), Linear Regression (LR), and

Neural Newark (NN) The best performance is in boldface 5 30 4.2 Equations for incer corrclation between input and output, with the

corresponding MSE when applied with the whole data set In the eqnations, x is the input and y is the output - ` MA

vii

RRF Radial Basis Functions

MSE Mean Square Error

viii

Chapter 1

Introduction

1.1 Overview and Motivation

Facial reconstruction is the work of recreating the face of an individual from his dis- covered skull This process is mainly used in crimainal investigations to facilitate vie- tim identification when no other means are available Besides, facial reconstruction

is also used in archaevlogy to verify the remains of historic figures or in anthropology

to approximate the look of prehistoric hominids

People have been recreating the face of sa unidentified individusl from their

disvovercd skulls for neorly a hundred yeurs, At the first days, locial reconstruction is

done using clay This method reqnires skillful experts who understand the structure

of skull and skin very well to nse clay to build np the depth of tissue on the skull

to that of a living individual ‘lhe experts first place the landmark dowels on the pre-delined craniofacial landmarks on the skull After that, clay is applied and the expert interpolates with clay between the landmark dowels to build up the skin ‘I'his method is called the Krogman method [Kro46] and is still used in non-automatic forensic fuciel reconstruction now The cxpert skill and amount of time required have motivated rescarchers Lo Ury to computerize the lochuique, A well-designed computer-aided facial reconstrnccion system has many advantages, including great reduction in time consumption Using such a system, we can produc

eral possihle facial models fram a given skill by using parameters determining the person's age, weight, and gender

Recently, the rapid development of 3D equipments and technology enable us to

advance into this field of research A lot of computerized methods for 3D facial

1

2 Chapter | Introduction

reconstruction have been proposed and developed, which make use of computer program to transform 3D scanned models of the skull into faces Many researches try to follow the manual approach, but make use of the computer to fasten the

process of revoustruction, In these methods they have the discovered skull scanned

to any other race with proper data collection

1.2 Our Contributions

In scope of a Master thesis, we propase a method for automatic discovery of connec-

tions between anthropometric features such as tissue thicknesses, distauce between

(wo pupils, nose height aud Uhe skull shape This work indudes the wicthod lor dutabase collecting, aud how we apply machine leaning Wo discover the relation- ships The couucetion mode} then ean be used te volve the probleu of reconstruct

ing the face from the skull Our results from evaluations also prove that there ia telationships between facial soft tissue thickness and the skull shape

1.3 Thesis Organization

The rest of this thesis

knowledge that is iuvolved ‘Lhe issues in this chapter mclude of previous work in

is organized as follows Chapter 2 presents aome hackground

facial reconstruction, current facial reconstruction systems We also review the work

1.3 Thesis Organization 3

of soft tissue thickness studies ond available soft tissue thickness data Chapter

3 describes the proposed method of automatic discovery of connections between Vietnameses anthropometric features Qur method of data organization, how to collect Ubis dawn, und how we muke use of Ukis data ace provided in this ehupter Chapter 4 presculs our results ud Ue comparison with other approaches, Chapter

5 coucludes our work and gives Lhe future research directions based om the results obtained eo far

This technique requires a forensic artisc who draws the picture of the recongtrneting face ‘There are two main 2D approaches, matching the skull into the drawing portrait and matching the skull into pictures or videos In the case af matching the skull into the drawing portrait, the artist first draws a simple version portrait based

on the skull’s metrics After that, he matches this drawing into the skull image and watch Ile then redraws or fixes some features until the skull image and the drawing

fit perfectly The process of this technique is shown in Figure 2.1

The other tachniqne, matching the skull into a picture or video, is used when people want to compare the face with the slcull ta identify the correlations or if the slcull belongs to the people in the picture or video (Figure 2.2)

4

2.1 Previous Work in Facial Reconstruction From Skulls

The most common manual approach to facial reconstruction is the clay modeling

approach In this method, people first put the landmark dowels on the predefined

craniofacial landmarks on the skull The lengths of these dowels are defined using

one of the available soft tissue thickness data in the beginning After that, they

attach clay to the skull in a way that the clay covers the dowels but still keeps a

continuous surface This method is also known as the Krogman [Kro46] method,

and it gives good result without any help of a computer However, this method

requires an expert with very good skill and still it takes a lot of time for a version

2.1 Previous Work in Facial Reconstruction From Skulls 7

Many rescarchers have worked on the facial reconstruction problem and have pro- vided different solutions using computcrs Mark Jones [Jon01] uses volumetric data and cross correlation techniques Matthew Cairns [Cai00 uscs statistical tools such

as Generalized Procrustes Analysis aud Principal Componems Analysis, Katrina Archer [Are97] trie to computerize Che inanual facial reconstruction process Aue other approach is presented by Kahler [ITTS03] in which he nse a physics based head model with skin surfaces, virtual muscles, a mass spring cystem and landmarks to reconstruct the face Bullock [Bul99] uses the Krogman method for placing virtual dowels on the digitized skull with the emission-based implicit modeling In this mod eling, every polygon of the skull model emits a real value which is the interpolated soft tissuc thicknesses at the landmarks associated with the polygon

Thore are many other techniques that share the same method of following steps The first step is generating the 3D wodel of the skull We can do tub with the help of digitalized cquipmeuts such ay the CT scanner or MRI scanner The CT technique enables accurate measurement of facial soft tissue thickness and is nsed widely in collecting soft tissue thickness dara Tn addition, we can generate the 3D model of the skull from CT images conveniently The second step is to locate the landmarks at the skull surface and the tissue depths at the landmarks Based on these landmarks, we can apply some regression techniques such as the ILI’, B-spline, and Non-uniform rational U-spline to generate the 3D grid of the skin surface ‘The last stop is to refine the reconstructed face, by tuning the features such as the cyes, nosy, cars and lips This iy a hurd work because these feutures are unpredictable by

the skull shape only

Figure 2.4 shows the process of the Iacial reconstruction propuscd by Mark Jones [Jou0L in which Le ompare voluiuetrie đạta of the 18 remaius with that

of w refercuce head Firstly, the discovered skull is seumued using » CT seunner to obtain volumetric dats After that, a reference head having the same sex, racial and age characteristics as the discovered skull is chosen Then a correspondence is created between the two heads using correlation techniques Finally, nsing this cor- respondence, the soft tissue from the reference head is mapped onto the discovered

slcull to produce the face of the unknown person

Figure 2.4: Process of Reconstruction using volumetric data

2.2 Facial Reconstruction Systems

2.2.1 System developed by Bjorn Anderson, Martin Val-

fridsson in 2005

This system uses the 3ds max software and contains 9 steps The first step is to

collect data from CT In this step, the skull is scanned in a CT scanner and the CT

slices are stored in DICOM format The second step is data segmentation, in which

CT slices are edited using segmentation software to remove artifacts such as metal

cavity fillings This segmentation software is also used to produce a 3D model to import into 3ds max, In the third step, the model is imported into 3ds max and

some pre-processing such as normalization and rotation is performed In the forth

2.2 Facial Reconstruction Systems 9

step, the landmark dowels are located by 4 using the GUI In the fifth step,

the holes in the cranium are covered Step 6 is to perform mesh calculations In

this step, based on the tissue depths at the landmark, the tissue depths at other locations are calculated The chin and neck are also constructed at this step Step 7

is the creation of nose, eyes, ears and lips Step 8 is the post process, in which they

2.2.2 System developed by Kolja Kahler and Jérg Haber

In December 2003, Kolja Kahler defended his PhD thesis with the title of "A Head Model with Anatomical Structure for Facial Modeling and Animation” In the

10 Chapter 2, Background

thesis, he studied the facial muscles, how they work and deform, ancl built a facial reconstruction software ‘I'his software is then developed [KHS03] by him and his colleagues at the University of Saarland ta become a complete system Beside fast

recoustruction, the system ulso cnables chunging the emotion of the fuec, based on

the 24 types of smuscle changes

diagram

2.3 Facial Landmarks

As described, most method uses the pre-defined points ou the head which are called

facial landmarks ‘here are a number of different landmark configurations with

varius numbers of landmarks, The sostly uscd landmwrks are the 32 landmarks desuribed by Rhine [Rhis

seribed in Table 2.1 and Figure 2.7, In our rescarch, we make uve of some extra,

- The list of these landmarks und ily position ure de-

landmarks such as exoranthion, endocanthion, alare, pronasale, basion, subnasale, and stomion There are twa types of landmarks, craniofacial and cephalometric They are the marks an the skull and corresponding marke on the skin surface, respectively In most date measurements, the distances between these pairs are

measurecl and stored.

12 Chapter 2, Background

Table 2.1: List of lacial Landmarks

Number Landmark Name

6 Upper Lip Margin

7 Lower Lip Margin

2.4 Important Facial Features

Facial reconstruction’s aim is to produce a, model of an individual's face that can

bs recognized by clore friends or family members Research presented by Wilkin- son [Wil04] shows that hair, face outline, eyes, nose and mouth are the important

features for the recognition of faces

2.5 Soft tissuc thickness studics 13

2.4.1 Ears

Wilkinson [Wild] stated that we cannot estimate the shape of the ears including their size, form, and projection based on the skull only, because there are no un derlying hone that describes the ear appearance Ilowever, the ear shape is not an

important feature in our face

2.4.2 Eyes

‘Ihere are some researches which show the eyes relationship Firstly, for the depth placument of the vyebull in the sockel, Wilkinson [Wil04] indicates that a straight

line Lelween Ube superivr an d inferior orbital margins will touch the front of the

cornea He also states that Une opening of the eye is G0 Lo 80 percent of the width

2.5 Soft tissue thickness studies

As meutioned, Une soll Lissue Uniekness dute play an important roic in [avial recon slruction, whutever our method is The issue of collecting soft Lissuc thickneys date

to clarify the relationship between soft tisene and the imderlying bony acrncrure of slenfl has been discussed hy forensic medicine experts for more than a himdred years

Tn 1883, Welcker [Wel33] obtained a datahase of soft tissue thicknesses hy inserting a thin blade into facial skin of cadavers at selected anatomical landmarks After that,

he measured the depth of the blades penetration Until middle 1980, all studies that

14 Chapter 2, Background

need to collect soft tissue thicknesses data at anatomical landmarks used eadaverous papulations and this ‘needle technique’ However, this type of approaches has some problems First of all, a dead person's tissues are not the same as in life clue to drying und cmbslning Secondly, the skin enn be deformed due lo the peneteution

of Ube necdle, Lastly, ib is hard to find the landinurks voreccly through soll Gssue when perlurming the needle insertion Since we nced to produce a model ay accurate

aa possible, all these matrers nmust: be taken into consideration

The ‘necdle technique’ cannot be used on living subjects, which leads to cr rors in measuremem Afler 1980, with the development of technolugy, non-invasive

medical systems become popular A variety of methods have been used to mea-

sure tissne depth in living subjects, including ultrasonnd, MRI, and CT In 1987,

George [Geo87] used lateral craniographs to record the depths of tissue at the midline anthropometric points And in 2000, Manhein et al [MLB*00] used ultrasound to

collect information for sample of children and adults of sexes, varying ages and differ

ent varieties El-Mchallawi and Soliman EMS01] and De Greef ct al [DOPV'! 06] also uscd ultrasound to conduct study In 2002, Sahni ct ai [SJG'02] uscd MRI

to oblain tissue depth dala of Indians, The most accurate measurement can be

oblained by using CT This techuique is faster and more avcurave as ib gives high quality images With the help of the computer, we can also construct the 3D model from the CT images In 1996, Phillips and Smuts [P$96] used CT technique to

obtain data of mixed population of South Africa There are many more related re-

searches that collect soft tissue thicknesses for study [lowever, most measurements

are collected from rather small populations due to the harm it may cause when tests

are carried out Ultrasound techniques sccm to be the most accurate and safe as it can be used without any considcrable throat for the candidate [Wil04].MRI has the advantage of collecting dala in 3D format, Soult tissue visualization ix excellent, but buuy lissue is not as well visualized as on the CT sean [VPSTT07] In addition, they

just gather tissue depths data at anthropometric landmarks, but give no informa-

tion abont any relationship between these depths and the skull shape Therefore, in most facial reconstruction systems, they just use the average thicknesses that can

be calculated from the database for every landmark

There are aorne researches which are available for Vietnamese These researches are made hy Le Vier Vung (2005), Xu Xuan Khoi (19963, Le Gia Vinh (2005), Pham

Huu Phung and Nguyen ‘Lrong ‘Ioan (2007) and are provided in form of average

values and variations bey have also made some conclusicns about Vietnamese

6 Available Soft ‘Tissue Thickness Data d5

reconstruction system wilh accurate soli tissue thickness prediction

2.6 Available Soft Tissue Thickness Data

There are several published soft tisene thickness data collections Some datasers

of American Blacks and American Whites are provided by Nhine “Rhidd] ‘hese datasets are divided into groups of different sex and weight, and show the average soft tissue thicknesses at Ithine's landmark for each population in each group Many Inter facial reconstruction systems used these date collections as the method to define the tissue depth However, these datascts were obtained from cadavers that they suller the disadvantuges described above

Tn 2000, Manhein et al [MIB*00] pnblished a study made on American Blacks and Cancasian Americans using the ultrasound technique This data is divided into

groups of age, with the landmarks similar to the ones made by Rhine However, the correctness of this data is considered higher than Rhine’s due to the method of

ultrasound in obtaining

The latest datasct scoms to be the T-tables (Tallied Facial Soft Tissue Depth

Data) provided by Stephan [$C10] The T-tables represent pooled soft tissue depth incans from many of previously published studies They started in 2008 and are well

updated up to now The T-tables provide three sets of soft tissue thickness data

with different age range, 0 to L! years, 12 to L7 years, and (8 years and heyond

Tn canrrast to any single soft tissne depth stndy that typically inclndes fewer than

AQ individuals, each of the T-rables report values for more than 3000 individuals

‘Uherefore, the ‘I-tables have an advantage that tolerates the measurement error in single study [lowever, some researches [ILW85] [Dum¥6] have shown that race,

sex, age and weight have moderate impacts on soft tissuc thickness The T-tables

divide date into groups of age only, which make it hard to use In addition, the data provided is the already averaged one, so we cannot separate the data into different groups.