STUDENT INTERNSHIP REPORT COMPUTER VISION ENGINEER

Tuyen Nguyen 4 Face Detector and Face Alignment Mr.. Tuyen Nguyen 6 Compare Euclidean distance and select threshold.. Pipeline project Face Detection Face Alignment Face Embedding

0

UNIVERSITY OF INFORMATION TECHNOLOGY

STUDENT INTERNSHIP REPORT

COMPUTER VISION ENGINEER

Student Internship : Le Thi Phuong Ngan

Ho Chi Minh, 14 – Jul - 2020

1

ABSTRACT

This report describes the activities carried out during a 8 weeks, final, fulltime internship program at the VCCorp

The document contains information about the organization and the responsibilities performed throughout the period between April and June 2020 More than a plain account of tasks, the objective of this text is to reflect upon the experiences collected during the internship from the perspective of a BSc student

The first part of the report offers an overview of the organization, followed by an outline

of all the duties carried out during this time Following, it proceeds to describe in some detail the most relevant projects carried out and their respective analysis Finally, the report wraps-up with a few closing remarks and conclusions from the experience

2

ACKNOWLEDGEMENT

First and foremost, I would like to say thanks to VCCorp for the opportunity given to me

as an intern in the Information technology section

Special thanks go to my academic supervisor Nguyen Thanh Anh Tuyen, head department Mr Huynh Ngoc Tin of AdTech-HCM team for their personal efforts, practical skills, professional guidance and direction towards successful internship

Finally, I would like to extend my heartfelt gratitude to Faculty of Software Engineer for their invaluable support throughout my training

Le Thi Phuong Ngan

Ho Chi Minh, 14 Jul 2020

3

FACULTY’S COMMENT

4

CONTENT

CHAPTER 1: WHERE I WORKED: VCCORP 5

CHAPTER 2: INTERNSHIP OVERVIEW ………6

1 Learn the company rules 6

1.1 Tools 6

1.2 Technical skills 6

2 Project implementation 6

3 Work schedule 6

CHAPTER 3: PROJECT 8

1 Introduce about project 8

2 Prepare data 8

2.1 Create list celebrity name 8

2.2 Collect data 9

3 Pipeline project 10

4 Face Detection 11

5 Face Alignment 11

6 Face Embedding 12

7 Select Threshold 13

8 Result 13

REFENCES……… ………14

CONCLUSION………15

5

CHAPTER 1: WHERE I WORKED: VCCORP

Founded in 2006, VCCorp Joint Stock Company (VCCorp) is a pioneer in the field of digital content and technology With more than 10 years of establishment and development, VCCorp has built a vast Internet ecosystem with a lot of creative and useful products in many fields (online advertising, e-commerce, online games , covering over 90% of Internet and mobile users, has contributed greatly to the development of Vietnam Internet in the past decade

To have a series of innovative and preeminent technology products today, VCCorp has built a strong workforce with over 1800 employees working and operating professionally in Hanoi, Ho Chi Minh City Chi Minh and other representative offices and branches in 06 most developed cities in Vietnam With the spirit of "Innovation-NonStop - Creative for the

community", VCCorp is a common home for young people who love technology and creative passion

6

CHAPTER 2: INTERNSHIP OVERVIEW

1 Learn the company rules

Time: 1 day

Contents: Introduce about the company, the organization of the company

Hear the person in charge of introducing the company, the process of establishment and development (as mentioned above), the process from high to low, the organization of the company

In addition, trainees are introduced to how to work in the company such as working time, the rules to comply, how to use email at work Tools and Technical skills 1.1 Tools

Time: 1 week

Contents:

During this time, the supervisor instructed trainees to learn about the tools that will help in the future Some of them are like VPN, Git,

1.2 Technical skills

Time: 5 days (1 week)

Contents: Keras, Linux, InsightFace,

2 Project implementation

After 2 weeks of training and practice, the trainee has mastered the basic knowledge about CNN, euclidean distance,… For the next 6 weeks, the trainer instructed the trainees to apply the knowledge they have learned to implement a celebrity identification project

Project details will be discussed in the following section

3 Work schedule

Comment

1

-Learn about the company

-Learn how to communicate, work via email

-Learn the tools that work in a company

Mr Tuyen Nguyen

2 Learn technical skills Mr Tuyen Nguyen

7

3

Create list celebrity name

Collect data

Mr Tuyen Nguyen

4 Face Detector and Face

Alignment

Mr Tuyen Nguyen

5 Get embedding of

training data

Mr Tuyen Nguyen

6

Compare Euclidean distance and select threshold

Mr Tuyen Nguyen

7 Test with test set and

verify data

Mr Tuyen Nguyen

8 Complete and optimize

code

Mr Tuyen Nguyen

8

CHAPTER 3: PROJECT

1 Introduce about project

Face recognition has become a common application in many current technologies Facebook has an auto-tagging feature on the face, the iPhone X uses face recognition to unlock, companies use facial recognition for attendance, and many security systems are also using the technology

In this project, I will work with a database of faces of famous people throughout Vietnam The problem is that from a given data set of 100 people, I need to build a model that predicts a new picture that corresponds to someone or from a completely new person

2 Prepare data

2.1 Create list celebrity name

Located on the same platform as Wikipedia is the Wikidata repository, a free, multilingual secondary database, specialized in structuring data collection to support other projects on the Wikimedia platform To get the Vietnamese name data on Wikipedia in the simplest way, you just need to go to Wikidata Query, perform 1 query as follows:

9

Result:

2.2 Collect data

After getting a list of people named on Wikipedia, I started to search their photos on Google Image Search Fortunately, this part of the script doesn't have to be crawled because I already have an open source called Google Images Download The use of open source is quite simple, just install the instructions at the home page of the package and run the following script:

10

For example, the image obtained by singer Toc Tien as shown below:

3 Pipeline project

Face

Detection

Face Alignment

Face Embedding

Select Threshold Save

embedding

11

4 Face Detection

The face detection algorithm that still uses most source face recognition is mtcnn because its accuracy is relatively fast However, because in the dataset there are many faces in the image are very blurry, bent down, so I chose the algorithm with higher accuracy is Single Stage Headless Face Detector Link https://github.com/deepinsight/mxnet-SSH

Pretrain model: https://github.com/deepinsight/insightface/wiki/Model-Zoo

5 Face Alignment

To align my face, I used the mtcnn algorithm but not the whole network but a part, mtcnn includes 4 consecutive subnets: Pnet, Rnet, Onet, Lnet I will not talk carefully about mtcnn model here because it is not the purpose of this article, I use 2 subnets behind Onet and Lnet, the purpose of these 2 networks is to detect 5 landmark and refine again However,

12

because of dataset nature, in Onet network I also reduced the threshold from 0.8 (origin) to 0.5 Mtcnn: https://github.com/deepinx/mtcnn-face-detection

Also can refer to some very good model align faces such as wingloss,

6 Face Embedding

I choose arcface algorithm as model face embedding (LResNet100E-IR, ArcFace @ ms1m-refine-v2) This is a pretrained model that has been trained with several million images

so the model has learned a very good (general) ability Retraining the model with this dataset I think will result in no good

Link: https://github.com/deepinsight/insightface

Pretrained model: https://github.com/deepinsight/insightface/wiki/Model-Zoo

Because I don't have the time and the power gpu base so I just use one model face embedding arcface Lresnet100 here and not ensemble model

Create embedding file:

With the photos in the training set without detecting the face, I will remove and do not use these embedding to classify the test set (1 wrong embedding in the train practice can lead to many mis-classified test photos and Reduce results by a lot) In addition, when creating the embedding train file to predict the test set, for each photo in the train set I augment by taking symmetrical image through the mirror (to select threshold, do not use this augment)

13

As for the images in the test set without detectable faces, I still create embedding for these images normally by processing images

7 Select Threshold

I use the pretrained facenet from this repo https://github.com/nyoki-mtl/keras-facenet

Calculate the embedding for each image, compare each embedding for each image in the test set with each group of embeddings belonging to the same person in the train set, find the shortest distance of each photo to each person

To insert class 1000 into predictions, I used a simple method of using thresholds

First I will draw the probability distribution of 2 variables:

The euclidean distance between 2 images if they belong to the same face

The euclidean distance between 2 images if they are on different sides

The goal is to find a suitable distance so that if the distance from one image to all images is higher than it, it will be unkown

I will choose the intersection between 2 sets ie 1.1

8 Result

Number of classes: 98

Accuracy: 95%

14

REFENCES

[1] Deep Face Recognition: A Survey - Mei Wang, Weihong Deng School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing, China

[2] MobileFaceNets: Efficient CNNs for Accurate RealTime Face Verification on Mobile Devices

15

CONCLUSION

This internship has been very rewarding for me on several aspects First the work I realized allowed me to go very deeply into machine learning/deep learning Thus it let me take great advantage of architecture course I had at the University

Also, working on my project is very interesting, first because it is not common to modify the network, and also because to do so we need a complete understanding of the hardware It is also interesting to how we can change the behavior of a program by doing that, and we need to

be extra careful not to change the logic of the program

Another aspect of the internship that fulfilled my expectation, even if it is not completely dependent of the internship itself, is that it allowed me to improve English’s skills Allowing me

to improve both my technical and daily life English

I would like to thanks especially Mr Huynh Ngoc Tin and Mr Nguyen Thanh Anh Tuyen for there help And of course there quiet impressive technical knowledge I want to thank Mr Nguyen Thanh Anh Tuyen for all his very helpful advises and especially about presenting and writing

I believe that if this experience has been as rewarding for me, Mr Huynh and Mr Nguyen are responsible of it for a very important part