Towards adversarial attack against embedded face recognition systems

VIETNAM NATIONAL UNIVERSITYHO CHI MINH CITY UNIVERSITY OF TECHNOLOGY FACULTY OF COMPUTER SCIENCE AND ENGINEERING BACHELOR THESIS Towards Adversarial Attack against Embedded Face Recognit

VIETNAM NATIONAL UNIVERSITY

HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY FACULTY OF COMPUTER SCIENCE AND ENGINEERING

BACHELOR THESIS

Towards Adversarial Attack against

Embedded Face Recognition Systems

Major: Computer Engineering

Committee: Computer EngineeringSupervisors: Dr Le Trong Nhan

Assoc Prof Quan Thanh ThoReviewer: Assoc Prof Tran Ngoc Thinh

—o0o—

Authors: Nguyen Minh Dang - 1752170

Nguyen Tien Anh - 1752076Tran Minh Hieu - 1752199

Ho Chi Minh City, July 2021

ĐẠI HỌC QUỐC GIA TP.HCM CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM

TRƯỜNG ĐẠI HỌC BÁCH KHOA

KHOA:KH & KT Máy tính _ NHIỆM VỤ LUẬN ÁN TỐT NGHIỆP

BỘ MÔN: KHMT Chú ý: Sinh viên phải dán tờ này vào trang nhất của bản thuyết trình

HỌ VÀ TÊN: Nguyễn Minh Đăng MSSV: 1752710 NGÀNH: KTMT LỚP: _

HỌ VÀ TÊN: Trần Minh Hiếu _MSSV: 1752199 NGÀNH: KTMT LỚP: _

HỌ VÀ TÊN: Nguyễn Tiến Anh MSSV: 1752076 NGÀNH: KTMT LỚP: _

1 Đầu đề luận án:

Towards Adversarial Attack against Embedded Face Recognition Systems

2 Nhiệm vụ (yêu cầu về nội dung và số liệu ban đầu):

✔ Investigate face authentication techniques

✔ Research and design the desired system based on NVIDIA Jetson Nano Developer Kit

✔ Research and propose an approach to apply adversarial attack technique to prevent attacker

to fool the system

✔ Implement a prototype and evaluate the performance

3 Ngày giao nhiệm vụ luận án:

4 Ngày hoàn thành nhiệm vụ:

5 Họ tên giảng viên hướng dẫn: Phần hướng dẫn:

1) Lê Trọng Nhân

2) Quản Thành Thơ

3) Nội dung và yêu cầu LVTN đã được thông qua Bộ môn

Ngày tháng năm

PGS.TS Quản Thành Thơ

PHẦN DÀNH CHO KHOA, BỘ MÔN:

Người duyệt (chấm sơ bộ): _

Ngày bảo vệ: Điểm tổng kết: _Nơi lưu trữ luận án: _

TRƯỜNG ĐẠI HỌC BÁCH KHOA CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM

KHOA KH & KT MÁY TÍNH Độc lập - Tự do - Hạnh phúc

-Ngày tháng năm

PHIẾU CHẤM BẢO VỆ LVTN

(Dành cho người hướng dẫn/phản biện)

1 Họ và tên SV: Nguyễn Minh Đăng

2 Đề tài: Towards Adversarial Attack against Embedded Face Recognition Systems

3 Họ tên người hướng dẫn/phản biện: PGS.TS Quản Thành Thơ

4 Tổng quát về bản thuyết minh:

6 Những ưu điểm chính của LVTN:

- The students addressed an emerging security problem in the area of face recognition The solution proposed by students include a selection of suitable hardware device and especially

an AI approach for black-box adversarial attack, whose performance overcomes the current state-of-the-art results To achieve this, the students has conducted a very insightful

literature review, gradually elaborated their suggested architecture and successfully

implemented their models with impressive performance

- The work in this thesis has been publish in two papers, one in a student scientific conferenceand especially in prestigious international conference, whose proceedings are published by Springer This should illustrate excellent result of the students’ work

7 Những thiếu sót chính của LVTN:

8 Đề nghị: Được bảo vệ  Bổ sung thêm để bảo vệ  Không được bảo vệ 

9 3 câu hỏi SV phải trả lời trước Hội đồng:

a

10 Đánh giá chung (bằng chữ: giỏi, khá, TB): Điểm : 10 /10

Ký tên (ghi rõ họ tên)

PGS.TS Quản Thành Thơ

VT姶云PI"A萎K"J窺E"DèEJ"MJQC E浦PI"JñC"ZÊ"J浦K"EJ曳"PIJ C"XK烏V"PCO

KHOA KH & KT MÁY TÍNH A瓜e"n壱r"- V詠"fq"- J衣pj"rj¿e

- Ngày 08 tháng 08 p<o""4221

RJK蔭W"EJ遺O"D謂Q"X烏"NXVP

*F pj"ejq"pi⇔ぜk"rjＶp"dkうp)

30"J丑"x "v‒p"UX< Nguyen Minh Dang MSSV: 1752170

J丑"x "v‒p"UX<"Nguyen Tien Anh MSSV: 1752076

J丑"x "v‒p"UX<"Tran Minh Hieu MSSV: 1752199

Ngành (chuyên ngành): M悦"vjw壱v Máy Tính

40"A隠"v k< Towards Adversarial Attack against Embedded Face Recognition Systems

50"J丑"v‒p"pi逢運k"rj違p"dk羽p: Assoc Prof Dr Vt亥p"Pi丑e"Vj鵜pj

60"V鰻pi"sw v"x隠"d違p"vjw{院v"okpj<

U嘘"vtcpi< 83 U嘘"ej逢挨pi< 6

U嘘"d違pi"u嘘"nk羽w:10 U嘘"j·pj"x胤< 35

U嘘"v k"nk羽w"vjco"mj違q< 104 Rj亥p"o隠o"v pj"vq p<

Jk羽p"x壱v"*u違p"rj育o+: 01 Adversarial attack system on Jetson Nano

70"V鰻pi"sw v"x隠"e e"d違p"x胤<

c They deployed a face recognition system on a Jetson Nano and proved it to work well

d 01 paper has been accepted by The 4th International Conference on Multimedia Analysis and Pattern Recognition (MAPR 2021)

Declaration of Authenticity

We hereby declare that this thesis titled "Towards Adversarial Attack against ded Face Recognition Systems" and the work presented in it are our own We confirmthat:

Embed-• This work was done wholly or mainly while in candidature for a degree at thisUniversity

• Where any part of this thesis has previously been submitted for a degree or anyother qualification at this University or any other institution, this has been clearlystated

• Where we have consulted the published work of others, this is always clearly tributed

at-• Where we have quoted from the work of others, the source is always given Withthe exception of such quotations, this thesis is entirely our own work

• We have acknowledged all main sources of help

• Where the thesis is based on work done by ourselves jointly with others, we havemade clear exactly what was done by others and what we have contributed ourselves

Ho Chi Minh City, July 2021

Firstly, we would like to show our deepest gratitude to our supervisors, ProfessorQuan Thanh Tho and Dr Le Trong Nhan, for their invaluable time, patience, and warmsupport They have spent so much effort guiding us, and their insightful feedback hashelped us realize the weaknesses in our work Furthermore, their enthusiasm has been anencouragement to help us move forward during the difficult stage of our research With-out the help from them, this thesis could not have come to reality

Secondly, we want to thank all the lecturers for all the knowledge and skills they vided us in the past four years Thank HCMC University of Technology and the Faculty

pro-of Computer Science and Engineering for creating such a wonderful incubating ment that has helped us grow as students as well as individuals

environ-Finally yet importantly, we thank our beloved friends and family for their immenseamount of love, support, and encouragement throughout the years

It has been an incredible journey, we wish you all good health and happiness in life

Nguyen Minh Dang, Nguyen Tien Anh, Tran Minh Hieu

Numerous studies have shown that deep neural networks (DNNs) are vulnerable toadversarial examples - malicious inputs that are carefully crafted to cause a model tomisclassify This phenomenon raises a serious concern, especially for Deep learning-basedsecurity-critical systems such as face recognition However, most of the studies on the ad-versarial vulnerability of DNNs have only considered the ideal scenarios (e.g., they assumethe attackers have perfect information about the victim model or the attack is performed

in the digital domain) As a result, these methods often poorly (or even impossible to)transfer to the real world and hamper future studies on defense mechanisms against real-world attacks To address this issue, we propose a novel physically transferable attack

on deep face recognition systems Our method can work in the physical world settingswithout requiring any knowledge about the victim model Our extensive experiments onvarious model architectures and training losses show non-trivial results and give rise tosome interesting observations that can be a potential research direction in the future toimprove the robustness of models against adversarial attacks

1.1 Overview 2

1.2 Thesis Scopes and Objectives 5

1.3 Our contributions 6

2 Background Knowledge 7 2.1 Deep Learning and Neural Networks 8

2.1.1 Artificial Neural Networks 8

2.1.2 Convolutional Neural Networks 10

2.2 Optimization Techniques 11

2.3 Face Recognition 13

2.4 Adversarial Machine Learning 15

2.4.1 Adversarial Examples 16

2.4.2 Properties of Adversarial Examples 16

2.4.3 A Taxonomy of Adversarial Attacks 17

2.4.4 Generating Adversarial Examples 21

2.5 Jetson Nano 22

2.5.1 Developer kit and Hardware 22

2.5.2 JetPack and libraries 24

3 Literature Review 28 3.1 Black-box adversarial attacks 29

3.1.1 Decision-based adversarial attacks: Reliable attacks against

black-box machine learning models 29

3.1.2 Efficient Decision-based Black-box Adversarial Attacks on Face Recog-nition 31

3.2 Adversarial attacks in the physical world 32

3.2.1 Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition 33

3.2.2 AdvHat: Real-world adversarial attack on ArcFace Face ID system 34 4 Methodology 35 4.1 Threat Model 36

4.2 Baseline Method 36

4.3 From Digital to Physical World Attack 38

4.4 Enhancing the Transferability of Transfer-based Attacks 39

5 Experiments 41 5.1 Experimental Settings 42

5.1.1 Datasets 42

5.1.2 Pre-trained Models 43

5.1.3 Evaluation Metric 44

5.1.4 Physical Evaluation 45

5.2 Experimental Results 46

5.2.1 Attack success rates in the physical world 46

5.2.2 Performance comparisons between digital and physical world 49

5.2.3 Sensitivity to epsilon and the number of ensemble models 50

5.2.4 Extended experiments on local adversarial attacks 51

5.2.5 Evaluation on NVIDIA Jetson Nano Embedded System 55

6 Conclusion and Future Works 57 Bibliography 67 Appendices 68 A FaceX-Zoo and LFW Dataset 69 A.1 Preparation and dependencies 69

A.2 Face cropping 70

A.3 Pre-trained models 70

A.4 LFW Dataset 71

B Deploying Face Recognition on NVIDIA Jetson Nano 72 B.1 Prerequisite and installation guide 72

B.1.1 Hardware requirements 72

B.1.2 Software dependencies 73

B.2 System descriptions 74

B.2.1 Face Detection 74

B.2.2 Face Representation 75

B.3 Evaluation 76

List of Figures

1.1 An overview of our work Given a black-box face recognition model, a pair

of source and target images, we aim to generate an adversarial noise that, when added to the source image, causes the model to misclassify them as

belonging to the same identity 4

2.1 A simple deep neural network with 2 hidden layers 8

2.2 Popular activation functions 9

2.3 Architecture of the LeNet-5 network 10

2.4 Gradient Descent in the physical view 12

2.5 A standard pipeline of an end-to-end face recognition system (Du et al., 2021) 13

2.6 Adversarial examples (Goodfellow et al., 2015) 15

2.7 An illustration of accessible components of the target model for each of the three threat models A white-box threat model assumes access to the whole model; a score-based threat model assumes access to the output layer; a decision-based threat model assumes access to the predicted label alone (J Chen et al., 2020) 18

2.8 Adversarial examples in the physical world 19

2.9 An example of local adversarial attack (Brown et al., 2018) 20

2.10 Jetson Nano Developer Kit 22

2.11 Developer kit module and carrier board (Nvidia, 2020) 23

2.12 TensorRT workflow 26

3.1 Basic intuition of the Boundary attack 30

3.2 An example of the targeted attack, along with the number of model calls 30

3.3 An example of the dodging(untargeted) attack and impersonate(targeted) attack 31

3.4 Attackers with the adversarial eye-frames 33

3.5 Adversarial stickers 34

5.1 Image preprocessing for the LFW dataset The first row shows the original images, and the second row shows the images after preprocessing 42

5.2 An example of our rebroadcast process We display the generated adver-sarial examples onto a monitor, then capture them using another device 45

5.3 Digital and Physical ASR backbone-wise in global setting 49

5.4 Digital and Physical ASR head-wise in global setting 49

5.5 ASR with respect to ε values in global setting 50

5.6 ASR with respect to the number ensemble models in global setting 50

5.7 Global and local perturbations of an image 51

5.8 Eye-glasses shape 51

5.9 Global and Local ASR backbone-wise in digital setting 52

5.10 Global and Local ASR backbone-wise in physical setting 53

5.11 Global and Local ASR head-wise in digital setting 53

5.12 Global and Local ASR head-wise in physical setting 54

5.13 ASR with respect to ε 54

5.14 ASR with respect to the number of ensemble models 54

A.1 FaceX-Zoo on Github 69

A.2 Backbone-wise models in FaceX-Zoo 70

A.3 Head-wise models in FaceX-Zoo 71

B.1 Pipeline of the cascaded framework that includes three-stage multi-task deep convolutional networks 74

B.2 FaceNet high-level model structure 75

B.3 Triplet loss intuition 76

List of Tables

2.1 Notable loss functions for Face Representation 152.2 Nvidia Jetson Nano module technical specification 232.3 Nvidia Jetson Nano carrier board components 24

5.1 Model accuracy evaluates on the LFW test set and their correspondingbest cosine similarity threshold 445.2 Backbone-wise ASR results ASR Baseline is the mean ASR when attackusing the baseline method ASR with M-E is the mean ASR when attackusing our method without including Diverse Input method ASR withM-E-DI is the mean ASR when attack using our method 475.3 Head-wise ASR results ASR Baseline is the mean ASR when attackusing the baseline method ASR with M-E is the mean ASR when attackusing our method without including Diverse Input method ASR withM-E-DI is the mean ASR when attack using our method 475.4 Head-wise ASR results ASR Baseline is the mean ASR when attackusing the baseline method ASR with M-E is the mean ASR when attackusing our method without including Diverse Input method ASR withM-E-DI is the mean ASR when attack using our method 485.5 Source - Target and Adversarial - Target l2 distance in global adversarialattack 555.6 Source - Target and Adversarial - Target l2 distance in local adversarialattack 55

B.1 Libraries/packages specifications 73B.2 The deployed system evaluation 77

List of Notations

Aij Matrix indexed for some purpose

Ai Matrix indexed for some purpose

Aij Matrix indexed for some purpose

An Matrix indexed for some purpose or the n-th power of a square matrix

A−1 The inverse matrix of the matrix A

A+ The pseudo inverse matrix of the matrix A

A1/2 The square root of a matrix (if unique), not elementwise

(A)ij The (i, j)-th entry of the matrix A

Aij The (i, j)-th entry of the matrix A

[A]ij The ij -submatrix, i.e A with the i-th row and j-th column deleted

a Vector (column-vector)

ai Vector indexed for some purpose

ai The i-th element of the vector a

a Scalar

det(A) Determinant of A

Tr(A) Trace of the matrix A

diag(A) Diagonal matrix of the matrix A, i.e (diag(A))ij = δijAij

eig(A) Eigenvalues of the matrix A

vec(A) The vector-version of the matrix A

kAk Matrix norm (subscript if any denotes what norm)

Chapter 1

Introduction

In this chapter, we give an overview of our thesis, define the thesis scopes, objectives, and summarize our contributions.

1.1 Overview

Deep learning is a branch of machine learning in which learning models are made up

of multiple layers The advent of deep learning has created numerous breakthroughs inhandling problems where traditional machine learning techniques perform poorly Com-puter vision tasks such as image classification (Dosovitskiy et al., 2020; Foret et al., 2020),object detection (Ghiasi et al., 2020; C.-Y Wang et al., 2020), face recognition (Deng,Guo, Zhou, et al., 2019; Schroff et al., 2015), semantic segmentation (Mohan et al., 2020;Yuan et al., 2020), and natural language processing tasks such as semantic analysis (Lan

et al., 2020; Raffel et al., 2020), question answering (Joshi et al., 2020; Yang et al., 2020),machine translation (Edunov et al., 2018; Zhu et al., 2020) Specifically, deep learning isthe dominant approach in many real-life applications such as virtual assistants (GoogleAssistant, Alexa, Siri), machine translation tools (Google Translate and IBM WatsonLanguage Translator), autonomous vehicles (Tesla, Audi, and BMW), or corporate facialrecognition systems to identify employees In recent years, deep learning has also beenapplied in highly complicated tasks such as analyzing the potential of drug molecules (Ma

et al., 2015), reconstruction of brain circuits (Helmstaedter et al., 2013), analyzing cle accelerator data (de Seixas T Ciodaro et al., 2012), effects of mutations in DNA (Lee

parti-et al., 2015) In addition, with the improvement in the computation power of hardwaresuch as GPU and TPU, the process of training and inference has become significantlysimpler and faster

The introduction of the Convolutional Neural Network (CNN) revolutionized deeplearning, especially in computer vision applications Specifically, for applications such asobject detection, image classification, face recognition, semantic segmentation the CNNnetwork has increased performance dramatically and has become the dominant approach.The CNN architecture is similar to the neurons’ communication pattern in the humanbrain and was influenced by the Visual Cortex organization Individual neurons respondonly in a small area of the visual field known as the receptive field to stimuli To coverthe entire visual region, a range of such fields overlap By applying appropriate filters, aCNN can successfully capture the spatial dependencies in an image The structure of aCNN network can model the image data set well thanks to the reduction in the number

of parameters involved and reusability of weights (Khan et al., 2020) In other words, thenetwork can be trained to understand the sophistication of the image better

In the aforementioned computer vision applications, face recognition, which is theprominent biometric technique for identity authentication, has been widely used in to-day’s fields such as military, finance, and information security Face recognition has longbeen a research topic in the machine learning community around the world since the1990s At that time, with traditional approaches such as holistic learning (Belhumeur

et al., 1997; Moghaddam et al., 1998), local handcraft (Chengjun Liu et al., 2002; chao Zhang et al., 2005), shallow learning (Cao et al., 2010; Lei et al., 2014), the achievedaccuracy is not high due to many reasons such as a lack of distinctiveness and compact-ness, limitation on robustness against the complex nonlinear facial appearance variations.However, thanks to deep learning and, in particular, CNN, the accuracy has improvedremarkably, and is comparable with human performance (Parkhi et al., 2015; Taigman

Wen-et al., 2014)

In recent years, several studies have shown that deep neural networks are vulnerable

to adversarial examples - malicious inputs that are carefully crafted to force the models

to make erroneous predictions (Goodfellow et al., 2015; Szegedy et al., 2014) Moreover,some adversarial examples are almost identical to the original images, making it difficult

to discern visually This raises a serious security concern especially when deep learninghave been widespread in everyday life applications

There have been several works study the adversarial vulnerability of deep face nition system (Erdogmus et al., 2013; Komkov et al., 2019) However, most of themhave only considered the ideal scenarios For example, (Sharif et al., 2016a) assumes thatthe attackers have perfect knowledge about the victim model, including its parameters,

recog-architecture, and gradients This type of attack is often classified as white-box attacks.

On the other hand, (Dong et al., 2019) proposed black-box attacks that do not requireany prior knowledge about the victim model, but they assume that the attack takes place

in the digital domain - where inputs are fed directly into the model (e.g., via an API)

This type of attack is also known as digital attacks Adversarial attacks in the white-box

or digital settings are relatively simple to achieve, but they are often ineffective or evenimpossible to be applied in real-world settings Firstly, attackers do not often have per-mission to obtain the model’s internal configurations Most of the time, only the labelpredicted by the model is accessible to the attacker Thus, attack in such limited settings

Figure 1.1: An overview of our work Given a black-box face recognition model, a pair ofsource and target images, we aim to generate an adversarial noise that, when added tothe source image, causes the model to misclassify them as belonging to the same identity.

is more challenging to achieve Secondly, real-world systems do not always provide anopen-access API, and the only way to attack a model is likely via a sensory input device(e.g., a camera) In the second case, the malicious input has to undergo two processes:

(1) digital-to-analog: attackers convert the generated adversarial example to the physical world; then (2) analog-to-digital : the model’s sensory input device reconstructs the phys-

ical adversarial example to the digital domain The above 2-step process is often referred

to as image rebroadcasting (Agarwal et al., 2018), and it has been shown to diminish the

effectiveness of adversarial examples due to environmental factors such as a change inlighting, contrast, and distance to the camera (Athalye et al., 2018)

In this work, we aim to investigate the vulnerability of deep face recognition systems in

a more realistic scenario That is, we assume that (1) the attackers only have access to themodel’s hard-labels outputs without any knowledge of its internal configurations; (2) theattack takes place in the physical domain Figure 1.1 illustrates what we aim to achieve.Given a black-box face recognition model, a pair of images from different people, we aim

to generate an adversarial noise that causes the model to misclassify them as belonging to

the same identity To tackle this problem, we propose a novel physical transferable attack method that can work without prior knowledge about the victim model, and subsequently,

the produced adversarial examples remain effective in the physical domain Most tantly, our method is efficient since it does not require any query to the victim model to

impor-generate adversarial examples We perform extensive experiments on the Labeled Face in

the Wild (LFW) dataset (G B Huang et al., 2007a), one of the most popular benchmark

datasets of face recognition tasks In addition, we evaluate our method on various trained state-of-the-art face recognition models with different architectures and training

pre-losses The pre-trained models are provided from the open-source repository FaceX-Zoo

(J Wang et al., 2021)

Although studying new attack methods seems dangerous and harmful, we argue that

it has important scientific value Firstly, it helps us gain valuable insight into how DNNsworks (Ilyas et al., 2019; Schmidt et al., 2018) Secondly, it serves as a base for furtherstudies on defense strategies to make deep face recognition systems more secure Finallyyet interestingly, adversarial attacks also have practical real-world applications, for exam-ple, in enhancing the security of CAPTCHAs (Shao et al., 2021) or protecting individual’sprivacy (Wu et al., 2020)

1.2 Thesis Scopes and Objectives

In this work, we aim to propose an adversarial attack algorithm against face nition systems in targeted physical black-box setting The face recognition systems weconsider in our thesis are state-of-the-art Deep Learning-based models trained with stan-dard training procedure in H Wang et al., 2018

recog-This work does not include face recognition systems equipped with an anti-spoofingmodule We also do not aim to propose a defense mechanism against our attack since it

is beyond the scope of our interest and adversarial defense is currently one of the mostchallenging unsolved problem (Carlini et al., 2019)

For concreteness, the goals of our thesis includes:

• Propose a targeted physical black-box attack algorithm on face recognition systems

• Evaluate the proposed attack on various model architectures and training losses

• Demonstrate the effectiveness of the proposed attack on an embedded face tion system

recogni-1.3 Our contributions

In summary, the main contributions of this thesis are as follows:

• We propose a novel attack algorithm on face recognition systems that works reliably

in the physical world without requiring any knowledge about the victim model norprobing the model’s outputs

• We thoroughly evaluate our method on various model architectures and traininglosses The results shows superior attack success rate against the baseline

• We demonstrate the effectiveness of our method by attacking a real-world embeddedface recognition system

• Parts of this thesis have been published at the 4th International Conference onMultimedia Analysis and Pattern Recognition (MAPR 2021) and the 8th Scienceand Technology Symposium for OISP Students at Ho Chi Minh City University ofTechnology

The remainder of our thesis is organized as follows Chapter 2 is a revision of eral foundation knowledge used in our thesis, including deep neural networks, deep facerecognition models, adversarial machine learning, and the taxonomy of adversarial at-tacks In chapter 3, we summarize recent related works on adversarial attacks againstface recognition models We analyze the weaknesses of the proposed methods and clarifythe distinctions between our work and previous works Chapter 4 presents our methodin-depth and explains how we approach the problem of facial adversarial attack in physicalblack-box settings In chapter 5, we describe our experiment setups and the results Here,

sev-we also give a discussion on several interesting observations from the results Finally, inchapter 6, we summarize our work, identify the limitations, and suggest several potentialresearch directions in the future

Chapter 2

Background Knowledge

In this chapter, we revise several background knowledge used in our thesis ing deep neural networks, optimization techniques, face recognition, adversarial machine learning, and NVIDIA Jetson Nano Developer Kit.

includ-2.1 Deep Learning and Neural Networks

2.1.1 Artificial Neural Networks

Figure 2.1: A simple deep neural network with 2 hidden layers

In the past two decades, neural networks have achieved remarkable results in manymachine learning tasks Due to their enormous potential of being a universal functionapproximator, neural networks can be used for a variety of computer vision tasks, such

as object classification, autonomous driving, and face recognition

Neural networks can be used to represent the relationship between a pair of inputand output by a complex function Figure 2.1 represent a simple neural network, whichconsists of three types of layers: one input layer, one output layer and a number of hiddenlayers The number of hidden layers in a neural network can range from 0 to as many as

possible For this reason, they are also called deep neural networks (DNNs) The total

number of layers in a neural network is usually denoted by L, which is derived by takingthe number of hidden layers and add one more, since we exclude the input layer Each

layer is made up of a number of neurons, which takes a vector of fixed length as input,

and produce a vector of fixed length as output Let’s denotes a(l−1) ∈ Rm the input ofthe layer l-th, and a(l) ∈ Rn the output of that layer, the forward propagation can beexpressed as followed:

a(l) = σ(l)(W(l)a(l−1)+ b(l)) (2.1)ˆ

Figure 2.2: Popular activation functions

where a(L) is the output of the final layer and ˆy is the model’s output, W(l)∈ Rn×mand b(l) ∈ Rn are the weight matrix and bias vector of appropriated dimension σ(l) are

non-linear functions that served as activation functions Activation functions have to be

non-linear If we allow linear transformation in the neural network, the output would be

a linear transformation of the input, which is not enough to form a universal functionapproximator (Leshno et al., 1993) Activation functions are an essential part of deeplearning Since they decide if a neuron should be activated, they affect a neural network’s

output, accuracy, convergence, and computational efficiency Some common element-wise activation functions are the logistic sigmoid function (sigmoid ), the hyperbolic tangent function (tanh) and the rectified linear unit function (ReLU ) Their mathematical formu-

lation are shown below and Figure 2.2 represents their graph

range of [−1, 1] However, both sigmoid and tanh often suffer from vanishing gradient

(the phenomenon where a deep neural network is unable to propagate useful gradientinformation from the output end of the model back to the layers near the input end ofthe model) A solution to this problem is by using the ReLU activation function, which

is linear for values greater than zero, meaning it has a lot of the desirable properties

of a linear activation function when training a neural network using backpropagation(Goodfellow et al., 2016) Yet, it is a nonlinear function as negative values are alwaysoutput as zero

2.1.2 Convolutional Neural Networks

Convolutional Neural Network (CNN) has become one of the most representativenetwork in deep learning (Li et al., 2020) Deep learning models based on CNNs haveachieved remarkable results in many areas, including but not limited to computer visionand natural language processing Hence, CNNs have become the dominant approach forapplications that involve computer vision or visual pattern recognition tasks (He et al.,2015; Krizhevsky et al., 2012) It was first introduced by Yann Lecun in (LeCun et al.,1990) and was later improved in (LeCun et al., 1998) by the same author In this section,

we will briefly review some of the basics of CNN

Figure 2.3: Architecture of the LeNet-5 network

CNNs are suitable for exploiting context information from the input Figure 2.3 shows

an example of a CNN, the LeNet-5 network (LeCun et al., 1998) A convolutional neural

network usually consist of convolutional layers, intertwined with pooling layers and maybe one or more fully-connect layers at the end The convolutional layer is used to learn the

representations of the inputs The output of convolution can be called feature maps.Convolutional layers can be composed of several convolution kernels to produce differentfeature maps A new feature map is obtained by first convolving the input with a learnedkernel, then applying an element-wise nonlinear activation function Note that the kernel

is shared by all spatial locations of the input This weight-sharing mechanism helpsreducing the model’s complexity and parameters, and avoiding overfitting Let’s denotethe p-th feature map at the output of the l-th layer by the matrix by H(l)p Kernels of

the convolutional layer can be represented by the form of a 4-dimensional kernel tensor

W(l), along with a 3-dimensional bias tensor B(l), are the parameters of the convolutionallayer Let W(l)pq and B(l)p be 2-dimensional slices of the kernel and bias tensors, the output

of the l-th convolutional layer can be expressed as:

The pooling layer reduces the dimension of the input feature maps This helps CNN

to achieve shift-invariance, i.e., robust to spatial displacements in the input and to avoidoverfitting It is usually placed between two convolutional layers Some common poolingoperations are average pooling (T Wang et al., 2012) and max pooling (Boureau et al.,2010) By stacking several convolutional and pooling layers, we could extract higher-levelrepresentations of the inputs Empirical studies have shown that kernels at the first fewlayers can extract low-level features such as lines and curves, while deeper layers are able

to encode more complex and abstract features (Zeiler et al., 2013)

The features at the final layer are then fed into a fully-connected network They takeall the neuron of the previous layer and connect them to every single neuron of the currentlayer to perform global senmatic information

2.2 Optimization Techniques

In Mathematical Optimization or Machine Learning, we often have to find the est or the maximum value of a function In general, finding the global minimum of lossfunctions in Machine Learning is very complicated, even impossible Instead, people oftentry to find the local minimum points, and to a certain extent, consider it the solution to

small-be found in the problem

The most common approach is to start from a point that we consider to be close

to the solution and use an iterative operation to progress to the desired point until thederivative is close to zero Gradient Descent (GD) and its variations are among the mostused methods

Suppose we need to find the global minimum for a loss function L(θ) where θ is avector, often used to denote a set of parameters of an optimal model The GD algorithmstarts with a prediction point θ0, then, in the tth loop, the update rule is:

Figure 2.4: Gradient Descent in the physical view

We can think more physically that if the ball’s velocity at point D is large enough,then when it reaches point C, the momentum can help it pass through the high pointand continue to drop to point B Therefore, Gradient Descent with Momentum (Qian,1999) helps find the optimal parameters not dropping into the inadequate local minimum

The requirement is to calculate the quantity so that it both carries information of theslope (i.e., the derivative) and carries information of the momentum, which is the previousvelocity Most simply, we can add (weighted) these two quantities to combines gradientand momentum:

where γ is a decay factor and usually takes a value of 0.9, vt−1 is the previous velocity,

∇θL(θ) is the slope of the previous point Then the new parameter is calculated as follow:

2.3 Face Recognition

Figure 2.5: A standard pipeline of an end-to-end face recognition system (Du et al., 2021)

A regular end-to-end deep face recognition system is made up of three parts: face tection, face preprocessing and face representation, as shown in Figure 2.5 Face detection

de-is the first procedure of the face recognition system It aims to find all the face regions

in a given image or video frames and provide bounding box coordinates (the box in which contains the face regions of the image) with a corresponding confidence score With the

development of deep learning, deep features have been extensively used in face detection,

such as MTCNN (K Zhang et al., 2016), Pyramidbox (Tang et al., 2018), RetinaFace

(Deng, Guo, Zhou, et al., 2019)

Then, the face preprocessing is proceeded to calibrate the faces to a canonical view and

crop them to a normalized pixel size There are two main processes in face preprocessing,

being face alignment and face frontalization (Du et al., 2021) Face alignment aims to use

spatial transformations to warp faces to a canonical position with reference of the faciallandmarks Hence, the step of finding facial landmarks from faces is important for face

alignment Furthermore, face frontalization is the process of synthesizing frontal facingviews of faces from non-frontal faces input.

The aligned and cropped faces are then mapped to a feature space via the face

rep-resentation module The mapping is trained so that images from the same identity will

have similar feature vectors while images from different identities will have their feature

vectors separated Finally, the similarity between source feature vectors versus a target

feature vector is measured via a distance metric (usually cosine similarity) If the

dis-tance surpasses a predefined threshold, the corresponding face image is considered thesame identity, otherwise, not the same identity

Early studies on face representation such as DeepFace (Taigman et al., 2014) and DeepID (Sun et al., 2014a) formulated the problem as a multi-class classification task, thus they employed the cross-entropy softmax loss as the loss function for the face repre-

sentation training phase However, soon after, researchers realized that the softmax loss isinsufficient to produce a discriminative representation of faces Since 2015, new classes of

training losses emerged, including Euclidean-distance-based loss, based loss and other variations of the softmax loss Characteristics and some examples of

angular/cosine-margin-the above loss functions are described in Table 2.1

Loss function type Descriptions

Euclidean-distance-based loss

These loss functions encourage discriminative features, inwhich the intra-class distance needs to be compact and theinter-class distance needs to be maximized Some examplesare DeepID2+ (Sun et al., 2014b), Triplet loss (Schroff et al.,2015), Contrastive loss (van den Oord et al., 2019)

angular/cosine-margin-based loss

These loss functions make the learned features seperatedmore stricly by introducing angular/cosine margins in be-tween them Some examples are L-Softmax loss (W Liu et al.,2017), AM-Softmax (F Wang et al., 2018a) , SphereFace(W.Liu et al., 2018), Arcface(Deng, Guo, Xue, et al., 2019)

softmax loss and its

variations

These loss functions is a improved version of the origin max loss, with the modification of features or weight nor-malizations Some examples are the L2-constrained Softmaxloss (Ranjan et al., 2017), Normface (F Wang, Xiang, et al.,2017), congenerous cosine (Y Liu et al., 2017)

soft-Table 2.1: Notable loss functions for Face Representation

2.4 Adversarial Machine Learning

Figure 2.6: Adversarial examples (Goodfellow et al., 2015)

Figure 2.6 illustrates the adversarial vulnerability of deep neural networks By adding

an imperceptible perturbations to the original image (on the left), the resulted image (on

the right) causes the classification model to produce erroneous prediction with ingly high confidence This phenomenon was first studied in Szegedy et al., 2014 Thepaper refer to these perturbed inputs “adversarial examples” This poses major securitychallenges for deep learning systems For computer vision systems, for example, a verysmall change in the input image can fool even the most advanced models (Kurakin et al.,2017a; Moosavi-Dezfooli et al., 2016); or with natural language processing systems, modi-fying a small phrase in a sentence can easily confuse Google’s negative comment detectionsystem (Hosseini et al., 2017) As a consequence, the topic of adversarial robustness hasgarnered a great deal of interest from researchers recently In this section, we will brieflyoverview adversarial machine learning including some definitions and several well-knownattack methods.

surpris-2.4.1 Adversarial Examples

In the above paragraph, we have informally defined an adversarial example x′ as aninput that is carefully manipulated by attackers to make the model produce incorrectpredictions Let denote f : Rm −→ {1 k} as a classifier mapping input vectors to adiscrete label set, x′ can be formally defined as:

2.4.2 Properties of Adversarial Examples

There are numerous debates on the origin of adversarial examples and their properties.Here we present several prominent properties of adversarial examples that have beenstudied in recent years:

• The transferability of adversarial examples: Adversarial examples are not arandom artifact of learning The same adversarial example can fool different modelstrained on different subsets of the same dataset (Szegedy et al., 2014)

• Adversarial examples exist due to the overly linear decision boundaries of currentstate-of-the-art models (Goodfellow et al., 2015).

• Adversarial examples do not scatter randomly in small pockets but rather exist inlarge, contiguous subspaces (Tramèr et al., 2017)

• Robustness against adversarial examples may be at odds with the standard accuracy

of deep models (Tsipras et al., 2019) Most approaches aim to robustify deep modelsoften come with a trade-off in standard accuracy

2.4.3 A Taxonomy of Adversarial Attacks

Adversarial attack is the process of generating adversarial examples Since adversarialexamples were first discovered in Szegedy et al., 2014, numerous different types of adver-sarial attacks have been proposed In this section, we introduce five of the most popularcategories of adversarial attacks

2.4.3.1 Categorize based on adversary’s phase of attack

• Poisoning attack: This type of attack, known as contamination of the trainingdata, occurs during the training phase of a machine learning model An adversarytries to poison the training data by injecting carefully design malicious samples,which can compromise the model’s performance (Biggio et al., 2013) Recent re-search has revealed that by manipulating the training data, one can even create

a "backdoor" inside the model, which can be activated on-command by attackers(Gu et al., 2019; Turner et al., 2019) This type of attack could appear when theadversary has access to the training data, e.g., via outsourced training

• Evasion attack: This is the most common type of adversarial attack that oftentakes place in the testing phase The attackers try to evade the system by inputtingperturbed samples to cause the model to misclassify or produce incorrect results.One notable example for this type of attack is in (Eykholt et al., 2018), the roadsign classifier gets confused just by sticking a few pieces of tape on the ’stop’ sign

2.4.3.2 Categorize based on adversary’s goal

• Untargeted Attack: The adversary tries to alter the output classification of aninput example to any class different from the original class For example, a legitimate

image of a "stop" sign will be predicted as any other class different from the class

of stop sign

• Targeted attack: The adversary tries to produce malicious inputs that force theoutput of the classification model to be a specific target class For example, anyinput image to the classification model will be predicted as a class of images having

"go" sign Targeted attacks are considered to be more difficult especially againstface recognition models since the perturbation directions are limited to just oneclass

2.4.3.3 Categorize based on adversary’s knowledge

Figure 2.7: An illustration of accessible components of the target model for each of thethree threat models A white-box threat model assumes access to the whole model; ascore-based threat model assumes access to the output layer; a decision-based threatmodel assumes access to the predicted label alone (J Chen et al., 2020)

• White-box attack: In a white-box setting, the adversary has perfect information

of the victim model, including its architecture, parameters, gradients, etc Theadversary can make full use of this information to produce adversarial examples

• Black-box attack: In contrast to the white-box attacks, the black-box setting

is considered when the information about the victim model is limited Black-boxattacks can be categorized further based on the amount of information available (seeFigure 2.7):

– Score-based black-box attack: Score-based black-box attacks are a relaxedform of true black-box where the attacker still has access to the class probability

distribution of the victim model.

– Decision-based black-box attack (or Query-based black-box attack):

In decision-based black-box attacks, the only information accessible is the label output of the model Several works have proposed decision-based attacks

hard-by probing the model’s outputs to estimate the gradient needed to calculatethe adversarial examples Current state-of-the-art decision-based attacks oftenrequire at least thousands of queries to complete (J Chen et al., 2020; Dong

et al., 2019)

– Transferable black-box attack: Transferable black-box attacks base on thetransferability of adversarial examples (see 2.4.2) The attacker first performswhite-box attacks on a surrogate model The generated examples are thentransferred to the victim model In this kind of attack, the attacker does notneed any information about the victim model nor require thousand of queries

to optimize adversarial examples

Compared to white-box attacks, black-box attacks are more practical and ably more difficult

consider-2.4.3.4 Categorize based on adversary’s domain

Figure 2.8: Adversarial examples in the physical world

• Digital attack: This type of attack occurs when the adversary can inject maliciousinput directly to a machine learning model in a digital domain, e.g., by submittingdigital images to a web-based system

• Physical attack: Inputs to a model are collected from sensory input devices, e.g.,through a phone’s camera or sensors Most of the studies so far fall into the category

of digital attack; however, (Kurakin et al., 2017b) has shown that even in thephysical world scenario, machine learning models are still vulnerable to adversarialexamples (see Figure 2.8) Physical attacks are generally more challenging to achievesince there are numerous uncontrollable environmental factors that may diminishthe effectiveness of adversarial examples (e.g., change in lighting, posing, cameraquality, etc.)

2.4.3.5 Categorize based on adversary’s perturbation space

Figure 2.9: An example of local adversarial attack (Brown et al., 2018)

• Global attack: In global attack, there is no limit to the region of adversarialperturbation The perturbation is optimized on the entire image One example ofthis type of attack is shown in Figure 2.6

• Local attack: In contrast, local adversarial perturbations are restricted to a defined region by the attacker (see Figure 2.9) Local adversarial examples are oftenused in physical world attack When comparing the difficulty between the two types,local attacks are more challenging depend on the area of perturbation The smallerthe area, the more difficult it is to attack (Brown et al., 2018)

pre-2.4.4 Generating Adversarial Examples

There are numerous attacking algorithms in the literature, here we present some ofthe pioneer works that have laid the foundation for this research direction and follow-upworks For an exhaustive survey on adversarial attacks, please refer to Chakraborty et al.,2018

2.4.4.1 L-BFGS

The work of (Szegedy et al., 2014) is the first to attack deep neural network imageclassifiers They formulate their optimization problem as a search for minimal distortedadversarial example x′, with the objective:

minimize kx − x′k22subject to f (x′) = t and x′ ∈ [0, 1]m (2.11)

The problem is approximately solved by introducing the loss function, which results inthe following objective:

minimizec kx − x′k22+ L(θ, x′, t))

In the optimization objective of this problem, the first term imposes the similarity between

x′ and x The second term encourages the algorithm to find x′which has a small loss value

to label t, so the classifier f is very likely to predict x′ as t By continuously changingthe value of constant c, they can find an x′ which has minimum distance to x, and at thesame time fool the classifier f To solve this optimization problem, they implement theL-BFGS algorithm (D Liu et al., 1989)

2.4.4.2 Fast Gradient Sign Method

One disadvantage of the L-BFGS method is its expensive computation cost, whichwould take a long time to produce one adversarial example The work in (Goodfellow

et al., 2015) has proposed a one-step method to generate adversarial examples faster Theformulation is:

x′ = x + ǫsign(∇xL(θ, x, y)) non-target

x′ = x − ǫsign(∇xL(θ, x, t)) target on t (2.13)

For a targeted attack setting, this formulation can be seen as a one-step of gradient descent

to solve the problem:

minimize L(θ, x′, t)subject to kx′− xk∞ ≤ ǫ and x′ ∈ [0, 1]m (2.14)

The objective function in (2.14) searches the point which has the minimum loss value tolabel t in x’s ǫ-neighbor ball, which is the location where model f is most likely to predict

it to the target class t In this way, the one-step generated sample x′ is also likely to foolthe model An example of FGSM-generated samples on ImageNet is shown in Figure 2.6.Because FGSM runs only one backpropagation step, the produced adversarial exam-ples may not be robust in many cases However, stronger adversaries can be generated

by applying multiple iterations of FSGM (Kurakin et al., 2017a, 2017b)

2.5 Jetson Nano

2.5.1 Developer kit and Hardware

The NVIDIA Jetson Nano Developer Kit in Figure 2.10 is an AI computer for makers,learners, and developers that brings the power of modern artificial intelligence to a low-power, easy to-use platform The power of Jetson Nano comes from the ability to let usersrun multiple neural networks in parallel for applications like image classification, objectdetection, segmentation, and speech processing (“Jetson Nano Developer Kit”, 2021)

Figure 2.10: Jetson Nano Developer Kit

Component Description

Memory 4 GB 64-bit LPDDR4 25.6 GB/s

MicroSD card use for main storage

Passive heatsink supports 10W module power usage at 25°C ambient temperature

Table 2.2: Nvidia Jetson Nano module technical specification

Figure 2.11: Developer kit module and carrier board (Nvidia, 2020)

Figure 2.11 shows the interfaces of the Jetson Nano Developer kit module The velopment kit is a carrier board (with all the ports) holding the attached module Thereare some highlight interfaces of the Jetson Nano Developer Kit that are listed Table 2.2show the technical specification of the Jetson Nano module, and Table 2.3 presents thecarrier board’s component

de-Component Description

Power LED lights when the developer kit is powered on

Camera connector enables use of CSI cameras Jetson Nano

Devel-oper Kit works with IMX219 camera modules,including Leopard Imaging LI-IMX219-MIPIFF-NANO camera module and Raspberry Pi CameraModule V2

4-pin fan control header Pulse Width Modulation (PWM) output and

tachometer input are supportedM.2 Key E used for wireless networking cards; includes in-

terfaces for PCIe (x1), USB 2.0, UART, I2S, andI2C

2.1×5.5×9.5 mm plug with positive ity

polar-USB 3.0 Type A support up to 1A total power delivery All

con-nectors are connected to the Jetson Nano modulevia a USB 3.0 hub built into the carrier boardPower over Ethernet (POE) header exposes any DC voltage present on J43 Ethernet

jack per IEEE 802.3af8-pin button header brings out several system power, reset, and force

recovery related signals

3.3V serial port provides access to the UART console

Table 2.3: Nvidia Jetson Nano carrier board components

2.5.2 JetPack and libraries

NVIDIA JetPack SDK is the most comprehensive solution for building AI applications

It includes the latest OS images for Jetson products, along with libraries and APIs,samples, developer tools, and documentation JetPack includes an operating system and

reference file system derived from Ubuntu It also includes following libraries, APIs, andsample applications:

• TensorRT and cuDNN for high-performance deep learning applications

• CUDA for GPU accelerated applications across multiple domains

• Multimedia API package for camera applications and sensor driver development

• VisionWorks and OpenCV for visual computing applications

in optimized C/C++, the library can take advantage of multi-core processing

OpenCV is being used for a very wide range of applications which include:

• Street view image stitching

• Automated inspection and surveillance

• Robot and driver-less car navigation and control

• Medical image analysis

• Video/image search and retrieval

• Movies - 3D structure from motion

• Interactive art installations

2.5.2.2 TensorRT

The core of NVIDIA TensorRT is a C++ library that facilitates high-performanceinference on NVIDIA graphics processing units (GPUs) It is designed to work in a com-plementary fashion with popular training frameworks such as TensorFlow, Caffe, PyTorch,