DEEP WAVELET SPARSE AUTOENCODER TO REMOVE ELECTROOCULOGRAPHY

5.1. Conclusions This study was presented a new method that contribute to enhance efficiency of extract information or diagnose from EEG signal by remove EOG artifacts. Firstly, the definition of EEG, EEG artifacts and EOG artifacts is presented. After that, the next section is about Wavelet Transform as well as Haar Wavelet Transform, a type of Wavelet Transform. The next is Independent component analysis for removing artifacts. The end of the related work chapter is about Deep learning and Sparse Autoencoder. The proposed method to remove EOG artifacts is on the next chapter. Many experimental results and evaluation results indicate that the proposed method effectively remove EOG artifacts (even better than ICA JADE). 5.2. Future Works In the future, the proposed method can be used to in a BCI system that helps eliminate EOG artifacts in an online way. It means the signal after record from device is processed immediately and then display it on the screen. And whats more, expanding to handle more types of artifacts as well as comparing with more methods to improve the proposed method. From there, it is possible to help the EEG signal is cleaner before it is taken for analysis, help doctors to provide better analytical results or even help automatically analysis and evaluation of machines by using EEG signals become more reliable and more accurate.

DEEP WAVELET SPARSE AUTOENCODER TO

REMOVE ELECTROOCULOGRAPHY ACKNOWLEDGEMENT

Firstly, I would like to express my sincere gratitude to my supervisor Assoc Prof

Le Thanh Ha of University of Engineering and Technology, Viet Nam NationalUniversity, HaNoi for his instructions, guidance and his research experiences

Secondly, I am grateful to thank my co-supervisor M.Sc Nguyen The Hoang Anh

of Viet Nam Academy of Science and Technology for invaluable assistance during ourworking time

Additionally, I am grateful to thank all the teachers of University of Engineeringand Technology, VNU for their invaluable lessons which I have learnt

I also thank my friends in K59CA class, University of Engineering andTechnology, VNU

I greatly appreciate the helps and support from Human Machine InteractionLaboratory of University of Engineering and Technology during this project

The research direction to improve the quality of human life is always a hot issue andhighly appreciated The main objective of these studies is to help improve the livingenvironment of people, helping the essential needs and desires of people to be served fasterand more accurately without spending too much effort, cost Therefore, based oninformation about people's wishes, computers can provide appropriate solutions to serve.The system using information of the electroencephalogram (EEG) signal may be a solution

in this case because in theory the body's wishes, thoughts, and actions are derived from thebrain and at a specific time, brain signal is an expression of those desires, thoughts, andactions But extract the information from EEG signal is plagued by many kinds of noise Inthis work, I will introduce a new method of removing electrooculography (EOG) artifacts, atypical type of noise and difficult to control, in EEG signals This helps improve the ability

to extract information from the obtained EEG signal The results of the proposed method arecompared with independent components analysis (ICA) method

Table of Content

List of Figures ix

List of Tables xi

ABBREVATIONS xii

Chapter 1 1

INTRODUCTION 1

1.1 Motivation 1

1.2 Contributions and thesis overview 2

1.2.1 Contributions 2

1.2.2 Thesis structure 2

Chapter 2 4

RELATED WORK 4

2.1 Electroencephalography (EEG) 4

2.1.1 Overview 4

2.1.2 EEG applications 5

2.2 EEG Artifacts and Electrooculography (EOG) Artifacts 6

2.2.1 EEG Artifacts 6

2.2.1.1 Internal artifacts 6

2.2.1.2 External artifacts 8

2.2.2 Electrooculography (EOG) Artifacts 8

2.3 Wavelet Transform and Haar wavelet 9

2.3.1 Wavelet Transform 9

2.3.2 Haar wavelet transform 12

2.4 Independent component analysis (ICA) and ICA JADE 12

2.4.1 Independent component analysis (ICA) 12

2.4.2 ICA JADE 15

2.5 Deep learning and Sparse Autoencoder 15

2.5.1 Deep learning 15

2.5.1.1 Overview 15

2.5.1.2 Neural network 16

2.5.2 Sparse Autoencoder 18

2.5.2.1 Overview 18

2.5.2.2 Network architecture 18

Chapter 3 20

3

METHODOLOGY: Deep Wavelet Sparse Autoencoder to remove

Electrooculography 20

3.1 Data set and Preprocessing data 20

3.1.1 Data set 20

3.1.2 EOG Detection with Haar Wavelet Transform 20

3.1.3 Preprocessing data 22

3.2 A Deep Wavelet Sparse Autoencoder to remove EOG artifacts 23

3.3 Training and correcting artifacts 25

Chapter 4 27

EXPERIMENTS AND RESULTS 27

4.1 Experimental setting 27

4.2 Evaluation metrics 27

4.2.1 Visual assessment 27

4.2.2 Power Spectral Density (PSD) 27

4.2.3 Frequency correlation 28

4.3 Results and discussions 28

Chapter 5 37

CONCLUSIONS 37

5.1 Conclusions 37

5.2 Future Works 37

References 38

Appendix 42

List of Figur

Figure 2.1 Recorded EEG signal 4

Figure 2.2 Example about eye-related artifacts [14] 7

Figure 2.3 Example about cardiac artifacts [9] 7

Figure 2.4 Example about muscle-related artifacts (Chewing artifact) [10] 7

Figure 2.5 First examples about EOG artifacts 9

Figure 2.6 Second examples about EOG artifacts 9

Figure 2.7 Some basic (mother) wavelets 10

Figure 2.8 Multi-resolution analysis using discrete wavelet transform 11

Figure 2.9 Haar wavelet shape [4] 12

Figure 2.10 The original EEG signal 13

Figure 2.11 The signal after using ICA method 14

Figure 2.12 Example about what components should be chosen to remove (set to 0) 14 Figure 2.13 Example about a neural network with 1 input layer, 1 output layer and 2 hidden layers [25] 17

Figure 2.14 A single neuron 17Y Figure 3.1 Emotiv EPOC+ [13] 20

Figure 3.2 Scalp locations covered by Emotiv EPOC+ [7] 20

Figure 3.3 Main steps of EOG detection using Haar Wavelet Transform 21

Figure 3.4 Example about length of single square-shaped 22

Figure 3.5 Example about single EOG with 4 haar-wavelet segments in length 22 Figure 3.6 Flowchart of algorithm 24

Figure 3.7 Haar Wavelet to decompose original signal at level 6 2 Figure 4.1 Applying EOG Detection to original EEG signal 28

Figure 4.2 Applying EOG Detection in second EEG signal record 29

5

Figure 4.3 Applying EOG Detection in third EEG signal record 29Figure 4.4 The results of DWSAE and ICA JADE compare with original signal29Figure 4.5 Compare signal corrected by DWSAE and original signal 30Figure 4.6 Compare signal corrected by ICA JADE and original signal 30Figure 4.7 Compare signal corrected by DWSAE and original signal in second

segment 31Figure 4.8 Compare signal corrected by ICA JADE and original signal in second segment 31Figure 4.9 Compare signal corrected by DWSAE and original signal in other record31Figure 4.10 Compare signal corrected by ICA JADE and original signal in other record 32Figure 4.11 Power Spectral Density (PSD) of original signal, signal corrected by ICA JADE and signal corrected by DWSA 34Figure 4.12 Frequency correlation between original signal and signal correct by ICA JADE 35Figure 4.13 Frequency correlation between original signal and signal correct by DWSAE 35

List of Tables

Table 4.1 The changing of coefficients under 32Hz between before and after using DWSAE 32

ABBREVATIONS

7

EEG Electroencephalogram

Electroencephalogram (EEG) have been used in various applications, includinghuman–computer interfaces, diagnosis of brain diseases, and measurement ofcognitive status For example, doctors use EEG signals obtained from patients fordiagnosis and classification then make decisions to improve the health of patients withepilepsy [32] For more, EEG signals can be used in conjunction with BCI systems tocontrol electrical devices such as wheelchairs, which will enable people withdisabilities to move on their own, without the need for assistance from others [1].However, while recording the electrical activity of the brain, gathering artifacts is verycommon and unavoidable Especially eye-related artifacts (EOG artifacts), a type thathas the most significant effect and is most difficult to control during EEG collectingprocess Therefore, artifacts removal is necessary for ensuring the outcome from theanalysis and evaluation of the received signal to be not seriously affected.

Nowadays, there are some methods to eliminate the artifacts in the obtained EEGsignal, can be mentioned with names like Independent component analysis (ICA) [19],Principal component analysis (PCA) [3] The ICA and PCA methods both analyze theobtained signal into components Some of these components are identified asartifactual components and then removing the artifacts is done by reconstructingcomponents without artifactual components But the difference is in ICA, components

is equally important, while in PCA the components are not It means, the firstcomponent of the PCA is the one that best explains the variability of data, the secondcomponent is the second best explanation and must be orthogonal to the first one,

However, PCA has a big disadvantage that cannot completely separate eye artifactsfrom brain signals, especially when they have comparable amplitudes ICA algorithmsare superior to PCA in removing a wide variety of artifacts from the EEG signal, even

in the case of comparable amplitudes but the main disadvantage of ICA is notautomatic The component based procedures used for artifact removal have to bechosen manually [18], [23] Therefore, the problem is that find a method to eliminateartifacts in the EEG signal that can work well like ICA method (even better) and can

be run automatically Deep Learning may be another approach to solve this problembecause Deep Learning could be trained to remove artifacts automatically Since theEEG signal is non-stationary, the frequency is always changing by time, therefore noresults can be confirmed to be completely accurate The results are often evaluatedthrough metrics such as Power Spectral Density (PSD), Frequency Correlation

1.2 Contributions and thesis overview

1.1.1 Contributions

Types of artifacts, especially Electrooculography (EOG) artifacts, are almostpresent in Electroencephalogram (EEG) recordings It may lead to a wrong orunpredictable judgment of a doctor when analyzing a patient's EEG signal Or, theseartifacts can interfere as well as minimize the possibility of obtaining informationautomatically from the recorded EEG signal The purpose of this thesis is to propose amethod to eliminate automatically EOG artifacts in obtained EEG signals bycombining Haar Wavelet Transform and Sparse Autoencoder with multi hidden layers.This can lead to applications such as: helping analysis, which are done by people ormachines, from EEG signals to be more efficient and accurate; contributing to creating

a Brain-computer-interface (BCI) system that remove EOG artifacts in online way(real time) not from EEG pre-recordings

1.1.2 Thesis structure

The rest of this thesis is organized as follows

Chapter 2 provides background knowledge that is related to Electrooculography(EOG) artifacts removal The main point in this chapter is to supply reader backgroundknowledge about EEG, artifacts and Electrooculography (EOG) artifacts, WaveletTransform in processing EEG signal, Independent component analysis (ICA) inartifacts removal, Deep Learning as well as Sparse Autoencoder

Next in chapter 3 described my method to remove Electrooculography (EOG)artifacts automatically from EEG signal.

Chapter 4 shows my experiments, the results of the proposed method as well asthe evaluation results

The thesis ends with the conclusion and future works in chapter 5

- Measures electrical activity generated by the synchronized activity of thousands

of neurons (in volts)

- Provides excellent time resolution, allowing to detect activity within corticalareas-even at sub-second timescales

The electrical activity generated by the brain has a feature that the voltagefluctuations are very small For a typical adult human, EEG signal is about 0.5 µV to

100 µV in amplitude [36] Therefore, the recorded data is digitized and sent to anamplifier The amplified data can then be displayed as a sequence of voltage values.EEG is one of the fastest imaging techniques available as it often has a high sampling

rate One hundred years ago, the time course of an EEG was plotted on paper Currentsystems digitally display the data as a continuous flow of voltages on a screen.

Nowadays, there are a lot of methods to study brain function, includingfunctional magnetic resonance imaging (fMRI), positron emission tomography (PET),magneto encephalography (MEG), nuclear magnetic resonance spectroscopy (NMR orMRS), electroencephalography (EEG), single-photon emission computed tomography(SPECT), near-infrared spectroscopy (NIRS), and event-related optical signal (EROS).Despite the relatively poor spatial sensitivity of EEG, it possesses multiple advantagesover some of these techniques:

- Hardware costs are significantly lower than those of most other techniques

- EEG sensors can be used in more places and are mobile

- Very high temporal resolution, in milliseconds In clinical and research settings,the common sampling rate is between 250 and 2000 Hz, but modern EEG datacollection systems are capable of recording at sampling rates above 20,000 Hz

- Subject movement in EEG is relatively accepted, unlike most otherneuroimaging techniques

- EEG does not aggravate claustrophobia

- EEG can be used in subjects who are incapable of making a motor response

- EEG is silent, which allows for better study of the responses to auditory stimuli

2.1.2 EEG applications

EEG has wide applications in areas such as psychology, physiology, cognitivescience, neuroscience, medicine and other life science-related fields An EEG candetermine changes in brain activity that might be useful in diagnosing brain disorders,especially epilepsy or another seizure disorder An EEG might also be helpful fordiagnosing or treating the following disorders: brain tumor, brain damage from headinjury, brain dysfunction that can have a variety of causes (encephalopathy),inflammation of the brain (encephalitis), stroke, sleep disorders

The best known application is in epilepsy An EEG test may be done in hospital

by a clinical neurophysiologist But it may also be done at home The doctors mightask patient to have an EEG test because of some following reasons: the patient mighthave epilepsy; patient has had epilepsy and they need to know more about it; they areunsure whether or not patient’s seizures are epilepsy; patient is being considered for

epilepsy surgery; they want to withdraw patient’s epilepsy medicines After test, theresults of an EEG can help doctors to make the right diagnosis and decide on the besttreatment They should always be interpreted by someone who specializes in readingEEG results Because reading an EEG incorrectly can cause giving the wrongdiagnosis or giving not the best treatment (even wrong) It is very harmful to patients ifseizures occur, therefore, another application of EEG in epilepsy is early detection andprediction of seizures from EEG signals [26] This application helps minimize theeffects of seizures caused to patients

Other well-known application is the study of sleep disorders in humans [27],[34] The Electroencephalogram (EEG) is one of the useful bio signals to detect thesleep disorders The sleep disorders are sub types of psychological disorders Somesleep related disorders: REM behavior disorder, periodic leg movements, sleepdisordered breathing, nocturnal frontal lobe epilepsy, insomnia, and bruxism Base onthe recorded EEG signals, the doctors can provide solutions to overcome or minimizethe consequences of sleep disorders

Moreover, some other applications about EEG are: Using EEG to controlwheelchair [35], Virtual Cursor Movement with BCI2000 [37], Human-machineinterfaces based on EMG and EEG applied to robotic systems [15]

2.2 EEG Artifacts and Electrooculography (EOG) Artifacts

2.2.1 EEG Artifacts

In practice, the EEG signals is not clean and almost contain artifacts Artifacts areconsidered unwanted signals or interference in a signal They are reason lead to wrong

diagnosis or mistake in capturing information from the EEG signal EEG signals

generally are contaminated by two types of artifact: internal artifacts and externalartifacts [29] The detail of the two artifacts are introduced below

2.2.1.1 Internal artifacts

Internal artifacts are the artifacts originated because of electrical activity of otherbody parts of the subject and obscure the EEG signals Some common types of internalartifacts:

- Eye-related artifacts (as shown in Figure 2.2): The noise is derived fromactions such as eye blinks, eye movements of the subject

- Cardiac artifacts (as shown in Figure 2.3): are produced by the heart include:electrical artifacts and mechanical artifacts

- Muscle-related artifacts (as shown in Figure 2.4): are derived from actions suchas: tongue movement, swallowing, grimacing, chewing

Figure 2.2 Example about eye-related artifacts [14]

Figure 2.3 Example about cardiac artifacts [9]

Figure 2.4 Example about muscle-related artifacts (Chewing artifact) [10]

2.2.1.2 External artifacts

Besides the internal artifacts, there are also external artifacts that affect therecorded EEG signal, the external artifacts are mainly from the environment and thedevices that used in the EEG recording process The main frequencies may cause anartifact by appearing as a 50-60 Hz component in EEG signal This same phenomenonmay appear as well in recordings where a battery is used as a power supply Some ofthe common types are:

- Phone artifacts: interference caused by telephone waves during recording EEGsignal

- Electrode artifacts: These artifacts are caused by the electrode is not tight orflickering, not enough power to measure

- Physical movement artifacts: This interference is caused by the movement ofother objects while doing experiments such as chairs, fans,

2.2.2 Electrooculography ( EOG) Artifacts

Electrooculography (EOG) artifacts are a type of artifacts that belongs to theinternal type and they are the type of eye-related artifacts EOG artifacts are the kind

of artifacts that almost always appear in EEG records because controlling all eyeactivities while collecting data, is extremely difficult The electric potentials createdduring eye movement and blinks can be orders of magnitude larger than the EEG andcan propagate across much of the scalp, distorting EEG signals Consequently, suchEOG artifacts will hinder the interpretation of EEG, it is thereby important to removethe EOG artifacts before further analysis of EEG

Eye blinking/movements generate spike-like shaped signal waveforms with thepeaks reaching up to 800 µV and each peak occurs in a very short period of 200ms-400ms [16] The frequency of EOG artifacts is in range of 0 to 16 Hz [20] Figure 2.5,2.6 are examples about EOG artifacts, the EOG artifacts are between two green lines

Figure 2.5 First examples about EOG artifacts

Figure 2.6 Second examples about EOG artifacts

2.3 Wavelet Transform and Haar wavelet

2.3.1 Wavelet Transform

The Wavelet Transform is similar to the Fourier Transform Wavelet Transformdecomposes the time signal into a formula consisting of lots of basic functions addedtogether From there a frequency graph can be constructed But the difference is aboutthe basic function In Fourier Transform, the basic function is sin() or cos() withinfinite long and in Wavelet Transform, the basic function is one of wavelets (miniwaves) with limited in time and frequency Some examples about wavelets are shown

in Figure 2.7:

Figure 2.7 Some basic (mother) waveletsGenerally, the continuous wavelet transform can be expressed by the followingequation:

where:

- F(a, b) is the wavelet transform coefficient of f(x)

- the ψ *(a, b) is the complex conjugate of ψ (x)

The wavelets are generated from a single basic wavelet ψ (x) by scaling and

translation has formula like that:

The coefficient is used because the energy has to be normalized across differentscales From that the equation (2.1) equivalent to the following equation:

Similar to the Fourier transform, continuous wavelet transformation is reversible

If the wavelet transform is in the form (2.1), the inverse wavelet transform has theform:

f(x) =

with satisfies the admissibility condition:

where is Fourier Transform of ψ(x)

Discrete Wavelet Transform (DWT) has relation with the idea of multi-resolutionanalysis The idea of multi-resolution analysis is to use digital filtering techniques inthe analysis process In particular, each signal is analyzed into two components:Approximate component A (Approximation) corresponding to low frequencycomponent and Detail component D (Detail) corresponding to high frequency

component, through two high and low pass filters as shown in Figure 2.8 In particular,high-pass filter uses wavelet function ψ(x) and low-pass filter using scaling functionΦ(x).

Figure 2.8 Multi-resolution analysis using discrete wavelet transform

The relationship between the scaling function and the wavelet function is givenby:

Φ(x) =

ψ(x) =

Filtering is carried out with different levels and in order to the calculated volumedoes not increase, when through each filter, the signal is sampled down to 2 For eachfloor, the signal has different resolution

At each filter layer, the expression of filtering is given by the formula:

where, f[n] is the signal, h[n] is the impulse response of low-pass filters corresponding

to the scaling function Φ(n) and g[n] is the impulse response of high-pass filterscorresponding to wavelet function ψ(n) These two filters relate to each otheraccording to the system:

h[N - 1 - n] =

with N is the sampling rate of the signal

The f[n] signal can be reproduced in reverse steps called inverse discrete wavelettransform (IDWT) given by:

f[n] =

2.3.2 Haar wavelet transform

Haar Wavelet Transform is a kind of Wavelet Transform with mother waveletfunction is Haar Wavelet Haar Wavelet has a "square-shaped" shape (Figure 2.9) andhas characteristics that are not continuous so it is not differentiable Because of this,Haar wavelet will be very useful in analyzing signals with sudden transitions Haar

Wavelet’s mother wavelet ψ (x) is defined by the following function:

ψ (x) =

and its scaling function Φ(x) can be described as:

Φ(x) =

Figure 2.9 Haar wavelet shape [4]

2.4 Independent component analysis (ICA) and ICA JADE

2.4.1 Independent component analysis (ICA)

Independent component analysis (ICA) is a well-known and classic method ofremoving almost types of EEG noise Basically, the ICA method assumes the receivedsignal as a mix from the sources of the signal For example, if EEG signal has 32channels, the signal in each channel is mixed from 32 independent components Morespecifically, if the symbol for each channel is ,,,…, and the components are , ,, , , then: = + + + … +

= + + + … +

= + + + … +

= + + + … +

In general, the expressions for each channel are of the form:

as x’ = W-1u’, where u’ is the matrix of activation waveforms, u, with rows representingartifactual components set to zero

Figure 2.10 The original EEG signal

Figure 2.11 The signal after using ICA method

Figure 2.12 Example about what components should be chosen to remove (set to 0)Figure 2.10 is an original EEG signal with a lot of artifacts, after using ICAmethod all artifacts are removed (as show in Figure 2.11) ICA gives quite good resultsabout reducing noise from EEG signal, but the biggest disadvantage of this method isnot completely automatic This method should be done manually in the step to selectwhich components to delete [18], [23] Moreover, choosing which components toremove is a problem because experienced people who know the type of noise by eyecan often choose the right components to delete, otherwise the results will becomeworse or lack of believe

Currently, the linear ICA model has been studied according to differentapproaches with many algorithms such as Infomax [5], JADE, SOBI, FastICA [17] In

my thesis, ICA JADE will be introduced in next section

2.4.2 ICA JADE

ICA JADE (Joint Approximation Diagonalization of Eigen-matrices) is one of thefamous algorithms to find hidden sources , ,, , from the mixed obtained signals ,,,…,based on diagonalization of fourth-order cumulant [8] The fourth-order cumulant can

be given by:

,,,where i, j, k, l = 1…n with n is number of sources and E {} is expectation of data Then sequentially obtain all of the fourth-order cumulants , which can be written as matrix m = [,,,…, ] The (i, j)th element of matrix m can be given by:

where M is n x n arbitrary matrix and is its (l,k)th element Thedecomposition of cumulant matrix should be satisfied:

where is the eigenvalue Then M is processed by joint approximatediagonalization to determine the unitary matrix which is used toobtain the estimation of source signals Finally, making each matrix to beapproximately diagonalized is necessary, and the basic criteria is to consider off-diagonal elements when it comes close to zero [33]

2.5 Deep learning and Sparse Autoencoder

2.5.1 Deep learning

2.5.1.1 Overview

Deep Learning is a method of machine learning It allows training a machine topredict outputs based on a set of inputs There are learning types that are supervised,semi-supervised or unsupervised [6], [22], [31] Currently, deep learning is widelyapplied in fields such as computer vision [11], speech recognition, natural languageprocessing, audio recognition and board game programs [28]

Supervised learning is an algorithm to predict the output (outcome) of a new data(new input) based on known pairs (input, outcome) This data pair is also known as(data, label) Supervised learning is the most popular group in machine learningalgorithms Mathematically, supervised learning is when having a set of inputvariables X={,,,…,} and a corresponding set of labels Y = {,,,…,} Predetermined data

pairs () are called training data From this training data collection, the goal is creating afunction f that maps each element from X to Y It means:

for The purpose is to approximate function f with the best as possible so that whenthere is a new data x, the result can be calculated by getting its corresponding label y =

f (x)

Besides supervised learning, there is a kind of deep learning that also trains butdoes not have label data It means learning does not need labels, which is exactlyunsupervised learning The algorithm will rely on the structure of the data to perform acertain task, such as grouping (clustering) or dimension reduction for convenience instorage and calculation

In addition to the two types of learning mentioned above, there is other type oflearning, named semi-supervised learning It crosses between the two types of learningabove, it means the data used for training has two part with one part has labels andother part has no labels

2.5.1.2 Neural network

Most modern deep learning models are based on an artificial neural network (Aneural network with artificial neurons) Neural Network (NN) consists of 3 mainlayers: Input layer and output layer including only 1 layer, hidden layer can have 1 ormore layers depending on the specific problem NN operates in a way that describeshow the nervous system works with connected neurons In NN, except for the inputlayer, all nodes of other layers are full-connected to the nodes of the previous layer.Each node of the hidden layer receives the input matrix from the previous layer andcombines it with the weight to produce the result