nghiên cứu đề xuất giải thuật tiến hóa đa mục tiêu dựa trên thông tin định hướng và ứng dụng

Multi-objective evolutionary algorithms MOEAs use apopulation of solutions to approximate the Pareto optimal set in a single run.. Combining decision maker’s preference with directions o

MINISTRY OF EDUCATION AND TRAINING MINISTRY OF NATIONAL DEFENSE

MILITARY TECHNICAL ACADEMY

NGUYEN LONG

A MULTI-OBJECTIVE EVOLUTIONARY ALGORITHM USING DIRECTIONS OF IMPROVEMENT AND APPLICATION

MINISTRY OF EDUCATION AND TRAINING MINISTRY OF NATIONAL DEFENSE

MILITARY TECHNICAL ACADEMY

A MULTI-OBJECTIVE EVOLUTIONARY ALGORITHM USING DIRECTIONS OF IMPROVEMENT AND APPLICATION

1 ASSOC PROF DR BUI THU LAM

2 ASSOC PROF DR NGUYEN VAN HAI

Hanoi - 2014

A multi-objective optimization problem involves at least two conflicting objectives and it has

a set of Pareto optimal solutions Multi-objective evolutionary algorithms (MOEAs) use apopulation of solutions to approximate the Pareto optimal set in a single run MOEAs haveattracted a lot of research attention during the past decade They are still one of the hottestresearch areas in the field of Computational Intelligence and they are the main focus of thisthesis

Firstly, the main concepts for multi-objective optimization are presented, then the thesis cerns about mentions the solving multi-objective optimization problems by multi-objectiveevolutionary algorithms This thesis also conducts a survey on the usage of directorial infor-mation in search’s guidance Through the survey, the thesis indicates that there is a need tohave more investigation on how to have an e↵ective guidance from both aspects:

con-1 Automatically guiding the evolutionary process to make the MOEA balanced betweenexploitation and exploration

2 Combining decision maker’s preference with directions of improvement to guide theMOEAs during optimal process toward the most preferred region in the objective space

To address this, the thesis builds up all its proposals based on a direction based objective evolutionary algorithm (DMEA), the most recent one with a systematic way tomaintain directions of improvement so some related issues on DMEA are raised and anal-ysed, hypothesised as primary research problems in this thesis

multi-At the highlighted chapters, the thesis discusses all the issues on using directions of ment in DMEA through thesis’s contributions:

improve-1 Design a new proposed direction based multi-objective evolutionary algorithm version

II (DMEA-II) with following improvement techniques:

• Using an adaptive ratio between convergence and spread directions

• Using a Ray based density niching method for the main population

• Using a new Ray based density selection scheme for dominated solutions selection

• Using a new parents selection scheme for the o↵springs perturbation

In order to validate the proposed algorithm, a series of experiments on a wide range oftest problems was conducted It obtained quite good results on primary performancemetrics, including the generation distance (GD), the inverse generation distance (IGD),the hypervolume (HYP) and the two set coverage (SC) The analysis on the resultsindicates the better performance of DMEA-II in comparison with the most popularMOEAs

2 Proposes an interactive method for DMEA-II as the second aspect of having an e↵ectiveguidance An interactive method is introduced with three ray based approaches: RaysReplacement, Rays Redistribution, Value Added Niching The experiments carried out

a case study on several test problems and showed quite good results

3 Introduces a SpamAssassin based Spam Email Detection System that uses

DMEA-II The proposed system helps users to have more good choices for the SpamAssassinsystem in configuration

The first of all, I would like to express my respectful thanks to my principal supervisor,Assoc.Prof Bui Thu Lam for his directly guidance to my PhD progress Assoc.Prof Buihas given me knowledge and passion as the motivation of this thesis His valued guidancehas inspired much of the research in the thesis

I also wish to thank my co-supportive Assoc.Prof Nguyen Van Hai for his suggestions andknowledge during my research, especially the relation between theories and real problems inwork I also would like to thank Prof Hussein Abbass, Assoc.Prof Tran Quang Anh andAssoc.Prof Dao Thanh Tinh for their invaluable support throughout my PhD I feel lucky

to work with such excellent people

I also would like to thank all of my fellows in the Department of Software Technology andEvolutionary Computation research group for their assistance and support

Last but not least, I also would like to acknowledge the support of my family, especially myparents Dr Nguyen Nghi, Truong Thi Hong, they worked hard and believed strongly in theirchildren I also would like to thanks my wife, sisters, brothers who always support me during

my research

Originality Statement

I hereby declare that this thesis is my own work, with my knowledge and belief the thesishas no material previously published or written by others Any contributions made to theresearch by colleagues, with people in our research team at Le Quy Don Technical University

or elsewhere, during my candidature is clearly acknowledged

I also declare that the intellectual content in this submission is the research results of my ownwork, except to the extent that assistance from others in conception or in style, presentationand linguistic expression is acknowledged

1.1 Overview 1

1.2 Research Perspectives 6

1.3 Motivation 6

1.4 Questions and Hypothesises 8

1.5 Thesis organization 9

1.6 Original Contributions 10

2 Background concepts and Issues 13 2.1 Common concepts 13

2.1.1 Multi-objective problems 13

2.1.2 Notations 14

2.1.3 General Definitions 14

2.1.4 Pareto Optimality 15

2.1.5 Weak Pareto Optimality 17

2.1.6 Dominance 17

2.2 Conventional methods 18

2.2.1 No-preference methods 19

2.2.2 A priori methods 19

2.2.3 A posteriori methods 20

2.2.4 Interactive methods 23

2.3 An overview of Multi-objective Evolutionary Algorithms 25

2.3.1 Non-elitist methods 25

2.3.2 Elitist methods 26

2.3.3 Performance measures 27

2.3.4 Test problems 29

2.4 Statistical testing 30

2.5 Search’s guidance in MOEAs 31

2.5.1 Technique of using guided directions 32

2.5.2 Advantages and disadvantages 45

2.6 Research Issues 48

2.6.1 Direction based multi-objective evolutionary algorithm (DMEA) 48

2.6.2 Issue 01: The disadvantages of the fixed ratio between types of directions 51 2.6.3 Issue 02: Lack of an efficient niching method for the main population 52 2.6.4 Issue 03: The disadvantages of using the weighted sum scheme 53

2.6.5 Issue 04: Using a ’hard’ niching method 53

2.6.6 Issue 05: Investigating on how the DM can interact with DMEA 53

2.7 Summary 54

3 A guided methodology using directions of improvement 55 3.1 Using an adaptive ratio between convergence and spread directions 55

3.2 Using a Ray based density niching for the main population 56

3.3 Using a ray based density selection schemes 59

3.4 Direction based Multi-objective Evolutionary Algorithm-II 60

3.4.1 General structure 60

3.4.2 Computational complexity 62

3.4.3 Experimental Studies 62

3.4.4 Results and Discussion 68

3.5 Analyzing e↵ects of di↵erent selection schemes for the perturbation 81

3.6 Summary 86

4 A guided methodology using interaction with decision makers 87 4.1 Overview 87

4.2 A multi-point Interactive method for DMEA-II 92

4.2.1 Rays replacement 93

4.2.2 Rays Redistribution 94

4.2.3 Value Added Niching 96

4.2.4 Experimental Studies 97

4.2.5 Results and Discussion 98

4.3 Summary 102

5 An application of DMEA-II for a spam email detection system 104 5.1 Overview 104

5.2 Spam email detection 107

5.2.1 SpamAssassin 107

5.2.2 Methodology 108

5.2.3 An interactive method 113

5.2.4 Computational complexity 113

5.2.5 Experimental Studies 114

5.2.6 Results and Discussion 115

5.3 Summary 123

6 Conclusions and Future Work 124 6.1 Conclusions 124

6.2 Future directions 129

List of Figures

2.1 An illustration of optimal Pareto 16

2.2 An illustration of weak optimal Pareto 17

2.3 An illustration of the weighted-sum approach 22

2.4 An illustration of the ✏-constraint approach 23

2.5 An illustration of performance metrics 28

2.6 An illustration of descent directions 33

2.7 An illustration of Pareto descent directions 34

2.8 An illustration of determination directions in di↵erent cases 34

2.9 An illustration of di↵erential directions 39

2.10 An illustration of directional convergence and directional spread 41

2.11 An illustration of the movement of a centroid 43

2.12 An illustration of convergence and spread directions 44

2.13 An illustration of the ray system 49

2.14 An illustration of the performance of DMEA 52

3.1 An illustration of the Ray-based Density 57

3.2 The obtained non-dominated of DMEA and DMEA-II 70

3.3 Results on DTLZ2, UF1, UF3 and UF8 71

3.4 Visualization of GD and IGD overtime for ZDT1, ZDT4 73

3.5 The chart for DMEA-II and DMEA comparison on GD, IGD and HYP 79

3.6 The chart for DMEA-II and other MOEAs comparison on GD 79

3.7 The chart for DMEA-II and other MOEAs comparison on IGD 80

3.8 The chart for DMEA-II and other MOEAs comparison on HYP 80

3.9 The chart for DMEA-II and other MOEAs comparison on SC 81

3.10 Visualization of GD and IGD over time for ZDT1, ZDT2 84

3.11 Visualization of GD and IGD over time for ZDT3, DTLZ3 85

4.1 An illustration of altering the reference point 90

4.2 An illustration of the use reference direction approach 92

4.3 An illustration of the rays replacement approach 94

4.4 An illustration of the rays redistribution approach 95

4.5 An Illustration of the value added niching approach 97

4.6 A visualization of the interactive method on ZDT1 99

4.7 A visualization of the interactive method on ZDT2 99

4.8 A visualization of the interactive method on ZDT3 100

4.9 A visualization of the interactive method on ZDT4 100

4.10 A visualization of the interactive method on ZDT6 101

5.1 An illustration of results with 30 and 100 rules for 272 emails 116

5.2 An illustration of results with 30 and 100 rules for 426 emails 117

5.3 An illustration of results with 30 and 100 rules for 286 multilingual emails 118

5.4 Results for the Rays Replacement approach with 30 rules 120

5.5 Results for the Rays Replacement approach with 50 rules 120

5.6 Results for the Rays Replacement approach with 100 rules 120

5.7 Results for the Rays Redistribution approach with 30 rules 121

5.8 Results for the Rays Redistribution approach with 50 rules 121

5.9 Results for the Rays Redistribution approach with 100 rules 121

5.10 Results for the Value Added Niching approach with 30 rules 122

5.11 Results for the Value Added Niching approach with 50 rules 122

5.12 Results for the Value Added Niching approach with 100 rules 122

List of Tables

3.1 The main features of test problems 66

3.2 Common parameter settings 67

3.3 Parameters settings 67

3.4 The average values of GD, IGD and HYP 72

3.5 The average value of GD 74

3.6 The average value of IGD 75

3.7 The average value of HYP 76

3.8 The comparison of DMEA-II and others on SC 77

3.9 The GD, IGD, HYP, SC results for DMEA-II and MOEA/D 78

3.10 The GD values of DMEA-II and DMEA-II* over the first 200 generations 83

3.11 The IGD values of DMEA-II and DMEA-II* over the first 200 generations 83

4.1 The main features of ZDT problems 98

5.1 Parameter settings 114

5.2 The result of SOOA with 30 and 100 rules for 272 emails 116

5.3 The result of SOOA with 30 and 100 rules for 426 emails 117

5.4 The result of SOOA with 30 and 100 rules for 286 multilingual emails 119

A.1 ZDT Problems 133

A.2 DTLZ Problems 135

A.3 UF Problems 139

AbbreviationsAbbreviation Meaning

MOP Multi-objective Optimization Problem

MOEA Multi-objective Evolutionary Algorithm

POF Pareto Optimal Front

POS Pareto Optimal Set

RD Ray based Density

DMEA Direction based Multi-objective Evolutionary Algorithm

DMEA-II Direction based Multi-objective Evolutionary Algorithm-II

NSGA-II Non-Dominated Sorting Genetic Algorithm II

SPEA2 Strength Pareto Evolutionary Algorithm 2

MOEA/D Multi-objective Evolutionary Algorithm Based on DecompositionMOGA Multi-objective Genetic Algorithm

NPGA Niched Pareto Genetic Algorithm

PAES Pareto-Archived Evolution Strategy

MOPSO Multi-objective Particle Swarm Optimization

PDE Pareto Di↵erential Evolution

DM Decision Maker

GD Generational Distance

IGD Inverse Generational Distance

SC Two Set Converge

SDR Spam Detection Rate

FAR False Alarm Rate

VSDSA Vietnamese spam detection based on SpamAssassin

CD Convergence Direction

SD Spread Direction

DC Directorial Convergence

DS Directorial Spread

!

BẢNG THUẬT NGỮ SỬ DỤNG TRONG LUẬN ÁN

Multi-objective Evolutionary Algorithm Giải thuật tiến hóa

in the absence of any further information, are all equally good An evolutionary algorithmshave been very popular for solving MOPs [16, 26] mainly due to their ease of use, work onpopulation and their wide applicability Evolutionary algorithms allow to find an entire set ofPareto optimal solutions in a single run of the algorithm, instead of having to perform a series

of separate runs as in the case of the traditional mathematical programming techniques.Recently, the guided techniques have been discussed, conceptualized and used to guide multi-objective evolutionary algorithms (MOEAs) during the search process towards the POS Gen-erally, guided information is derived from population, individuals, archives, decision makers.Then those information are used to guide MOEAs during their evolutionary process quicklytowards the POS The good guidance will control MOEAs to obtain the set of solutions to-wards POSs in a good quality of convergence and diversity This is a difficult task since theevolutionary process allows randomness so it is hard to maintain the balance between conver-gence and diversity properties during the search This thesis will discuss the determinationand the e↵ective usage of the guided information in MOEAs

1.1 OVERVIEW

Evolutionary Algorithms Evolution via natural selection of a randomly chosen lation of individuals as a search through the space of possible chromosome values In thatsense, an evolutionary algorithm is a stochastic search for an optimal solution to a givenproblem The evolutionary search process is influenced by the following main components of

popu-an evolutionary algorithm (EA) [17]:

• Population: Since EAs work with a population of individuals, it is important to definethis structure in the first place For this issue, a population of individuals is defined toencode a finite set of possible solutions for a problem

• Individual: It is a data ensemble encoding a solution for the problem It might contain

a structure for a solution (a set of problem variables), objective values, and several otherproperties such as a fitness value, index, rank, etc Here, the representation of a solution

is vital for the operations of the algorithm In general optimization problems, thereexist three major representations for this structure:

– Binary : The solution is represented by a string of bits Sometimes, this string

is called a genotype The values at each bit location or locus are called alleles.This genotype is generally composed of one or several chromosomes where eachchromosome is a composition of several genes For real-valued problems, thisgenotype will be decoded into an array of real values (equivalent to a solution

of the problem) using a mapping function This array is usually considered as aphenotype

– Real-valued : For this type of representation, the solution is represented by anarray of real values This array is considered as a chromosome and each element ofthis array is a gene Here the genotype and phenotype are identical Each element

of the array is considered a gene

– Graph: In some cases, the problem in question is to find a topology, network,

or program of function containing a set of symbols Here, a graph such as a treerepresentation is more suitable The genotype-phenotype mapping is even morecomplicated than the one with binary representation

in the tree.

• Crossover operator: This production operation allows the combination of geneticmaterials from two or more parents to create o↵spring In evolutionary computation,crossover is used to create new individuals that have gene values from selected parents.The e↵ect of this operator is to potentially combine good elements of parents For abinary representation, two selected parents may swap parts of their binary strings tocreate two o↵spring For a real-valued representation, genes from two parents may becombined mathematically to form a new child For a tree representation, branches oftwo trees are swapped

• Selection operator: The selection operator in EAs is used to select promising dividuals to contribute to next generations It relies on the fitness values associatedwith the individuals Note that fitness and objective values are di↵erent concepts Theobjective value is the one obtained directly from the objective function of the problem,while the fitness value is to show how good an individual is in relative comparison withother individuals in the population Selection strategies such as fitness proportion, ortournament, are usually based on fitness values

in-These components are combined to form a generic EA shown in Algorithm 1 There, t is thegeneration counter, n is the population size, C(t) is the main population at tth generation The steps of an EA are applied iteratively until some stopping conditions are satisfied Eachiteration of an EA is referred to as a generation

1.1 OVERVIEW

Algorithm 1: Generic Evolutionary Algorithm

• Let t = 0

• Create and initialize an population C(0) with size of N, to consist of N individuals

• While stopping condition(s) not true do

– Evaluate the fitness f (xi(t)), of each individual xi(t) with i2 [1, N], t is thecurrent iteration

– Perform reproduction to create o↵spring

– Select the new population C(t + 1)

– Advance to the new generation, i.e t = t + 1

• End

Based on di↵erent representations, conventional EAs have been categorized as follows [130]:

• Genetic Algorithms (GA): model genetic evolution and use binary representation

• Evolution Strategies (ES): geared towards modeling the strategy parameters thatcontrol variation in evolution and use real-valued vectors

• Evolutionary Programming (EP): derived from the simulation of adaptive ior in evolution (phenotypic evolution), currently evolutionary programming is a wideevolutionary computing dialect with no fixed representation

behav-• Genetic Programming (GP): based on genetic algorithms, but individuals are grams (represented as trees)

pro-Recently, researchers extended EA’s paradigms to Di↵erential Evolution (DE)[89], ParticleSwarm Optimization (PSO)[24] and Ant Colony Optimization (ACO)[111] etc

In genetic algorithms, problems are encoded in a series of bit strings that are manipulated

by the algorithm In evolutionary, the decision variables and objective functions are useddirectly Both of genetic or evolutionary algorithms apply the principles of evolution found

1.1 OVERVIEW

in nature to find an optimal solution for an optimization problem

In EAs, niching methods are used to allow EAs to maintain a diverse population of viduals EAs that incorporate niching methods are capable of locating multiple, optimalsolutions within a single population E↵ective niching methods are critical to success of EAs

indi-in classification and machindi-ine learnindi-ing, multi-modal optimization, multi-objective tion, and simulation of complex and adaptive systems Niching is also useful for findingbetter, single solution to hard problems, the intermediate formation and maintenance of di-verse sub-solutions is often critical to the solution of hard problems In [67] Mahfoud suggests

optimiza-a cloptimiza-assificoptimiza-ation boptimiza-ased on the woptimiza-ay thoptimiza-at multiple niches optimiza-are found in optimiza-a EA:

• Spatial or Parallel Niching methods: Niching methods belonging to this categoryfind and maintain multiple niches simultaneously in a single population Examples ofparallel niching methods are Sharing, Crowding function approach and Clearing method

• Temporal or Sequential Niching methods: These niching methods find multipleniches iteratively or temporally For example the Sequential Niching method findsmultiple niches iteratively

The idea of niching is applicable in optimization of constrained problems In such problems,maintaining diverse feasible solutions is desirable so as to prevent accumulation of solutionsonly in one part of the feasible space, especially in problems containing disconnected patches

be an even better one in other regions Without exploration, the algorithm’s search ability

is limited Or, the search may be trapped in very low reward areas that the algorithm wouldavoid without exploration On the other hand, if the algorithm explores too much, it cannotstick to a region, hence slowing down convergence Thus, it is important to find a good

1.3 Motivation

In optimization area, using evolution algorithms (EAs) brings a lot of e↵ectiveness to solveoptimization problems In fact, evolution algorithms work on population and stochasticmechanism so evolution algorithms can be e↵ectively used to solve difficult problems whichhave complex optimal sets in objective space EAs have a widely randomized range so theymake the search being not biased towards local optima, that is why EAs are suitable forglobal optimization problems When solving multi-objective problems, EAs are adaptivelyand e↵ectively used to obtain a set of approximated Pareto optimal solutions However, EAsalso have some difficulties such as: the obtained solutions are approximated Pareto optimalsolutions so they are not really desired optimal solutions for the problems It also requires

a high number of generations to get a good set of solutions To avoid these disadvantages,

a hybridization model that combines MOEAs with search mechanisms to improve the formance quality of the algorithms The search techniques are discussed and widely used

per-in multi-objective optimization such as: particle swarm optimization (PSO) [96], ant colony[111] These techniques are used to guide the evolutionary process quick towards Paretooptimal fronts (POFs) in objective space ( or POSs in decision space), and to avoid beingtrapped in local optima This guidance helps MOEAs to be improved in their exploitationand exploration characterises and the quality of the obtained solutions Using guided infor-mation is a promising technique to get good approximated solutions, it helps MOEAs to beimproved in their quality and capacity In fact, there are many approaches in using guidedinformation in MOEAs, one of these kinds of guided information is directional information inMOEAs, namely gradient based directions [45, 38, 5, 107], di↵erential evolution [2, 62, 65, 23],

1.3 MOTIVATION

directions of improvement [49, 18, 14, 15, 19]

Solving MOPs by gradient based directions is early discussed and used in di↵erence proaches In fact, gradient based multi-objective algorithms have some advantages: Thisalgorithms can be used to solve complex di↵erentiable MOPs, gradient based directions areused so it makes multi-objective algorithms to be good convergence rate, when incorpo-rating with evolution strategy in a hybridization MOEA, the algorithms can have a goodconvergence rate and avoid the local optimums during the search However, there are somedifficulties in using gradient based directions such as: The algorithms can not be used withnon-di↵erentiable MOPs, it requires a hight performance cost to determine gradient baseddirections There are several difficulties for gradient based algorithms such as: determiningdescent, Pareto descent and directed directions, keeping the balance between exploitationand exploration globally

ap-To date, evolutionary algorithms which use concept of di↵erential direction is known as apowerful and e↵ective algorithm to solve single optimization problems However, in multi-objective optimization The usage of di↵erential directions has some difficulties: MOEAswith DE have hight convergence rate but it is difficult to keep diversity for the population.This disadvantage can be solved if some mechanisms for maintaining diversity of the popula-tion are incorporated with the algorithms Another difficulty is that MOEAs with DE onlywork on real decision space, so it can not be used when decision space is binary space Thisdifficulty can be solved when using an additional codding technique for a space transforma-tion

Using directions of improvements in MOEAs is known as an e↵ective technique since the aim

of directions of improvement Directions of improvement are used to guide the evolutionaryprocess to make the population to be quickly converged and uniformly distributed towardsthe POF It helps to improve the convergence rate and diversity for obtained population.Almost of the difficulties in using of gradient based directions and di↵erential directions will

be overcome by using directions of improvement: It is quite simple to determine directions

of improvement since these directions are determined by dominance relationship of solutions(or individuals) in population ( or an external population); Directions of improvement areused for a movement of solution follows two aspects: being closed and uniformly distributed

1.4 QUESTIONS AND HYPOTHESISES

the POF It helps to ensure convergence rate and diversity of the population so it promises toobtain a good approximated POS, the primary aim of improving MOEAs in multi-objectiveoptimization area However, there are some difficulties in using directions of improvement:keeping the balance between exploitation and exploration is difficult since the evolutionaryprocess follows the stochastic mechanism This difficult might be a reason of reducing con-vergence rate and diversity of the population Almost directions of improvement are used in

a local search model for MOEAs, so it might not be an e↵ective algorithm for global objective optimization

multi-In summary, the usage of direction for guiding MOEAs is a promising approach There

is a need to have more investigation on how to have an e↵ective guidance from both pects: 1) Automatically guiding the evolutionary process to make MOEAs balanced betweenexploitation and exploration 2) Combining decision maker’s preference with directions ofimprovement to guide MOEAs during optimal process towards the most preferred region inobjective space The previous discussions represent the motivation of this thesis

as-1.4 Questions and Hypothesises

In MOEAs, using directions of improvement has been concerned in much research, sometechniques for using directional information in MOEAs are proposed in [89, 2, 3, 18, 14,

15, 19, 52] However, we need to use guided information for the evolutionary process in

an e↵ective way to help MOEAs be good performance in optimal approximation This is ahard problem that many researchers have been tried to solve These are the focal points ofthe research reported in this thesis In other words, this thesis aims to address the followingquestion: How to design an e↵ective guidance for MOEAs to move quickly towards

a suspected optimum or decision makers’ preferred region and also to avoid beingtrapped too easily in a basin surrounding a local optimum? In order to answer thisquestion, this thesis gives some hypothesises:

• When incorporating evolutionary techniques with directions of improvement, thosetechniques have again the e↵ect on the balance between exploitation and exploration

of the algorithms There is a need to have a guidance for the evolutionary

1.5 Thesis organization

This thesis is organized in six chapters, the remainder of the thesis is arranged as following:

• Chapter 2 It is devoted to summarize common concepts and methods related tomulti-objective optimization (MO) Further, a description of MOEAs is given Twogenerations of MOEAs elitist and non-elitist are described, then various aspects ofperformance metrics and well-known test problems for MOEAs are presented Up

to date, a significant number of MOEAs are reported in depth At the end of thechapter, the recent general developments and research issues on search’s guidance areaddressed At the highlighted part of the chapter, using directional information forsearch’s guidance in MOEAs is discussed For more details, the chapter also indicatesseveral issues in using directions of improvement in a selected direction base multi-objective evolutionary algorithm (DMEA)

• Chapter 3 The primary characteristics of MOEAs when they work with elitist tions are: maintaining elitist solutions during the evolutionary process; using nichingmethods to maintain diversely production and archive during the search; using selec-tion strategies for MOEAs to select solutions for next generations These importantcharacteristics make MOEAs to be efficient when solving MOPs Chapter 3 describesand analyses these important characteristics of DMEA All issues of using directions ofimprovement related to these characteristics in DMEA which are indicated in Chapter

solu-2 are solved This leads to a new version of DMEA, namely DMEA-II In order to

vali-1.6 ORIGINAL CONTRIBUTIONS

date the proposed algorithm, a series of experiments on a widely range of test problemsfor proposed DMEA-II is presented and analyzed in the final part of the chapter Theexperimental results indicate that DMEA-II has better performance over the originalDMEA The thesis also conducts a comparison between DMEA-II’s performance withother 5 MOEAs on four metrics: GD, IGD, HYP and SC DMEA-II with the aboveproposed techniques was competitive in comparison with these algorithms with respect

to both convergence and spread Several analyses on the behaviors of the algorithmwere thoroughly investigated

• Chapter 4 This chapter proposes a guided methodology using interaction with sion makers for DMEA-II with three ray based approaches: Rays Replacement, RaysRedistribution, Value Added Niching on DMEA-II This chapter suggests a way fordecision makers to join to evolutionary processes, decision makers’ preference informa-tion is used to guide the MOEAs to be converged to their preferred region in objectivespace To validate the proposed method, the experiments are presented and discussed

deci-• Chapter 5 An application of DMEA-II for Spam Email Detection System is introduced.The proposal is a multi-objective optimization approach for generating sets of feasibletrade-o↵ solutions for an anti-spam email system (using Apache SpamAssassin) Theexperiments on Vietnamese language databases and rules are implemented The resultsindicated that, when solving the problem using DMEA-II, it achieved more efficientresults but also created a set of ready-to-use rule scores

• Chapter 6 Conclusions and future works are given

1.6 Original Contributions

Using evolutionary algorithms (EAs) for approximating solutions of MOPs (MOPs) has been

a popular topic in the field of evolutionary computation, since EAs can o↵er simultaneously

a set of trade-o↵ solutions To date, there have been a large set of MOEAs in the literatureaddressing a widely range of problems with di↵erent properties Directions of improvementhave been discussed, conceptualized and used to guide MOEAs during the search process

1.6 ORIGINAL CONTRIBUTIONS

towards POF The major concern of the thesis is how to use directions of improvement in ane↵ective way to guide the evolutionary process of MOEAs in both aspects: 1) Automaticallyguiding the evolutionary process to make MOEAs balanced between exploitation and explo-ration 2) Combining decision maker’s preference with directions of improvement to guidethe MOEAs during optimal process towards the most preferred region in an objective space.Overall, through a series of analysis and validation, the main contribution of the researchintroduced in the thesis can be summarized as follows:

1 Design a new direction based multi-objective evolutionary algorithm II) with new features:

(DMEA-• Using an adaptive ratio between convergence and spreading directions: This gests a thought on an adaptation of the balance between convergence and spread.Instead of using a fixed ratio, this thesis suggests using an adaptive ratio based onthe number of non-dominated solutions in the archive at the current generation

sug-• Using a ray based density niching method for the main population: This thesisproposes the new measurement called ’Ray-based Density’ (RD) for counting thenumber of rays that a solution is the closest Based on the concept of RD, thethesis proposes a new niching procedure, the density value for all non-dominatedsolutions in the combined population is calculated and kept to use for the selectionfor the main population during generations

• Using a ray based density selection scheme: The thesis proposes to use a newselection scheme using the ray-based density in the same way as DMEA does forthe archive: all rays are scanned and calculated the distance from all solutions ofcombined population between current archive and o↵spring to the current ray toget the closest solutions for each rays until getting a specified number of dominatedsolutions for the next generation

• Analyzing e↵ects of di↵erent selection schemes for the perturbation: This thesissuggests to replace original selection scheme by a new one in which parent at aloop will be changed by a solution in the population Note that, the index for thatsolution ranges from 1 to the half of population size With new selection scheme,

1.6 ORIGINAL CONTRIBUTIONS

the work’s reasoning is that more selection pleasure on DMEA is placed so thatthe convergence will be balanced with diversity

2 Propose a new interactive method for DMEA-II: To use guided information in

an e↵ective way for guiding the evolutionary process to make the population towardsDM’s preferred region, this thesis proposes a ray based interactive method for DMEA-IIwith three ray based approaches:

• Rays Replacement: The furthest rays from DM’s preferred region are replaced bynew rays that generated from a set of reference points

• Rays Redistribution: Redistribute the system of rays to be in DM’s preferredregion

• Value Added Niching: Based on the distances from non-dominated solutions inarchive to DM’s preferred region, the niching values for the solutions is increased

to be priority selected

3 Propose an application of DMEA-II for a Spam Email Detection System: Aframework which applied DMEA-II to solve the problem of a Spam Email DetectionSystem (a SpamAssassin based system) as an applied MOEAs for real applications,which validates the usefulness of distributions on this thesis When solving the problemusing DMEA-II, it achieved more efficient results but also created a set of ready-to-userule scores These scores support di↵erent levels of the trade-o↵ between SDR andFAR By obtained solutions, it gives users more flexibility and efficiency for systemconfiguration

is optimal with respect to every objective function This means that the objective functionsare at least partly conflicting.

The image of the feasible region is denoted by Z(= f (S)) and called feasible objective region,which is a subset of objective space Rk The elements of Z are called objective (function)vectors or criteria vectors and denoted by f (x) or z = (z1, z2, , zk)T where zi = fi(x) for

i = 1, , k are objective (function) values or criteria values

i for all i = 1, , n Correspondingly x < x⇤ stands for

xi < x⇤i for all i = 1, , n

In some connections it assumes that the feasible region is formed of inequality constraints,that is, S = {x 2 Rn|g(x) = (gl(x), g2(x), , gm(x))T  0} An inequality constraint gj issaid to be active at a point x⇤ if gj(x⇤) = 0, and the set of active constraints at x⇤ is denoted

by J(x⇤) = {j 2 {1, , m}|gj(x⇤) = 0}

There are some general definitions for the types of MOPs

Definition 1 When all objective functions and constraint functions forming the feasibleregion are linear, then the MOPs is call linear It is multi-objective programming problem or

A problem is called a nonlinear multi-objective optimization problem if there is at least one

of the objective or constraint functions is nonlinear

To define a convex MOPs, the convex functions and convex sets are defined:

Definition 2 A function fi : Rn ! R is convex if for all x1, x2 2 Rn valid that fi( x1 +(1 )x2) fi(x1) + (1 )fi(x2) for all 0  1 A set S ⇢ Rn is convex if x1, x2 2 S

Definition 3 The MOP is convex if all objective functions and the feasible region are

2.1 COMMON CONCEPTS

An important concept in optimization, namely optimality is given in this section The cision variable space is mainly focused in single optimization problems However, in themulti-objective context, the objective space is most interested since it is usually of a lowerdimension than the decision variable space

de-The objective functions are conflicting and possibly incommensurable, so the finding of singlesolution that would be optimal for all the objectives simultaneously is impossible MOPs are

in a sense of ill-defined There is no natural ordering in the objective space since it is onlypartially ordered (meaning that, for example, (1, 1)T can be said to be less than (3, 3)T, buthow to compare (1, 3)T and (3, 1)T) [69] This is always the case when vectors are compared

in real numeral spaces Anyway, some of the objective vectors can be extracted for tion In [36] a definition is given: Objective vectors are those where none of the componentscan be improved without deterioration of at least one of the other components However, thisdefinition is usually called Pareto optimality after the French-Italian economist and sociolo-gist Vilfredo Pareto, who in 1896 developed it further [86] However, in some connections,like in [113], the term Edgeworth-Pareto optimality is used for the aforementioned reason.Koopmans was one of the first to employ the concept of Pareto optimality in [55] A moreformal definition of Pareto optimality as follows:

examina-Definition 4 A decision vector x⇤ 2 S is Pareto optimal if there does not exist anotherdecision vector x 2 S such that fi(x) fi(x⇤) for all i = 1, , k and fj(x) < fj(x⇤) for at

An objective vector z⇤ 2 Z is Pareto optimal if there does not exist another objective vector

z 2 Z such that zi  z⇤

i for all i = 1, , k and zj < zj⇤for at least one index j; or equivalently,

z⇤ is Pareto optimal if the decision vector corresponding to it is Pareto optimal For anexample of Pareto Optimality, S⇢ R3 is a feasible region, Z ⇢ R2 is feasible objective regionwhich is the image of S in objective space The illustration is shown in Fig 2.1 Thereare usually a huge number of Pareto optimal solutions This set is called Pareto optimalsolutions or a Pareto optimal set (POS) This set can be non-convex and non-connected Inaddition to Pareto optimality, several other terms are used for the optimality concept such

2.1 COMMON CONCEPTS

Fig 2.1: The feasible region S and feasible objective region Z and the Pareto optimal vectors

as non-inferiority, efficiency and non-dominance The POS in the objective space is calledPareto optimal front (POF) In general, Pareto optimality is used as a concept of optimality,unless stated otherwise

The definitions of a global Pareto optimality and a locally Pareto optimal:

Definition 5 A decision vector x⇤ 2 S is locally Pareto optimal if there exists > 0 suchthat x⇤ is Pareto optimal in ST

B(x⇤, ) An objective vector x⇤ 2 Z is locally Pareto optimal

if the decision vector corresponding to it is locally Pareto optimal 2

Here, B(x⇤, ) is a neighbourhood of x⇤ such that there is no x 2 STB(x⇤, ) for which

fi(x) fi(x⇤) for all i = 1, k and for at least one index j is fj(x) < fj(x⇤)

There are a couple of theorems based on above definitions:

Theorem 1 Let the MOP be convex Then every locally Pareto optimal solution is also

Proof: See [69], page 12

Theorem 2 Let the MOP have a convex feasible region and quasiconvex objective functionswith at least one strictly quasiconvex objective function Then every locally Pareto optimal

Proof: See [69], page 12

According to the definition of Pareto optimality, moving from one Pareto optimal solution

to another necessitates trading o↵ This is one of the basic concepts in multi-objectiveoptimization

2.1 COMMON CONCEPTS

This section gives the definition of weak Pareto optimality, the relationship between thisconcept is that the properly POS is a subset of the POS which is a subset of the weakPOS A vector is weak Pareto optimal if there does not exist any other vector for which allcomponents are better

Definition 6 A decision vector x⇤ 2 S is weak Pareto optimal if there does not exist anotherdecision vector x2 S such that fi(x) < fi(x⇤) for all i = 1, , k 2

An objective vector z⇤ 2 Z is weak Pareto optimal if there does not exist another objectivevector z 2 Z such that zi < x⇤

i for all i = 1, , k or equivalently, if the decision vectorcorresponding to it is weak Pareto optimal In Fig 2.2 represents the set of weak Paretooptimal objective vectors The fact that the POS is a subset of the weak POS can also beseen in the figure The Pareto optimal objective vectors are situated along the line betweenthe dots

Fig 2.2: An example of Weak Pareto optimal vectors

In fact, approximation approaches usually use the concept of dominance to compare solutionswith multiple objectives: An individual x1 is said to dominate x2 if x1 is not worse than x2

on all k objectives and is better than x2 on at least one objective If x1 does not dominate x2

and x2 also does not dominate x1, then x1 and x2 are said to be non-dominated with respect

to each other If using the symbol “ ” to denote that x1 x2 means x1 dominates x2, and

2.2 CONVENTIONAL METHODS

the symbol “7” between two scalars a and b to indicate that a 7 b means a is not worsethan b, then dominance can be formally defined in [26]:

Definition 7 (Dominance): x1 x2 if the following conditions are met:

fj(x1)7 fj(x2) for all j 2 {1, 2, , k}; and,9j 2 {1, 2, , k} : fj(x1) fj(x2) 2

In general, if an individual is not dominated by any other individual in the population, it

is called a non-dominated solution All non-dominated solutions in a population form thenon-dominated set as formally described in the following definition:

Definition 8 (Non-Dominated Set): A set S is said to be the non-dominated set of a ulation P if the following conditions are held:

pop-S ✓ P ; and,For all x⇤ 2 S@x 2 Rn : x x⇤ 2

2.2 Conventional methods

In real world multi-objective optimization, from the set of trade-o↵s generated by an rithm, a single solution is usually chosen by the Decision Maker (DM) That is why the DM’spreferences are so important since they will a↵ect the multi-objective optimization processand will delimit the search space to certain regions of interest if such information is incorpo-rated in the optimization process However, this is not an easy task, there are several varioustechniques have been developed for such purpose In research operations, several classifica-tions of multi-objective optimization techniques (i.e., mathematical programming techniques)have been proposed However, the most popular taxonomies are based on the stage of thesearch at which the DM’s preferences are incorporated In [26] gives a classifications by isdescribed next:

algo-• No-preference methods:For methods not using preference, the DM will receive thesolution of the optimization process They can make the choice to accept or reject it

The no-preference methods are suitable in the case that the DM does not have specificassumptions on the solution The approach of global criterion can be used to demonstratethis class of methods [69] For this approach, the MOPs are transformed into single objectiveoptimization problems by minimizing the distance between some reference points and thefeasible objective region.In the simplest form (using Lp-metrics), the reference point is theideal solution and the problem is represented as follows:

where z⇤ is the ideal vector, and k is the number of objectives, 1  p < 1

When p = 1, it is called a Tchebyche↵ problem with a Tchebyche↵ metric and is presented

as follows:

min max

From the equation, one can see that the obtained solutions depend very much on the choice

of the psvalue Also, at the end the method will only give one solution to the DM

2.2 CONVENTIONAL METHODS

the search process has begun In many real world applications, the DM might be uncertainabout his/her preferences of the problem and a priori techniques are not appropriate in thiscase since they are unsuitable There exist several proposals, some of the most approachesare [69]:

• Lexicographic order: In order, sorting the objective functions according to theirimportance, the objectives are optimized in sequence beginning with the most impor-tant and finishing with the less important The performance of the algorithm is highlydependent on this ordering given by the DM

• Linear aggregating functions: The results of the di↵erent objective functions arecombined into a single fitness value The obtained single value from means of a linearcombination of the objective functions, although that is exits another way to do it Thislinear combination is assigned a relative importance to each of the objective functions.Designing an algorithm that uses a linear combination of objectives might be easy, andalso efficient, but it is impossible for a linear function is to generate non-convex portions

of the Pareto front regardless of the weights used

• Goal programming: The targets that expected to accomplish for each objective aregiven by DM Based on these values, some additional constraints are incorporated intothe problem

In posteriori methods, DM is required to indicate the preferences and select a single solution

or a subset of solutions from a whole set of solutions provided by the MOEA Based on thismethod, there are many proposed MOEAs that are introduced A well-diversified approxi-mation set along the Pareto front provided to DM is major concerned in algorithmic Then

a certain solution is chosen based on certain preferences and priorities and might require afurther analysis In this method a better understanding of the problem as well as its objectivespace is allowed However, in this method, the MOEA produces a large set of solutions Itmight require a high computational cost In fact, a posteriori multi-criteria decision mak-ing approaches are among the most difficult to implement and use [69] There exist several

2.2 CONVENTIONAL METHODS

proposals within this method, the typical approaches are shown:

• Weighting approach: In this approach, a scalar optimization problem is solved inwhich the objective function is a weighted sum of the components of the original vector-valued functions So, varying the components of the objective function can generateall the solutions along the Pareto front

• ✏-Constraint approach: In this approach, one single objective is kept to optimizeand manage the rest of the objectives as constraints within a specific value Withthe approach,the set of solutions in non-convex parts of the Pareto front is found butusually at a very high computational cost

For the Weighting approach, all objectives are combined into a single objective by using aweighted vector The problem in Eq 2.1 is now transformed as follows:

minf (x) = w1f1(x) + w2f2(x) + + wkfk(x)|x 2 S (2.4)

where i = 1, 2, , k and S 2 Rn, wi is weighted vector for ith objective The weighted vector

is usually normalized such that Pk

i=1wi = 1 Then the above equation becomes:

subject to fi(x) ✏i where i = 1, 2, , k, i6= j and D 2 Rn.

In this method, the ✏ vector is determined and used the boundary (upper bound in the case

of minimization) for all objectives i For a given ✏ vector, this method will find an optimalsolution by optimizing objective j By changing ✏, we will obtain a set of optimal solutions.Although, this approach alleviates the difficulty of non-convexity, it still has to face theproblem of selecting appropriate values for the ✏ vector, since it can happen that for a given

✏ vector, it does not exist any feasible solution An example is given in Fig 2.4 where e2, e3

will give an optimal solution respectively, while e1, e4 will result in no solution at all

op-• Sequential multi-objective problem solving approach: Also known as SEMOPS.

In this approach a surrogate objective function is used, it based on the goal levels thatgiven by the DM and aspiration levels, these are di↵erent levels of the objectives whichthe DM desires to achieve

• Goal sequence approach: Suggested as a approach for articulating alternative erence scenarios for the optimization problem at hand In this approach, the articulation

pref-of hard and spref-oft preferences on each objective function is allowed such as priorities andconstraints using logical connectives (”AND and ”OR”) to guide the search towardsmultiple trade-o↵s regions

• Tchebyche↵ approach: In this approach, multi-objective problems are convertedinto a single objective problem, this multiple objectives approaches are combined into

2.2 CONVENTIONAL METHODS

a single one by weighted distances

In general, the interaction can be described step-by-step [69] as follows:

• Step 1: Find an initial feasible solution

• Step 2: Interact with the DM

• Step 3: Obtain a new solution (or a set of new solutions) If the new solution (or one

of them) or one of the previous solutions is acceptable to the DM, stop Otherwise, go

However, there are many disadvantages of using conventional methods such as:

• Weighted sum method: A uniformly distributed set of Pareto optimal solutions can not

be found by using a uniformly distributed set of weight vectors This method can not

be used for non-convex MOP

• ✏ - constrains: It is hard to chose vector ✏ since it must be lied within maximum orminimum values of the individual objective function When the number of objectivesincreases, the user needs more information to chose the vector ✏

Overall, when using conventional methods, the user needs prior knowledge of MOPs to form them into single objective problems A single solution is found for each run and it re-quires fitness function to be linear, continuous and di↵erentiable MOPs having discontinuousand concave POF can not be solved with these methods

trans-2.3 AN OVERVIEW OF MULTI-OBJECTIVE EVOLUTIONARY

a set of solutions instead of only one, makes the measurement of MOEA convergence moredifficult since one individual’s closeness to POFs does not act as a measurement for the entireset Unsurprisingly, then, convergence and diversity are commonly used performance criteriawhen algorithms are assessed and compared to each other [140]

Many MOEAs have been developed To date, there are several ways to classify MOEAs.However, this thesis follows the one used in [25] where they are classified into two categories:

• Step 1: Initialize a population P