Incorporation of human decision making preference into evolutionary multi objective optimization

17 3 Decision Making Preference in Multi Objective Optimization 18 3.1 Features of Human Decision Making.. The fitness functions inthese algorithms are designed with the assumption that

Incorporating Decision Maker Preference in

Multi-Objective Evolutionary Algorithms

Lily Rachmawati

B Eng Hons., NUS

A THESIS SUBMITTED

FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

DEPARTMENT OF ELECTRICAL ENGINEERING

NATIONAL UNIVERSITY OF SINGAPORE

2008

There are many people whom I wish to thank for the support they have rendered

me throughout the course of the Doctoral program.

I gratefully acknowledge the financial support National University of pore during the course of the program I sincerely thank my supervisor, Dr Dipti Srinivasan, for the suggestions and encouragements that helped shape the research direction I also thank the many anonymous referees, whose valuable feedback have contributed greatly to the research work accomplished.

Singa-I am also thankful to the family and friends that have supported me in direct and indirect means from the beginning of the course I would like to extend my especial gratitude to my parents, sister and brother, for the countless advice and encouragement they have given Many thanks as well to Ms Jessica Kusuma,

Ms Riyanti Teresa, Mr Lisman Komaladi, Mr Arief Adhitya, Mr Steven Halim, and many others who have offered prayers, encouragement and support during the writing and revision of this thesis Finally, and most importantly, I would like to thank the almighty God for His enduring grace and love.

1.1 Motivation of Research 3

1.2 Objectives and Scope of Research 6

1.3 Contribution of the Research 9

1.4 Outline of the Thesis 11

2 Concepts and Terminology 12 2.1 Multi-objective Optimization Problem 12

2.2 Pareto Optimality 13

2.3 Pareto Dominance 14

2.4 Decision Maker Preference 16

2.5 Conclusion 17

3 Decision Making Preference in Multi Objective Optimization 18 3.1 Features of Human Decision Making 20

3.2 Desirable Properties of Preference-based Evolutionary Search 23

3.3 Approaches based on the Relative Importance of Objectives 28

3.3.1 MAUT-based Algorithms 29

3.3.2 Lexicographic Ordering 37

3.3.3 Outranking-based Algorithms 38

3.4 Approaches based on a Goal/Reference Vector 41

3.5 Approaches based on Optimality of Trade Off 45

3.6 Other Approaches 50

3.6.1 Population-based Indicator 50

3.6.2 Fuzzy Logic 50

3.7 Issues in Preference Incorporation into Evolutionary Algorithms 51 3.7.1 a priori and Interactive Preference Incorporation 52

3.7.2 Coevolutionary and Fitness-based Preference Integration 56 3.8 Conclusion 58

4 Fitness Functions in Multi objective Evolutionary Algorithms 60 4.1 Deriving Total Order from Partial Order 62

4.2 Aggregative Fitness Functions 66

4.2.1 Random and Dynamic Weighted Aggregation 66

4.2.2 Maximin Function 67

4.3 Dominance-based ranking with niching 68

4.3.1 NSGA-II 70

4.3.2 SPEA2 71

4.3.3 PAES 71

4.3.4 NPGA 72

4.3.5 Distributed Pairwise Comparison 73

4.3.6 SEAMO 74

4.3.7 ε-MOEAs 74

4.3.8 Steady State Replacement 76

4.4 Conclusion 78

5 Imprecise Goal Vectors 79 5.1 Fuzzy Sets 80

5.2 Preference Representation 81

5.2.1 Goal Vectors 82

5.2.2 Degree of Imprecision 83

5.2.3 Invalid Imprecise Goal Vectors 83

5.3 Preference Elicitation 84

5.4 Preference Integration 85

5.5 Empirical Evaluation 89

5.6 Conclusion 95

6 Knee Solutions 97 6.1 Preference Representation 99

6.2 Weighted Sum Niching 100

6.3 Parallel Local Weighted-Sum Based Optimization 103

6.3.1 Stage-One Optimization: Information Gathering 104

6.3.2 Estimation of potential knee solutions 105

6.3.3 Stage-Two Optimization: Integration of Preference into MOEA 108

6.3.4 Interactive Preference Articulation 110

6.4 Computational complexity 111

6.5 Empirical Evaluation 113

6.5.1 Performance Metric 114

6.5.2 Simulation Details 118

6.5.3 Covergence, Accuracy and Diversity: A Comparative Anal-ysis 124

6.5.4 Weighted-sum Niching and Parallel Local Weighted-Sum Optimization: Further Remarks 130

6.6 Conclusion 134

7 Relative Importance of Objectives 137 7.1 Preference Representation 139

7.1.1 Preference structure 139

7.1.2 Mathematical Interpretation 141

7.2 Elicitation of Preference 146

7.3 Integration of Preference information into MOEA 149

7.4 Empirical Evaluation 151

7.4.1 Simulation Details 152

7.4.2 Result and Discussion 157

7.5 Conclusion 161

8 Conclusion 163 8.1 Contributions 163

8.1.1 Imprecise Goal Vector 164

8.1.2 Optimality of Trade-Off 164

8.1.3 Relative Importance of Objectives 165

8.2 Future Work 166

The objective of the majority of work in Evolutionary Multi ObjectiveOptimization (EMOO) is to develop Multi-Objective Evolutionary Al-gorithms (MOEA) able to find a well-distributed approximation of thePareto optimal front An evenly distributed set of non dominated so-lutions equips the decision maker with the trade-off behavior associatedwith the problem at hand so that (s)he could select from the set the mostsuitable solutions

The aposteriori manual application of human preference to the tion of solutions is rendered impractical in higher dimensional problems

selec-as a very large number of solutions would be required to approximate theentire extent of the Pareto front meaningfully An apriori/interactive in-corporation of preference information into a MOEA averts the problem

by concentrating on a subset of the Pareto front The approach avails thedecision maker with a higher resolution in the region of interest in theobjective space and aids the progression of the elite population towardsthe true optima

Human preference in multi-objective decision making contains tainties and anomalies that are to be taken into account in a formal model

uncer-of preference The uniqueness uncer-of the evolutionary computation approachrenders the direct adoption of modelling and implementation techniquesdeveloped for classical optimization approaches unsuitable This the-sis documents research effort into the articulation and incorporation ofpreference information into EMOO Models of preference formulated interms of the importance ranking of objectives, an imprecisely specifiedreference vector, and objective trade off and their implementations inMOEAs are reviewed Three preference incorporation schemes encom-passing the representation, elicitation and implementation of preferenceinformation are also proposed The approaches are designed for easyadoption into major state-of-the-art general-purpose MOEAs The first

guides the population of solutions towards an imprecisely specified goalvector The second directs the search to regions of optimum trade-off inthe Pareto front The third incorporates importance ranking of objectivefunctions into MOEAs The proposed mathematical model of preferencecaters to incomparability and features a functional correspondence be-tween explicated importance ranking of objectives and a specific subset ofthe Pareto front The preference elicitation algorithm devised facilitatesscalable explication of preference.

The preference incorporation techniques are validated in an empiricalstudy that involves difficult test problems and comparison with simi-lar preference-based algorithms as well as baseline MOEAs Preference-based performance metrics are also proposed where applicable to measurethe concord between obtained solutions and explicated preference

List of Figures

2.1 Ideal and Nadir objective vectors 14

3.1 Selection of best solution 25

3.2 Guided dominance (Shaded: area dominated by X i ) 31

3.3 Dotted: Region dominated by solution, Stripes: Region non-dominated with respect to solution 33

3.4 Dotted: Region dominated by solution, Stripes: Region non-dominated with respect to solution 34

3.5 Knee Solution 45

3.6 Knee solution: a bulge of the Pareto front 47

3.7 Knee in Concave Region 49

4.1 Block Diagram: Evolutionary algorithm 61

4.2 Pareto dominance imposes partial order on the objective space 64 4.3 Bi-objective example with Maximin Fitness (given in brackets) 68 5.1 Case 1: Vector T lying in dominating region 87

5.2 Case 2 : Vector T lying in non-dominated region 87

5.3 Case 3 : Vector T lying in dominated region 88

5.4 Pareto front of problem ZDT1 91

5.5 Pareto front of problem ZDT2 91

5.6 Pareto front of problem ZDT3 92

5.7 Results for problem ZDT1 94

5.8 Results for problem ZDT2 95

5.9 Results for problem ZDT3 95

5.10 Results for problem ZDT4 96

6.1 Knee Region 101

6.2 Multiple Knee Regions 102

6.3 Block Diagram 104

6.4 Test Problem DEB2DK-2 with K = 1 to 4 121

6.5 Captures of the population at different instants in stage 2 (DO2DK, K=2, s=1) 131

6.6 Result obtained for test problem DO2DK (K=1,s=0) with par-allel local optimization (δ = 0.1, 0.2, 0.5) 132

6.7 Result obtained for test problem DO2DK (K=2,s=1) with pro-posed approach (δ = 0.1, 0.2, 0.5) 133

6.8 Result obtained for test problem DO2DK (K=4,s=1) with pro-posed approach (δ = 0.1, 0.2, 0.5) 133

6.9 Result obtained for test problem DO2DK (K=2,s=1) with weighted sum niching in [92] 134

6.10 Result obtained for test problem DO2DK (K=4,s=1) with weighted sum niching in [92] 134

7.1 Left: f 1 P f 2 , Middle: f 1 If 2 , Right: f 2 P f 1 142

7.2 List and graphs of c 1 and c 2 143

7.3 Example 1: Three dimensional view (target subset highlighted) 144 7.4 Example 1: View from f 1 -f 2 plane (target subset highlighted) 144

7.5 Example 1: View from f 1 -f 3 plane (target subset highlighted) 145

7.6 Example 1: View from f 2 -f 3 plane (target subset highlighted) 145

7.7 List and graphs of c1, c2 and c3 146

7.8 Pareto front of problem DTLZ2 154

7.9 Pareto front of problem DTLZ5 157

7.10 Generational Distance in Six Objective Problems 160

C.1 Attained solutions for DTLZ8 by (i) -MOEA = [0.02, 0.02, 0.02], (ii)-MOEA =[0.0095, 0.0095, 0.0095], (iii) SSRS, (iv) NSGA-II 176

List of Tables

5.1 Generational Distance 93

6.1 Levels of uncertainty 107

6.2 Additional Computational Cost Incurred 113

6.3 Various Termination Criterion for Stage One 124

6.4 Generational Distance: Result Set 1 125

6.5 Generational Distance : Result Set 2 126

6.6 Non-Dominance Ratio : Result Set 1 127

6.7 Non-Dominance Ratio: Result Set 2 128

6.8 Proportion of Knee Regions Lost 129

7.1 Preference Settings 152

7.2 Non-dominance Ratio 158

C.1 Test Problems 172

C.2 Setting of Parameter  173

C.3 Hypervolume of Dominated Space 178

C.4 Computation Time (in seconds) 179

Journal Papers

1 Srinivasan, D and Rachmawati, L., ”Efficient Fuzzy Evolutionary Based Approach for Solving the Student Project Allocation Problem”, in IEEE Transactions on Education, accepted for future publication.

Algorithm-2 Rachmawati, L and Srinivasan D., ”A Multi-objective Evolutionary rithm with Controllable Focus on the Knees of the Pareto Front”, inIEEE Transactions on Evolutionary Computation, accepted.

Algo-3 Rachmawati, L., ”Incorporating The Notion of Relative Importance of Objectives in Evolutionary Multi-Objective Optimization”, submitted to IEEE Transactions on Evolutionary Computation.

International Conference

1 Rachmawati, L and Srinivasan D., ”A Hybrid Fuzzy Evolutionary rithm for A Multi-Objective Resource Allocation Problem”, in Proceedings

Algo-of the 2005 Hybrid Intelligent Systems Conference, pp 55-60, 2005.

2 Rachmawati, L and D Srinivasan, ”A Fuzzy Evolutionary Algorithm for Combinatorial Optimization of Soft Objectives”, in Third International Conference on Computational Intelligence, Robotics and Autonomous Sys- tems, 13 - 16 Dec 2005, Singapore.

3 Srinivasan, D and Rachmawati L., ”An efficient multi-objective tionary algorithm with steady-state replacement model,” in Proceedings the Genetic and Evolutionary Computation Conference-GECCO06, pp 715-722, 2006.

evolu-4 Rachmawati, L and Srinivasan D., ”A multi-objective evolutionary rithm with weighted-sum niching for convergence on knee regions”, Pro- ceedings the Genetic and Evolutionary Computation Conference-GECCO06,

Algo-of Congress on Evolutionary Computation, 2006.

7 Rachmawati, L and Srinivasan D., ”Dynamic resizing for grid-based ing in evolutionary multi objective optimization”, in Proceedings of Congress

archiv-on Evolutiarchiv-onary Computatiarchiv-on, pp 3975 - 3982, 2007.

8 Rachmawati L and Srinivasan D., ”Multi-Objective Evolutionary assisted Automated Parallel Parking”, in 2008 World Congress on Com- putational Intelligence - WCCI 2008, 2008.

Algorithm-Chapter 1

Introduction

An overwhelming majority of real life problems are multi objective innature Multiple conflicting criteria that are incomparable and non-commensurable must be optimized in the design of various engineeringsystems for which no single utopia solution exists In the absence ofadditional information, the optimum solutions constitute a non-singularset characterized by performance trade off among the various objectives.These solutions are optimal in the Pareto optimal sense, i.e improve-ment in any one objective necessarily entails degradation in another forany single solution [31]

Evolutionary computation techniques have been applied to Multi jective Optimization Problem (MOP) for nearly three decades now [18].Research in the area focuses largely on obtaining a well-distributed ap-proximation of the Pareto optimal front The approximation constitutes

ob-a representob-ative sob-ampling of the trob-ade off surfob-ace, from which the cision maker may select a final solution This thesis presents a studyinto incorporating preference information to direct the optimization inevolutionary multi objective optimization Schemes to represent, elicitand implement preference guidance into Multi Objective EvolutionaryAlgorithms (MOEA) are also proposed and validated in this work Inthe following sections the motivation, objectives and contributions of theresearch undertaken are presented in detail

de-1.1 Motivation of Research

Trade off occupies a central position in MOPs Given the inevitability

of performance trade off between objectives, the availability of multipleoptimum solutions is highly desirable In comparison to conventionalmathematical programming approaches, Evolutionary Algorithms (EA)possesses the advantage of being able to pursue multiple trade off solu-tions in a single run The algorithm’s population-based approach allowsthe generation and propagation of multiple trade off solutions by means

of the application of recombination and mutation operators to solutionsretained through an elitist selection

The elitist selection in a MOEA is facilitated by a population-basedfitness function, which evaluates the quality of a feasible solution to theMOP at hand in terms of its performance relative to other feasible so-lutions in the current population Population-based formulation avertsthe need for an aggregation of objective functions, which often restrictsapplicability and efficacy of the algorithm For example, the convex-ity and differentiability of objective functions are requisite features ofthe applicability of aggregative value functions guiding conventional op-timization methods such as the Step method [5],Geoffrion method [52],the Interactive Surrogate Worth Trade off (ISWT) method [13, 14], andand the Sequential Proxy Optimization Technique (SPOT) [109] Algo-rithms based on the weighted-sum constructs of Multi-Attribute UtilityTheory (MAUT) [62] cannot find intermediate solutions in a concavePareto-front In contrast, an MOEA with the population-based fitnessfunction can successfully discover optimum trade off solutions for anytype of problem This is exemplified by highly popular MOEAs such asNSGA-II [32], SPEA2[129] and PAES [63]

Current research in Evolutionary Multi objective Optimization (EMO)

is largely focused on attaining a well-spread approximation of the entireoptimal trade off surface [18] The sampling of the trade off surface isconsidered desirable as it equips the decision maker with the requisiteinformation to select a final solution based on preference [17] Numerous

MOEAs proposed in the literature successfully converge on the optimumtrade off surface (e.g [2, 6, 32, 35, 63, 129]) The fitness functions inthese algorithms are designed with the assumption that decision makingpreference is absent.

In multi objective optimization practice, however, human decisionmaking preference is pivotal in that it resolves the conflict between ob-jectives to enable the selection of a final solution from the set of tradeoff solutions Except for a few preliminary works that tailor goal pro-gramming (e.g [45, 116]) and the weighted-sum formulation of trade off(e.g [8, 10]) for use in evolutionary techniques, preference informationhas been largely relegated to the post-optimization stage That is, ac-cording to a commonly employed taxonomy in [128], most MOEAs are aposteriori techniques

a posteriori techniques aim to discover a representative sampling ofthe optimum trade off surface to be presented as candidate solutions tothe decision maker once the optimization is completed In a large major-ity of MOEAs, the fitness function is designed as a two-tiered evaluation

of a dominance-based ranking and diversity in order to obtain an distributed approximation of the optimum trade off front [31] Equippedwith the trade off information, the decision maker(s) selects the mostsuitable solution based on his/her preference for the particular problem

evenly-By deferring preference-based decision making until optimization isaccomplished, a number of challenging computational issues are averted.However, new problems are introduced The computational cost incurred

in supplying the decision maker with a set of alternatives exhibiting ing trade offs is considerable in many-objective problems The increase

vary-in the number of objectives results vary-in an exponential vary-increase of thesearch space and the number of solutions required to adequately approx-imate the Pareto optimal front becomes intractable with large numbers ofobjectives Computationally expensive population-wide Pareto rankingand diversity preservation measures as well as solution archival commonlyfound in the MOEAs are expended to discover and retain solutions whichmay be of low relevance to the decision maker Streamlining efforts have

been introduced (e.g in [58]) to reduce computational cost but furtherresearch is necessary for significant improvement in this respect Besidesefficiency, search efficacy also becomes an issue as algorithms based onthe Pareto optimality criterion could not achieve much progress [18] inmany-objective problems So many solutions would be non-dominatedwith respect to each other that no effective selection pressure is applied

in the MOEA Further, the a posteriori analysis and exploration of theobtained approximation of the trade off front could prove intractable todecision makers in many-objective problems

A priori and interactive incorporation of preference into multi tive search algorithms delimits search directions using explicated prefer-ence information While a priori algorithms assume a stable and con-stant preference throughout the optimization, progressive techniques al-low preference to be modified during the search Mathematical model ofdecision making preference is requisite in both a priori and interactiveapproaches

objec-The subjective nature of human preference in multi objective decisionmaking results in imprecision The representation of preference in math-ematical form must accommodate the imprecision Although many pref-erence models have been proposed and employed in conventional multiobjective optimization, e.g goal programming, MAUT [62] and Out-ranking [99, 100, 101, 102, 103, 11], direct and uncritical adoption of themodels into EA neglects the unique capacities and search characteristic

of evolutionary approaches

The population-based characteristic of EA implies that a priori andinteractive incorporation of preference could be harnessed to produce ahigher resolution in the sampling of the optimal region of interest In con-trast, non ad-hoc conventional preference models constitute aggregationsinto a single criterion that yields a single optimum solution if adopted

in its original form An undiscriminating use of preference-motivatedcriterion may also degrade the efficacy of the evolutionary search, whichultimately relies on genetic variations of existing solutions to generatebetter solutions

While it is clear that a focus on a subset of the objective space forded by preference-guided optimization is beneficial, there are manychallenges in incorporating preference information into MOEAs Thebenefits of preference-directed exploration of the search space and thelack of work investigating the representation, elicitation and implemen-tation of preference information into MOEAs motivate the research workdocumented in this thesis.

The overall goal of the research is to investigate a priori / progressiveincorporation of human decision making preference into MOEAs Thework focuses on the representation and elicitation of human preferenceinformation as well as the means of effectively integrating the preferenceinformation to guide a multi objective optimization

Human decision making preference in a multi objective context couldassume several different formulations depending on the needs at hand.Relative importance of objectives, reference vector and optimum tradeoff are three major preference notions found in classical multi objectiveoptimization and Multi Criteria Decision Making (MCDM) literature

A formalization of preference notions which are by nature uncertain, isrequired for effective use of preference information in multi objective op-timization Once a mathematical model of preference is constructed,

an algorithm to facilitate the instantiation of the designated model ofpreference is also required to assist the human decision maker in sys-tematically explicating his/her preference coherently Implementationtechniques pertaining to the modification in the MOEA to accommodatepreference information also need to be devised Explicated preference in-formation should be integrated into the evolutionary optimization suchthat the focus on the desired subset of the objective space is gainedwithout adverse effects on the ability of the evolutionary algorithm toconverge on the optimum

Although human decision making preference has been studied for along time, particularly in the areas of MCDM and Multi Criteria DecisionAid (MCDA), preference models and techniques developed in these areascannot be indiscriminately imported for use in MOEAs Conventionalmulti objective optimization approaches aim at finding a single satisficingsolution, which may or may not be Pareto optimal The uniqueness ofthe evolutionary computation approach warrants the design of differentsuitable preference models.

The thesis is concerned with the modeling of preference in a tion suitable for exploitation in evolutionary techniques and the relevantelicitation methods It also aims at enabling the integration of preferencemodels into the current state-of-the-art general purpose MOEAs to allowimmediate deployment of preference models in widely-used MOEAs

formula-The main objectives and milestones of the research work are ated below:

enumer-1 A study into the characteristics of the fitness functions employed instate-of-the-art MOEAs This study investigates Pareto-dominancebased ranking and the diversity preservation measures built intothe fitness function of successful MOEAs Representative work inthe design of the fitness functions of the MOEAs is examined andimprovements are proposed based on the study The milestonesinclude:

(a) the introduction of a chain-based model as a characterization ofthe induction of a total order from a partial order in dominance-based ranking,

(b) the design and validation of a steady-state replacement modelfor elitist archival in an MOEA,

(c) the investigation into diversity preservation techniques and archivalpolicy of MOEAs with respect to the capacity for finding andretaining solutions close to the true optima, and

(d) the design and validation of a dynamic resizing heuristic forgrid-based diversity preservation, a computationally efficient

and highly scalable niching method suitable for many-objectiveproblems.

2 A study into the characteristics of preference models in classicalMCDM and MCDA practices The milestones include:

(a) The review of preference models commonly encountered in multiobjective optimization practice and their implementation inclassical optimization methods and

(b) The formulation of necessary and desirable properties in theincorporation of decision maker preference into an evolutionarymulti objective optimizer

3 A framework for the representation, elicitation, and implementation

of decision maker preference for knee solutions Major milestonesinclude:

(a) the construction of a representation framework for specifyingpreference at verying levels of uncertainty by means of a lin-guistic variable and

(b) the implementation of the preference model in the fitness tion of state-of-the-art elitist MOEAs Multiple weighted ag-gregations of objectives impose a selection pressure favoringsolutions with optimum trade off, where the appointed weightcoefficients also constitute the mathematical denotation of thevarying levels of uncertainty The resulting focus on the regionswith optimum trade off is controllable in its extent

func-4 A framework for the representation, elicitation, and tion of decision maker preference expressed in terms of the relativeimportance of objectives Major milestones include:

implementa-(a) The construction of a representation framework based on nary importance relations catering to pair-wise importance com-parison of objectives The axiomatic representation framework

bi-allows precedence, indifference and incomparability of tive importance to be expressed and synthesized in a coherentpartial ranking of objectives The mathematical denotation

objec-of objective importance ranking accommodates uncertainty bymeans of a set-based functional correspondence between objec-tive importance ranking and the target subset of the trade offsurface

(b) The design of an algorithm to elicit the overall preference bymeans of (1) direct partial ranking of objectives involved, and/or(2) assertion of pair-wise relative importance Coherence of theoverall preference is maintained throughout the elicitation, andscalability with respect to the number of objectives is ensured.(c) The design and investigation into three different ways of in-tegrating preference information into a well-known baseline,NSGA-II [32]

The main contributions of the thesis are as follows:

1 The development of a steady-state replacement model to facilitateelitism in MOEAs with simpler computational requirements andcompetitive performance with Pareto-ranking based methods Thereplacement model has also been validated in an empirical studyinvolving difficult test problems, with significantly favorable results

2 The development of a dynamic resizing heuristic for use in an elitistMOEA with grid-based diversity preservation measure The resizingheuristic allows the sizes of sub-divisions to be altered as the span

of the non-dominated fronts vary during the course of the search.This facilitates effective diversity preservation while averting frontdeterioration

3 The development of frameworks for the representation and

elici-tation of decision maker preference in terms of an imprecise goalvector An integration method has also been developed by sub-stantially building on the existing work in this area to facilitateconvergence on a user-specified subset with designated imprecisionlevels expressed in the diversity of solutions.

4 The development of frameworks for the representation and tion of decision maker preference for solutions with optimum ob-jective trade off Implementations of the explicated preference instate-of-the-art MOEAs have also been proposed and validated in anempirical study with various difficult test problems, some of whichwere devised in the course of this research

elicita-Previous work that feature multi objective optimization algorithmswith a focus on the region of optimum trade off in the Pareto-front (e.g in references [9] and [27]) does not include a mechanism

to control the extent of focus Further, the method proposed inthis thesis exhibits better scalability with respect to the number

of objectives and enhanced efficacy in problems with non-convexPareto front

5 The development of frameworks for the representation and tion of decision maker preference in terms of the relative impor-tance of objectives Implementations of the explicated preference

elicita-in state-of-the-art MOEAs have also been proposed and validated

in an empirical study The work in objective importance ranking isunique in that it accommodates incomparability, an important no-tion in practical multi objective optimization, provides a succinctregion-based mathematical interpretation of the importance rank-ing of objectives in the objective space, and facilitates the scalableelicitation and implementation of preference

6 The development of a performance metric to measure the bility of obtained solutions with the explicated decision maker pref-erence

compati-1.4 Outline of the Thesis

This thesis is divided into 8 chapters In chapter 2 the thesis commenceswith the definition of terms and concepts that would be employed re-peatedly throughout the work This includes a review of the features ofhuman preference in multi objective decision making Chapter 3 of thethesis presents a review of preference models widely-employed in multiobjective optimization and current work in preference incorporation inEMOO Salient issues in the incorporation of preference are also identi-fied and studied

In chapter 4 an investigation into the state-of-the-art in fitness mulation in general-purpose MOEAS is documented The aim is to iden-tify properties essential to successful multi objective optimization with

for-an evolutionary approach The knowledge motivates the strategic for-andeffective incorporation of preference information into the framework ofexisting MOEAs Chapters 5 to 7 discuss three preference incorpora-tion schemes pertaining to imprecise goal vectors, optimum trade off andobjective importance ranking respectively Empirical studies with testproblems of varying difficulties validate the efficacy of the approachesproposed Chapter 8 outlines future work and concludes the thesis

Chapter 2

Concepts and Terminology

This chapter reviews the salient concepts in evolutionary multi-objectiveoptimization The terminology and notation presented here are employedthroughout the remainder of this work

The multi-objective optimization problem (MOP) that constitutes thesubject of this work is stated as follows:

Definition 1 MOP

Find the vectors of decision variables X∗= [x1, x2, xN] that satisfy

p inequality constraints: g i X∗≥ 0, i = 1 , 2 , , p

q equality constraints: h i X∗= 0, i = 1 , 2 , , q And minimize M conflicting objective functions:

F = [f 1 X∗, f 2 X∗, , f M X∗] where f m : R n → R

The objective functions in F define a mapping of an N -dimensional decision variable vector in the variable space to an M -dimensional objective vector in the objective space The inequality and equality constraints gi(X) and hi(X) delineate a non-empty feasible region S in the variable space, that corresponds

to the feasible region in the objective space denoted by Z.

The objective functions in F are in general non-commensurable and parable Non-commensurability implies that objective functions are defined in different units and scales while incomparability implies that in the absence of explicit decision maker preference, no objective in F is more important than another The conflict among the objectives precludes the simultaneous op- timization of all objectives and therefore the existence of the ideal solution which is optimum in terms of all objective functions The incomparability, non-commensurability and conflict between objectives result in a partial order among solutions in the objective space.

Pareto optimality criterion encapsulates the non commensurability, bility and conflict between objectives The criterion was formulated by Pareto and Edgeworth in the course of working on economic efficiency and income distribution [83] The definition of Pareto-optimality and related concepts are described in the following:

incompara-Definition 2 Pareto-optimality

A solution vector X∗is Pareto-optimal if there does not exist another decision vector

X ∈ S such that X performs at least as well as X ∗ in all objectives and better in

at least one objective, i.e.

∀m ∈ [1, M ], f m (X) ≤ f m (X∗) ∧ ∃l ∈ [1, M ], f l (X) < f l (X∗) (2.1)

An objective vector Z∗ = F(X∗) is Pareto-optimal if the corresponding decision vector is Pareto-optimal.

Definition 3 Weak Pareto-optimality

A decision vector Xi∈ S is weakly Pareto optimal if there does not exist another decision vector X j ∈ S such that ∀m ∈ [1, M ], f m (X j ) < f m (X i ) The Pareto- optimal set is a subset of the weakly Pareto-optimal set.

Definition 4 Pareto-front

The Pareto-front is the set of objective vectors corresponding to Pareto-optimal solutions, {F (X∗)} A continuous Pareto-front is convex if for every M objective vectors in the Pareto-front, there does not exist a vector taken from the hyper plane containing the M objective vectors that dominates at least one member of the Pareto-front Otherwise, it is non-convex.

Figure 2.1: Ideal and Nadir objective vectors

Definition 5 Extreme solutions

An extreme solution (Xextreme∗ ) is a Pareto optimal solution that corresponds to the maximum attainment in an objective function f m , m ∈ [1 M ] in the Pareto-optimal set, i.e.

or imperfect objective vector, denoted as Fo = [f1o, , fmo, , fMo ] The ideal and nadir objective vectors enclose the complete Pareto-optimal set and are illustrated

in Figure 2.1.

The multi-dimensional objective space of a MOP is ill-structured The conflict, incomparability and non-commensurability of constituent objectives render a simple ordering of vectors impossible Pareto-dominance, derived from Pareto- optimality, provides a criterion by which solution vectors may be compared The definition of Pareto-dominance and related concepts are given in the following.

Definition 7 Pareto-dominance

For any two solution vectors X i , X j ∈ S, solution vector X i dominates the vector

X j if and only if X i performs at least as well as X j in all objectives and strictly better than X j in at least one objective, i.e.

X i ≺ X j ⇔ ∀m ∈ [1 M ], f m (X i ) ≤ f m (X j ) ∧ ∃l ∈ [1 M ], f l (X i ) < f l (X j )

(2.3) Pareto-dominance is a binary relation with the following properties:

Definition 8 Weak Pareto-dominance

A solution X i weakly dominates solution X j if X i performs at least as well as X j

in all objectives, i.e.

X i  X j ⇔ ∀m ∈ [1 M ], f m (X i ) ≤ f m (X j ) (2.4) Definition 9 Strong Pareto-dominance

A solution Xi strongly dominates solution Xj if Xi performs better than Xj in all objectives, i.e.

X i  X j ⇔ ∀m ∈ [1 M ], f m (X i ) < f m (X j ) (2.5) Pareto-dominance relations impose a partial order in the objective space and solutions which are not described by the dominance relation are non-dominated Non-dominance is intransitive and symmetric In the following two related but distinct concepts, the non-dominated set of solutions and the set of non- dominated solutions are defined.

Definition 10 Non-dominated set of solutions

A set of solutions in which no member pair exhibits Pareto dominance relation is called a non-dominated set of solutions, denoted by XN D.

@X i , X j ∈ X N D , X i ≺ X j (2.6)

Definition 11 Set of non-dominated solutions

The set of non-dominated solutions, denoted by X∗,consists of solutions that are not dominated by any other solution in the feasible region, i.e.

∀Xi∈ X ∗

A set of non-dominated solutions is also a non-dominated set of solutions, as no member of the set dominates another The inverse is not true, i.e a non-dominated set of solutions is not always a set of non-dominated solutions A member of X N D

could be dominated by a non-member solution.

Human preference occupies central position in resolving the conflict between the various objectives in a multi-objective optimization problem Mathematically, all Pareto-optimal solutions are incomparable and any one of them constitutes

an acceptable solution to the MOP at hand In practice, only one Pareto optimal solution is to be adopted as the final solution.

Selection of one of the Pareto optimal solutions requires information not contained in the objective functions themselves This subjective information

is furnished by the Decision Maker, a person or a group of persons with the additional insight into the practical requirements of the solution to the problem

at hand, the ability to express preference between alternative solutions in a non-dominated set and the responsibility for the selection of the final solution The work presented in this thesis is concerned with human decision making preference that imposes an ordering of alternatives in a non-dominated set on the basis of a practical insight into the problem at hand The term preference in this thesis is employed to denote the internal judgment of decision maker(s) that introduces an ordering of members in a non-dominated set of solutions based

on the corresponding overall performance in F.

In the description of preference, the concepts of binary relation and ence structure are pivotal The definitions of the two are given in the following Definition 12 Binary Relation

prefer-Let S be a finite set of solutions X1, Xi, Xj , XN A binary relation R on the set

S is a subset of the Cartesian product SXS, that is a set of ordered pairs (X i , X j ) such that X i and X j are in S : R ⊆ SXS For an ordered pair (X i , X j ) that belongs to R, the following notations are interchangeably used: (X i , X j ) ∈ R, or

X i RX j

Definition 13 Preference Structure

Let S be a finite set of solutions X 1 , X i , X j , X N A preference structure P consists of a group of binary relations R defined on the set of alternatives S such that for each pair in S, one and only one binary relation from the set structure

is satisfied A preference structure defines a partition of the set SXS For each preference structure there exists a unique relation, the characteristic relation, from which the binary relations composing the preference structure can be deduced That

is, the collection of binary relations can be defined through the epistemic states of the characteristic binary relation.

In this chapter the concepts and terminologies in Multi-Objective Optimization are revisited Features of human behavior in multi-objective decision making and issues in the representation of preference have been described.

These features and concerns must be adequately addressed in the tion of preference models and the elicitation and integration of preference for use in MOEAs The next chapter discusses the best-known mathematical mod- els of three preference notions: the importance ranking of objectives, proximity

construc-to a desired performance vecconstruc-tor and optimality of trade-off The attributes of each preference model and existing modifications and adoption into MOEAs are discussed.

Chapter 3

Decision Making Preference

in Multi Objective

Optimization

This chapter presents a review of the a priori / interactive incorporation

of preference into Multi Objective Optimization with particular focus onevolutionary approaches Preference information has been widely utilized

in classical a priori / interactive optimization algorithms [77] ing formulations, such as the distance to a pre-specified goal vector or avalue function, direct the search in these methods

Aggregat-The use of preference information in evolutionary multi objectivemethods was first recorded in [43], where goal dominance was defined toguide the population of solutions towards a reference vector specified bythe decision maker A number of other researchers have since proposedother approaches to incorporate decision maker preference in MOEAs,with the aim of approximating in greater detail a region of interest in thePareto front

Preference-based multi-criteria decision making is characterized bypeculiarities and imprecision The use of preference information in an

MOEA requires the accomplishment of three tasks:

1 Preference modeling A model represents the internal preference

of the decision maker in a mathematical form The modeling ofpreference can be achieved by direct comparison of alternative so-lutions or induction of preference relations between solutions fromtheir associated attributes [120] The former method is impracticalwith increasing number of alternatives and objectives This work

is concerned only with the latter, where preference is represented

by a mathematical model in terms of the attributes of alternativesolutions

2 Preference elicitation Elicitation facilitates the systematic sion of the decision maker’s internal preference in terms of the se-lected model A separate elicitation algorithm may be requireddepending on the complexity of the preference model

expres-3 Preference integration Preference information expressed in terms

of the selected model must be integrated into the optimization gorithm to inform the search or solution generation process Inconventional approaches the integration of preference is simple InMOEAs, preference could be embedded in the fitness evaluation as amodified Pareto-dominance criterion (e.g [8, 9, 61]) or a bias on thedistribution of solutions (e.g [10]) Alternatively, preference could

al-be integrated by means of penalty functions in a co evolutionaryframework (e.g [24])

In the following sections, issues in the modeling, elicitation and gration of preference in EMOO are discussed The rest of the chapter

inte-is organized as follows Section 3.1 presents the pertinent features inhuman decision making behavior Issues surrounding incorporation ofpreference conclude the chapter in section 3.2 These issues concern nec-essary features in preference modeling, elicitation and integration as well

as the validation of algorithm performance Sections 3.3 to 3.5 presentthe most-widely employed preference models in evolutionary approaches

The models produce an ordering of solutions by means of evaluations ofassociated attributes and could be classified into three main groups Thefirst formulates preference in terms of the relative importance of objec-tives, the second in terms of trade-off property, and the third in terms ofproximity to a reference objective vector The design, instantiation anduse of these models in evolutionary approaches are discussed according

to the preference notions formalized Section 3.7 presents a discussion ofthe various issues in preference incorporation and concludes the chapter

Human behavior in multi objective decision making is characterized by

a number of peculiarities that include the following:

1 Limited span of short term memory [113] Although repeated cutions of a task could improve the span of working memory [40],decision makers often choose to simplify decision task by replac-ing some of the objectives with constraints, eliminating some of theobjectives, or grouping of alternatives [69], to compensate for thisaspect

exe-The partiality of preference internal in the decision maker may sult in contradictions for complex decision tasks, particularly wheninexperienced decision makers are involved [69] It is necessary thatpreference models and elicitation algorithm be designed to allowpartial information to be combined into a coherent structure

re-2 Neglect of small quantitative differences in the evaluation of natives The intransitivity of preference, an oft-cited irregularitythat prevents the development of a mathematical model with welldefined analytical properties, is attributable to this property Tver-sky in his study on the stable intransitivity of choice [117] conducted

alter-an experiment where subjects were presented with successive pairs

of alternatives where slight improvement in gain was accompanied

by a slight increase of cost Persistently, the alternative with the

higher gain was preferred Yet, when given alternatives from thefirst and last pairs presented, the one from the first, associated withlowest gain, was persistently preferred as the subjects were not will-ing to render the entailed increase in payment for a correspondingimprovement in gain The experiment showed intransitivity of pref-erence.

Intransitivity of indifference, where applicable, is also attributable

to the neglect of small quantitative differences, as is demonstrated

by the famous coffee example in [74] Luce [74] gives the followingexample to illustrate the unsuitability of transitivity in indifference:since a decision maker cannot make a difference between coffee with

n grains of sugar and n + 1 grains of sugar, transitivity dictates thatthe decision maker is indifferent between coffee with 1 grain of sugarand coffee with 1000 grains of sugar

Besides intransitivity of preferences, the inability to take into count the small differences in the evaluation also accounts for theirrational behavior of eliminating dominating alternatives by con-servation of the dominated [66]

ac-3 Absence of preconceived decision rules Von Winterfeldt and wards [119] noted that it is unlikely that utilities and numbers ex-pressing subjective evaluation of alternatives are stored in the minduntil elicited Learning time and process are usually required forthe decision maker to come up with a good set of decision rules

Ed-4 Multi objective decision making is often performed by a group ofdecision makers with independent preferences Arrow’s Impossibil-ity Theorem is arguably the most important result in group decisionmaking behavior It is given in the following:

Let ND be the number of decision makers Assume that ND ≥ 2 andthere are at least three alternative solutions to choose from Eachindividual decision maker has an ordering, denoted by Cd, 1 ≤ d ≤

ND The collective choice problem is defined by producing a group

preference C from the set of individual preference, i.e.:

as it was defined in the original set

(c) For every Xi and Xj there is a set of individual ordering suchthat the group prefers Xi to Xj

(d) There is no individual decision maker d such that if he prefers

Xi to Xj, the group also prefers Xi to Xj for all Xi and Xj.That is, there is no dictator in the group whose individualpreference constitutes the entire group’s preference

(e) Assume that for a set of individual ordering the group prefers

Xi to Xj, and the set of odering is modified such that

i individual orderings of the alternatives other than Xi arenot changed

ii individual orderings involving Xiare not changed or changed

compar-and Coello [20] compar-and Scott et al [111], in the context of ing design problems perfect independence of individual preferenceshardly ever occurs Interaction between members of the group al-lows preferences to be adjusted Further, Arrow’s theorem dealswith the consensus on the preference-based ordering of candidatesolutions A consensus ranking of available alternatives is in prac-tice less important than an agreement on the best solution.

Evo-lutionary Search

This section summarizes the desirable properties of preference-based EMOObased on the features of human decision making presented in the preced-ing section and the characteristics of evolutionary approaches

Convergence to the Pareto-optimal front is paramount in multi jective optimizers A mathematical proof of convergence cannot be fur-nished for evolutionary algorithms due to their stochastic nature In-stead, preference incorporation into an MOEA should be shown to pre-serve the Pareto-dominance relations between members of the population

ob-as noted in [17] This feature ob-ascertains that progression towards the truePareto optima is promoted In addition, a few other properties are im-portant in incorporating preference into an MOEA as enumerated in thefollowing:

1 Functional mapping between preferences explicated in the matical model and the desired Pareto-optimal solution(s)

mathe-The preference-based ordering of candidate solutions in this work

is obtained indirectly from the associated attributes The mappingbetween the preference represented by the mathematical model em-ployed and the ordering produced in the solution and objective spaceshould be functional in nature That is, for a particular instanti-ation of the preference model, there is a consistent ordering of the

candidate solutions in an MOP.

Further, the functional mapping should be surjective Surjectivitydemands that for any possible ordering of non-dominated solutionsthere is one corresponding instantiation of the preference model.The requirement could be relaxed as it has been pointed out earlier

in view of the Arrow’s Impossibility Theorem to pertain only tothe designation of the best solutions instead of the precise ordering.The surjective requirement could thus be formulated as follows Forall solutions Xi in the non-dominated set, there should be an instan-tiation of the preference model such that the solution is designated

as the most preferred solution

The functional mapping also enables concord between expressedpreference and obtained solutions to be measured, thus allowingthe efficacy of preference-guided optimization to be evaluated

2 Set-based denotation of preference in the objective space

A set-based denotation of preference in the ordering of solutionsmimics the grouping of alternatives commonly performed by humandecision makers [69] As outlined in the previous section, impreci-sion and anomalies are inherent in human decision making behav-ior The mismatch between preference asserted within the providedmathematical framework and actual preference over candidate solu-tions could occur because of the uncertainties and anomalies that acoherent mathematical model could not accommodate For exam-ple, a decision maker that prefers solutions with optimum trade-offmay not regard the solution lying precisely at the acute point of theknee (X1 in Fig 3.1) as the most desirable An adjacent solutionwith slightly worse trade-off property (X2 in Fig 3.1), may be moreacceptable due to practical considerations

The availability of a set of alternatives enhances the possibility thatthe decision maker would find the most suitable solution while obvi-ating the need for repeated execution of the optimization algorithm

As such, an even distribution of solutions to approximate the subset

Figure 3.1: Selection of best solution

of interest in the Pareto-front is also desirable Set-based denotation

of preference also harnesses the population-based feature of MOEAs

to an advantage over other multi objective optimization algorithms.Surjectivity requirement of the functional mapping between attributeand solution spaces implies that the preference model should accom-modate the designation of any solution as part of the preferred sub-set Let the mathematical model of preference be Ψ(ψ1, ψm, f1, fm).Completeness requires that there exists a set of values for the pa-rameters ψ1to ψ1 in the model such that ∀Xi ∈ X∗, Xi is a member

of the subset of the Pareto optima corresponding to the preference

3 Scalability with respect to the number of objectives

Preference-guided evolutionary multi objective search is especiallyimportant in high-dimensional applications where an intractablylarge number of solutions would be required to approximate mean-ingfully the Pareto-optimal front Even if the approximation could

be furnished in sufficient detail, an a posteriori examination ofthe concordance between available solutions and preference requiresenormous effort and resources in such problems Scalability is im-

perative in the modelling, elicitation and implementation of ence in MOEAs.

prefer-4 Non-inferior convergence

The convergence property of the preference-guided evolutionary gorithm should be at least as good as that of a comparable generalpurpose counterpart that seeks to approximate the entire Pareto-front Evolutionary algorithms find an optimum solution by effec-tive exploration of the search space and exploitation of the immedi-ate surrounding of an existing candidate solution Exploration andexploitation is performed by the generation of new solutions fromsolutions in the current population Preference-based focus shouldnot degrade the search power of the general purpose optimizationalgorithm

al-In algorithms designed for multi-modal as well as general purposemulti objective optimization, niching is commonly applied to pre-vent the population from being trapped in a local optima and/or

a subset of the Pareto front The role of niching measures is toinduce exploration of the entire extent of the objective space Inpreference-guided multi objective evolutionary search, the approx-imation of a restricted subset of the Pareto-front is desired Theexclusion of solutions that are not compatible with explicated deci-sion maker preference could inflict adverse effects on the efficacy ofthe algorithm to promote convergence on the Pareto front

The surjective functional property desirable in a preference-based gorithm is largely a requirement in the design of the preference model.Integration of preference information in the implemented MOEA shouldalso facilitate the discovery of solutions in the desired subset of thePareto-front, and allow the retention of these solutions

al-A set based denotation of preference could be achieved by ing a binary relation that expresses incomparability between alternatives

incorporat-in the preference structure Incomparability was incorporat-introduced to handleuncertainty resulting from ambiguity, imprecision or lack of information

The most traditional preference structure is a crisp structure dating the precedence and indifference relations, in which for any pair ofalternatives Xi, Xj ∈ S, either one of the following applies:

The set-based denotation of an instantiated preference model impliesthat in the implementation of preference there is a need for a mechanism

to control the extent of focus in the MOEA Branke first drew attention

to the issue in his collaboration with Deb on guided MOEA and biasedcrowding distance [10] The extent of focus represents the degree ofuncertainty in the instantiated preference model A means to control theextent of the subset obtained at the end of the optimization run facilitates

a comprehensive sampling of the specified target subset Thus far, onlyBranke and Deb’s work provide a control variable to vary the extent ofthe pursued subset of the Pareto front

Desirable properties in the control mechanism in a preference plementation scheme include the ability for fine-grained control over theextent of focus, the decoupling of preference-based focus from the conver-gence and diversity of obtained solution set, and robustness over varyingPareto-front geometry

im-Fine grained control allows incremental change in the extent of thePareto-front subset obtained by the MOEA Decoupling of control effectsfrom the search progression towards the true optima and the preserva-tion of diversity within the target area is desirable The extent of the