An ant colony optimization approach for phylogenetic tree reconstruction problem

ACO technique is widely used in various types of combinatorial optimization problems including in bioinformatics Dorigo and Stutzle, 2004.. 2000.In molecular phylogenetics, the sequences

Vietnam National University, Hanoi

College of T echnology

H uy Q uang D inh

A n A nt Colony O p tim izatio n

A p p ro ach for P hylogenetic Tree

R eco n stru ctio n P ro b le m

M ajor : Inform ation Technology

C o n te n ts

1.1 M o tiv a tio n 1

1.1.1 C om putational B io lo g y 1

1.1.2 Phytogeny R e c o n s tru c tio n 2

1.2 Thesis Works and S t r u c t u r e 3

2 P h y lo g e n e tic T re e R e c o n s tru c tio n 4 2.1 Phylogenetic T r e e s 4

2.2 Sequence Alignment 7

2.2.1 Biological D a t a 7

2.2.2 Pairwise and Multiple sequence a l i g n m e n t 8

2.3 Approaches for phylogeny re c o n stru c tio n 11

2.4 Maximum Parsimony Principle 11

2.4.1 Parsimony C o n c e p t 11

2.4.2 Counting evolutionary changes 12

2.4.3 Rem arks on Maximum Parsimony A p p ro a c h e s 14

2.5 Finding the best tree by heuristic searches 15

2.5.1 Sequential Addition Methods 15

2.5.2 Tree Arrangement M e t h o d s 16

vi

C on ten ts _ _ vn

2.5.3 O ther heuristic search m e t h o d s 19

3 A n t C olony O p tim iz a tio n 20 3.1 The Ant Algorithms 20

3.1.1 Double bridge ex p erim en ts 20

3.1.2 Ant S y s t e m 22

3.1.3 Ant Colony S y s te m 24

3.1.4 Max-Min Ant S y stem 25

3.2 Ant Colony Optimization M e ta -h e u ristic 27

3.2.1 Problem R e p re s e n ta tio n 27

3.2.2 Artificial A n t s 28

3.2.3 Meta-heuristic S c h e m e 29

3.3 Remarks on ACO A p p lic a tio n 30

3.4 ACO approaches in p h y lo g en etics 31

4 P h ylogen etic Inference w ith A nt C olony O ptim ization 33 4.1 Related W orks 33

4.2 Tree Graph D escription 34

4.2.1 BD Tree C o d e 34

4.2.2 State Graph D e sc rip tio n 38

4.3 Our ACO-applicd A p p ro a c h 39

4.3.1 Pheromoue Trail and Heuristic Information 40

4.3.2 Solution Construction Procedure .40

4.3.3 Pheromonc Update Chosen Procedure 42

4.4 Simulation Results 42

4.4.1 Simulated Data 43

4.4.2 Real D a t a 44

4.5 D iscussion 46

A p p e n d ix 57

.1 Probabilistic Decision R u le 57

.2 Tree encoding from a BD tree c o d e 57

.3 BD tree code Decoding a lg o rith m 58

.4 ACC) Solution Construction P r o c e d u r e 58

.5 Pltcromone Trails Update P r o c e d u r e 59

.6 Algorithm for calculating evolutionary changes 60

List of Figures

1.1 The exponentially growth of nucleotide d a ta b a s e s 2

2.1 A looted tree of life 5

2.2 Unrooted tree representation of annelid relationships 6

2.3 Three possible topologies of unrooted tree for four t a x a 7

2.4 An example of four types of nucleotide mutations (Nei and Kumar, 2 0 0 0 ) 9

2.5 Multiple Sequence Alignment E x a m p le 10

2.6 An example for Fitch a lg o rith m 13

2.7 An example of sequential addition m e t h o d 16

2.8 An example of Nearest-Neighbor Interchange O p e ra tio n 17

2.9 An example of Subtree Pruning and Regrafting Operation 18

2.10 An example of Tree Bisection and Rcconnncction O p e r a t io n 18

3.1 Experimental setup for the double bridge e x p e r im e n t 21

3.2 Results gained in the double bridge experiment 22

4.1 Example of encoding a tree from a given BD tree code .36

4.2 Graph structure description with a*,iV p la n e 38

4.3 An example of tree building on the a, N p la n e 39

4.4 A found tree with 17 real species .45

ix

List of T ables

generated true tree arc identical in the simulated d ata instances 434.2 Simulation results with real d ata of our proposed a p p ro a c h 44

x

D atabase of Jap an (D D BJ) (see Figure 1.1) The size of the GenBank database is extrem ely large: over 65 billion DNA ba.se pairs in 61 million molecular sequences

J T his drastic grow th of biological d a ta requires com putational tools for biological data (.so-called bioinformatics tools) being capable of handing a large-scale analysis The term s bioinformatics and com putational biology are often used interchange­ably It is further emphasized th a t there is a tight coupling of developments and knowledge between the more hypothesis-driven research in com putational biology and technique-driven research in bioinformatics 2

A lot of approaches in com puter science have been applied to solve more and more complex problems in com putational biology (Baldi and Brunak, 2000); unfortunately almost, all such problems are N P-hard or NP-complctc Therefore, heuristic search

m ethods play an im portant role in tackling the com binatorial optim ization problems

1 htt p: / / www n cb i n lm nih.gov / G e n b a n k /

2h tt p: / / www b is ti nill gov/ C om puB ioD ef p d f

1

1.1 M o tiv a tio n 2

Figure 1.1: The exponentially growth of nucleotide databases

Growth of the International Nucleotide Sequence Database Collaboration

B.IS« P & rs by 'S H n fla rk ii— ** t M S i — O O BJ —•

h ttp ://w w w ncbi.nlm.nih.gov/Genbank/

Recently, Ant Colony Optimization (Dorigo 1992) has been proposed and shortly afterwards has been recognized as one efficient method for finding an approximate solution for NP-hard problems The first application is traveling salesman problem

by inspiring by the real ants’s behavior when traveling from the colony to the food resource and transporting the food back ACO technique is widely used in various types of combinatorial optimization problems including in bioinformatics (Dorigo and Stutzle, 2004)

1.1.2 P h y lo g en y R ec o n stru c tio n

Since the t ime of Charles Darwin, evolutionary biology has been a main focus among biologists to understand the evolutionary history of all organisms Where the re­lationship of the structure of the organisms is often expressed as a phylogenetic tree (Haeckel 1866) Since the mid of twentieth century, the emergence of rnolec-

1.2 T h e s is W o rk s a n d S t r u c t u r e 3ul,u biology has given rise to a new branch ot study based on inolccular scqucnce (e.g DNA or protein) Moreover, phylogenetic analysis helps not only elucidate the evolutionary pattern but also understand the process of adaptive evolution at the molecular level (Nei and Kumar 2000).

In molecular phylogenetics, the sequences of the contemporary species arc given and one asks for the tree topology (including the branch lengths) which explains the data It is commonly accepted that phytogenies arc rooted bifurcating trees, where the root is the most common ancestor of the contemporary species The leaves represent contemporary species, and the internal nodes stand for spéciation events Among plenty of approaches to rcconstruc phylogenetic trees, the statistic-based methods have been recognized as sound and accurate methods Determining the best phylogénies based on optimality critcrions such as maximum parsimony, mini­mum evolution and maximum likelihood was proved as NP-hard and NP-completc problems (Graham and Foulds, 1982; Day and Sankoff, 1986; Chor and Tullcr, 2005)

In this thesis, we will build a general framework to apply ACO principle into phylo­genetics and mainly deal with maximum parsimony However, such approach can be easily adapted to any objective function Our contribution is the formal description

of framework to apply ACO mctaheuristics to solve the phylogcny reconstruction problem Attempts to solve the phylogenetic reconstruction problem using ACO gained only a poor results partly because of the poor construction graph (Ando and

mure general graph representation to overcome this problem

Except the introduction and conclusion, the thesis is organized into 3 chapters The first chapter sketches the major problem of reconstructing phylogenetic trees from given biological sequences The second chapter will show the general building block of ACO technique and application for solving the combinatorial optimization problems The third chapter describes the main outcome of the thesis It will de­scribe our approach and some initial experiences to employ ACO into phylogenetics

C h a p te r 2

P h y lo g e n e tic T ree R e c o n s tru c tio n

The goal of the phylogenetic tree reconstruction problem is to assemble a tree rep­resenting a hypothesis about the evolutionary relationship among a set of genes, species, or other taxa In this chapter, we will briefly introduce the main concept

of phylogenetics and the state-of-the-art methods In particular, we will concen­trate on the maximum parsimony principle used as an objective function for our optimization approach discussed in chapter 4

2.1 P h y lo g e n e tic T rees

According to Charles Darwin’s evolution theory, all species have evolved from an­

phylogenetic trees in phylogenetics terminology arc the one way to display the evolu­

ships among a number of species having a common ancestor Figure 2.1 depicts the phylogenetic tree of life consisting of three domains of all existing species: Bacte­ria Archaea, and Eukarya In a phylogenetic: tree, each internal node represents

¿in unknown common ancestor th a t split into two or more species, its descendants Each external node or leaf represents a living spec ies, each branch has a length cor­responding to the time between two splitting events or to the amount of changes that accumulated between two splits

4

2 1 P h y lo g e n e tic T re e s 5

0 r e * «

& p i r o c h « « » t t»ac«efw »

http://w w w biologydaily.com /biology/A rchaebactena

Figure 2.1 and 2.2 constitute examples of a rooted and a unrooted tree, respectively The real unrooted phylogenetic tree of Annelida, the segmented worms including three m ajor groups: Polychaeta, Oligochaeta (earthworms etc.) and Hirudinea (leeches), represents the most conservative representation of our understanding of annelid relationships in Figure 2.2 In a rooted tree, one has the information about the position of ancestral node W hereas in the unrooted case, no such information

is available and one can thus see how related the taxa are connected in the tree Phylogcnctic applications usual produce an unrooted tree To identify the root po­sition one often inserts an outer group one or several extra taxa not closely related

to the original taxa, and observes the branch it joins to the tree From now on, we only focus on unrooted trees

each internal node has the degree three, while a m ultifurcating tree allows internal node of arbitrary degree Typically, one assumes a bifurcating tree, i.e a speciation event, in the past leads to two lineages Hence for the rest, of the thesis, we mean

2.1 P h y lo g e n e tic Trees 6

Figure 2.2: Unrooted tree representation of annelid relationships

Aeolosomatidao+Pocamodrilidae

http://www.tolwcb.org/Annclida

phylogenetic trees as unrooted and bifurcating The branching pattern of a tree

is called a topology or tree structure In phylogenetic analysis, the branch lengths represent the evolution time a species needs to evolve into another specics

to this data structure used, the traversals on trees is easily performed by applying

the very important role in phylogcnctic analysis This traditional framework not

2.2 S e q u e n c e A l i g n m e n t 7

Figure 2.3: Three possible topologies of unrooted tree for four taxa

only helps build a perfect structure for phylogenetic trees but also provides a lot

of efficient strategies from traversal to searching for optimization stuffs in the main reconstruction problem

T h e n u m b e r of p h y lo g en etic tre e s

In general, the number of possible topologies for a bifurcating unrooted tree of

in taxa is given by

for in > 3 (Cavalli-Sforza and Edwards, 1967; Felsenstein, 1978) There are only

unrooted tree is often called quartet In fact, finding the best topology based on

are more than thirteen billion trees (Felsenstein, 2004) Therefore, heuristic searches are essential when the number of taxa becomes large

2.2.1 Biological D a ta

The data in biology and nature is very diverse and abundant Nowadays, one can study the evolutionary relationships of organisms by comparing their deoxyribonu­cleic acid (DNA) since the blueprint of all organisms is written in DNA (or ribonu­cleic acid RNA in some cases of viruses) (Nci and Kumar, 2000) DNA consists of the four types of nucleotides: Adenine, Cytosine, Guanine and Thymine classified into either purine (A and G) or pyrimidine (C and T) bases; Uracil is replaced by Thymine when considering the RNA sequences Besides, another type of genetic

2m-3(ra - 3)!

2.2 S e q u e n c e A l i g n m e n t 8

Table' 2.1: Twenty different types of amino acids with corresponding code

sequences, amino acids including twenty different kinds listed in Table 2.1 (Brown

<:t al 2002) are widely used in phylogenetic analysis Both types of molecular sequences (nucleotides and amino acids) play an im portant role in molecular phy­

2004) From here, we assumed th at the biological sequence d ata is molecular data

2.2.2 P a irw ise an d M u ltip le sequence a lig n m e n t

As we known, one of the most important features in evolution is replicating gene in

an organism According to evolutionary theory, the genes in the later generation is not exactly copied from those in the previous generation be cause of the errors dur­

controlled by the genetic information carried by DNA, any mutational changes in these character are due to some changes in DNA molecular sequences (Nei and Ku­mar, 2000) There arc four basic types of changes in DNA: substitutions, insertions, deletions and inversions (Nei and Kumar, 2000) where all types except for inversions are point mutations (Vandammc, 2003)

ACC TAC TTT GCT G - Thr Tyr Phe Ala (B) Deletion

Thr Tyr Leu Leu

i— *—i

ACC TTT ATG CTG Thr Phe Met Leu

the character A is substituted by C causes Tyrosine (Tyr) amino acid is re­placed by Serine (Ser) in the new sequence Nucleotide substitutions can be

divided into two classes: transitions and transversions A transition is the

substitution of a p u rin e (A or G) for another purine or the substitution of

a p y rim id in e (T or C) for another pyrimidine Other types of nucleotide substitutions are called transversions

• In se rtio n s: inserting one or more characters into the sequence In Figure

acid After that, two new amino acids (Phenylalanine (Phe) and Alanine (Ala)) replace two consequence Leucine(Leu) amino acids before the unknown

• D eletio n s: deleting one or more characters from the sequence In the example

sequence creates tin' new amino acid Cysteine (Cys) and a triple of characters

general input to phylogony reconstruction programs is MSA (Felsenstcin, 2004) In

sequences are assigned into the same column (so-called site), defines a MSA (Wa­terman 2000) Figure 2.5 illustrates an example MSA with Human, Chimpanzee, Gorilla, Rhesus Cow Dog, Mouse and Bird In this example, a t least three point

Human and Chimpanzee, t he character G can be deleted in Dog gene or inserted in

0 (m ") respectively in building the multiple sequence alignment by dynamic pro­

ot site s Approximation methods have been proposed in case of larger number of

2.3 A p p ro a c h e s for pliylogeny re c o n stru c tio n 11

hast d and distance-based. Distance-based approaches reconstruct, phylogenies for a set of species S based on the pairwise distance matrix D = {d (u ,v )} where d (u v ) is

The first type of them is introduced by (Cavalli-Sforza and Edwards, 1967) and (Fitch and Margoliash 1967), unfortunately they require a very huge computation times Hence, we did not used the distance-based approaches for applying ACO approach to solve phylogeny reconstruction problem

Another one of character-based approaches besides the Maximum Parsimony ap­proach discussed in the next section is Maximum Likelihood Maximum Likelihood approach is more and more widely used for inferring the phylogenies The results

on computer simulations showed that maximum likelihood methods often give the

ft al 2005) Using maximum likelihood can obtain the better experimental results, however due to limited time, we apply Maximum Parsimony criterion for easier com­puting process YYc did that because we want to consider the performance of ACO approach compared to another approaches based on the same objective function

2.4.1 P a rs im o n y C oncept

Maximum Parsimony (MP) was proposed by Edwards and Cavalli-Sforza (1963)

m inim um net am ount o f evolution” In general, the goal of the MP methods is to select phylogenies that minimize the total number of substitutions along all branches

1996)

Mathematically, the general maximum parsimony problem is defined as follows

2.4 M a x im u m P a rsim o n y Prin ciple 12

sites), find all trees T that minimize the tree length

in

.7 = 1 (u.v)

i lie tiee T. the coefficient ir, assigns a weight to the given site, x u],x v] represent

cliarart.er-st.Hte if a or r is internal node, <li.ff(y,z) is a cost function of a transfor­

We have to distinguish between the optimality criterion (minimal tree length under an assumption of the permissible character-state changes) and the actual algorithm used to search for optimal trees in parsimony analysis (Farris, 1970, i.e,) The optimality criterion is an objective function to guide the search whereas the algorithms can be different but attem pt to optimize the same MP function The

tree 7’

2.4.2 C o u n tin g evolutionary changes

Among various met hods for counting the minimal number of state changes on a given phylogeuy the most popular ones are Fitch’s algorithm for lion-weighted parsimony (1 itch 1971) and Sankoff's one for weighted case (Sankoff, 1975) In both algo-

minimum changes required and then add up the weighted site changes As a conse­

possible assignment of an internal node of its two children were already assigned some characters

Figure 2.6: An example for Fitch algorithm

In t he following we will give a description of Fitch algorithm in (Fitch, 1971) for

a each site in ca.sc of non-weighted parsimony The total length of tree is the sum

of returned algorithm value for every site

S, — {.-I}) Initialize the tree length to zero

according to the following rules:

(b) Otherwise (S t n Sj = 0) let k's state set equal the union of those state sets (i.e St — 5, U S ;). Increase the tree length by one unit

ot the terminal node placed at the root),the traversal has been completed; proceed to step 4 Otherwise return to step 2

4 If the state set to the terminal node at the root of tree is not contained in thestate set just assigned to the node at the basal fork of the tree, increase the tree length by more one unit

2.4 M a x im u m P a rsim o n y P rinciple 14

In tin example (Figure 2.G), there are totally three union operations in traversal

{ CT} U {.4} The remaining immediate descendants arc created by intersection

operation with the common character T Therefore, the tree length for the given

site is three

2.4.3 R em ark s on M axim um P arsim o n y A pproaches

Although the maximum parsimony approaches do not have statistical properties

play an important role in phylogenetic analysis First, MP often consumes much less computation than other statistical-based approaches That will be of great benefit when the tree becomes larger to provide a first view how the tree will look like Second, analysis on morphological data is normally carricd out with MP- bascd methods Beside the strong points of MP approaches, there are still some disadvantages

The first one is that MP does not use all sequence information because there

sites2 arc informative for topology construction in other tree-building methods even

likelihood methods (Nci and Kumar, 2000) The second disadvantage is th at MP approaches do not fully account for multiple mutations because of not implying a model of evolution as other statistical methods such as maximum likelihood

Early descriptions of MP methods were (Kluge and Farris, 1969), (Farris, 1970), (Fitch 1971) and (Sankoff, 1975) Heuristic searches described in the next section have bet'ii proposed to reduce computational burden in Maximum Parsimony meth­ods such as latched-based methods (Nixon, 1999), hill-climbing searches based on

inod-'T ln 'iv m ust b e a t least tw o d ifferen t k in d s of n u cleo tid es, each re p re s e n te d a t le a s t tw o tim e s

"N u c le o tid e s ite a t w hich o n ly u n iq u e n u c le o tid e ex ist

•*Sit t* have th e sa m e n u c le o tid e fo r all ta x a

2.5 F in d in g th e b e s t tr e e by h e u ris tic searches 15(’in parsimony computer programs such as F an is’s Hennig86, Fclscnstciu’s PHYLIP-

and bioiiifurniatics communities whereas PAUP* is the most, popular package used (Swofiord 2002)

2.5 F in d in g th e b est tree by h euristic searches

As we have seen, it is impossible to examine all possible tree topologies Instead, one usually applies the heuristic searches In phylogenetic analysis, the greedy

liill-cliinbing techniques such as sequential addition or star decomposition methods are widely used (Felscnstein, 2004; Nei and Kumar, 2000) Tree rearranging, so- called branch swapping methods arc also widely used However, such methods

algorithms only accept the modification to the current partial solution with higher

avoid being trapped into local optimal such as Simulated Annealing (Stamat.akis,

2002) that were successfully employed for phylogenetics We will review theses methods in this section

2.5.1 S eq u en tial A d d itio n M e th o d s

Almost all heuristic searches for finding the best trees start with either a random

possible trees bv adding species one at a time at a already constructed tree , each

in all possible places From the starting tree with three species, two more branches arc added to the tree when having the fourth species branch off from the middle of any the three branches Each of three possibilities has five possible ways that the next species can be added, and so on

parsimonious trees with score 9 arc found from that chosen tree by inserting the fifth

2.5 F in d in g th e b e st tre e by heuristic searches 16

Figure 2.7: An example of sequential addition method

E

Figure from chapter 4 in the book ’’Inferring Phvlogenies” of Fesenstcin

rearrangement strategies described in the next subsection The similar strategy for building tree is applied in our works (discussed in chapter 4), only one modification

is that we used probabilistic decision rule for the adding order instead of a random order when the species arc added is arbitrary in sequential addition

2.5.2 Tree A rra n g em en t M eth o d s

Those are the fundamental techniques that take an initial estimate of the tree and

• ne any "better" neighbors, we take them and continue to rearrange them The

pro-2.5 F in d in g th e b e st tre e by heu ristic searches 17

Figure 2.8: An example of Nearest-Neighbor Interchange Operation

Such a tree is at a local optimum in the very large tree space Local rearrangement operations can be used to measure the difference between phylogenetic trees (Wa­terman and Smith, 1978) In addition, it provides a simple and efficient travel way through the space of possible phylogenetic trees for finding the best one based on arbitrary objective function (Fclscnstcin, 2004)

There are three main types of rearrangements (see Figure 2.6,2.7,2.8 for visual comparison between these three operations).These techniques are very useful to find both the most parsimonious tree and the best one based on other criteria in very large tree space They are applied in many heuristic searches including Ant Colony Optimization discussed in the next chapter

• Nearest-Neighbor Interchanges (N N I) NNI in effect swaps two adjacent branches

on the tree This operation is implemented by erasing an interior branch on the tree and connecting the two branches to it at each end; hence there are a total of five branches which arc erased This leaves four subtrees disconnected from each other and four subtrees can be hooked into a tree in three possible ways (Felsensteiu 2004) There are 2(n - 3) neighbors can be examined from each unrooted tree to find the best, one because for each tree having n tips we

DAI HOC QUÖ C G IA HÄ NÖI TRUNG TÄM THÖNG TIN THU VIEN

- A HLs/gf —

2.5 F in d in g th e b e s t tre e by heuristic searches 18

Figure 2.9: An example of Subtree Pruning and Regrafting Operation

Figure 2.10: An example of Tree Bisection and Reconnnection Operation

• Subtree Pruning and Regrafting (SPR ) A branch of a provisional tree is cut

point of pruned subtree is then grafted onto each branch of the residual tree

to produce a new topology A new tree topology is generated by grafting the cutting point of the pruned subtree onto each branch of the residual one

• Tree Bisection and Reconnnection (T B R ) Two subtrees are generated from a provisional tree by cutting at a branch Then they arc reconnected by joining two branches, one of which is from each correspondence subtree; hcnce a newtree topology is generated

2.5 F in d in g th e b e st tre e bv h e u ristic sea rc h es 19

2.5.3 O th e r heu ristic search m e th o d s

Simulated Aiuicalui.fi (SA) is a generic probabilistic meta-algorithm for the global optimization problem, namely locating a good approximation to the global optimum

controlled cooling of a material to increase the size of its crystals and rcduce their defects The heat causes the atoms to bccomc unstuck from their initial positions (a local minimum of the internal energy) and wander randomly through states of highei energ\ the slow cooling gives them more chances of finding configurations with lowei internal energy than the initial one SA is applied successfully in solving phylogeueiic tree reconstruction problem (Stamatakis, 2005) and (Barker, 2004) with promising further experimental results

G< n.<’hc A lyoritkm s(G A ) or evolutionary computation is one of the most pop­ular and effective methods in solving complex optimization problems The first application in general optimization was inspired largely by (Holland, 1975) through simulations of evolution by biologists and engineers GA is used for solving phy-

Junction that reflects the optimality of the tree Optimizing branch lengths on each tree and using recombination operator that swapped particularly good subtrees be­tween is used in the first GA application in phylogenetics (Matsuda, 1996) (Lewis, 1998: Moilanen, 1999) used SPR rearrangement and recombining by choosing a sub­tree in one tree and deleting those species from the other and inserting the subtree

with similar n'combination operator GA is also easy performed with parallel com­

Milinkovifch 2002)

C h a p te r 3

A nt Colony O ptim ization

Aut Colony Optimization ( ACO) is a ineta-hcuristic approach inspired by the study

of real ant colonies' behavior while finding the shortest path from their nest to food source' and vice versa Despite its recent proposal in 1992, ACO has been applied

to various classic NP-hard problems and the experiments showed very promising results ((Dorigo and Stutzlc, 2004) and references therein) In this chapter, we will briefly survey the ACO with respect to its biological motivation and its application for solving combinatorial optimization problems

3.1.1 D ouble bridge experim ents

The idea of Ant AlgoritInns is based on the brilliant result of a controlled experiment

connecting a nest of ants and a food source (Figure 3.1) In the first ease (a), the two bridges have the same length whereas in the second case (b), the upper bridge is shorter than the lower one With the same experimental setup (number

of ants, environmental conditions) in both cases, the experimental results presented the percentage of ants’s traffic on each bridge was observed over time and the results were presented in Figure 3.2.1

'B o t h figures 3.1 a n d 3.2 in c h a p te r 1 (D o rig o a n d S tu tz le , 2004)

20

3.1 T h e A n t A lg orith m s 21

Food

Figure 3.1: Experimental setup for the double bridge experiment

(a) Branches have equal length., (b) Branches have different length

Tlit' outcome of the first, experiment showed that there was a little difference between two i raffic flows on the two bridges while in the second case, more ants were stimulated to move on the shorter bridge These results can be explained as follows (Dorigo and Stntzle 2004) While mutually walking between the nest and the food source, the ants deposit phcromonc on the ground They can smell the pheromone and they tend to choose, the direction with the strongest pheromone concentration Hence, the pheromone trail forms a kind of indirect communication which attracts the ants due to the foraging behaviors Initially, there are no pheromone on the ground and i he ants moved in a random manner Time by time they changed the environment around via pheromone The more frequently a path is visited, the higher amount of the pheromone this path contains In the first experimental setup, an ant will reach the food at the same time no mater which bridge was chosen Whereas in the second ease, it will arrive there faster with the upper bridge Therefore, wi i h t he same number of ants, the shorter bridge in the second experiment will have higher pheromone intensity and visited more frequently overtime than the longer one as a consequence

The result of the presented experiments was the motivation of the simulated AGO algorithm as first proposed in (Dorigo, 1992) The next, section will give an in-

i roduction to the algorithmic skeleton of the so-called ant system and its application

to the famous Traveling Salesman Problem

Figure 3.2: Results gained in the double bridge experiment

in) In cast- o f two equal-length branches, the ants use one branch or the other in approximately lln stunt num ber o f Inals, (b) In almost trials in ease o f two different branches, the great

m ajority o f ants chose the short branch

3.1.2 A n t System

The first ant system was particular designed for solving the Traveling Salesman Problem (T S P ) (Dorigo, 1992; Dorigo ami Stut.zle 2004) The famous Traveling

a salesman has to find the shortest tour through each city once and finish at the

gorithm classes Almost all later ACO algorithms are inherited from the algorithm

by adding the improvements on the two basic phases: solution construction and

uiit-quantity, and ant.-cycle, see (Dorigo and Stutzlc, 2004) We will present the last one which is ac tually referred to AS nowadays because of its better analysis results among three above versions

where i , j arc two cities indexed from 1 to tl , is initialized to tq There are m artificial nuts were arbitrarily placed at the starting city The algorithm iterates through a

t he solution construction and phcromone update procedure are as follows:

S o lu tio n C o n stru c tio n Each ant selects another city according to the so-

c a l l e d random proportional, rule (Dorigo tt al 1996) or State Transition Rule (Hoang and Dinli, 2002: Dinli et al 2006)): The kth ant at city i will move to the city j

3.1 T h e A n t A lg o rith m s 23with the following probability:

(3.1)

where:

• ru is tin- pheromonc trail intensity of edge (i j )

• iftl — j - wlieic dKJ is the distance between the city i and j r/y is called theheuristic information

pheromonc trail and the heuristic information In the most simple case, weset o and 0 equal to 1

• N-' is the set of neighboring citics of the city i that the ant kth has not visited

probability, the current ant selects the ’’best" city by applying probabilistic decision

indexed from 1 to ¡>\ then we calculate P = Y^j=i Aj> after that, the 1 < q < p city

in [0,1) Every ant will complete its tour according to this probabilistic rule after N iterations,

P h e ro m o n c T rails U p d a te P ro c e d u re After all the ants performed the solution construction, the pheromonc trails arc updated The process is done by decreasing the pheromonc intensity rtJ on all edges by a constant factor first, and then adding all pheromonc trails the ants have walked during their tour:

vet

3.1 T h e A n t A lg o rith m s 24

deposits on the edge (i j ) it has visited:

where Q is a parameter La is the length of the path the kth ant has traveled so far.The relative performance of AS, compared to other meta-heuristics, tends to decrease dramatically when the number of cities increases (Dorigo and Stutzle, 2004) Hence almost all further researches on AGO were concentrated on how to improve the AS a.s previously mentioned at the beginning of this section

3.1.3 A n t Colony System

Ant Colony System (ACS: Dorigo and L.M.Gambardella, 1997) is the improved al­gorithm of AS based on two modifications to the solution construction and pheromone trail updates divided into three main points (Dorigo and Stutzle, 2004) First, the main idea starts from exploiting the search experience accumulated by the ants more strongly than AS does via the use of the mure aggressive action transition rule Sec­

o n d the pheromone updates including both depositing and evaporating are placed only on the edges that belong to the best-so-far tour And third, decreasing some pheromone on the edge which is visited by each ant to increasing the exploration of another paths More details about two improvements will be presented as below

S ta te T ra n s itio n R u le For providing that the ant not only can explore the new edges but also exploit the existed accumulated information of the problem, in ACS when located in city i, the ant k"1 select the city) s according to the so-called

pseudorandom proportional rule as below:

where q is a random variable uniformly distributed in [0,1], r/o(0 < go < 1) is a

3.1 T h e A nt A lg orithm s 25

In equation (3.1): the a parameter is clioscn by 1 here Eciuation 3.4 means that with probability i/o the ant makes the best possible move as indicated by the learnt, pheromone trails and the heuristic information (in this case, ants exploit the learnt information (Dorigo and Stutzle, 2004) called reinforcement learning information ( Diiili <:l U.I 2006)) while with probability (1 - </o) it performs a biased exploration

of the edges Adjusting the parameter i/o allows the modulation of the degree of explorai ion and t he choice of whether to concentrate the search of the system around

i lie best-so-far solution or to explore other tours

P h e ro m o n e U p d a tes ACS uses two kinds of update rules

• Local U p d a te : while building its own solution, each ant visits one edge and update the pheromone trail of this edge according to:

• G lobal U p d a te : is applied for the edges belonged to the best tour(solution)

where L lti(t) is the length of tour W(t).

The disadvantage of AC'S is that if some edges art1, not visited by ant from the time / their pheromone trail is not changed although this pheromone trail ought to have been decreased comparing to the edges that arc visited more frequently

3.1.4 M ax -M in A n t System

In Max-Min Ant System (M M A S: Stutzle and Hoos 1997), the solution construc­

1 On the other side, four main improvements an- introduced (Dorigo and Stutzle, 2004)

r/lobally arc allowed to deposit the pheromone

3.1 T h e A n t A lg o rith m s 26

• riii' MM AS limit oil the possible range of pheromone trail values to the inter­val TmrU]. where r,„,„ and rmax arc two paia.iiict.ers That prevents anunexpected stagnation situation in which all ants follow the same tour caused

by the above modification

with a small pheromone evaporation rate T hat allows a more thorough ex­ploration of solutions at the beginning of the search

• The MMAS re-initializcs the pheromone trails whenever the system reaches saturation or when all the ants can not generate improved solutions for the long period (exceeding a certain number of iterations)

U p d a te o f P h e ro m o n e T rails After each iteration, the pheromone trail associated wit h each edge will be globally updated according to the following formula where tj(\\'(t)) = L ( W ( t ) ) ~ l where L ( W( t ) ) is the length of the best, tour W(t ),

T i j * - (1 - p ) T i j + & i , j (3.7)

where

py{W{t ) ) if edge ( i , j ) € W( t )

in the edges belonging to the best solution On the one hand, un-visited and visited edges which do not belong to the best solution arc not distinguished Hence, the search space is reduced if rmin is not chosen large enough On the other hand, if rm,„ is too big the algorithm tends to be a random search based on the heuristic informal ion while the reinforcement learning based on the pheromone trail informa­tion is decreased However, the MMAS is still widely applied for solving NP-hard problems thanks to its simplicity and effectiveness

O th e r e x te n s io n s to a n t s y s te m

There are many extended ant systems of two presented ant systems (Dorigo and

ii.2 A n t Colony O p tim iz a tio n M eta -h e u ristic 27update of plicromone trails since the solution construction procedure of the AS and

the update procedure so that it is more flexible, easier parameter choosing, and

disadvantages These studies have contributed to diversifying the ACO algorithm classes and applications (Dorigo and Stutzlc 2004)

3.2 A nt Colony O ptim ization M eta-heuristic

'This section will introduce the ACO incta-heuristic technique Thanks to may successful applications, the ACO has becomc a standard method for ’’solving” com­binatorial optimization problems Combinatorial optimization is the problem of determining values for discrete variables that maximize or minimize a given objcc-

Salesman Problem mentioned in the previous section Almost all combinatorial op­timization problems are computational difficult Hence they require heuristic search met hods or the so-called meta-heuristics such as local search, hill climbing, simulated annealing, genetic algorithms, tabu search

In the next, we will discuss the ACO as a recently proposed method for solving NP-ha.nl problems This section will introduce the general framework for applying ACO to solve NP-hard problems from building the state graph based on problem definition to performing all next steps Firstly, to apply ACO technique for solving

(pupli or the state graph. This is the important factor for converting various types of combinatoria] optimization problems to the one which is to find the optimal paths

iu the given weighted graph The section will describe the general applied problem dest ript ion of ACO

3.2.1 P ro b le m R ep resen tatio n

A> we have seen, the first step is presenting the solution of combinatorial optimiza­tion problem a.s the set of partial states such that there is a way to collcct the states