UAV mission planning problem modeling and solution methods

LIST OF FIGURES Figure 1.1 Dot-on-coordinate mission planning model and Voronoi Diagram solution Figure 1.2 Interface of Polygon-obstacles model Figure 3.1 The evolution process of giraf

ACKNOWLEDGEMENT

I would like to express my sincere gratitude to my supervisor Associate Professor ZHANG Yunfeng, for his invaluable guidance throughout this project From literature survey methodology at the beginning, to the crucial steps taken throughout the project, his advices kept

me on the right direction and inspired me greatly

In addition, thanks are given to PhD candidates Mr GENG Lin and Dr WANG Yifa, who had been very supportive for my research

I would also like to show my appreciation for the financial support in the form of a research scholarship from the National University of Singapore

TABLE OF CONTENTS

ACKNOWLEDGEMENT i

TABLE OF CONTENTS ii

ABSTRACT vii

LIST OF FIGURES ix

LIST OF TABLES xii

CHAPTER 1

INTRODUCTION 1

1.1 The History of UAVs 1

1.1.1 World War I 1

1.1.2 World War II 2

1.1.3 Post-World War II, through pre-Vietnam war 2

1.1.4 From Vietnam War to Gulf war 2

1.1.5 From Gulf war to present 3

1.2 Modern Research Areas Concerning UAVs 3

1.2.1 Sensor technologies 3

1.2.2 Control technologies 4

1.2.3 Mission management system 4

1.2.4 Mission planning 4

1.3 UAV Path Planning 5

1.3.1 Multiple travel salesman problems with time window constraints (mTSPTW) model 6 1.3.2 Dots-on-coordinate model 7

1.3.4 Three-dimensional (3D) mission model 9

1.4 Integrated Task Assignment and Path Optimization 12

1.4.1 Dubins‟ vehicle 14

1.4.2 The dynamic network flow model 15

1.4.3 The tree representation model 16

1.4.4 The graph representation model 16

1.5 Surveillance in Complex Urban Environment 16

1.6 Research Objectives 17

1.7 Limitations and Assumptions of Research 18

1.8 Organization of the Thesis 18

CHAPTER 2

RESEARCH MOTIVATIONS 20

2.1 UAV Path Planning in Complex and Realistic Environment 20

2.2 UAV Cooperative Task Assignment 21

2.3 UAV Surveillance in Complex Urban Environment 21

2.4 Genetic Algorithms as a Solution Tool 22

2.5 Java Monkey Engine (JME) as System Implementation 22

CHAPTER 3

INTRODUCTION TO GENETIC ALGORITHM 24

3.1 Encoding of the GA Chromosome 26

3.2 The Initialization 27

3.3 The Fitness Function 27

3.4 The Genetic Operators 28

3.5 The Selection Process 29

CHAPTER 4

A GA-BASED UAV PATH PLANNER

IN COMPLEX REALISTIC ENVIRONMENT 31

4.1 The Proposed Model for UAV Path Planner 32

4.1.1 The description of terrain 32

4.1.2 The description of the UAV path 35

4.1.3 The description of threats 39

4.2 The Design of Genetic Algorithm 41

4.2.1 The encoding of solution 41

4.2.2 Initialization 41

4.2.3 The fitness function 43

4.2.4 The selection mechanism 47

4.2.5 The genetic operators 47

4.3 Simulation Experiments 50

4.3.1 Experiments on different fitness functions 50

4.3.2 Experiments on functions of different GA operators 55

4.3.3 Experiments on magnitude of probabilities for the GA operators 61

4.3.4 Experiments with different UAV and UAV path characters 62

4.4 Discussions 67

CHAPTER 5

A GENETIC ALGORITHM

FOR UAV COOPERATIVE TASK ASSIGNMENT 69

5.1 The Problem Description 69

5.2 Dubins Vehicle 70

5.3 Graph Representation 73

5.3.1 Discrete UAV heading angle on target 73

5.3.2 The graph representation 74

5.4 The Solution Encoding 76

5.5 The Initialization 77

5.6 The Fitness Function 79

5.7 The Genetic Algorithm Operators 81

5.8 The Simulation Experiments 83

5.8.1 Diversity of the random solution generator 84

5.8.2 Experiments on different probabilities of genetic operators 84

5.8.3 Experiments with different fitness functions 87

5.8.4 Experiments with different UAV flight dynamics 89

5.8.5 Stability of the solution algorithm 96

5.9 Summary 97

CHAPTER 6

AN OCCLUSION-AWARE MODEL FOR UAV SURVEILLIANCE

IN COMPLEX URBAN ENVIRONMENT 99

6.1 Problem Description 99

6.1.1 The environment 99

6.1.2 The sensor capability of UAVs 100

6.1.3 The objective function 100

6.2 The Two-Stage Optimization 101

6.3 The Occlusion Aware Model 101

6.4 System Implementation of the Occlusion Aware Model 102

6.5 Limitations of the Two-Stage Method 104

6.6 Discussions 105

CHAPTER 7

CONCLUSIONS AND FUTURE WORK 106

7.1 Conclusions on the Study of the UAV Path Planning Problem 106

7.2 Conclusions on the Study of the Multiple UAV Task Assignment Problem 107

7.3 Conclusions on the Study of the UAV Surveillance Problem 108

7.4 Recommendation for Future Work 108

REFERENCES 110

ABSTRACT

The research work reported in this thesis addresses the fixed-wing UAV mission planning problem More specifically, the study focuses on 3 sub-problems: single UAV path planning, multiple UAV cooperative task assignment, and UAV surveillance in an urban environment The study for each specific problem includes the development of problem modeling method and optimization solution method, followed by simulation experiments for verification

The first part considers the problem of finding an optimal path for a UAV in a realistic environment The realistic model used in this project includes terrain, radar zone, flight prohibited zones, and UAV capability limits The flight path is represented by piecewise cubic Bezier curves A genetic algorithm (GA) is developed to find the optimal solutions The unique design of the GA, including solution encoding, GA operators, and fitness function, allows the search to escape from those local minimums and manage to find the optimal or near-optimal solution in a robust manner

The second part considers the problem of multiple UAV coordination to accomplish assigned tasks on different targets To accommodate the fact that UAVs can be heterogeneous, a multiple graph representation scheme is proposed to model the solution map for each UAV A

GA with unique design of encoding scheme and GA operators is developed to find optimal solutions The testing results from various simulation experiments show that the GA is able to solve this coordination problem in an effective and efficient manner

The third part considers the problem of UAV surveillance in an urban environment To overcome the limitation of previously reported models, an occlusion-aware model is developed

to simulate the environment under surveillance (terrain and buildings) An efficient ray-casting based collision checking algorithm is also developed to determine the visibility of any given

observation position of the UAV Again, heterogeneous UAVs can be modeled Due to the limitation of time, however, the solution method to find the optimal set of observation points for UAVs is yet to be developed

In summary, the study on the 3 different UAV mission planning problems represent a major step towards developing advanced modeling and solution tools to deal with close-to-reality problems Although there are plenty of challenges to be tackled before the final goal can be achieved, this effort has built solid foundations for future extension

LIST OF FIGURES

Figure 1.1 Dot-on-coordinate mission planning model and Voronoi Diagram solution

Figure 1.2 Interface of Polygon-obstacles model

Figure 3.1 The evolution process of giraffes

Figure 3.2 Illustrations of gene and chromosome

Figure 3.3 Example of solution encoding for path planning problems

Figure 4.1 Grayscale height map representation of Oahu Island

Figure 4.2 Rendered 3D representation of Oahu Island

Figure 4.3 Illustration of UAV flight constraints

Figure 4.4 Cubic Bezier interpolation

Figure 4.5 Illustrative example of mission planner over Oahu Island, Hawaii

Figure 4.6 A chromosome with two intermediate waypoints

Figure 4.7 The cross-over operator

Figure 4.8 The local mutation operator

Figure 4.9 The strong mutation operator

Figure 4.10 The deletion operator

Figure 4.11 Test 1: simple obstacle evasion test for static fitness

Figure 4.12 Test 2: concave obstacle evasion test for static fitness

Figure 4.13 Different solutions with same static fitness value

Figure 4.14 Test 3: concave obstacle evasion test for dynamic fitness

Figure 4.15 Test 4: fitness plot for random generation

Figure 4.16 Test 5: fitness plot for cross-over operator

Figure 4.17 Test 6: comparative study of local and strong mutation operators

Figure 4.18 Test 7: simple test without deletion/smoothening

Figure 4.19 Test 8: simple test with deletion/smoothening operator

Figure 4.20 Test 9: aggressive UAV maneuverability

Figure 4.21 Test 10: conservative UAV maneuverability

Figure 4.22 Test 11: fitness evolution for different values of d

Figure 5.1 The minimum path problem

Figure 5.2 Illustration of a feasible path

Figure 5.3 Dubins path is one of the four possible choices when

Figure 5.4 Two additional choices of Dubins path at close starting and ending distance

Figure 5.5 Illustrative example of a mission involving two UAVs and two targets

Figure 5.6 Discrete headings of UAV

Figure 5.7 Two examples of paths with different heading angle on target

Figure 5.8 The graph representations with 3 part discretization

Figure 5.9 One solution of the example

Figure 5.10 Algorithm for the random solution generation

Figure 5.11 Cross-over operator on same solution across different UAVs

Figure 5.12 Mutation operator changes heading angle

Figure 5.13 Illustration of swap operator

Figure 5.14 Fitness evolution for different GA operators

Figure 5.15 Best fitness evolution for different GA operator probabilities

Figure 5.16 Average fitness evolution of the different GA probabilities

Figure 5.17 Result comparison for different fitness functions

Figure 5.18 Fitness value evolutions for different fitness functions

Figure 5.19 Simulation output for turning radius 1

Figure 5.20 Fitness evolution of the solution of experiment with turning radius 1

Figure 5.22 Fitness evolution of the solution of experiment with turning radius 3

Figure 5.23 Simulation output for turning radius 7

Figure 5.24 Fitness evolution of the solution of experiment with turning radius 7

Figure 5.25 Simulation output for turning radius 10

Figure 5.26 Fitness evolution of the solution of experiment with turning radius 10

Figure 5.27 Simulation output for heterogeneous turning radius

Figure 5.28 Fitness evolution of the solution of experiment with heterogeneous turning radius Figure 5.29 The best fitness of 10 same-parameter simulations

Figure 5.30 The average fitness of 10 same-parameter simulations

Figure 6.1 Visibility test with analytical method

Figure 6.2 Illustration of ray detection

Figure 6.3 Occlusion aware model

LIST OF TABLES

Table 1.1 3D models by components

Table 1.2 Models and solution algorithms in previous works

Table 4.1 Parameters defining the piecewise cube Bezier curve

Table 4.2 Static fitness value assignment

Table 4.3 Dynamic fitness value assignment

Table 4.4 Symbols for probability of each EA operator

Table 4.5 Controlled constants for experiments on fitness function

Table 4.6 Homogeneous assignment of fitness multipliers

Table 4.7 Final assignment of fitness multipliers

Table 4.8 Controlled variables for experiments on EA operators

Table 4.9 EA operator probabilities for immigration testing

Table 4.10 EA operator probabilities for cross-over testing

Table 4.11 EA operator probabilities for local mutation testing

Table 4.12 EA operator probabilities for local and strong mutation testing

Table 4.13 EA operator probabilities for deletion/smoothening testing

Table 4.14 Fitness values after 50 iterations with different EA probabilities

Table 4.15 Final adopted EA operator probabilities

Table 4.16 Aggressive UAV maneuverability

Table 4.17 Conservative UAV maneuverability

Table 4.18 Different distances between two consecutive waypoints

Table 5.1 Constant parameters for genetic operators‟ test

Table 5.2 Solution quality of the different GA operators

Table 5.4 Constant parameters for UAV flight dynamics tests

Table 5.5 Result statistics

Table 5.6 Parameters used for repeated simulations testing stability

CHAPTER 1 INTRODUCTION

1.1 The History of UAVs

This section firstly introduces the definitions UAVs and differentiates them from

conventional aircrafts It then outlines the essential technologies required for them A revision of

the historical development of UAVs is also given in detail

There are two ways to categorize aerial vehicles The first way is according to their

method of guidance and the second is according to whether they are expendable In terms of

guidance technology, an aerial vehicle can be either manned, remote piloted or auto piloted

Expendable aerial vehicles are not reusable and are categorized as missiles The UAVs discussed

in this thesis are auto or remote piloted and recoverable James [1] summarizes three important

technologies in order to make UAVs successful for operational use: (1) an aerial platform

capable to maneuver to an appropriate objective; (2) a guidance system that would permit

over-the-horizon unmanned aerial vehicle operations; (3) a payload that can perform a useful mission

once the platform gets to its objective

1.1.1 World War I

Aerial balloon was a primitive form of UAV, used in the siege of Venice by the Austrian

troops in August 1849 In early 20th century, the invention of airplane by Wright brothers was a

major breakthrough in technology that brings UAVs into practical use The directional mobility

of an airplane is a capability that balloons do not have Thereafter, the first essential technology

is met Aerial vehicles are able to reach their objectives However, the guidance of such vehicles

torpedoes” appeared Designed by the U.S and Britain, these are aircrafts filled with explosives designed to fly for a set distance and crash However, due to the inaccuracy of the guidance system, none of the attempts during WWI resulted in operational application of the UAVs

1.1.2 World War II

Each side had significant development in UAV technology during the war The most significant and notorious was the German V-1 [1] The V-1 was a self-guided monoplane filled with explosives that would fly a pre-set heading and time, at which time the engines would cut off and the aircraft would go into a dive, exploding on impact Hitler‟s V-1 campaign marked the first large-scale operational employment of unmanned aircraft Studies have shown that the cost

of the Germen was only one forth of that of Britain, thanks to the use of UAVs On the other hand, the allies achieved radio control of the unmanned aircrafts However, without a computer, automatic control is not feasible

1.1.3 Post-World War II, through pre-Vietnam war

During this period, there were some significant improvements in guidance technologies and flight endurance, which is a direct consequence of the escalating cold war between the US and the Soviet Union Therefore, cruise missiles and photo-reconnaissance aircrafts were developed However, the low reliability of such aircrafts makes them not practical for operational use

1.1.4 From Vietnam War to Gulf war

In terms of capabilities, the UAVs in this period improved in both flight capacities and accuracies Unmanned target drones increased in performance with speeds all the way up to mach 4 and service ceilings of nearly 30000 meters Their payload also increased, allowing the

carriage of bombs In terms of guidance technology, they began as programmed drones Later, they were capable to receive order when in flight This period was an important one during which the technology gap between UAVs and the manned aircrafts had been removed

1.1.5 From Gulf war to present

The increasing computer processing capabilities, data transmission rates and miniaturization technology were the driving forces behind the development of UAV capabilities during this period Computer processing power allowed for smaller chips to be installed onto the aircraft, increasing the payload It also enabled auto-piloting by assimilating all the decision variables in an efficient way Fast data transmission rate ensures minimum delay in the information obtained from the controller on the ground This ensured the real-time decision making by commanders Miniaturization decreased the probability that a UAV will be detected

in its mission This greatly ensured the secrecy of the mission Moreover, miniaturization widens the scope of missions that UAVs are capable to carry out

1.2 Modern Research Areas Concerning UAVs

Due to the widening scope and increasing complexity of UAV missions, the researches

on UAVs span a very broad range Some of the important research areas include sensor and control technology, mission management system and UAV mission planning

1.2.1 Sensor technologies

The development in sensor technologies aims at enhancing the UAV capabilities and broadening the mission types a UAV is capable of For the reconnaissance UAVs, the complexities of the sensors decide the UAV‟s operational altitude, which is pertinent to its

balance between weight and complexity of its sensors [3] The equipment of chemical sensor, for example, enables the Raven UAV – a hand launched close range reconnaissance platform - to detect and track a chemical plume in a variety of atmospheric conditions [4]

1.2.2 Control technologies

Researches in control technologies look at how to better take charge of the UAVs while they are a certain distance away There are two aspects associated with the control technology challenges The first is how to send orders to the aircrafts while they are distant from the control center The second is how to achieve real-time positioning of the aircraft while they are over the horizon Cooperative control investigates the distributed control of multiple UAVs over the horizon [5] Integration of onboard and ground camera is a proposed method for real-time positioning of UAVs [6] Another study proposes UAV control by means of a single axis rate gyro, an absolute pressure sensor and a GPS receiver [7]

1.2.3 Mission management system

Researches on mission management system target at facilitating the battlefield commanders in high-level control and decision making Cassandras and Wei proposed a simulated battle space and a dynamic target assignment scheme which achieves the commander‟s order with optimal performances [8] Another example is the airspace integration through network among the UAV, the mission management station and the air traffic control [9]

1.2.4 Mission planning

UAV mission planning, which is also the focus of this study, considers optimization of flight paths in a dynamic environment in which numerous mission objectives are defined There are two kinds of problems in this area of research The first considers the optimization of the

UAV flight paths, which is called UAV Path Planning The second considers the cooperation of different UAVs in achieving a collection of mission objectives which is called UAV Cooperative Task Assignment More details on these two problems are discussed in the following section

1.3 UAV Path Planning

UAV mission planning is in itself a very broad research area in which the mission coordination among multiple UAVs and the path planning of UAVs are two fundamental areas These two areas are chosen in this study as the results can be used for future research on mission planning, e.g., multiple UAV mission planning, real-time, and variable velocity UAV path planning For example, multiple UAV mission planning requires a target allocation step before applying the results of this study Real-time planner runs parallel threads simultaneously, of which the primary threads are to be developed in this study Variable velocity path planner makes the flight constraint a new variable – which is assumed to be constant throughout a single UAV take-off

The previously reported work on UAV path planning is studied from the following two different perspectives:

(1) The robustness of the model This examines the UAV path planning model in terms of its accuracy, complexity and authenticity

(2) The soundness of solution algorithms This concerns whether the solution algorithm is fast enough in terms of finding the optimum solutions

At the same time, these two perspectives are highly correlated A very complicated model can describe the real-life situation well, yet it is more demanding on the solution algorithms On

could be found much easier It is such trade off that makes the researches in UAV path planning challenging, yet interesting

In general, finding the optimal solution to the route-planning problem is nondeterministic-polynomial-time complete (NP-complete) Moreover, the UAV path planning problems are characterized by its huge solution space Early researchers focused on modeling the problem in two dimensional (2D) spaces More solution-demanding three dimensional (3D) models appeared only after the 2D case studies were mature enough and powerful solution algorithms were developed Even today, 3D models still have potential for further developments Some important modeling methods reported in the literatures are briefly reviewed in the following sections

1.3.1 Multiple travel salesman problems with time window constraints (mTSPTW) model

The first modeling approach on the UAV mission planning problem derives from the famous mTSPTW [10] In this model, the UAV has predefined starting and ending positions and

a number of targets to be visited The route connecting any pair of targets is pre-defined Each target is to be visited within a specified time window Tabu search was proposed to be the solution method for such model [11, 12] Tabu search enhances the performance of a local search method by using memory structures: once a potential solution has been determined, it is marked

as tabu so that the algorithm does not visit that possibility repeatedly Such local search procedure is used iteratively to move to better solutions until the stopping criterion is reached

This model links the UAV mission planning problem to the mature problem of mTSPTW, which allows for the application of various previous results However, mTSPTW is more of a mathematical problem than a practical model It hardly describes any real world UAV mission

situations Therefore, the path planning result is of little practical use On the algorithm side, tabu search is not capable of solving more complex models proposed subsequently

1.3.2 Dots-on-coordinate model

This proposed model (see Figure 1.1a) describes the mission setting on a 2D coordinate

It is called “dots-on-coordinate” in this study One or several UAVs (black triangles) with their fixed initial positions are to visit single or multiple targets (squares) The constraints are threats

in the form of dots The closer the UAV is to a dot, the greater the probability of destruction The objective is to find a path or multiple paths that visits the targets while minimizing the probability of destruction of the UAV

Figure 1.1 Dot-on-coordinate mission planning model and Voronoi diagram solution [13]

Studies had targeted both single [14] and multiple [13, 15, 16] UAV mission planning in such setting One proposed solution algorithm is the two-stage Voronoi diagram (see Figure 1.2b) and virtual forces [13] Voronoi diagram is a way of space repartition by straight lines such that a point on any given line has equal distance to the two dot-threats closest to it Hence, finding the

Voronoi diagram In the first stage, graph search is applied to find the best combination of Voronoi edges as a rough solution Virtual forces are imaginary repelling forces received by the paths from the dot-threats The closer a path is to a threat, the greater the force Better paths are subject to smaller repelling forces The second stage includes the trimming of the rough solution with virtual forces A gradient decent method is applied here to arrive at the path that has the smallest repelling force The solution algorithm proposed for target assignment is the satisfying and social welfare paradigms [17] Multiple UAVs are first decomposed into teams with assigned targets to each team The welfare of each team is then optimized in parallel, allowing for re-allocation of targets in the solution process Another solution method proposed for solving the Voronoi diagram is the A-star algorithm Genetic algorithms are also proposed for the second step of virtual forces [18]

1.3.3 Polygon-obstacles model

Another popular problem is similar to the Dots-on-coordinate model But instead of having dots as the threats, polygons are used to enclose areas as restricted zones (see Figure 1.2) The objectives are waypoints on the 2D map to be visited and the constraints are the shaded areas that should not be visited Two improvements in the model were made as compared to the Dots-on-coordinate model First, the dimension of the search space is greatly increased from the Voronoi edges to the entire 2D space (Except for the prohibited zones) Second, the coupling of the UAV missions are taken into account [19] This means that, for instance, the elimination of a hostile surface to air missile site will have a positive effect on all the UAVs concerned

Figure 1.2 Interface of the Polygon-obstacles model [20]

One proposed solution method is the Mixed-integer Linear Programming [19-23] However, being a linear programming method, the computation becomes very costly when the complexity

of the problem increases Another proposed method is the A-star algorithm [24-26] Being a heuristic search algorithm, its efficiency is not robust to the increase in complexity in the model

1.3.4 Three-dimensional (3D) mission model

Recently, evolutionary algorithm (EA) becomes a popular choice for solving route planning problem Since it greatly increases the computational power, complex 3D models are widely used for the UAV mission planning problems The major components of a 3D model are terrain, UAV path, and threats

For terrain modeling, one method is to artificially generate a terrain surface [27-30] The

terrain elevation of a point z(x, y) is given by:

) cos(

) sin(

) cos(

) cos(

) sin(

) sin(

)

,

where a, b, c, d, e, f, g are parameters defining the terrain shape Such method is not demanding

on the memory and guarantees the smoothness of the terrain surface However, manually

generated terrains lack authenticity as compared to real terrain Yet, it is not certain whether memory overflow issues will arise when real terrain is applied

In other reported works, real-world elevation data is used for terrain representation 36] Digital Elevation Model (DEM) is commonly used terrain model in this case It is a continuous representation of the ground surface landform through a matrix that contains the elevation data at the corresponding coordinates The merit of such representation is that the DEM data is a common form of geographic data representation As a result, DEM data of different parts of the world is widely available However, the rendering of large area of 3D terrain from DEM data is memory-intensive Moreover, DEM may lead to discontinuity of the terrain Although highly unusual, this will result in inaccuracy of the path planning procedure

[31-The choice of path model directly affects the solution algorithm and the complexity of the entire mission planning model Paths can be in the form of point-wise linear segments

determined by a list of 3D absolute Cartesian points (x, y, z) [32, 36] However, such paths are

not smooth and hence hard to be followed by UAVs In other works, Bezier curve [31] or spline curve [37] is used to represent the path to be followed by the UAV Other curve models, such as cubic spline and 3rd degree B-spline curves, were also used for path representation [33-35] In this study, piecewise cubic Bezier curve is used for the representation of the UAV path due to computational efficiency

B-In terms of the constraints imposed on path planning, some focused only on the collision avoidance with the terrain [27-31], while others introduced additional simple circular threat model (representing enemy radar zone) that the UAV path must avoid [32, 36] However, such radar model differs with any real-world radar For real radar, its probability of object detection decreases with range Some took a further step to incorporate a realistic model of both the enemy

radar and the flight prohibited zones (FPZs) [33-35] Yet with increased complexity, the computational load increases significantly, thus affecting computational efficiency

As for the solution method to work with the 3D model, EA is typically the optimization method in which the coding of the points controlling the smooth path is critical Some EAs codify the 3D points with absolute Cartesian coordinate [27, 30, 31] Such coding scheme suits the EA operators well; however, the search space would be enormous with such setting Others

use relative polar coordinates for (x, y) [29, 34, 35], which would greatly reduce the search space

However, with such representation, changing any intermediate point will induce the change in all the subsequent points Relative polar coordinate encoding is thus hard to handle for local modifications Table 1.1 summarizes the merits and disadvantages of the various modeling approaches for different components of the 3D UAV path planning model

Table 1.1 Various 3D modeling approaches for UAV path planning

Terrain

Parametric artificial terrain

Continuous, not demanding on memory

Circular Radar Zone Easy to calculate Unrealistic Combined realistic radar

Demanding on the solution algorithm

EA Coding Absolute 3D Cartesian coordinate Easy to be used for the EA operators The search space is large

Relative polar coordinate Reduced Search Space

Local changes are hard

to handle

Table 1.2 summarizes the models and solution algorithms adopted in some of the

important previous works on UAV mission planning

Table 1.2 Models and solution algorithms in previous works

Model Solution Algorithm Related Literatures

Dots-on-coordinate

Voronoi Diagram Graph search, A-star

[13-18]

Virtual Forces

Gradient Descent, genetic Algorithms

Polygon-obstacles & coupled

UAV missions

Mixed-integer linear

Three-dimensional mission Evolutionary Algorithms [27-36]

1.4 Integrated Task Assignment and Path Optimization

In a generic UAV task assignment problem, the UAVs are required to perform three tasks (classify, attack, and verify) on each of the targets in the battlefield The requirement of flyable trajectories dictates a lower bound on the turn radius and speed of the aerial vehicles Thus, the generic problem is denoted bounded speed task assignment problem (BSTAP) [38] This generic problem was further defined more specifically in [39] and is adopted in this research It is assumed that the terrain has already been searched by other means and several targets have been

found Let T = {1, 2, …, N t } be the set of targets found, V = {1, 2, …, N v } be the UAVs

performing tasks on these targets The set of tasks that need to be performed by the UAV team

on each target is M = {Classify, Attack, Verify} and we denote N m as the number of such tasks

Target classification, consisting of maximizing the correct target recognition under given observation ability, can be performed only if the vehicle follows a trajectory that places its sensor

footprint on the target After a target has been successfully classified, one or more UAVs attack

it by releasing appropriate weapons Following target attack, cooperative damage verification is performed In general, the objective is to minimize the total length of the paths of the UAVs combined

In the previous works, some assumptions had been used in the description of the basic problem With these assumptions, the problem is presented in its simplest form Relaxing assumptions will lead to more complicated problem formulation The implications for relaxing the assumptions will be discussed further The assumptions for the basic type of generic problems are as follows

(1) Each UAV flies at its own, fixed altitude so that there are no collisions among the UAVs (2) All UAVs are homogenous and fly at a constant speed They hence have the same minimum turning radius

(3) Each UAV is capable of carrying out all three tasks on each target

(4) Each UAV has only three motions: (i) flying straight ahead; (ii) turning left; or (iii) turning right at the minimum turning radius

(5) There is no constraint of fuel and weapon system Therefore, each UAV can travel for as long as it is assigned and visit as many targets as it can

(6) It takes one UAV only one time to put its sensor footprint above (to fly over) a target in order to classify the target

(7) It takes one UAV only one time to fly over a target in order to destroy it

(8) It takes one UAV only one time to fly over a target in order to verify that the target is destroyed

(10) The targets are homogenous There is no priority in destroying any of them

(11) Each target can be destroyed at any time There is no time window within which a target

is to be destroyed

(12) There is no communication delay or interruptions among the UAVs In the case that one UAV detects a target; the target information is shared immediately with all other UAVs

on the same battlefield

While other assumptions are easy to understand, assumption (4) is further discussed in detail here It is the so called “Dubins vehicle” problem [40], which explicitly defines the minimum-length path given the starting and ending positions and tangents of a UAV path

R everywhere, if and only if,

2 1 1 2

(

for all s1 and s2 in the interval of definition of X

A curve X in real Euclidean n-space parameterized by arc length has average curvature

always less than or equal to  1

R provided that its first derivative X' exists everywhere and

satisfies the Lipschitz condition in Eq (1.2) Such path of minimum length between points u and

v with starting and ending conditions of U and V is called an R-geodesic The following theorem

is the main theorem of Dubins‟ research [40]:

Theorem 1 – Dubin’s Vehicle

Every planar R-geodesic is necessarily a continuously differentiable curve which is either: (1) an arc of a circle of radius R, followed by a line segment, followed by an arc of a

circle of radius R; or

(2) a sequence of three arcs of circles of radius R; or

(3) a sub-path of a path of type (1) or (2)

If applied to UAV paths, the theorem basically says that given the starting and ending positions and directions, there exists only one minimum-length path, which is among the six options Among the six possible paths, one is the minimum length path according to the theorem

In the following sections, three variations of the generic problem, reported in the previous works, are discussed One or many assumptions mentioned above are relaxed in each variation

As a result, different problem formulations and solution algorithms are proposed

1.4.2 The dynamic network flow model

In [41, 42], the air vehicle resource allocation problem is solved using dynamic network flow optimization models The assumptions (5), (6) and (10) are relaxed The UAVs have only sufficient fuel to fly for 30 minutes Differences in onboard weapons and the attack angle also restrict the attack of a target The classification of target is a probabilistic event that depends on the direction of the sensor on top of the target Only when the classification probability of a target is greater than 90% can the target be attacked Otherwise it has to be classified again Moreover, targets are not homogenous and each target is assigned a value associated with its

model for solving the problem and Schumacher et al [42] employed a linear program based on the capacitated trans-shipment problem to solve the task allocation problem

In [38, 39], the problem of cooperative multiple task assignments is discussed Further to the relaxing of assumptions (5) as in [41, 42], assumption (7) is also relaxed A second attack must be performed on a target if its destruction is not verified or if multiple attacks are required

on the target However, assumptions (6) and (10) are still used As for solution method, Shima et

al [39] employed genetic algorithm and Rasmussen [38] used a state-space search algorithm

1.4.3 The tree representation model

Rasmussen and Shima [43] proposed a tree search algorithm for finding the solution for this classic problem There are two drawbacks of the tree representation First, the exhaustive expansion of all the branches of the tree is very memory-intensive Usually, a heuristic search is applied for tree models to avoid exhaustive search This, however, could easily lead to local minimums Second, the tree structure is very hard to use for the representation of target visiting sequence, which is an important factor that will affect the solution qualities

1.4.4 The graph representation model

The graph representation has certain advantages as compared to the tree representation

In such representation scheme, each solution is composed of a combination of edges of the graph The construction of the graph is not difficult and requires limited memory space The implementation of the graph representation will be discussed in detail in Chapter 5

1.5 Surveillance in Complex Urban Environment

The integrated task assignment problem discussed above considers the coordination among multiple UAVs to complete pre-defined tasks on spot targets Another class of problems

is to find an optimal way to cover an area with complex surface conditions For example, in the urban areas, skyscrapers usually block the line of sight of the sensors in the sky

The scenario is best described by the work of [44] A team of UAVs with onboard sensor

is the given resource, the sensors have limited covering angles In other words, the sensor covering area is a cone-like space below the UAV The team of UAVs is to survey a given urban area with tall skyscrapers, with the objective to minimize the aggregate non-exposure time on the ground The problem formulation in [44] will be described in Chapter 7 in detail

The drawback of [44] is two folded First, it proposed a two-stage solution which decouples the problem into two separate optimization problems This greatly impedes the ability

of the algorithm to find global optimum Second, its occlusion aware model allows only for flat terrain and fixed shapes of buildings, making the problem unrealistic This study focuses on developing an occlusion aware model that is both efficient and allows for more generalized problem definition

1.6 Research Objectives

This thesis covers three different aspects of the researches on UAV mission planning The first is the trajectory and path planning of single UAVs This study has gone very deep and realistic into this area as compared to previous researches The objective is to design a realistic and complex optimization model and a robust solution algorithm efficient in solving the optimization problem The second objective is the coordination among multiple UAVs, in which reasonably simple assumptions on the UAV trajectories are made so as to focus the computational resources on the task allocation to different UAVs Finding the optimal paths of

urban environment in which the line of sight of UAV sensors can be blocked by skyscrapers This lays a foundation for the future application of path planning and cooperative task assignment

1.7 Limitations and Assumptions of Research

This research project considers the mission planning of the aircraft on a macro scale The problem studied does not include the design of the aircraft itself All experiments are done through computer simulations, with no actual fly-testing using the physical aircrafts In terms of solution algorithm, this study focuses only on the genetic algorithm for optimization Although three different topics are studied in this thesis, the general limitation and assumptions of the research are listed below

(1) The UAV is assumed to be a point, rather than a mass that has volume Therefore, the collision detection is on a single point

(2) The UAV is assumed to fly at constant velocity As a result, the turning radius constraint is constant in each simulation

The assumptions specific to each topic are described in detail in each section

1.8 Organization of the Thesis

The thesis is organized into 7 chapters The current chapter is an overview of the thesis and the literature reviews Chapter 2 explains the motivations of conducting the research It points to the weakness and room for further development in the current research It also provides reason for the use of genetic algorithm as a solution tool Chapter 3 explains the principles and mechanisms of Genetic Algorithm Chapter 4 provides a detailed description of the implementation of the path planning model Moreover, the merit of the model and the Genetic

Algorithm as a solution is discussed in correlation with the results from the simulation experiments Chapter 5 starts by introducing the model for the problem of UAV cooperative task assignment It then furthers the discussion by justifying the model and solution through simulation experiment Chapter 6 introduces the implementation of an occlusion-aware model in the urban surveillance problem The generation of a vantage point set will later be used as the target waypoints for multiple UAVs in the coordinate task assignment problem Chapter 7 summarizes the major findings from the thesis and points to the possible future directions of research

CHAPTER 2 RESEARCH MOTIVATIONS

This chapter starts by describing the motivation of conducting research in the three areas

studied The motivations concern more on the modeling and formulation of the problem

Following this, the genetic algorithm (GA) as a solution tool is discussed

2.1 UAV Path Planning in Complex and Realistic Environment

Despite much effort that has been put on the 3D UAV mission planning and some

success has been achieved, there are plenty of rooms for improvement The following

observations from the previous reported works serve as the motivations for the current study:

(1) Among the reported optimization methods for path planning, there is no constraint on the

solution search space initially Instead, random solutions are generated and evolved from

a very large space The constraints are added during the evolution process as penalties for

unqualified individuals Although genetic algorithms are powerful, the large initial search

space still adversely affects the path planning efficiency This study seeks to reduce the

initial search space by forcing each individual in the initial generation to obey the flight

constraints

(2) The previously reported methods have limitations in solving path planning problems with

concave obstacles This study attributes it to the problem in evaluating each individual

solution A new method is proposed to escape concave cases

(3) Most of the previously reported methods are “result-driven” They are eager to achieve

good results This study, however, aims also at fully understanding the path planning

problem Different parameters of the path planner are studied in detail so as to understand how they affect the solution and algorithm convergence

2.2 UAV Cooperative Task Assignment

For multiple UAV cooperative task assignment, the following issues are not fully addressed in previous literatures

(1) The efficiency of graph representation as compared to tree representation of the problem

is not demonstrated in the previous literatures This study will demonstrate through simulation experiments the merits of such

(2) The UAVs considered in previous literatures all have homogenous flight dynamics However, this thesis considers UAVs with different flight dynamics through the application of multiple graph structures

Apart from the above two aspects, the development of a 3D display system for the solution is also a motive for the current research This is because all the previous literatures could only represent the solution in 2D coordinates, which prevents the further development of the problem in the future

2.3 UAV Surveillance in Complex Urban Environment

The occlusion aware model is an important part of this class of problem This is a model that checks the visible areas on the ground from a sensor positioned in the sky In the literatures, the check for each sensor would require iteratively going through all the buildings concerned Moreover, the ground must be flat for such model This gives raise to motivations to develop a model that utilizes the “ray detection” function of the JME3 package The efficiency is greatly

increased with the application of the “ray detection” Moreover, the direct use of the 3D package makes the result-rendering easier This is an advantage for result-visualization

2.4 Genetic Algorithms as a Solution Tool

Genetic algorithms (GA) are a type of stochastic and non-gradient search optimization method It enables searching efficiently a decision state space of possible solutions, as long as the search space is not extremely rugged As an optimization tool, GA has the following characteristics

(1) It does not require exhaustive search of the whole solution space in order to arrive at an optimal or near optimal solution

(2) It does not require the calculation of gradient for each iteration

(3) It is exceptionally suitable for solving multi-objective optimization problems

This study is motivated to use GA as a solution tool due to the nature of mission planning problems Firstly, the solution space of this type of problems is usually infinite and discontinuous Therefore, the exhaustive search algorithms are not suitable in this case Secondly, it is hard to measure the improvement on solution quality through gradient, rendering the gradient descent methods unfeasible Thirdly, there are usually multiple objective functions to be optimized, which favor the use of GA

2.5 Java Monkey Engine (JME) as System Implementation

Java is the main programming language for implementing the methods developed in this study As an object-orientated programming language, it is simple to use All the unnecessary complexities of the languages such as C++ are removed Java enables the development of robust applications on multiple platforms in heterogeneous, distributed networks Moreover, due to the

3D nature of such project, the package JME is used in rendering the results The JME package has the following merits

(1) Robust terrain package makes it easy to render the terrain from data It also has functions for height measurements and scaling These are used in terrain collision detection

(2) Compatibility with other Java packages

(3) Plug-in of Java-swing components that allows for the programming of Graphical User Interface (GUI) with JME

CHAPTER 3 INTRODUCTION TO GENETIC ALGORITHM

The idea of GA comes from the natural evolution process The famous reasoning behind

the long neck length of giraffes can be used here as an illustration Giraffes with longer necks are

capable to reach higher branches of trees They therefore have access to more abundant food

supplies as compared their shorter-necked peers Fig 3.1 illustrates the evolution process of two

generations of giraffes The parent generation is constituted of giraffes of great diversity – long

and short neck ones The offspring of that generation are therefore also of diversity However,

due to the scarcity of food, some offspring may die out According to the assumption –

shorter-neck ones have less food, the survival rate for longer-shorter-necked giraffes is much higher than that of

shorter-neck ones This step in the evolution process is called “selection” The environment

selects the most suitable individuals and therefore the most suitable genes After generations of

evolution, shorter-neck giraffes died out and only the longer-neck giraffes survived and hence

their genes passed down

The GA, which is also called evolutionary algorithm, is designed according to such

process It involves five essential steps

(1) The encoding of the solution – the mathematical representation of the solution This

resembles the construction of the genes of the giraffes

(2) Initialization – randomly generates a first generation with diversity This compares to the

first generation of giraffes as shown in Fig 3.1 The first generation should consist of

solutions with diversity

(3) The fitness function – the mathematical way of quantifying how suitable is the solution to

the problem In the giraffe example, giraffes with longer necks are “more fit” to the environment They hence have a larger fitness value as compared to shorter-necked ones

(4) The genetic operators – this resemble the process of producing offspring The common operators include crossover, mutation, elitism, and immigration

(5) The selection – this is the analogy of the natural selection process that retains the superior

individuals and kills the inferior ones

Figure 3.1 The evolution process of giraffes Each iteration of the algorithm produces a new generation The algorithm stops when certain convergence criteria are met Thereafter, the individual with the best fitness will be retained as the final solution

Although a particular GA must be designed for each specific problem The principle mechanisms of this class of algorithms are the same This section introduces the important components of GAs

3.1 Encoding of the GA Chromosome

The encoding is the representation of solutions by a list of data resembling that of the

genes on a chromosome As shown in Fig 3.2, a chromosome is an organized structure of DNA and protein found in cells It is a single piece of coiled DNA containing many genes Each gene can be in control of a particular biological feature of the cell, and hence some features of the entire biological entity If the chromosome and its interpretation mechanism of a biological entity are known, this entity can hence be created

Figure 3.2 Illustrations of Gene and Chromosome [45]

Derived from these biological facts, in GAs, both the chromosome interpretation process and its inverse process are important The encoding process involves designing the genes and chromosome to represent the solutions The decoding process involves interpreting the chromosome into a solution In some of the previously reported works, GA is widely used to solve path planning problems Using the waypoints‟ coordinates as genes for the chromosome is

a common way of solution encoding for the path planning problems For the example path shown

in Fig 3.3a, the encoding by its waypoints is shown in Fig 3.3b

(a) Path example (b) Path encoding Figure 3.3 Example of solution encoding for path planning problems

3.2 The Initialization

The initialization is a process that randomly generates a set of solutions, which is called

the “first generation” Usually, each individual solution of the first generation satisfies a number

of constraints However, the solutions of the first generation do not have to satisfy all the constraints This is because some constraints can later be met through evolutionary process For a well designed GA, the initialization process must be able to produce a first generation of great diversity Yet, the production of subsequent generations from the first one must also be efficient

3.3 The Fitness Function

The fitness function is a mapping that maps the solution space into positive real numbers

It basically describes how “optimal” the solution is, given the problem setting The fitness function design must accurately reflect the solution quality In the giraffe problem, it is obvious that bigger fitness values should be assigned to longer-neck giraffes However, higher dimensional and multiple objective problems do not have obvious fitness function assignment The fitness function assignment is usually the result of trial and error In general, a good fitness function must distinguish better solutions