(Luận án) Research on image processing methods of detecting and tracking several military targets to employ for controlling autonomous weapon systems

Phase1:Determinationofmathematicalmodelsfortargetrepresentation imagef eatures themathemati calmodel fortargetrepr esentationbyi mage Phase 1:Determination of mathematical models for tar

DEPARTMENTOFDEFENSEACADEMYOFMILIT ARYSCIENCEANDTECH NOLOGYMILITARY

SUMMARIZESM A T H E M A T I C S D O C T O R A L T H E

S I S

HaNoi-2017

Atd a y m o n t h year2017

Canlearndissertationatthelibrary:

- LibraryofAcademyofMilitaryScienceAndTechnology-MinistryofDefense

- NationalLibraryofVietnam

OPENINGMonitoring systems for automatical target detection are an important part ofmanyhightechnologyweaponsystems.Theyincreasetheeffectivenessofweaponsystems,anddecreasetheactionswhichareussuallymanipulatedbyhands,especiallyinhardconditionsofenvironment.Researchanddevelopmentofmonitoringsystemsforautomaticaltargetdetectionplayanimportantroleinupgradingtheoldweaponsystemsandprovidegroundsfordevelopingnewgeneration weapon systems The dissertation of “Research onimage processingethods ofdetecting and tracking several military targets to employfor controllingautonomous weapon systems” is usedto fittheneedsof ourarmy

 The main purpose of this dissertation is developing an automatical systemforautomatic detection and tracking military targets (tanks, vehicles…) using theimageprocessingtechnologyandrecognizingalgorithmstocontrolweaponsystems

 The main object of this research is monitoring systems of differentweaponsystems, the targets of which are ground military target like tanks and othermotorvehicles

 Theresearchareaofthisdissertationfocusesonresolvingthefollowingquestions:-What are the targets of particular types of weapons? What specialcharacteristicsdotheyhave in comparisionwith other backgroundobjects?

-How to automatically detect and recognize target in the images underdifferentconditions of image collecting?

-Howtocorrectlytrackthedetectedtargetinrealtime?

 Scientificsignificancesofthedissertation:

- Proposeasolutiontodevelopamonitoringsystemforautomaticaltargetdetection,usingimageprocessingtechnologyandintelligentrecognitionalgorithms

- Proposeanewautomaticmilitaryt a r g e t detectionmethodu s i n g videoimagesequences

- Proposeanewautomaticmi lit ar y

targettracking method usingvideoimage sequences

 Practicalsignificancesofthedissertation:

- Thisdissertationservesasanimportanttheoreticalbasisfordevelopingmonitoringand tracking military target systems in order to improve or upgradetheoldgenerationweaponsysemsanddevelopnewhightechnologyweaponsystems

image features specify the target

the input images

- The study in this dissertation is also a solution using in replacing themonitoringpartsinhightechnologyweaponsystems

 Content of the dissertation: Opening, 04 chapters, conclusionandrecomendation,publishedworks,references

CHAPTER 1 OVERVIEW OF TARGET DETECTION AND

This problem's input are imagescollected from camerasa n d i t s o u t p u t i s

t h e target regions with images in input image frames In the automatic targetdetectionand tracking system, target detection is the first problem need to besolved It isconsidered asthe firststepintheprocessoftargettracking

Phase1:Determinationofmathematicalmodelsfortargetrepresentation

imagef eatures

themathemati calmodel fortargetrepr esentationbyi mage

Phase 1:Determination of mathematical models for target

representation.Thisphase is performed on the template images to build the

mathematical model fortarget representationbyimage features

templat eimages

Phase 2:Identification of the target in the input images.This phase detects

thetargetimageregionsontheinputimagesusingmathematicalmodelfortargetrepresentationidentifiedinPhase1

The tracking section presents in detail the type of image features and themethodoftargetdetection withdifferent mathematicalmodels

1.1.2 Imagefeatures

This section presents the types of image features commonly used torepresentobjectsin automatictargetdetection.Therearethreemain typesoffeatures:

1.1.2.1 Colorfeatures:

Color features refer to one of the important features to characterize the surface

ofthe target Color features of a pixelPis a vectorf= ( f 1, f2,… ,fn),wherefiis thevalue

of a color componentiin positionPin a certain color space or in manydifferent color spaces For an image regionsR,the color features commonly used torepresentRintargetdetectioniscolor histogram.

c a t i o n s f o r target detection and tracking by input video images Based on calculations, shapefeatures are

classified into two main categories:i ) r e g i o n - b a s e d s h a p e

f e a t u r e s which are selected based on the pixels located on the contour of object;ii) contour-based shape features selected based on the information of the

pixels located on thecontourandinsidethe contourofthe target

1.1.3 Targetdetectionmethods

Some authors classify the target detection method based on image features [52],[106], while many other authors classify based on mathematical model fortargetrepresentation [87], [ 107], [108] In this thesis, we rely on both imagefeatures andmathematicalmodeltoclassifythesemethodsintofourcategoriesasfollows:

exploit information (color and texture) at the level of pixel to separate theinputimagesinto differentimageregionscontaining pixels withsimilarfeatures.

Remarks:Generally, segmentation-based methods have high accuracy in

targetdetectionandhavesimpleprocessesforlearningtargetrepresentationmodelparameters.However,therearesome shortcomings as follows:

- Slowspeedofcalculation,becausetheprocessofimagesegmentationisrequiredtoconsiderallthepossibilities ofeachpixel

-The advantages of the algorithms: adaption to the change of objects in

thebackground, but low accuracy when the target is changed by differentlightingconditions orsuddenspeed anddirection ofmovement

- Background model-basedalgorithms have significantly high computing

speedand efficiency where there are fewer changes in the backgrounds However,

detection include: i)Neural networks;ii) SVMSupport VectorMachines;iii)AdaBoost.

Remarks:The above methods are easy to implement and highly effective for

thecases where the target has image features with high difference from thebackgroundobjects.Their maindisadvantages are:

- They require a large enough template data set of targets and backgroundobjectsfortraining.Thisisverydifficulttocollectenough

- They will have low accuracy in target detection in case of smalldifferencebetweentheimagefeaturesfortargetpresentationandimagefeaturesforpresentation ofother backgroundobjects

Remarks:The above method is used relatively popularly because of its

highaccuracy The effectiveness of this method depends largely on the set offeaturesrepresenting target The biggest disadvantage of this method is its slowcalculatingspeed, especially in case of large size and number of set of featuresrepresentingtarget

1.2 Targettracking

1.2.1 Targettrackingproblem

Target tracking is a problem of identifying the orbital motion of one ormoretargets over time by localizing the target in each frame [52] The main features

include:-Inputs:Theimagesequenceso v e r t i m e ; i n f o r m a t i o n about the target;

information about background objects -Outputs:The position ofthetargetoforbital

motioninthe input images

1.2.2 Targettrackingmethods

Based on the features for target representation and models for orbitalmotiondemonstration of target [52], the target tracking methods are classified intothreemain typesasfollows:

1.2.2.1 Point-basedtargettracking

Point-based target tracking method presents the target in the image as apoint(center point of the target) or a set of points (using features on the targetcontour).There are many algorithms for target tracking and they are divided into 2

categories:DeterministicalgorithmsandStatisticalalgorithms.

Remarks:Theadvantageofpoint-basedtargettrackingalgorithm isitsfastcalculating

speed, suitable for applications where motion speed and trajectory of thetargetchange slowly over time However, these algorithms have low accuracyincaseofconstantlychangingorbitalspeedandmovementofthetarget.Ontheother

hand, the use of information in some pixels to identify the target is sensitivetobackgroundnoise.

1.2.2.2 Surfacefeature-basedtrackingmethods

The methods of this class approximates the target image regions as arectangularorellipticalonesandusesurfacefeatures(colorandtexture features)torepresentthe target Most of the traditional tracking methods use grayscale information topresent the targetandcross-correlation matching to identify the target.I n s t e a d

o f justusingthegrayscalevaluesonly,recenttargettrackingmethodsuseacombination

of manydifferentsurfacefeatures

Remarks:Surface feature-based tracking methods solve the problem of

targettracking similarly as target detection which is based on motion features, sothat itadapts to the change in speed and direction of movement of the target.However, theaccuracy and calculating speed of these methods depend largely on the selection ofthe image featuresfor target presentation The accuracy in target tracking is lowifcolororgrayscale features areusedonlywhenlight conditionsinthebackgroundare changed In case of too complicated features, the computing speed will beslower

1.2.2.3 Shape-basedtrackingmethods

Thesemethodscanbedividedintotwomaincategories

- The first categoryuses a shape specification to present the target as the

templatein the first frame based on the detected target and then applies matchingtechniques totrackthetargetinthe next frames

template Thesecondcategoryrepresentsashiftinthespaceofthetargetcontoursbetween

consecutiveframes inastate-space model

Remarks:The shape-based tracking methods have high accuracy However,

theyalsohave highcomplexityandslowcomputingspeed

1.3 Characteristicsofmilitarytargetdetectionandtracking

Therearesomeoutstandingcharacteristicsinmilitarytargetdetectionandtrackingcompared with civiliantargetdetectionand tracking asfollows:

- Firstly,the colors of military targets are often similar to those of

backgroundobjects such as grass and tree regions, making it difficult to separate themilitaryfrombackgroundobjectsinthe images

- Secondly,military target detection and tracking are often conducted at a

distanceof hundreds of meters to kilometers; therefore the collected images oftencontainmanybackgroundobjectswithnoise

Target detection Target tracking

Video image sequences

Target location for each image

- Thirdly,militarytargetdetectionandtrackingsystem isrequiredtohaver

eal-timecalculatingspeedand highaccuracy

Theabovecharacteristicsarealsotherequirementstosolvetheproblemoftargetdetection andtrackinginthis thesis

Figure1.5: Blockdiagramof militarytargetdetectionandtracking

1 Image acquisition block:This block includes dedicated cameras capable

ofcapturingdistantscenes withhigh imagequality

2 Target detection block:based on video image sequences collected from

imageacquisition block, this block is responsible for identifying the presence ofmilitarytargets (people, tanks and military vehicles) in the scene Output of this

block isinputtothe initialstep oftargettrackingblock.

3 Target tracking block:while the output of thedetection targetblock indicatesthe

presence of military items in the scene, in the next video image sequence,

thesystemwillstarttotrackthetarget andtargetdetection blockwillstop working.

1.4.2 Orientationsforthetasksofthethesis

Therefore, target detectionandtracking methodsb a s e d o n i m a g e s

p r o p o s e d i n this thesis is required to solve the above-mentioned difficulties.The mains tasks ofthethesisaredefinedas:

Task1:Conductresearchanddevelopamethodtodetectmilitarytargetseffectivelywit

hfastercomputingspeedfromremotelycollectedvideoimagesequences

Task 2:From the target image regions identified in the first image

sequences,conduct research and develop a military target tracking method with highcomputingspeed andaccuracyinthe next videoimage sequences

1.4.3 Solvingorientationsformilitarytargetdetectionandtrackingproblems

As above described in Section 1.1 and 1.2, it is difficult to meet therequirementsof military target detection and tracking problems Based on researchand analysis ofthe advantages of the methods and military target features, the direction ofsolvingmilitarytargetdetectionandtrackingproblemsintheinputvideoimagesequencesisidentifiedasfollows:

- Firstly,to narrow the searching space of the target in the input images using

theimagefeaturesandthe movingfeaturesofthetarget

- Secondly,to choose image features reflecting the unique properties of the

targetcompared to background objects In this thesis, a combination of different

imagefeatures are used for target presentation including: 1)Color

features,2)Direction-based featuresand 3)Shape features.This combination of

features will reflect thepartial andentire features ofthe target

- Thirdly,to choose mathematical models for representation of image

featureswithfastercalculatingspeedandhighaccuracy.Mathematicalmodelsforrepresentationofimagefeaturesfocusonprobabilisticmodels,mathematicalmeasurementoftemplatematchingandclassificationmodelsusedintheidentification algorithms

- Fourthly,to gather large enough data sets containing various

environmentalscenes under different imaging conditions to: 1) Build target templatedata set forlearning; 2) Review and analyze the effectiveness of target detection andtrackingalgorithms

This chapter presents a new method for detecting military targets in videoimagesequences In this method target detection from the source images isperformed intwo steps:

- The first step is extracting the image’s regions where there’s posibility oftarget’spoints Those regions are call region of interest, abbreviated ROI ROIs areselectedbymovementfeatures

ROI extraction Target’s identification Image’s areas Input image sequences

- In the second step, target is located usingthe ROIs To do this, we usethecombination of colour and shape features to show all the ROIs in amathematicalmodel This model calculates the similarity between thecharacteristics of all ROIsand the sample characteristics of the targets, which aredefined before in learningdata sets

The proposedmethod has been applied to different types of military targets suchastanks and motor vehicles More over, this method is also evaluated by thepraticalresultsincomparisonwithotherobjectdetection methods

𝑤𝑟,𝑤g,𝑤𝑏–co lor weight values, satisfied 𝑤 𝑟+𝑤g+𝑤𝑏= 1.Themovement mask

of all the objects intheframe is extractedasbelow:

𝑀(𝑥,𝑦)={1,0,k h i 𝐷(𝑥,𝑦)<𝑟khi𝐷(𝑥,𝑦)≥𝑟𝑀 𝑀

(2.3)

t h e image size are filtered Image 2.2 (b) show the result of ROI extraction fromtwoinput frame intheimage2.2 (a).

2.2.2. Targetidentification

2.2.2.1 Method

R= {R 1, R2, , RN}is a set of ROIs which are extracted in 2.2.1 The process

oftarget identification is performed successively with eachR j On each region

ofinterest (ROI)R i , is a rectangleWdefined fromF i+kwhich has the center pointcoincidedwithcenterpointofRj(seeImage2.3).ThereafterrectangleWissegmented

into homogeneous by color regionsS={S 1,S2, , SM} Using the setS,the target is

being identified as the area hZ<S,Zis the set of hormogeneous colorregions

{S k ,S g, } which connet between each other to make a bigger imagearea,satisfiedt h e two conditions(2.4) and(2.5):

𝑋𝑆(𝑍,

3 Find the image regionZ<Sfrom the subsetsX<Sso thatZhas themaximum

sameness by image features in comparision with target(satisfiedthecondition(2.4))

(c) Theresultofeachstepintheloop ofAlgorithm1