Level 2 Fusion Formation Association Metric with Uncertainty

Level 2 Fusion: Formation Association Metric with Uncertainty STEPHEN C.. One way to interpret formation has been to use the image processing technique of invariant moments.. These momen

Level 2 Fusion: Formation Association Metric with Uncertainty

STEPHEN C STUBBERUD Boeing Corporation Anaheim, CA 92805 UNITED STATES OF AMERICA

KATHLEEN A KRAMER Electrical Engineering Program University of San Diego

5998 Alcalá Park San Diego, CA 92110-2492 UNITED STATES OF AMERICA

Abstract – Level 2 data fusion, also referred to as Situational Assessment, defines relationships between entities

In a target tracking scenario, one part of interpreting the relationships is the formation that related entities form One way to interpret formation has been to use the image processing technique of invariant moments These moments can be used to compare the actual formation of the targets to known formations In the previous implementation of this metric, the invariant moments for the tracks were computed based on the estimated location However, tracks are often estimated with both a location and uncertainty This uncertainty can have an impact in the comparison of the formations In this paper, we begin the incorporation of uncertainty into the formation metric and determine its potential for benefit in formation estimation

Keywords – Invariant moments, Level 2 fusion, Formation association, Information fusion, Uncertainty

1 Introduction

Level 2 fusion, also known as Situational

Assessment, develops and interprets the relationships

between entities [5] One approach to implement an

automated Level 2 fusion system was presented in

[7] Figure 1 shows the proposed functional flow of

a Level 2 fusion system, allowing the problem to be

decomposed into smaller and more easily

addressable components

Detect Predict Associate Hypothesis Update

Hypothesis Management

Fig.1:A Level 2 Fusion Architecture

One of the key components that was defined in this

approach was that of association Unlike Level 1

association that compares measurements to existing

tracks as discussed in [2], Level 2 association also

incorporates the interpretation of the relationships

that are developed between entities

To implement the Figure 1 approach, the concept of

a state representation to define the Level 2 object was

presented The proposed state was defined as

a

b

c

d

 

 

 

 

 

 

where

unit

unit

unit

unit

x

y

a

x

y





  

1

1

#

#

#

#

n

n

ofclass

ofclass b

ofclass ofclass







formation

c

formation change

1 2

group extent

d

group extent

In [8] and [9], association metrics were developed for each component While each of these techniques has been shown to work well as defined, the

implementations were developed without incorporating the uncertainty associated with the tracks The question of how uncertainty can be incorporated into these metrics has been raised as a logical next step in the development of an automated Level 2 fusion system

In this paper, we begin this investigation Our first step in the use of uncertainty in the association metrics investigates the inclusion of uncertainty into the formation metric Section 2 summarized the formation metric and how we can interpret uncertainty in this context This is followed in Section 3 with the proposed approaches and associated reasoning Section 4 summarizes the results of our tests cases while Section 5 presents our conclusions and proposed continuation of this research

2 The Formation Metric And Viewing Uncertainty

Tracking systems are often developed using a Kalman filter [1] The state estimates of the target localizations are computed as a mean value and an associated error covariance In Figure 2, the targets are plotted over time as points that represent the mean estimates It was from this interpretation of the results, points plotted on a display that an operator can interpret, that the initial concept of image processing as an approach to interpret the formation was first considered The upper left corner shows the last set of position estimates that are of interest

Fig 2: Tracking of 3 Targets Over Time

To make the determination, point locations for each group were compared against an image of a known

formation using the following seven invariant

moments:

(1)

(4)

3

3

(5)

(6)

4

(6)

3

(7)

where

  p q ( , )

pq x x y y f x y dxdy

   

(8)

10

00

m

x

m

01

00

m

y

m

( , )

p q pq

m   x y f x y dxdy

   

These moments, as described in [3] and [4], will be the same for two images if they can be considered translated, rotational, and scaled versions of each other To create a metric for this comparison, we employed a norm of the error between the two sets of the moments In [9], we showed that this approach was useful even in the presence of noisy results This is important because these so-called noisy formations emulate the real behavior of vehicles in the field Also, the estimates from tracking systems are corrupted by noise

The formation metric combines the error of the seven invariant moments between the estimated formation and those of a known formation, using a norm, as shown in equation (12)

known known known

known known known

m













(12)

While the point estimates have performed well in previous experiments, the concept of incorporating the uncertainty into the metric has been raised as a potential method for improvement A number of the techniques have been discussed The seemingly most promising approaches to the problem are 1) to use the uncertainty ellipse for the target estimates to derive the metrics and compare to results using the point estimates and 2) to create a convex hull from the target group and compare that to a formation’s convex hull

Here, we address the first approach As with the point-to-point comparison of [9], we again look at the display of the Level 1 fusion or track objects While the target localization is considered a point, the error covariance can be represented as an error ellipse The Kalman filter assumes that the distribution of the target estimate is Gaussian Ellipses, similar to those of Figure 3, are often drawn around the target’s estimate to represent this

distribution Each ellipse typically represents two standard deviations from the mean and is also referred to as a 2-sigma ellipse The 2-sigma ellipse

represents a 0.865 probability that the target shall be

in that region

Fig 3: Associated uncertainty of position shown as a

2-sigma ellipse The ellipse is determined from the positive definite

covariance matrix, C The eigenvalues of this 2x2

matrix represent the semi-major and semi-minor axes

of the ellipse The rotation of the ellipse is based on

the eigenvector of the larger eigenvalue

Mathematically, the matrix C is defined as

2

2

(13)

where x and y are the standard deviations of the x-

and y-coordinate of the estimates.

Fig 4: Four ellipses that are the same according to

the invariant moments

In Figure 4, we see four scaled ellipses Each has the

same invariant moments as the others Whether

scaled, rotated, or translated, these ellipses are considered identical

This allows an ellipse, rather than a point, to serve as the image for each target By looking at a point as a scaled version of a larger ellipse, we can compare the results of the moments for each

3 Implementation of Uncertainty in the Formation Metric

In image processing, the density function f(x,y) used

in equations (8) and (11) is replaced by an intensity function In [9], the intensity function was set to

1 for , target location ( , )

0 else

x y

f x y  

This made implementation simple in that integration was performed over a small set of delta functions Here, an intensity function must be defined and a numerical integration must be implemented for each ellipse

Numerical integration was performed by approximating the ellipse as a set of points Figure 5 shows an example of the technique that was used

Fig 5: Points Defined Over Area of Ellipse Next, we selected three approaches for computing the intensity values of each point

1 Each value is computed based on the Gaussian distribution of the target at that point

1 1 1

x x x x C

y y y y

e C





Then, the values of the points for a given

ellipse are normalized to sum to 1

2 Similar to method 1, but the values are

weighted according to the area of the

sub-ellipse associated with that point The values

are normalized to sum to 1

3 Each value is based on a uniform distribution

over the ellipse

4 Summary of Results

For our evaluation, we defined five primary

formations: horizontal column, vertical column, a

box, a wide wedge, and a narrow wedge Figures 6,

7, and 8 show the latter three of these These three

were used to evaluate the effectiveness of each of the

uncertainty techniques

Fig 6: Box Formation

Fig 7: Wide Wedge Formation

Fig 8: Narrow Wedge Formation Test cases were generated by applying Gaussian-distributed error to the standard formations using different levels of noise Different numbers of sub-ellipses (5, 10, 15, and 20) were also tested In most cases seven targets were in the formation, but cases

of only five were also tested Each test run included

52 different noise sequences This number was selected based upon 90-90 results of order statistics [6] Measures of effectiveness included whether the correct formation was identified, the average metric observed over the 52 tests, the minimum metric observed, and the maximum

One such test case included applying noise to the box formation Figure 9 shows the randomly generated noise and associated ellipses for one of the sequences within the test run These tests were based upon using 10 sub-ellipses and a maximum noise sigma of 0.5

Fig 9: Box Formation With Random Noise On Each

Target Table 1 shows the results observed in that case The four elements of each entry correspond respectively

to the metric basis used – techniques 1, 2, and 3

(defined above), and the point-based metric that does

not take uncertainty into account In this case all

techniques resulted in an accurate identification of

formation

0.0004

0.0058

0.0011

0.1035 0.1038 0.1151 0.0903

0.0413 0.0365 0.0481 0.0359

wide

wedge 7.2257 7.2328

7.2221

7.2287

7.2822 7.2822 7.2779 7.2880

7.2550 7.2570 7.2519 7.2582

narrow

wedge 4.2539 4.2551

4.2496

4.2590

4.3160 4.3163 4.3104 4.3236

4.2826 4.2850 4.2786 4.2868

Table 1: Metric Results from

Box Formation Test Case

Results from numerous test runs showed that

technique 2 provided the best results among the three

uncertainty techniques However, the results

obtained were similar to those obtained from the

point-based technique Varying the number of

ellipses did not improve performance significantly

5 Conclusions

In this paper, we addressed the initial use of

uncertainty in the Level 2 fusion formation metric

Our initial results indicate that, while the methods

are accurate, they provide little, if any, improvement

over the point-to-point technique originally

developed

The use of the added information of uncertainty

should, however, provide an improvement We plan

to continue to look at the incorporation of uncertainty

into the metric Technique number 2 will provide the basis of this approach We also intend on

investigating the convex hull approach and to use a multiple point-to-point technique

References:

[1] Blackman, S., Multiple-Target Tracking with

Radar Applications, Artech House, Norwood,

MA, 1986

[2] Blackman, S and R Popoli, Design and

Analysis of Modern Tracking Systems, Artech

House, Norwood, MA, 1999

[3] Hu, M., “Visual Pattern Recognition by Moment

Invariants,” IRE Transactions on Information

Theory, pp 179-187, February 1962.

[4] Maitra, S., “Moment Invariants,” Proceedings

of the IEEE, Vol 67, No 4, pp 697 - 699, April

1979

[5] Steinberg A., C Bowman, F White, “Revisions

to the JDL Data Fusion Model”, Proceedings of

the SPIE Sensor Fusion: Architectures, Algorithms, and Applications III, pp 430-441,

1999

[6] Stubberud, A.R., Class Notes, Engineering Probability:ECE 186, University of California, Irvine 1996

[7] Stubberud, S., P.J Shea, and D Klamer, "Data Fusion: A Conceptual Approach to Level 2

Fusion ( Situational Awareness)," Proceedings

of SPIE, Aerosense03, Orlando, FL., April,

2003

[8] Stubberud, S., P.J Shea, and D Klamer,

"Metrics for Level 2 Fusion Association,"

Proceedings of Fusion 2003, Cairns, Australia,

July, 2003

[9] Stubberud, S and P.J Shea, "More Metrics for

Level Fusion Association," Proceedings of the

16th International Conference on Systems Engineering 2003, Coventry, England,

September, 2003