Vision Systems - Applications Part 10 pps

Bearing-only Simultaneous Localization and Mapping for Vision-Based Mobile Robots 357The estimation of the landmark position is more accurate when the intersection of two vision cones i

Bearing-only Simultaneous Localization and Mapping for Vision-Based Mobile Robots 351

r is independent onα Formulas are derived for computing the PDF (Probability Density

Function) when an initial observation is made The updating of the PDF when further

observations arrive is explained in Section 5.2.A

5.1 Method description

Let p(r,α) be the PDF of the landmark position in polar coordinates when only one

observation has been made We characterize p(r,α) when r and α are independent Let

β denote the measured landmark bearing Assume that the error range for the bearing is

ε

± The landmark position is contained in the vision cone which is formed by two rays

rooted at the observation point with respect to two bearings β − andε β+ (see Figure 11) ε

Figure 11 The vision cone is rooted at the observation point The surface of the hashed area

is approximately r dr dα for small d and drα

The surface of the hashed area in Figure 11 for small dr and d can be computed as α

[π r+dr 2−πr2]dπα= [ r dr+ dr 2]dα≅r dr dα

)(22

12)

(Because the probability of the landmark being in the vision cone is 1, we have

³ ³ββ−+εε max α α=

min

1)

,(

R

In Equation (7), Rmaxand Rmin are the bounds of the vision range interval We define F(R) as

the probability of the landmark being in the area{(r,α)|r∈[Rmin,R],α∈[β−ε,β+ε]},

We define Ψ R( ,Δ) as the probability of the landmark being in the dotted area in Figure 11

SinceΨ(R,Δ)=F(R+Δ)−F(R), we have

³ ³−+Δ +

=Δ

R F

∂

∂

=

∂Δ+

(10) Because of the independence of α and r , p(r,α)can be factored as

)()(),(rα f r gα

Without loss of generality, we impose that ³g(α)dα=1 After factoring, Equation (8)

becomes =³R

R f r r dr R

F

min

)()

( Because of the property of the integration, we have

R R f R R F

)()(

=

∂

∂

(12) From Equations (10) and (12), we deduce that f(R+Δ (R+Δ)= f(R)R Therefore,

0)()(

)

(

=Δ++

f

R

f

R By making Δ goes to zero, we obtain R f'(R)+ R f( )=0

The equality f'(R) f(R)=−1R can be re-written as [log(f(R))]' = −[log(R)]' After

integrating both sides, we obtain

−

=

R

e c R c R R

f

c

log1

log)log(

))(log(

Where c is a constant, let ξ=e c, we obtain

R R

f( )=ξFrom Equations (7) and (11), ξ can be calculated and thus

min max

1

R

=ξ

r R R r f

)(

1)

(

min max−

=Therefore, p(r,α) can be re-written as

)()(

1)

,(

min max

α

r R R r

Bearing-only Simultaneous Localization and Mapping for Vision-Based Mobile Robots 353

σβ

αα

22)(exp)(

1)

,(

2 2

min max

Figure 12 The PDF of the landmark position following Equation (14)

5.2 Utilization of the PDF for bearing-only SLAM

We illustrate the application of the PDF for our bearing-only SLAM system Section 5.2-A

describes how the PDF can be updated with a second observation In Section 5.2-B, we

present experimental results on a real robot

A Updating the PDF with a second observation

When a second observation is made after a linear motion, the landmark position falls in the

uncertainty region which is the intersection of two vision cones rooted at the first and the

second observation points O1 and O2 We denote with p1 and p the PDFs of the landmark 2

positions computed from Equation (14) with respect to O1 and O respectively Let p2

denote the PDF of the landmark position after fusing the sensory readings from O1 andO 2

From the work of Stroupe et al (2000), we havep=p1p2 ³p1p2

We want to approximate p with a Gaussian distribution q To compute the parameters of

q , we generate a set S according to the PDF p by the Rejection Method (Leydold, 1998) We

determine the maximum probability density pmax of p by computing p1p2at the

intersection of two bearings The sampling process selects uniformly a sample point s and a

random number {ν ν∈0,1} If p(s) pmax <ν , s is rejected, otherwise s is accepted and added to S The sampling process is repeated until enough points are accepted Figure 13

shows the generated samples in the uncertainty regions of four landmarks

The mean x and the covariance matrix C of q are obtained by computing the mean and the covariance matrix of S as previously done by Smith & Cheeseman (1986) and Stroupe et al

(2000) In Figure 13, the contour plots present the PDFs of landmark positions

The estimated PDFs in Figure 13 are expressed in the robot-based frame F Since the robot R

is moving over time, its frame changes too Therefore, it is necessary to change the coordinate systems to express all the estimations in the global frame F We use the method L introduced in Section 4 to transfer the samples in S from F to R F After the change of L

frames, the uncertainties of L1 and L are transferred to other objects The samples of other 2

objects are taken to approximate the PDFs of the object positions in F Figure 14 shows the L

distribution of the samples after the change of frames The contour plots present the PDFs of the object positions in the global frame F associated to the points L (L1,L2)

Figure 13 The PDFs and the contour plots of four landmarks in the robot-based frame F ; in R

this example, the uncertainty region of each landmark is a bounded polygon The generated samples are distributed in the uncertainty regions

Bearing-only Simultaneous Localization and Mapping for Vision-Based Mobile Robots 355

Figure 14 After the change of frames from F Rto F L, the PDFs and the contour plots of O ,1 2

O and L ,3 L are presented in the global frame 4 F L

B Experimental Results

Our method was evaluated using a Khepera robot equipped with a colour camera (176 x 255 resolution) The Khepera robot has a 6 centimetres diameter A KheperaSot robot soccer playing field, 105 x 68 square centimetres, was used for the experiment arena (see Figure 15) There were four artificial landmarks in the playing field The first and second landmarks were placed at the posts of a goal, 30 centimetres apart from each other

The objective of the experiment is to evaluate the accuracy of the method by estimating the positions of the third and the fourth landmarks At each iteration, the robot was randomly placed in the field The robot took a panoramic image and then moved in a straight line and captured a second panoramic image Landmark bearings were extracted from the panoramic images using colour thresholding

A total of 40 random movements were performed in this experiment Rmin was set to 3 centimetres, Rmax was set to 120 centimetres, and the vision error ε was ± 3 degrees Figure 16(b) shows the errors of the estimated landmark positions For L , the estimated error was 3

reduced from approximately 9 centimetres at the first iteration to less than 1 centimetre at the last iteration For L , the error was reduced from 14 centimetres to 2.77 centimetres The 4

experiment shows that the estimated error of landmark position is sensitive to the relative distance with respect to L and L

Figure 15 Experimental setup, the landmarks are the vertical tubes

Figure 16 (a) Diagram of the experimental setup (b) The errors of the estimated landmark positions

We made another experiment to test the sensitivity of the errors of the landmark positions with respect to the different directions of the robot’s moving trajectories We let the robot move in four different directions with respect to three landmarks In Figure 17(a), stars denote the landmark positions and arrows denote the moving trajectories The robot repeated 10 iterations for each trajectory

The errors on L in four trajectories after the tenth iteration were 2.12, 1.17, 1.51, and 13.99 3

centimetres respectively The error of the fourth trajectory is large because the robot moves along a line that is close to L Therefore, the vision cones at the first and the second 3

observations are nearly identical

Bearing-only Simultaneous Localization and Mapping for Vision-Based Mobile Robots 357The estimation of the landmark position is more accurate when the intersection of two vision cones is small This is the case of the second trajectory where the intersection is the smallest among all trajectories

Figure 17 (a) Trajectories of the robot for the experiment to study the relationship between moving directions and estimated errors of landmark positions (b) The errors on L at each 3

iteration

Although the intersecting area ofL for the first and the third trajectories are the same, the 3

intersecting areas of L and 1 L2for the first trajectory are much bigger than the areas from the third trajectory This is the reason why the estimated error from the third trajectory is smaller than the one for the first trajectory

6 Conclusion

In this chapter, we proposed a vision-based approach to bearing-only SLAM in a dimensional space We assumed the environment contained several visually distinguishable landmarks This approach is inspired from techniques used in stereo vision and Structure From Motion Our landmark initialization method relies solely on the bearing measurements from a single camera This method does not require information from an odometer or a range sensor All the object positions can be estimated in a landmark-based frame The trade-off is that this method requires the robot to be able to move in a straight line for a short while to initialize the landmarks The proposed method is particularly accurate and useful when the robot can guide itself in a straight line by visually locking on static objects

2-Since the method does not rely on odometry and range information, the induced map is up

to a scale factor only In our method, the distance ||L1−L2|| of two landmarks is taken as the measurement unit of the map The selection of L and L is critical for the accuracy of

the map In Section 3.1, the mathematical derivation shows that the estimated error of a landmark position is proportional to the range of the landmark and the inverse of the relative change in landmark bearings Choosing L and 1 L2 with larger change in bearings produces a more accurate mapping of the environment

In the sensitivity analysis, we showed how the uncertainties of the objects’ positions are affected by a change of frames We determine how an observer attached to a landmark-based frame F can deduce the uncertainties in L F from the uncertainties transmitted by an L

observer attached to the robot-based frame F Each estimate of landmark uncertainties R

requires a pair of the observations in a straight movement The simulation in Section 4.3 shows how the uncertainties of landmark positions are refined when the robot moves in a polygonal line

With dead reckoning, the error of the estimated robot’s location increases with time because

of cumulated odometric errors In our method, we set O and 1 O (pair of observation 2

points in a straight movement) at [0, ]' and [1, ]' in F There is no dead reckoning error R

on O1 and O by construction In practice, the robot’s movement may not be perfectly 2

straight However, the non-straight nature of the trajectory can be compensated by increasing the size of the confidence interval of the bearing

The induced map created by our method can be refined with EKF or PF algorithms With EKF, the uncertainty region computed from the geometric method can be translated into a Gaussian PDF With PF, the weights of the samples can be computed with the formulas derived in Section 5.1 Since the samples are drawn from the uncertainty region, the number of samples is minimized

The accuracy of our method was evaluated with simulations and experiments on a real robot Experimental results demonstrate the usefulness of this approach for a bearing-only SLAM system We are currently working on the unknown data association when all landmarks are visually identical In future work, we will deal with the problems of object occlusion and non-planar environments That is, the system will be extended from a 2-dimensional to a 3-dimensional space

7 References

Bailey, T (2003) Constrained Initialisation for Bearing-Only SLAM, Proceedings of IEEE

International Conference on Robotics and Automation, pp 1966 - 1971, 1050-4729

Bailey, T & Durrant-Whyte, H (2006) Simultaneous Localization and Mapping (SLAM):

Part II, IEEE Robotics and Automation Magazine, page numbers (108-117), 1070-9932

Costa, A.; Kantor, G & Choset, H (2004) Bearing-only Landmark Initialization with

Unknown Data Association, Proceedings of IEEE International Conference on Robotics and Automation, pp 1764 - 1770

Davison, A (2003) Real-time simultaneous localization and mapping with a single camera,

Proceedings of International Conference on Computer Vision, pp 1403-1410, Nice,

October

Davison, A.; Cid, Y & Kita, N (2004) Real-time 3D SLAM with wide-angle vision,

Proceedings of IAV2004 - 5th IFAC/EURON Symposium on Intelligent Autonomous Vehicles, Lisboa, Portugal, July

Bearing-only Simultaneous Localization and Mapping for Vision-Based Mobile Robots 359Fox, D.; Burgard, F.; Dellaert, W & Thrun, S (1999) Monte Carlo localization: Efficient

position estimation for mobile robot, Proceedings of National Conference on Artificial Intelligence, pp 343-349

Goncavles, L.; Bernardo, E d.; Benson, D.; Svedman, M.; Karlsson, N.; Ostrovski, J &

Pirjanian, P (2005) A visual front-end for simultaneous localization and mapping,

Proceedings of IEEE International Conference on Robotics and Automation, pp 44-49

Huang, H.; Maire, F & Keeratipranon, N (2005a) A Direct Localization Method Using only

the Bearings Extracted from Two Panoramic Views Along a Linear Trajectory,

Proceedings of Autonomous Minirobots for Research and Edutainment (AMiRE), pp

201-206, Fukui, Japan

Huang, H.; Maire, F & Keeratipranon, N (2005b) Uncertainty Analysis of a Landmark

Initialization Method for Simultaneous Localization and Mapping, Proceedings of Australian Conference on Robotics and Automation, Sydney, Australia

Jensfelt, P.; Kragic, D.; Folkesson, J & Bjorkman, M (2006) A Framework for Vision Based

Bearing Only 3D SLAM, Proceedings of IEEE International Conference on Robotics and Automation, pp 1944- 1950, 0-7803-9505-0, Orlando, FL

Kwok, N M & Dissanayake, G (2004) An Efficient Multiple Hypothesis Filter for

Bearing-Only SLAM, Proceedings of IEEE International Conference on Intelligent Robots and Systems, pp 736- 741, 0-7803-8463-6

Lemaire, T.; Lacroix, S & Sola, J (2005) A practical 3D Bearing-Only SLAM algorithm,

Proceedings of IEEE International Conference on Intelligent Robots and Systems, pp

2757-2762

Leonard, J J & Durrant-Whyte, H F (1991) Simultaneous localization for an autonomous

mobile robot, Proceedings of IEEE Intelligent Robots and System, pp 1442-1447, Osaka,

Japan

Levitt, T S & Lawton, D M (1990) Qualitative navigation for mobile robots, Artificial

Intelligence, Vol 44, No 3, page numbers (305-360)

Leydold, J (1998) A Rejection Technique for Sampling from log-Concave Multivariate

Distributions, Modelling and Computer Simulation, Vol 8, No 3, page numbers

(254-280)

Menegatti, E.; Zoccarato, M.; Pagello, E & Ishiguro, H (2004) Image-based Monte Carlo

localisation with omnidirectional images, Robotics and Autonomous Systems, Vol 48,

No 1, page numbers (17-30)

Montemerlo, M.; Thrun, S.; Koller, D & Wegbreit, B (2003) FastSLAM 2.0: An improved

particle filtering algorithm for simultaneous localization and mapping that

provably converges, Proceedings of International Joint Conferences on Artificial Intelligence, pp 1151-1156, Morgan Kaufmann, IJCAI

Mouragnon, E.; Lhuillier, M.; Dhome, M.; Dekeyser, F & Sayd, P (2006) 3D Reconstruction

of complex structures with bundle adjustment: an incremental approach,

Proceedings of IEEE International Conference on Robotics and Automation, pp 3055 -

3061, Orlando, USA

Murray, D & Jennings, C (1998) Stereo vision based mapping and navigation for mobile

robots., Proceedings of IEEE International Conference on Robotics and Automation, pp

1694 - 1699, New Mexico

Nister, D.; Naroditsky, O & Bergen, J (2004) Visual Odometry, Proceedings of IEEE Computer

Society Conference, pp I-652 - I-659, Princeton

Rizzi, A & Cassinis, R (2001) A robot self-localization system based on omnidirectional

color images, Robotics and Autonomous Systems, Vol 34, No 1, page numbers (23-38)

Sabe, K.; Fukuchi, M.; Gutmann, J.-S.; Ohashi, T.; Kawamoto, K & Yoshigahara, T (2004)

Obstacle Avoidance and Path Planning for Humanoid Robots using Stereo Vision,

Proceedings of International Conference on Robotics and Automation, pp 592 - 597, New

Orleans

Se, S.; Lowe, D & Little, J J (2002) Mobile Robot Localization and Mapping with

Uncertainty Using Scale-Invariant Visual Landmarks, International Journal of Robotics Research, Vol 21, No 8, page numbers (735 - 758)

Siegwart, R & Nourbaksh, I R (2004) Introduction to Autonomous Mobile Robots, The MIT

Press, Cambridge, Massachusetts

Sim, R.; Elinas, P.; Griffin, M & Little, J J (2005) Vision-based slam using the

rao-blackwellised particle filter, Proceedings of IJCAI Workshop on Reasoning with Uncertainty in Robotics, Edinburgh, Scotland

Smith, R & Cheeseman, P (1986) On the Representation and Estimation of Spatial

Uncertainty, The International Journal of Robotics Research, Vol 5, No 4, page

numbers (56-68)

Sola, J.; Monin, A.; Devy, M & Lemaire, T (2005) Undelayed Initialization in Bearing Only

SLAM, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 2751-2756

Spero, D J (2005) Simultaneous Localisation and Map Building: The Kidnapped Way,

Intelligent Robotics Research Centre, Monash University, Melbourne, Australia Stroupe, A W.; Martin, M C & Balch, T (2000) Merging Probabilistic Observations for

Mobile Distributed Sensing, Pittsburgh, Robotics Institute, Carnegie Mellon University

Usher, K.; Ridley, P & Corke, P (2003) Visual Servoing of a Car-like Vehicle - An

Application of Omnidirectional Vision, Proceedings of IEEE International Conference

on Robotics and Automation, pp 4288- 4293, 0-7803-7736-2

Wooden, D (2006) A Guide to Vision-Based Map Building, IEEE Robotics and Automation

Magazine, June 2006, page numbers (94-98)

Zunino, G & Christensen, H I (2001) Simultaneous localization and mapping in domestic

environments, Proceedings of International Conference on Multisensor Fusion and Integration for Intelligent System, pp 67 - 72

Object Recognition for Obstacles-free Trajectories Applied to Navigation Control

W Medina-Meléndez, L Fermín, J Cappelletto, P Estévez,

G Fernández-López and J C Grieco

Universidad Simón Bolívar, Grupo de Mecatrónica, Valle de Sartenejas, Caracas

Venezuela

1 Introduction

In this chapter applications of image and video processing to navigation of mobile robots are presented During the last years some impressive real time applications have been showed to the world, such as the NASA missions to explore the surface of Mars with autonomous vehicles; in those missions, video and image processing played an important role to rule the vehicle

Algorithms based on the processing of video or images provided by CCD sensors or video cameras have been used in the solution of the navigation problem of autonomous vehicles

In one of those approaches, a velocity field is designed in order to guide the orientation and motion of the autonomous vehicle A particular approach to the solution of the navigation problem of an autonomous vehicle is presented here In the first section of this introduction

a state of the art review is presented, after it, the proposed algorithm is summarized; the following sections present the procedure Finally, some experimental results are shown at the end of the chapter

1.1 Review of Navigation of Autonomous Robots using Vision Techniques

In the area of autonomous navigation using vision techniques, the previous works Victor & Sandini,1997), and (Nasisi & Carelli, 2003) are corner stones In the first mentioned study, robot control techniques are explored, using both cameras on board of the robots and external cameras In that work it is shown that it is possible to accomplish effective control actions without doing a complete processing of the image captured or without the calibration of the camera In the job of Nasisi & Carelli, a set of equations needed to establish relationships among a bidimensional image captured by a video camera and its corresponding tri-dimensional image is obtained, an equation set that is important when a single camera is being used The jobs of S Skaar et al., (who participated in the 2003 Mars Exploration Rover experiment of NASA), over the concept of Camera Space Manipulation (CSM) (Skaar et al., 1992) and the concept of Mobile Camera Space Manipulation (MCSM) (Seelinger et al., 2002), must be cited The MCSM method consists of the estimation of the relationship among the characteristics position of the manipulator and its corresponding points in the image spaces of the two cameras mounted over the robot; the CSM concept is quite similar but with more restrictions Both methods, the CSM and the MCSM require not

(Santos-only the parameters of the cameras to be completely known, but also the kinematics model

of the manipulator, even if they don’t require the complete calibration of the camera and the manipulator These methods require a set of cameras, while the methodology proposed in (Santos-Victor & Sandini,1997) and (Nasisi & Carelli, 2003) involves only one

One vital characteristic of every navigation strategy is the way that the decisions are taken

on it when the sensory system indicates the presence of obstacles on the robot trajectory Different obstacle avoidance strategies have been presented; among those strategies the use

of electrostatic fields - where the robot is modeled as a positive electric charge, as also the obstacles, and the objective of the trajectory is modeled as a negative charge - must be mentioned (Dudek & Jenkin, 2000) and (Khatib, 1985); the Coulomb law is applied to determine the trajectory of the mobile robot

In 1995, Li and Horowitz presented the concept of velocity fields (Li & Horowitz, 1995); in such a case, a vector velocity is defined over the trajectory of the robot for each possible position coding a specific task Using control schemes with the velocity field as the reference for the system, has allowed approaches to the problem employing different control schemes instead of the classic following trajectory problem In this approach, the improvement of coordination and synchronization among the degrees of freedom of the robot are much more important than the time to execute a given task In the Figure 1 an example of this strategy is presented

Figure 1 Velocity Field for a manipulator in the plane

With the introduction of the concept of velocity fields, also emerged the strategy of Passive Velocity Field Control (PVFC) In this control strategy the tasks are coded using the velocity and energy that the system could transfer to the environment and that it is limited by a constant (Li & Horowitz, 1999) Li and Horowitz presented the properties of the PVFC strategy doing special emphasis on the contour following problem in (Li & Horowitz, 2001a) and (Li & Horowitz, 2001b)

In many research works related with velocity fields, the stability of the control schemes is pursued; Cervantes et al (Cervantes et al., 2002) proposed a Proportional – Integral Controller based on compensation errors techniques where it is not necessary to have a deep knowledge of the robot dynamics, obtaining semi global asymptotically stable conditions on the following errors of the velocity fields Moreno and Kelly, in (Moreno & Kelly, 2003a),

Object Recognition for Obstacles-free Trajectories Applied to Navigation Control 363(Moreno & Kelly, 2003b) and (Moreno & Kelly, 2003c), proposed different control schemes using velocity fields focusing mainly on the asymptotically stable conditions

Other investigations related to velocity fields have focused in other areas, such as adaptive control (Li, 1999) In that work, Li proposed an adaptive control scheme using velocity fields

as references for robots with unknown inertia parameters Also in (Dixon et al., 2005) an adaptive derivative control is presented in order to follow a velocity field In almost all the previous cases, the works had as an objective the design of control schemes taking a velocity field as a time-invariant reference, obtained after a theoretical or analytical procedure Only few works in the literature have considered the case when the field is time-dependent

Figure 2 Robot system employed in (Kelly et al., 2004a)

Figure 3 Robot system employed in (Kelly et al., 2004b)

Yamakita et al., in (Yamakita & Suh, 2000) and (Yamakita & Suh, 2001), applied PVFC to cooperative robots, obtaining field velocity generating algorithms for tasks where it is necessary to follow certain orders of a master robot For this work the authors created an

augmented dynamic system using an inertia flywheel, in order to avoid the problems of control following by velocity fields Recently, Kelly and collaborators in (Kelly et al., 2004a) and (Kelly et al., 2004b), used a camera as a main sensor in a control scheme by velocity fields in order to control a mobile robot and a robot manipulator Particularly in (Kelly et al., 2004a) is presented a controller by velocity fields for a manipulator of two degrees of freedom that incorporates an external camera to capture the manipulator movements and the work environment On the captured image a theoretical velocity field that the manipulator is capable to follow is pre-defined In the Figure 2 an example for this case is presented.

In (Kelly et al., 2004b), the control is applied to a wheeled mobile robot where the video camera is located over the robot Like in the previous case, the camera is located in such a way that covers the environment and the robot itself In the Figure 3 can be visualized the system under study

1.2 The proposed approach

During last years, as it was mentioned, the use of artificial vision in robot tasks has been increasing In many applications, such as rescue jobs, the use of vision to generate velocity fields is under study The dynamic modification and generation of the velocity field itself, changing it with the environment modifications, is a new research area If timing is not an issue in the control objectives, then velocity field control, where the speed can be adjusted as desired, is a potentially important technique For instance, if the objective is to control a vehicle’s trajectory at 80 Km/h, the velocity field can be adjusted to that speed and modified

in order to avoid obstacles detected by a camera (either on board or external), while keeping close to the original trajectory in order to reach the final objective This approach could be of crucial importance in rescue tasks where a flying machine could be “understanding” the environment, and changing the velocity field references for another robot on the ground Another potential application is in automatic warehouses where changing the velocity field references could be assigned different and changing tasks to the manipulator In the future, the dynamic velocity field generator that is presented in this work will be modified to in order to allow the generation of a 3-dimensional velocity field Also, the algorithm is going

to be used to coordinate the tasks of cooperative robots for Futbot, submarine vehicles coordination, cooperative multi-robot observation, etc

In order to perform the tests, an experimental setup has been integrated This experimental setup consists of:

1 A Hemisson differential mobile robot, created by the Swiss company “K-Team Corporation”

2 2.4 GHz wireless video camera, model XC10A and its wireless receptor VR31A This is the only sensor employed in the experiments in order to detect the robot and the obstacles The camera has a resolution of 640 x 480 pixels and offers up to 30 frames per second (fps)

3 Image acquisition card model NI-PXI-1407 using the Standard RS-170 for monochromatic video signals All the characteristics can be seen in Table 1

4 A PXI module from National Instrument is employed to process the data The module

is the NI-PXI-1000B

Object Recognition for Obstacles-free Trajectories Applied to Navigation Control 365

Bus PXI/CompactPCI

640 × 480 RS-170 Spatial Resolution

768 × 576 CCIR

Video input Standard RS-170, CCIR

Table 1 Characteristics of the NI-PXI-1407

On the NI-PXI-1000B run all the LabView DLL’s and Matlab applications

The working environment consists of an area of approximately 6 square meters where the robot navigates The wireless video camera was located 2.5 meters over this area In Figure 4

is shown a sketch of the experimental arrangement

Figure 4 Environment employed for the experiments

The problem for the generation of a dynamic velocity field was divided in three different

steps:

1 A generator of the initial velocity field

2 The processing by the artificial vision system

3 The generator of the evading velocity field

1.3 Description of the generated system

Initially the user must define the task without taking into account the obstacles that might appear in the trajectory of the mobile robot In order to indicate the trajectory a generator was developed for the initial velocity field This generator is presented later in this chapter

In order to avoid the obstacles presented during the trajectory of the mobile robot, it is necessary to identify their existence, determining its location, size and orientation In order

to solve this problem the artificial vision system was The information supplied by the artificial vision system is employed to modify the initial velocity field using the generator of the evading field Once the desired task is well known and also the position and orientation

of the obstacles the system creates the local velocity fields that surrounds each obstacle; the

final navigation field is obtained adding the initial field with the evading fields pondering

each one properly

In first place and before testing the system in the experimental setup, the results were

validated using the Easy Java Simulations platform The results are presented also in this

chapter

2 Vision System Implementation

This section describes two implementations of the vision system for the robot and obstacle

identification process In both cases the robot detection was made by using a pattern

matching technique, and for the obstacle detection were employed two different

approaches

The first implementation is based on the hypothesis that any object present in the scene can

be described as any of the following regular shapes: circle, square and rectangle In this case

it was utilized a classification strategy for its identification

For the other system, the obstacle detection was made through a particle analysis, in order

to cope with those problems arisen by using the previous approach in images containing

obstacle with irregular shapes

2.1 Robot identification by pattern matching

The pattern matching process consists of the fast detection into the image of those regions

that have a high concordance with a previously known reference pattern This reference

pattern contains information related with edge and region pixels, thus allowing the deletion

of redundant information contained into a regular shape

y

t u v f x y t x u y v

Where f is the input image of size LxM, and the sum is done over (x, y) in the window

containing the sub-image t localized at (u, v), of size JxK By expanding d2 we obtain:

2 2

2

L

x M

2 ,

L x M

y f x y is almost constant, then the resulting term for the cross correlation is the measure of similitude or concordance

between the image and the pattern For this case u = 0, 1,…, M-1, and v = 0, 1,…, L-1, and

u

Object Recognition for Obstacles-free Trajectories Applied to Navigation Control 367

Figure 5 shows the correlation process assuming that the origin for f is placed in the upper

left corner Therefore, the correlation process consists on moving the template t through the

area of the input image and to calculate the value for C By this method, the maximum value

for C points the position where is the highest similitude between t and f.

Figure 5 Window movement during the correlation process

Due to the dependence of the correlation values to intensity variation in the image, is

required to compute the normalized version for the correlation given by (4):

0 1 0

1 0 1 0

2 2

,

1 0 1

,,

,,

L x M y v

L x M

t v y u x t f

y x f

t v y u x t f y x f v

u

Where t is the average intensity value for the pixels in the image and f ,v is the average of

f(x,y) in the template The obtained value for R is normalized between -1 and 1, and is

independent to intensity of f and t images

2.1.2 Pattern recognition process

The detection pattern process is divided in to main subsystems: learning and recognition

itself

The learning phase analyzes the template image in order to extract features that can

improve recognition process compared to standard approach

The pattern learning algorithm can be described as follows (National, 2004):

• Pseudo-random Image sub-sampling: By this method, it can be obtained more uniformity on sampling distribution through the image, without using a predefined

grid With a uniform grid information like the presence of horizontal and vertical

borders could be lost In the other hand, completely random sampling can produce large areas with poor sampling or over-sampled areas

• Stability analysis: The pixels sampled by pseudo-random algorithm are analyzed in order to verify stability (uniformity) on their vicinities Based on this information, each pixel is classified according to its uniform vicinity size (i.e 3x3, 5x5) By doing so, it is reduced the number of comparisons required for further matching process

• Features identification: it is an edge detection process, which output is used to detailed adjustment for the placement of the recognized pattern

• Rotation invariance analysis: is based on the identification of a circular intensity profile

on the pattern image, that will be used later to localize rotated versions of the same pattern because those versions will have the same profile with an displacement factor proportional to its original rotation

The learning phase is computationally complex It can take several seconds to be completed However, this process is done only once and its result can be stored for further applications The pattern recognition algorithm consists on two processes:

• The utilization of a circular intensity profile obtained in the learning phase, in order to localize rotated and displaced versions of that profile through the image

• The utilization of those pixels obtained in the pseudo-random sub-sampling, that are employed in a correlation process between the candidates previously identified, giving

it a score that is used to determine if that candidate can be classified as a pattern match

2.1.3 Pattern recognition subsystem – Implementation

As it was stated in the previous section, the pattern recognition subsystem is separated in to stages: learning and matching; each of them was implemented as an VI in LabVIEW v7.1® The corresponding VI for the matching stage also contains the implementation for the obstacle detection However, in this section it will be described only the pattern recognition system

• Initialization: A blank image with 8 bits per pixel is created

• Image file loading: Over the previously created image it is loaded the file image containing the desired pattern

• Image conversion: The image is converted into a grayscale one, to be processed by the learning module

• Learning module configuration: it is configured so it generates information for the pattern recognition rotation invariant

• Storing: The information related to the pattern learning in a new PNG image file

Object Recognition for Obstacles-free Trajectories Applied to Navigation Control 369

Figure 6 Block diagram of Pattern Learning Process

Figure 7 shows the VI created for this learning subsystem

Figure 7 VI for the learning stage

This VI has neither explicit input nor output parameters, because it relays upon the loading

of an image file for the learning process After de VI construction, it is created a Dynamic Link Library (DLL) using the Math Interface Toolkit from LabVIEW® This provides a DLL file with the subroutines so they can be called from Matlab®

Pattern recognition

The recognition process consists in taking a snapshot from the workspace, loading the file containing the learning pattern previously obtained, and searching such pattern on the captured image Later, it is determined the position and orientation of the pattern The recognition process is described in Figure

Figure 8 Block Diagram of the pattern matching process

• Initialization: two blank images with 8 bpp are generated One image corresponds to the video capture and the other one is for the patter image loading It is also started the video acquisition through the NI IMAQ 1407 module

• Image Capture: It is acquired a real image from the workspace, the resulting image in grayscale

• Image resizing: In order to improve position calculation and detection speed, the captured image is resized from 640 x 480 to 480 x 480 by cropping it This image size corresponds to a physical working area of 4 m2.

• Information loading: It is loaded the information related to the learning pattern contained in the PNG image file stored in the previous system

• Matching module configuration so it performs the invariant rotation pattern search

• Matching process: This process will be done with the acquired image and the image loaded with the learning information In case of a successful detection of the desired pattern, it will be obtained the position of the coincidence in the working space and its rotation, as shown in Figure 4.7

Figure 9 Representation of position and rotation convention

As it was done for the VI of the learning stage, it was created a DLL file from the pattern recognition VI, which we will identify as “detection engine” The matching process has no input parameters; it only loads the image containing the information in the learning process The output parameters are the position and orientation for the detected pattern

2.2 Obstacle Detection

2.2.1 Classification technique

The obstacle detection is based on the classification of binary particles, in which an unknown particle is identified by comparison of their most relevant features against a set of characteristics that conceptually describes previously known classes samples

The obstacle detection based on classification techniques has two main phases: the learning process, which was implemented utilizing the NI Vision Classification Training tool, and the classification process itself

Learning Process

The training process consists in the recollection of a set of samples images that contains possible obstacles that can be found and detected by the camera From these samples is obtained a set of features, known as characteristics vector that describe unequivocally each class of known sample For example, it could be the object circularity or elongation Once determined the characteristics vector and collected the samples, each object class is labeled