Advances in Theory and Applications of Stereo Vision Part 1 docx

Sergio Taraglio and Stefano ChiesaImpact of Wavelets and Multiwavelets Bases on Stereo Correspondence Estimation Problem 17 Asim Bhatti and Saeid Nahavandi Markov Random Fields in the C

ADVANCES IN THEORY AND APPLICATIONS

OF STEREO VISION

Advances in Theory and Applications of Stereo Vision

Edited by Asim Bhatti

Published by InTech

Janeza Trdine 9, 51000 Rijeka, Croatia

Copyright © 2011 InTech

All chapters are Open Access articles distributed under the Creative Commons

Non Commercial Share Alike Attribution 3.0 license, which permits to copy,

distribute, transmit, and adapt the work in any medium, so long as the original

work is properly cited After this work has been published by InTech, authors

have the right to republish it, in whole or part, in any publication of which they

are the author, and to make other personal use of the work Any republication,

referencing or personal use of the work must explicitly identify the original source.Statements and opinions expressed in the chapters are these of the individual contributors and not necessarily those of the editors or publisher No responsibility is accepted for the accuracy of information contained in the published articles The publisher

assumes no responsibility for any damage or injury to persons or property arising out

of the use of any materials, instructions, methods or ideas contained in the book

Publishing Process Manager Iva Lipovic

Technical Editor Teodora Smiljanic

Cover Designer Martina Sirotic

Image Copyright Alex Staroseltsev, 2010 Used under license from Shutterstock.com

First published January, 2011

Printed in India

A free online edition of this book is available at www.intechopen.com

Additional hard copies can be obtained from orders@intechweb.org

Advances in Theory and Applications of Stereo Vision, Edited by Asim Bhatti

p cm

ISBN 978-953-307-516-7

free online editions of InTech

Books and Journals can be found at

www.intechopen.com

Sergio Taraglio and Stefano Chiesa

Impact of Wavelets and Multiwavelets Bases

on Stereo Correspondence Estimation Problem 17

Asim Bhatti and Saeid Nahavandi

Markov Random Fields in the Context of Stereo Vision 35

Lorenzo J Tardón, Isabel Barbancho and Carlos Alberola

Type-2 Fuzzy Sets based Ego-Motion Compensation

of a Humanoid Robot for Object Recognition 71

Tae-Koo Kang and Gwi-Tae Park

Combining Stereovision Matching Constraints for Solving the Correspondence Problem 89

Gonzalo Pajares, P Javier Herrera and Jesús M de la Cruz

A High-Precision Calibration Method for Stereo Vision System 113

Chuan Zhou, Yingkui Du and Yandong Tang

Stereo Correspondence with Local Descriptors for Object Recognition 129

Gee-Sern Jison Hsu

Three Dimensional Measurement Using Fisheye Stereo Vision 151

Stereo Measurement of Objects

in Liquid and Estimation of Refractive Index

of Liquid by Using Images of Water Surface 189

Atsushi Yamashita, Akira Fujii and Toru Kaneko

Detecting Human Activity

by Location System and Stereo Vision 203

Yoshifumi Nishida, Koji Kitamura

Global 3D Terrain Maps for Agricultural Applications 227

Francisco Rovira-Más

Construction Tele-Robotic System with Virtual Reality (CG Presentation of Virtual Robot and Task Object Using Stereo Vision System) 243

Hironao Yamada, Takuya Kawamura and Takayoshi Muto

Navigation in a Box Stereovision for Industry Automation 255

Giacomo Spampinato, Jưrgen Lidholm, Fredrik Ekstrand, Carl Ahlberg, Lars Asplund and Mikael Ekstrưm

New Robust Obstacle Detection System using Color Stereo Vision 279

Iyadh Cabani, Gwenặlle Toulminet and Abdelaziz Bensrhair

A Bio-Inspired Stereo Vision System for Guidance of Autonomous Aircraft 305

Computer vision is one of the most studied and researched subjects of recent times and has gained paramount att ention over the last two decades with exponentially grown focus on stereo vision Lot of activities in the context of stereo vision are gett ing reported and published on the vast research spectrum, including novel mathematical ideas, new theoretical aspects, state of the art techniques and diverse range of applications These reported ideas and published texts serve as fi ne introductions and references

to individual mathematical ideas, however, they do not educate research trends of the overall fi eld This book addresses the aforementioned concerns in a unifi ed manner

by presenting diverse range of current research ideas and applications, providing an insight into the current research trends and advances in the fi eld of stereo vision The book presents wide range of innovative research ideas and current trends in stereo vision The topics covered in this book encapsulate research trends from fundamental theoretical aspects of robust stereo correspondence estimation to the establishment of novel and robust algorithms, as well as the applications in wide range of disciplines The book consists of 17 chapters addressing diff erent aspects of stereo vision Research work presented in these chapters tries to establish either the correspondence problem from a unique perspective or new constraints to keep the estimation process robust Understanding of the theoretical aspects and the algorithm development in solving for the robust solutions are connected Algorithm development and the relevant applications are also tightly coupled as generally algorithms are customised to achieve optimum performance for specifi c applications Despite of this tight coupling between theory, algorithms and applications, presented ideas in this book could be classifi ed into three distinct streams

First fi ve chapters (1 to 5) discuss correspondence estimation problem from theoretical perspective New ideas employing approaches such as evolutionary, wavelets and multiwavelets theories, Markov random fi elds and type-2 fuzzy sets are introduced For instance, Chapter 2 proposes the use of multiwavelets in addressing the correspondence estimation problem and initiates a new debate by discussing the implicit potential of multiwavelets theory and embedded att ributes of multiwavelets bases in the context

of stereo vision Chapter 3 discusses the consideration of local interactions to defi ne Markov random fi elds to recover 3D structure from stereo images Chapter 4 proposes fuzzy information theoretical approach based on type-2 fuzzy sets for the estimation and extraction of features of interest Chapter 5 proposes novel combination of matching constraints to address the correspondence estimation problem

Similarly, chapters 6 to 10 present innovative algorithms employing novel ideas and technologies inspired by the nature Particularly interesting are biologically inspired technologies and techniques, such as address-event based stereo vision with bio-inspired silicon retina imagers and dimensional measurement using fi sheye stereo vision Chapter 10 presents a novel idea of measurement of objects in liquids by making use of refractive index of liquid These unique ideas and algorithms truly inspire new researchers to look outside the box and redefi ne the current research problems and trends

Chapters 11 to 17 provide a diverse range of applications, including human activity detection, 3D terrain mapping, navigation, obstacle detection and bio-inspired autonomous guidance Although these applications are targeted to the domains of surveillance, agriculture, mobile robotics, manufacturing and unmanned air vehicles, presented techniques can easily be applied to other disciplines A major problem with robust stereo vision algorithms is the computational complexity, which compromises their real time performance This issue is addressed in chapter 17 by introducing FPGA-based architecture to execute stereo vision algorithms at 100 Hz, much faster than real time

In summary, this book comprehensively covers almost all aspects of stereo vision and highlights the current trends Diverse range of topics covered in this book, from fundamental theoretical aspects to novel algorithms and diverse range of applications, makes it equally essential for established researchers as well as experts in the fi eld

At this stage of the book completion, I would like to extend my gratitude and appreciation to all the authors who contributed their invaluable research to this book

to make it a valuable piece of work Finally, from all research community, I would like

to extend my admiration to INTECH Publisher for creating this open access platform to promote research and innovation and making it freely available to the community

Dr Asim Bhatt i

Centre for Intelligent Systems ResearchInstitute of Technology Research Innovation

Deakin UniversityVic 3217, Australia

Evolutionary Approach to Epipolar Geometry

Estimation

Sergio Taraglio and Stefano Chiesa

ENEA, Robotics Lab, Rome

Italy

1 Introduction

An image is a two dimensional projection of a three dimensional scene Hence a degeneration

is introduced since no information is retained on the distance of a given point in the space

In order to extract information on the three dimensional contents of a scene from a single

image it is necessary to exploit some a priori knowledge either on the features of the scene,

i.e presence/absence of architectural lines, objects sizes, or on the general behaviour ofshades, textures, etc Everything becomes much simpler if more than a single image isavailable Whenever more viewpoints and images are available, several geometric relationscan be derived among the three dimensional real points and their projections onto thevarious two dimensional images These relations can be mathematically described under theassumption of pinhole cameras and furnish constraints among the various image points Ifonly two images are considered, this research topic is usually referred to as epipolar geometry.Naturally there is no mathematical difference whether the considered images are taken at thesame time by two different cameras (the stereoscopic vision problem) or at different times

by a single moving camera (optical flow or structure from motion problem) In Robotics boththese cases are of great significance Stereoscopy yields the knowledge of objects and obstaclespositions providing a useful key to obtain the safe navigation of a robot in any environment

(Zanela & Taraglio, 2002) On the other hand the estimation of the ego-motion, i.e the measure

of camera motion, can be exploited to the end of computing robot odometry and thus spatialposition, see e.g (Caballero et al., 2009) In addition the visual sensing of the environment isbecoming ubiquitous out of the ever decreasing costs of both cameras and processors and thecooperative coordination of more cameras can be exploited in many applicative fields such

as surveillance or multimedia applications (Arghaian & Cavallaro, 2009) Epipolar geometry

is then the geometry of two cameras, i.e two images, and it is usually represented by a

3 x 3 fundamental matrix, from which it is possible to retrieve all the relevant geometrical

information, namely the rigid roto-translation between camera positions The estimation ofthe fundamental matrix is based on a set of corresponding features present in both the images

of the same scene Naturally the error in the process is directly linked to the accuracy in thecomputation of these correspondences In the following a novel genetic approach to epipolargeometry estimation is presented This algorithm searches an optimal or sub-optimal solutionfor the rigid roto-translation between two camera positions in a evolutionary framework Thefitness of the tentative solutions is measured against the full set of correspondences through

a function that is able to correctly cope with outliers, i.e the incorrectly matched pointsusually due to errors in feature detection and/or in matching Finally the evolution of the

1

2 Theoretical background

Let us briefly review the relevant geometrical concepts of the pinhole camera model and ofepipolar geometry

2.1 Pinhole camera

A point M= (X, Y, Z, 1)T in homogeneous coordinates in a world frame reference and the

correspondent point m= (x, y, 1)T on the image plane of a camera are related by a projectivetransformation matrix:

with (c x , c y)the optical centre of the camera, f its focal length, α and β take into account

the pixel physical dimensions and γ encodes the angle between x and y axis of the CCD

(skew) and is usually set at 0, i.e perpendicular axes The matrix[R|t]is a matrix relating

the camera coordinate system with the world coordinate one, i.e the camera position t and rotation matrix R:

point M and its projections m and mon the two focal planes of the cameras, the three pointsdefine a planeΠ which intersects the two image planes at the epipolar lines l m and l mwhile

e and eare the epipoles, i.e the image point where the optical centre of the other cameraprojects itself The key point is the so called epipolar constraint which simply states that if the

object point in one of the two images is in m, then its corresponding image point in the other image should lay along the epipolar line l mSuch a constraint can be described in terms of a3x3 fundamental matrix through the:

m TFm=0 (5)

2 Advances in Theory and Applications of Stereo Vision

Evolutionary Approach to Epipolar Geometry Estimation 3

Fig 1 Epipolar geometry

The fundamental matrix F contains the intrinsic parameters of both cameras and the rigid

transform of one camera with respect to the other and thus describes the relation betweencorrespondences in terms of pixel coordinates A similar relation can be found for the so calledessential matrix where the intrinsic parameters of the cameras are not considered and therelation between correspondences is in terms of homogeneous coordinates The algorithmsfor the estimation of epipolar geometry deal with actual pixel positions as produced byactual lenses and cameras Therefore the interest of such algorithms is in the fundamentalmatrix rather than in the essential one The standard approach for the computation of thefundamental matrix is based on the solution of a homogeneous system of equations in terms

of the nine unknowns of the matrix F:

where

f= (f1, f2, f3, f4, f5, f6, f7, f8, f9)T (7)and

If nine or more correspondences are known the system is overdetermined and a solution can

be sought in a least square sense; in a subsequent step, from the found fundamental matrix, thegeometrical information is derived, exploiting the knowledge about the two camera matrices(equation 3) The number of independent unknowns varies among the different approachesemployed for the computation Some approaches don’t take into account the additionalrank-two constraint on the fundamental matrix (8 point algorithms) and some do (7 pointalgorithms) Naturally the former considers the rank constraint in a subsequent phase; finallythe solution is derived with an unknown scale factor Let us now suppose that the rigid motion

of one camera with respect to the other is a-priori known, it is then possible to build directly

the fundamental matrix Let us consider the essential matrix E defined as (Huang & Faugeras,

3 State of the art

As described in Section 2 the starting point for epipolar geometry estimation is represented

by a set of correspondences between two images of the same scene as taken from differentviewpoints The existing techniques to exploit this pairwise information for fundamentalmatrix estimation can be classified in three broad classes: linear, iterative and robust.Longuet-Higgins in 1981 (Longuet-Higgins, 1981) opened the way to the computation ofscene reconstruction from epipolar geometry through a linear approach The basic procedure

is the so called Eight-Point Algorithm, an algorithm of low complexity but prone to greatsensitivity to noise in the data, i.e error in the pixel position of the correspondences, and tothe possible presence of outliers, i.e incorrectly matched points The outliers are usually due

to error in feature detection and in matching and are in large disagreement with the inliers,i.e the correctly matched points Further refinement by Hartley (Hartley, 1995) allowed asensible amelioration of the original algorithm through a simple normalization of image data.The linear approach solves a set of linear equations relating the correspondences throughthe fundamental matrix, i.e solves equation (6) If a large number of correspondences isavailable, the solution is sought in a least square sense or through eigen analysis determiningthe fundamental matrix through eigen values and vectors, see (Torr & Murray, 1997) Theiterative methods basically try to minimize some kind of error signal and can be classified

in two groups: those minimizing a geometrical distance between points, and their iscorresponding epipolar lines and those based on the gradient The most widely usedgeometrical distances are the Euclidean distance and the Sampson one They both measurewith slightly different means the distance between a correspondence and its relative epipolarline in a symmetric way Since two are the correspondences, the distance from the firstone to the epipolar line originating from the other is computed and then the positions arereversed and the distance of the second from the epipolar line originating from the first one iscomputed and added to the former Finally all the contribution are added up and considered

in an average value The minimization can be carried out with different approaches: classical

4 Advances in Theory and Applications of Stereo Vision