Single lens multi ocular stereovision using prism 1 4

Two different approaches are developed to understand and model this system: one based on a camera calibration technique and another based on geometrical analysis of ray sketching.. This

ACKNOWLEDGEMENT

The author would like to express his most sincere appreciation to:

Associate Professor Kah Bin LIM, the supervisor of my Ph.D study, for giving me such an interesting and fruitful project to improve and demonstrate my ability, and for his continuous supervision and valuable foresight and insight on this project

Mr Voon Pong LEE, for his excellent early contribution on single-lens stereovision using mirrors and initiation on single-lens stereovision using biprism (2F-filter); and Mr Raymond Lye Choon NG, for his cooperation on the preliminary discussion on binocular stereovision using biprism

Mr Yee, Mrs Ooi, Ms Tshin, Miss Hamidah and Mr Zhang and all the staff

in Control and Mechatronics Laboratory of the Mechanical Engineering Department, for their kind support

All colleagues and friends in Control and Mechatronics Laboratory, with whom this project has become such a meaningful and memorable experience

TABLE OF CONTENT

2.1 C ONVENTIONAL T WO C AMERA S TEREOVISION T ECHNIQUE 5

4.1.1 FORMATION OF VIRTUAL CAMERAS 21

6.1.2 DETERMINING THE VIRTUAL CAMERA BY GEOMETRICAL ANALYSIS OF RAY

SUMMARY

This thesis investigated a passive single-lens stereovision system using prism (filter) Each image captured by this system is split into multiple different sub-images and these sub-images are taken as images simultaneously captured by one group of virtual cameras which are generated by the prism Hence this system is able

to obtain multiple different views of the same scene using a single camera in one shoot The differences among these views, called disparities, are exploited to perform depth recovery This system can also be called a virtual stereovision system corresponding to a virtual camera concept According to the numbers of virtual cameras generated, binocular stereovision system, trinocular stereovision system and multi-ocular stereovision system are discussed separately

Two different approaches are developed to understand and model this system: one based on a camera calibration technique and another based on geometrical analysis of ray sketching The latter approach requires no complex camera calibration, thus saving a large implementation effort without compromising accuracy

One real system is implemented and experiments are designed and conducted

to test this concept The result shows that both approaches are effective

While this stereovision system has the advantages of low cost, compactness, simultaneous image capturing, no camera synchronization problem, etc, it has the limitation of small baseline due to the dimension of prisms used Hence this system

is more suitable for close-range stereovision

To our knowledge, the approaches developed in this thesis to study and implement the single-lens binocular stereovision system are novel Furthermore, the designs of the single-lens trinocular and multi-ocular stereovision systems and the

approaches used to understanding these two systems that are reported in this thesis are novel

Parts of this thesis have been previously published in papers

LIST OF TABLES

T ABLE 4.1 R ECOVERED DEPTH BY BINOCULAR STEREOVISION , λ = 40 MM 41

T ABLE 5.1 R ECOVERED DEPTH BY TRINOCULAR STEREOVISION , λ = 40 MM 93

T ABLE 6.1 R ECOVERED DEPTH BY MULTI - OUCLAR STEREOVISION , 4 FACE FILTER , λ=45 MM 113

LIST OF FIGURES

F IGURE 2.1 M ODELING OF TWO - CAMERA STEREOVISION SYSTEM 6

F IGURE 2.2 T HE CONCEPT OF EPIPOLAR LINE AND EPIPOLAR PLANE 7

F IGURE 2.3 A SINGLE - LENS STEREOVISION SYSTEM USING A GLASS PLATE 9

F IGURE 2.4 A SINGLE - LENS STEREOVISION SYSTEM USING THREE MIRRORS 10

F IGURE 2.5 A SINGLE - LENS STEREOVISION SYSTEM USING TWO MIRRORS 11

F IGURE 3.1 C AMERA CALIBRATION MODELING 16

F IGURE 4.1 S INGLE - LENS STEREOVISION SYSTEM USING A BIPRISM 22

F IGURE 4.2 G ENERATION OF VIRTUAL CAMERAS USING A BIPRISM ( TOP VIEW ) 23

F IGURE 4.3 R AY MAP OF VIRTUAL - CAMERA CONFIGURATION 31

F IGURE 4.4 S YSTEM SETUP 36

F IGURE 4.5 C ALIBRATION BOARD 36

F IGURE 4.6 C ALIBRATION OF REAL CAMERA 37

F IGURE 4.7 C ALIBRATION OF VIRTUAL CAMERA 38

F IGURE 4.8 D ISPARITY I NFORMATION 39

F IGURE 4.9 F IELD OF VIEW : CONVERGENT SYSTEM (ω ′ 1 < γ) 43

F IGURE 4.10 F IELD OF VIEW : DIVERGENT SYSTEM (ω ′ 1 > 2γ) 43

F IGURE 4.11 F IELD OF VIEW : DIVERGENT SYSTEM (γ < ω ′ 1 < 2γ) 44

F IGURE 4.12 A CASE OF CONVERGENT FIELD OF VIEW 44

F IGURE 5.1 P OSITIONING A 3F FILTER IN FRONT OF A CCD CAMERA 55

F IGURE 5.2 O NE IMAGE CAPTURED BY THE SINGLE - LENS TRINOCULAR SYSTEM 55

F IGURE 5.3 P OSITION RELATIONSHIP BETWEEN REAL CAMERA AND 3F FILTER 64

F IGURE 5.4 S YMBOLIC I LLUSTRATION OF VIRTUAL CAMERA MODELING USING GEOMETRICAL ANALYSIS 65 F IGURE 5.5 W ORKFLOW OF DETERMINING THE VIRTUAL CAMERA VIA GEOMETRICAL ANALYSIS 68

F IGURE 5.6 P LANE PMN 71

F IGURE 5.7 T EMPORARY COORDINATE SYSTEM T AND T′ USED IN FINDING LINE MN 73

F IGURE 5.8 P LANE LNM 76

F IGURE 5.9 T EMPORARY COORDINATE SYSTEM R AND R′ USED IN FINDING LINE NL 77

F IGURE 5.10 P LANE KJS 79

F IGURE 5.11 I LLUSTRATION OF THE SHORTEST SEGMENT CONNECTING TWO NON - INTERSECTING , AND NON - PARALLEL LINES 82

F IGURE 5.12 P LANE F′P′K′ 84

F IGURE 5.13 C ALIBRATION OF VIRTUAL CAMERAS 91

F IGURE 6.1 S YMBOLIC ILLUSTRATIONS OF MULTI - FACE FILTERS WITH 4 AND 5 FACES 97

F IGURE 6.2 O NE IMAGE CAPTURED BY THE SINGLE - LENS MULTI - OCULAR SYSTEM (4 FACES ) 99

F IGURE 6.3 C ALIBRATION OF VIRTUAL CAMERAS (4 FACES FILTER USED ) 112

F IGURE A 1 E PIPOLAR CONSTRAINT 130

F IGURE A 2 E PIPOLAR CONSTRAINT ( USING DIFFERENT CAMERA MODE ) 131

F IGURE A 3 I LLUSTRATIONS OF E PIPOLAR C ONSTRAINTS IN T RINOCULAR S TEREOVISION 136

F IGURE A 4 A SIMPLE PIN - HOLE CAMERA MODEL ( SIDE VIEW ) 138

F IGURE A 5 A SIMPLE PIN - HOLE CAMERA MODEL WITH TWO CROSSING OBJECT LINES 139

F IGURE A 6 S YMBOLIC ILLUSTRATION OF 3F FILTER STRUCTURE 142

F IGURE A 7 3F FILTER 3D STRUCTURE , WITH FRONT AND SIDE VIEW 143

LIST OF SYMBOLS

λ = B ASELINE , I E THE DISTANCE BETWEEN THE TWO CAMERA OPTICAL CENTRES

γ = T HE ANGLE BETWEEN TWO CAMERA OPTICAL AXES

f = E FFECTIVE REAL CAMERA FOCAL LENGTH

f′ = E FFECTIVE REAL VIRTUAL CAMERA FOCAL LENGTH

N R = R EFLECTIVE INDEX OF PRISM

N CX = N UMBER OF COLUMNS OF SENSOR ELEMENTS IN X - DIRECTION IN THE CCD

N CY = N UMBER OF COLUMNS OF SENSOR ELEMENTS IN Y - DIRECTION IN THE CCD

N FX = N UMBER OF PIXELS IN A LINE AS SAMPLED BY THE COMPUTER IN X - DIRECTION

N FY = N UMBER OF PIXELS IN A LINE AS SAMPLED BY THE COMPUTER IN Y - DIRECTION

d X = D ISTANCE BETWEEN ADJACENT CCD ELEMENTS IN X - DIRECTION

d Y = D ISTANCE BETWEEN ADJACENT CCD ELEMENTS IN Y - DIRECTION

ρ = Y AW ANGLE ( ROTATION ABOUT Y AXIS )

ν = P ITCH ANGLE ( ROTATION ABOUT X AXIS )

ζ = T ILT ANGLE ( ROTATION ABOUT Z AXIS )

G = T RANSLATION MATRIX

R = R OTATION MATRIX

P = T RANSFORMATION MATRIX

(x w ,y w ,z w ) = W ORLD COORDINATE S YSTEM

(x cam ,y cam ,z cam) = C AMERA COORDINATE S YSTEM

k 1 , k 2 = C AMERA LENS DISTORTION COEFFICIENTS

Marr [1] depicts 3D vision as follows: ‘Form an image (or a series of images)

of a scene, derive an accurate three-dimensional geometric description of the scene and quantitatively determine the properties of the object in the scene’ This means that 3D computer vision consists of three stages: Data Capturing, Reconstruction and Interpretation

Stereovision system usually employs two or multiple cameras to capture different views of a scene When a point in the scene is projected into different locations in each image plane, the difference in position of its projections is called the

disparity The depth information of the point can be detected by using the properties

of individual cameras, geometric relationships between the cameras and the disparity,

yielding Reconstruction

To effectively determine camera properties including its intrinsic and extrinsic parameters, various camera calibration techniques have been developed Camera calibration usually requires accurate calibration patterns and dedicated software In addition, the use of two or multiple cameras results in high cost, difficulty in the system setup and the camera synchronization problem To avoid and or alleviate these problems, a group of techniques called single-lens stereovision have been

The correspondence search in computer vision is to determine the pixels corresponding to the same object point in different views acquired by the cameras from different view angles of the same scene Correspondence is problematic because of occlusion, repeated patterns, image noise, poor illumination and image quality, high computing load, etc Current techniques often used in correspondence searching include various geometrical constraints, correlation based analysis and feature based analysis Trinocular stereovision enables to check the hypothesized correspondence points using additional epipolar constraints However the extra camera increases difficulty of system implementation, camera calibration and synchronization

This thesis presents our investigation of a single-lens stereovision using prism Though only one single CCD camera is used, this vision system is able to capture multiple views (two, three or more views) of the same scene simultaneously and these views can be taken as the images captured by virtual cameras which are generated by the prism The disparities among these views are exploited to perform depth recovery like usual stereovision systems

This system can be further categorized into three types according to different numbers of virtual cameras generated: single-lens binocular stereovision system, single-lens trinocular stereovision system and single-lens multi-ocular stereovision system which will be discussed separately

Firstly the single-lens binocular stereovision system is presented, followed by the description on trinocular stereovision system which combines the advantages of the single-lens stereovision and the trinocular stereovision Finally, the understanding of this single-lens trinocular stereovision is generalized and a single-lens multi-ocular stereovision system is created

The advantages of this single-lens stereovision system are obvious As compared to common two or multiple camera stereovision systems, it has a more compact setup, lower cost, simpler implementation process, easier camera synchronization since only one camera is used, and also simultaneous image capturing without any complicated hardware, etc Moreover this is a passive vision system and it does not require any active assistance such as structured illumination or any additional visual cues to be provided by the system nor the environment Finally, the trinocular stereovision can facilitate the correspondence searching These advantages motivate the investigation of this system

Two different approaches are developed to model the single-lens stereovision system and in particular, its virtual cameras: one of them is based on a camera calibration technique and the other one is based on geometrical analysis of ray sketching The first approach is of secondary importance as it involves cumbersome calibration implementation and operation which can be avoided by the latter approach The geometrical analysis based approach provides an interesting way of understanding the system as it gives much simpler system implementation with acceptable accuracy in depth recovery

One real system has been implemented and the experiments are carried out to test the single-lens binocular, trinocular and multi-ocular stereovision systems and to verify the validity of this system The results can prove the effectiveness of the both approaches used to model these systems We believe that most of the work presented

in this thesis, including the way of modeling the single-lens binocular system and the design of the single-lens trinocular and multi-ocular systems are novel, and we also believe that this system are practically useful in science and industrial area

The thesis is organized as follows: Chapter 2 gives a literature review on single-lens stereovision technique; Chapter 3 describes the theory of calibration technique which is used by this system; Chapter 4 describes the single-lens binocular stereovision system; Chapter 5 and 6 describe the single-lens trinocular stereovision system and the single-lens multi-ocular stereovision system, respectively; finally Chapter 7 and Chapter 8 give the conclusion and comments on future work More information can be found in the Appendices

CHAPTER 2 LITERATURE REVIEW

This section firstly gives a brief review on the theories of computer stereovision and camera calibration that are the basic concepts used through the thesis, and then presents a detailed review on the single-lens stereovision techniques

2.1 Conventional Two Camera Stereovision Technique

2.1.1 Stereovision Using Two Cameras

A conventional stereovision process used in depth recovery can be summarized into three following major steps [2]: (1) detection of features in each image, (2) matching of features between the images under certain geometric and other stereo correspondence constraints, and (3) calculation of depth using the disparity values and the geometric parameters of the imaging configuration A simple canonical stereo system using two parallel cameras is modeled as shown in Figure 2.1

The geometry of the projections leads to the recovery of the coordinate of the scene point:

r l r

l

r l

r l

r l

x x

f z

x x

y y y x

x

x x x

Figure 2.1 Modeling of two-camera stereovision system

A useful concept often used in stereovision is the epipolar line, which

increases the efficiency of correspondence search between two image planes The

essence of this theory is: given an object point p and its projection in the left image p l,

then the corresponding right image point pr must be located on the corresponding epipolar line The epipolar line is formed by the intersection of the epipolar plane with the right image plane The epipolar plane is defined as the plane that passes

through the points p l , C l and C r , where C l and C r are the optical centers of the left and right camera lens respectively, as shown in Figure 2.2 A more detailed review and a discussion on its mathematics for the consideration of implementation are given in Appendix A

Right Image plane Left Image plane

Right Optic Centre

Left Optic Centre

Figure 2.2 The concept of epipolar line and epipolar plane

2.1.2 A Review on Camera Calibration Technique

The purpose for camera calibration is to find the relationship between the camera image plane coordinates and the world coordinates This relationship is defined by the camera intrinsic parameters, such as camera focal length and lens distortion, and the extrinsic parameters, such as relative position and orientation with respect to the external world The essence of a simple calibration technique can be

described as: a world point w, T

w w

or

44 43 42 41

34 33 32 31

24 23 22 21

14 13 12 11

z y x

a a a a

a a a a

a a a a

a a a a

G, R and P contain terms which include the intrinsic parameters and extrinsic

parameters, which are important in the characterization of a machine vision system

Reference [3] gives details of the calibration procedures The matrix containing a ij , i

= 1 to 4, j = 1 to 4, is known as the calibration matrix

Due to the importance of stereovision, a great amount of research work has been devoted to this topic A concise introduction of stereovision can be found in the book by Trucco and Verri [4] More explanations and discussions can be found in the books by Faugeras [5], Hartley et al [6], and Sonka et al [7]

2.2 The Single-Lens Stereovision Technique

Many different single-lens stereovision techniques have been developed because of the significant potential advantages of this technique over conventional two or multiple camera stereovision systems These techniques can be classified into

two groups The techniques of the first group use some optical devices, such as

mirrors to achieve the stereovision effect; the techniques of the second group exploit

some known cues from the vision system itself such as known camera movement or

from its environment such as known illumination conditions Both categories rely on

triangulation knowledge to explain the generation of their stereovision effect They are separately introduced in the following sections

Passive vs active methods is another possible criterion to classify these techniques; simultaneous stereo images (or their equivalents) capturing or non-simultaneous stereo images capturing can also be a criterion

Here the discussion is not extended to the techniques of stereo from shading/pictorial information and photometric stereo

2.2.1 Single-lens Stereovision Systems Using Optical Devices

Nishimoto and Shirai [8] proposed a single-lens stereovision system which can obtain stereo images as shown in Figure 2.3 In this system a glass plate is positioned in front of the camera so that its rotation will cause the optical axe of the camera slightly shifted because of its refraction Hence stereo image pairs can be captured but with small disparities

Figure 2.3 A single-lens stereovision system using a glass plate

Mirrors are often used to assist in achieving single-lens stereovision effect Teoh and Zhang [9] described a single-lens stereo-camera system which employs three mirrors as shown in Figure 2.4 Two mirrors are positioned to at a 45° relative

to the optical axis of the camera, and a third mirror is positioned in front of the

camera lens and can be rotated to be parallel to either of the fixed mirrors in sequence and two different images are thus obtained from one static scene

Figure 2.4 A single-lens stereovision system using three mirrors

The systems described above require the camera to take two separate shots to obtain one pair of stereo images, their applications are probably limited to static scene or slow changing environment only (even though fast rotation speed of the glass or mirrors reduces the negative effect of this limitation) Gosthasby and Gruver [10] described another mirror-based single-lens stereovision system as shown in Figure 2.5, which can overcome this problem The acquired images are reflected by the mirrors and transformation processes of these images are needed before carrying out the correspondence and depth measurement as in a normal two camera stereovision system

And one system which has movable mirror components to control its view scope is introduced by Inaba et al [11] Nene and Nayar [12] performed further analysis of this kind of mirror based stereovision system In their study a single camera is positioned to point towards not only planar mirrors, but also hyperboloidal, ellipsoidal and paraboloidal mirrors By using non-planar reflecting surfaces, such as hyperboloids and paraboloids, a wide field of view (FOV) can be achieved

Figure 2.5 A single-lens stereovision system using two mirrors

Compared with the design of capturing images in mirrors only, the design of capturing images in mirrors plus the direct object image of the object were also proposed by some researchers Zhang and Tsui [13] proposed such an implementation by positioning the mirror beside the object Francois et al [14] further refined the concepts of stereovision from a single perspective of a mirror symmetric scene and concluded that a mirror symmetric scene is equivalent to observing scene with two cameras and all traditional analysis tools of binocular stereovision can be applied

2.2.2 Single-lens Stereovision System Using Known Cues

A good example of this kind of method is by using known illuminations Segan et al [15] designed a system which used a camera and a point light source to track a user’s hand in 3-D space The light source needed to be calibrated in this system and the projections of the hand and its shadow were used as the cues to find stereo information

Another good method uses known geometry of the object in the scene as the cue to recovery depth information Nakazawa et al [16] devised a method which

points were projected onto one image plane while moving the single camera by hand

It required four coplanar points such as the corners of a sheet of A4-sized paper in the scene as the cue Another attempt using a similar approach was presented by Suzuki

et al [17] The most important cue used in their algorithm was the invariant relative positions of the representative points of a rigid body

Moore and Hayes [18] presented a simple method of tracking the position and orientation of objects from a single camera by exploiting the perspective projection model Three coplanar points on the object need to be identified, which are the cues for the stereos, and their distances from the camera lens are measured

There are many other different techniques using camera movement as the cue

to retrieve the stereo information In the work by LeGrand and Luo [19], an estimation technique which retains the non-linear camera dynamics and provides an accurate 3-D estimation of the positions of selected targets within the environment is presented When this method is applied to robot navigation, the key to this algorithm

is that during pursuit, the robot continuously takes centroid measurements of the target and uses the estimation algorithm to calculate the target’s position This implies that the stereo information is generated from the motion information which is acquired through the movement-sensor attached to the robot

Adelson and Wang [20] described one method which achieves single-lens stereovision effect using a concept called plenoptic camera Their system retains the structure by the light impinging on the camera sensor plane by placing a set of miniature cameras formed by a pinhole array or a lenticular array Cardillo et al [21] described another single-lens stereovision method which was based on the investigation on the blurring effects of camera focusing Another interesting example

is the work by Ye, et al [22] In their work, they only use one sing camera with a telecentric lens to captured stereo views of the IC chip by rotating the chip

It can be seen that a great amount of attention and effort has been drawn to single-lens stereovision technique In Control and Mechatronics Laboratory of the Department of Mechanical Engineering, NUS, continuous effort has been made into single-lens stereovision A mirror based binocular stereovision system was designed successfully by Lee [25] a few years ago A preliminary discussion on a biprism (2F-filter) based binocular single-lens stereovision was done by Lee [25], Xiao [26] and

Ng [27], and recent and more comprehensive work is discussed in depth in this thesis

In this thesis the newest approaches of understanding this kind of binocular stereovision and also designs of trinocular and multi-ocular stereovision system are presented, which are believed to be novel

Lee and Kweon et al [23][24] proposed a single-lens stereo system using one biprism which has a similar setup of the binocular system that is presented in this thesis, but in the aspect of the approaches used to understand such a system, readers will see fundamental differences between the methods reported here and theirs, and some differences are high-lighted in this thesis Lee and Kweon assume an arbitrary point in the view zone of this vision system is transformed into two virtual points in 3-D space, from which this system can be understood, and its equivalent stereo system can be found, but the approaches reported in this thesis directly assume the existence of two virtual cameras and determine them using the law of refraction (for the approach using geometrical analysis of ray sketching) The two different approaches give many different understandings of system properties including the relationship between the image disparity and the depth Moreover, this thesis also

this system Here, alternatives to understand such a system are also presented to some details

CHAPTER 3 CAMERA CALIBRATION

This chapter describes the basic theory of the camera calibration technique used in this single-lens stereovision system Camera calibration is a process to obtain the camera intrinsic and extrinsic parameters The intrinsic parameters are inherent in

a camera system, which normally include the effective focal length, lens distortion coefficients, scaling factors and position and orientation of the coordinates of the camera; the extrinsic parameters include the translation and orientation information

of the camera frame with reference to a specified world coordinate system The extrinsic parameters can be used to determine the relative position between two cameras, which is an essential knowledge of a stereovision system The classical method of calibration solves the perspective transformation matrix which contains the intrinsic and extrinsic information of interest This is accomplished by associating an enough number of known non-coplanar 3D world coordinates with their corresponding 2D image coordinates captured by camera These points are usually obtained from a set of objects with known relative positions and dimensions in camera view zone, and are called calibration patterns A list of the works from the pioneers and recent contributors on calibration techniques can be found in [31] - [37]

In our system, camera calibration is required to determine the real camera properties and also the virtual cameras properties required by one of the approaches used to model the system, which is presented in the following chapters

The camera technique used here is based on the work by Tsai [37] with very minor modification as this is a very well-known calibration method and is found to be suitable to serve our purposes A short description on this calibration technique is

directly useful to the understanding of this system and calibration based approach of determining this system Similar coordinate systems are created in the future analysis

of the single-lens stereovision system The bases of this camera calibration are the camera modeling and the 4-step transformation from 3D world coordinates to 2D camera coordinates which include all the parameters of interest These 4 transformation steps are illustrated in Figure 3.1

Figure 3.1 Camera calibration modeling

Step 1: Rigid body transformation from the object world co-ordinate system

(x w ,y w ,z w ) to the camera 3D co-ordinate system (x cam ,y cam ,z cam)

,

T z y

x R z

y

x

w w w

,

6 5 4

3 2 1

=

x

T T

T T r

r r

r r r

r r r

The matrix R represents the rotation required to make the three axes of the

world coordinates system coincide the three corresponding axes of the camera

coordinates system The vector T represents the translation required to move the

origin of the world coordinate system to coincide with the origin of the camera coordinate system

Step 2: Transformation from 3D camera co-ordinates (x cam ,y cam ,z cam) to ideal

(undistorted) image coordinates (X u ,Y u) using perspective projection with pin hole camera geometry

cam

cam u

cam

cam

y f Y z

x f

X = = ,

(3.2)

where f is the effective focal length

Step 3: Radial lens distortion

u y d u

2

2 1

4 2

where k 1 and k 2 are the distortion coefficients

Step 4: True image coordinate (X d ,Y d ) to computer screen coordinate (X f ,Y f) transformation

)

( )

(X f ,Y f): row and column numbers of image pixel on computer screen;

(C x ,C y): centre of computer screen;

d x′ = d x N cx /N fx;

d y′ = d y N cy /N fy;

d x : distance between adjacent CCD sensor elements in X direction;

d y : distance between adjacent CCD sensor elements in Y direction;

N cx : number of CCD sensor elements in X direction;

N cy : number of CCD sensor elements in Y direction;

N fx : number of pixels in a line as sampled by the computer in X direction;

N fy : number of pixels in a line as sampled by the computer in Y direction

Due to a variety of factors, such as slight timing mismatch between image

acquisition hardware and computer monitor, one uncertainty parameter s x is introduced in equation (3.5) to accommodate this uncertainty

Now a centralized computer screen coordinate S(X, Y) can be defined, where: X=X f - C x Y=Y f -C y

(3.6)

Experimentation on calibration shows that a perfect pinhole camera model would be proper enough to simulate the real camera employed in this system because,

its lens distortion ratios are negligible, which means k 1 ≈ k 2 ≈ 0, and its principle point

(defined as the intersection between camera image plane and camera optical axis) is

accurately located at the camera image plane center, and in addition, s x ≈ 1, etc This conclusion is also one of the bases of the approach using geometrical analysis to model this system, which can be seen later in this thesis

By combining the last three steps of the transformation, the (X,Y) centralized computer screen coordinates are related to (x cam , y cam , z cam), the 3D coordinates of the

object point in camera coordinate system, by the following equations with the

assumptions that k 1 = k 2 = 0 and s x = 1:

z w w w

x w w w

T z r y r x r f X

d

+++

+++

=

9 8 7

3 2 1 '

,

z w w w

y w w w

T z r y r x r f Y

d

+++

+++

=

9 8 7

6 5

(3.7)

By assuming k 1 = k 2 =0 and s x=1, the number of unknowns to be solved is 13

In this way, only 7 world points are needed to yield a non-trivial solution

To determine the relative orientation (described by ρ, υ, and ζ) of the camera with respect to the world co-ordinates in the calibration progress, the following relationship is used:

,coscoscos

sinsinsincoscos

sincossin

sin

sincossin

sinsincoscossin

sincoscos

sin

sincos

sincos

cos

9 8 7

6 5 4

3 2 1

+

−+

++

ρ ζ υ ζ υ

ρ ζ υ

ζ

υ ρ υ

ρ ζ υ ζ υ

ρ ζ υ

ζ

ρ ρ

ζ ρ

ζ

r r r

r r r

r r r

R

(3.8)

where ρ = yaw angle (rotation about y w axis),

υ = pitch angle (rotation about x w axis),

ζ = tilt angle (rotation about z w axis)

This ends the description on the basic theory used in calibration

A very simple calibration based on the pin-hole camera model is created and described in Appendix B for fast implementation and trials