Industrial Robotics Theory Modelling and Control Part 14 pdf

28 Visual Feedback Control of a Robot in an Unknown Environment Learning Control Using Neural Networks Xiao Nan-Feng and Saeid Nahavandi Many methods1-11 have been so far proposed to c

• finger_shape(G)=number,shape i,size i,i=1, , p, expresses the shape of the

gripper in terms of its number p of fingers, the shape and dimensions of each finger Rectangular-shaped fingers are considered; their size is given

"width" and "height"

• fingers_location(G, O)={x ci(O),y ci(O),rz i(O)},i=1, ,p,indicates the relative

loca-tion of each finger with respect to the object's centre of mass and minimum ertia axis (MIA) At training time, this description is created for the object's model, and its updating will be performed at run time by the vision system for any recognized instance of the prototype

in-• fingers_viewing(G, pose_context i,i=1, p) indicating the way how ble" fingers are to be treated; fingers are "invisible" if they are outside the field of view

"invisi-• grip=1, ,k are the k gripper-object Gs_m(G, O) distinct grasping models a priori trained, as possible alternatives to face at run time foreground context situations

A collision-free grasping transformation CF(Gs_m i,O) will be selected at run time

from one of the k grip parameters, after checking that all pixels belonging to

i

FGP_m (the projection of the gripper's fingerprints onto the image

planex , vis y vis, in the O -grasping location) cover only background-coloured

pix-els To provide a secure, collision-free access to objects, the following vision sequence must be executed:

robot-1 Training k sets of parameters of the multiple fingerprints model

O) MFGP_m(G, for G and object class O , relative to the k learned grasping

stylesGs_m i(G, O),i=1, ,k

2 Installing the multiple fingerprint model MFGP_m(G, defining the shape, O)

position and interpretation (viewing) of the robot gripper for clear-grip tests,

by including the model parameters in a data base available at run time This must be done at the start of application programs prior to any image acquisi-tion and object locating

3 Automatically performing the clear-grip test whenever a prototype is recognized

and located at run time, and grips FGP_m i,i=1, k have been a priori fined for it

de-4 On line call of the grasping parameters trained in the Gs_m i(G, O) model, which corresponds to the first grip FGP_m found to be clear i

The first step in this robot-vision sequence prepares off line the data allowing to position at run time two Windows Region of Interest (WROI) around the cur-rent object, invariant to its visually computed location, corresponding to the

two gripper fingerprints This data refers to the size, position and orientation of the gripper's fingerprints, and is based on:

• the number and dimensions of the gripper's fingers: 2-parallel fingered grippers

were considered, each one having a rectangular shape of dimensions

g

g ht

wd , ;

• the grasping location of the fingers relative to the class model of objects of interest

This last information is obtained by learning any grasping transformation for a

class of objects (e.g "LA"), and is described by help of Fig 9 The following frames and relative transformations are considered:

• Frames: (x0,y0): in the robot's base (world); (x vis,y vis): attached to the image plane; (x g,y g): attached to the gripper in its end-point T; (x loc,y loc): default object-attached frame, x loc ≡MIA (the part's minimum inertia axis);

• Relative transformations: to.cam[cam]: describes, for the given camera, the

lo-cation of the vision frame with respect to the robot's base frame; vis.loc:

de-scribes the location of the object-attached frame with respect to the vision frame; vis.obj: describes the location of the object-attached frame with respect

to the vision frame; pt.rob: describes the location of the gripper frame with

re-spect to the robot frame; pt.vis: describes the location of the gripper frame with respect to the vision frame

As a result of this learning stage, which uses vision and the robot's joint

encod-ers as measuring devices, a grasping model

{d.cg alpha z_off rz_off}

) ( G,"LA" = , , ,

centre of mass C and minimum inertia axis MIA (C and MIA are also available

at runtime):

))Gdir(C,,(_

G),dist(T,_

G)),dir(C,MIA,(G),

els, which means that no other object exists close to the area where the gripper's fingers will be positioned by the current robot motion command A negative result of this test will not authorize the grasping of the object

For the test purpose, two WROIs are placed in the image plane, exactly over the areas occupied by the projections of the gripper's fingerprints in the image plane for the desired, object-relative grasping location computed from

) GP_m(G, LA"" ; the position (C) and orientation (MIA) of the recognized object must be available From the invariant, part-related data:

cg d ht wd wd off

rz

alpha, , LA, g, g, , there will be first computed at run time the current coordinates xG, yG of the point G, and the current orientation angle

grasp

Figure 9 Frames and relative transformations used to teach the GP_m(G, LA"" ) rameters

pa-The part's orientation angle.aim=∠(MIA,x vis) returned by vision is added to the learned alpha

alpha angle.aim

x beta= ∠(dir(C,G), vis)= + (5)

Once the part located, the coordinates xC, yC of its gravity centre C are

avail-able from vision Using them and beta, the coordinates xG, yG of the G are puted as follows:

com-)sin(

.),

Now, the value ofangle.grasp=∠(x g,x vis), for the object's current orientation and accounting for rz off from the desired, learned grasping model, is obtained from angle.grasp=beta+rz.off

Two image areas, corresponding to the projections of the two fingerprints on the image plane, are next specified using two WROI operations Using the ge-

ometry data from Fig 9, and denoting by dg the offset between the end-tip

point projection G, and the fingerprints centres CWi, ∀i=1,2,

2/2

).cos(

G

cw1 x dg angle grasp

x = − ⋅ ;xcw2 =xG +dg⋅cos(angle.grasp)

(7))

.sin(

G

cw1 y dg angle grasp

y = − ⋅ ; ycw2 = yG +dg⋅sin(angle.grasp)

The type of image statistics is returned as the total number of non-zero ground) pixels found in each one of the two windows, superposed onto the ar-eas covered by the fingerprints projections in the image plane, around the ob-ject The clear grip test checks these values returned by the two WROI-generating operations, corresponding to the number of background pixels not occupied by other objects close to the current one (counted exactly in the grip-per's fingerprint projection areas), against the total number of pixels corre-sponding to the surfaces of the rectangle fingerprints If the difference between

(back-the compared values is less than an imposed error err for both fingerprints –

windows, the grasping is authorized:

If ar1[ ]4 − ar fngprt ≤errANDar2[ ]4 − ar fngprt ≤err,

clear grip of object is authorized; proceed object tracking by ously

continu-altering its target location on the vision belt, until robot motion is pleted

5 Conclusion

The robot motion control algorithms with guidance vision for tracking and grasping objects moving on conveyor belts, modelled with belt variables and 1-d.o.f robotic device, have been tested on a robot-vision system composed from a Cobra 600TT manipulator, a C40 robot controller equipped with EVI vi-sion processor from Adept Technology, a parallel two-fingered RIP6.2 gripper from CCMOP, a "large-format" stationary camera (1024x1024 pixels) down looking at the conveyor belt, and a GEL-209 magnetic encoder with 1024 pulses per revolution from Leonard Bauer The encoder’s output is fed to one

of the EJI cards of the robot controller, the belt conveyor being "seen" as an ternal device

ex-Image acquisition used strobe light in synchronous mode to avoid the

acquisi-tion of blurred images for objects moving on the conveyor belt The strobe light is triggered each time an image acquisition and processing operation is executed at runtime Image acquisitions are synchronised with external events

of the type: "a part has completely entered the belt window"; because these events generate on-off photocell signals, they trigger the fast digital-interrupt line of the

robot controller to which the photocell is physically connected Hence, the VPICTURE operations always wait on interrupt signals, which significantly improve the response time at external events Because a fast line was used, the most unfavourable delay between the triggering of this line and the request for image acquisition is of only 0.2 milliseconds

The effects of this most unfavourable 0.2 milliseconds time delay upon the integrity

of object images have been analysed and tested for two modes of strobe light triggering:

• Asynchronous triggering with respect to the read cycle of the video camera, i.e as soon as an image acquisition request appears For a 51.2 cm width of the image field, and a line resolution of 512 pixels, the pixel width is of 1

mm For a 2.5 m/sec high-speed motion of objects on the conveyor belt the most unfavourable delay of 0.2 milliseconds corresponds to a displacement

of only one pixel (and hence one object-pixel might disappear during the

dist travel above defined), as:

(0.0002 sec) * (2500 mm/sec) / (1 mm/pixel) = 0.5 pixels

• Synchronous triggering with respect to the read cycle of the camera, ing a variable time delay between the image acquisition request and the strobe light triggering The most unfavourable delay was in this case 16.7 milliseconds, which may cause, for the same image field and belt speed a potential disappearance of 41.75 pixels from the camera's field of view

induc-(downstream the dwnstr_lim limit of the belt window)

Consequently, the bigger are the dimensions of the parts travelling on the veyor belt, the higher is the risk of disappearance of pixels situated in down-stream areas Fig 10 shows a statistics about the sum of:

con-• visual locating errors: errors in object locating relative to the image frame

)

,

(x vis y vis ; consequently, the request for motion planning will then not be issued;

• motion planning errors: errors in the robot's destinations evaluated during

motion planning as being downstream downstr_lim, and hence not

author-ised,

function of the object's dimension (length long_max.obj along the minimal inertia axis) and of the belt speed (four high speed values have been considered: 0.5

m/sec, 1 m/sec, 2 m/sec and 3 m/sec)

As can be observed, at the very high motion speed of 3 m/sec, for parts longer than 35 cm there was registered a percentage of more than 16% of unsuccessful object locating and of more than 7% of missed planning of robot destinations (which are outside the CBW) for visually located parts, from a total number of

250 experiments

The clear grip check method presented above was implemented in the V+ gramming environment with AVI vision extension, and tested on the same ro-bot vision platform containing an Adept Cobra 600TT SCARA-type manipula-tor, a 3-belt flexible feeding system Adept FlexFeeder 250 and a stationary, down looking matrix camera Panasonic GP MF650 inspecting the vision belt The vision belt on which parts were travelling and presented to the camera was positioned for a convenient robot access within a window of 460 mm

pro-Experiments for collision-free part access on randomly populated conveyor belt have been carried out at several speed values of the transportation belt, in the range from 5 to 180 mm/sec Table 1 shows the correspondence between the belt speeds and the maximum time intervals from the visual detection of a part and its collision-free grasping upon checking [#] sets of pre taught grasping modelsGs_m i(G, O),i=1, ,#

0 4 8 12 16 20 Object locating errors [%]

long_max.obj [cm]

0.5 m/sec 1m/sec 2m/sec 3m/sec

0 4 8 12 16

20 planning errors [%]

long_max.obj [cm]

Figure 10 Error statistics for visual object locating and robot motion planning

Belt speed [mm/sec] 5 10 30 50 100 180 Grasping time (max) [sec] 1.4 1.6 1.9 2.0 2.3 2.5 Clear grips checked[#] 4 4 4 4 2 1 Table 1 Corespondance between belt speed and collision-free part grasping time

6 References

Adept Technology Inc (2001) AdeptVision User's Guide Version 14.0, Technical

Publi-cations, Part Number 00964-03300, Rev B, San Jose, CA

Borangiu, Th & Dupas, M (2001) Robot – Vision Mise en œuvre en V+, Romanian

Academy Press & AGIR Press, Bucharest

Borangiu, Th (2002) Visual Conveyor Tracking for "Pick-On-The-Fly" Robot Motion

Control, Proc of the IEEE Conf Advanced Motion Control AMC'02, pp 317-322,

Maribor

Borangiu, Th (2004) Intelligent Image Processing in Robotics and Manufacturing,

Roma-nian Academy Press, ISBN 973-27-1103-5, Bucarest

Borangiu, Th & Kopacek, P (2004) Proceedings Volume from the IFAC Workshop

Intelli-gent Assembly and Disassembly - IAD'03 Bucharest, October 9-11, 2003, Elsevier Science, Pergamon Press, Oxford, UK

Borangiu, Th (2005) Guidance Vision for Robots and Part Inspection, Proceedings

vol-ume of the 14th Int Conf Robotics in Alpe-Adria-Danube Region RAAD'05, pp

27-54, ISBN 973-718-241-3, May 2005, Bucharest

Borangiu, Th.; Manu, M.; Anton, F.-D.; Tunaru, S & Dogar, A (2006) High-speed

Ro-bot Motion Control under Visual Guidance, 12th International Power Electronics

and Motion Control Conference - EPE-PEMC 2006, August 2006, Portoroz, SLO Espiau, B.; Chaumette, F & Rives, P (1992) A new approach to visual servoing in ro-

botics, IEEE Trans Robot Automat., vol 8, pp 313-326

Lindenbaum, M (1997) An Integrated Model for Evaluating the Amount of Data

Re-quired for Reliable Recognition, IEEE Trans on Pattern Analysis & Machine

In-tell

Hutchinson, S A.; Hager, G.D & Corke, P (1996) A Tutorial on Visual Servo Control,

IEEE Trans on Robotics and Automation, vol 12, pp 1245-1266, October 1996

Schilling, R.J (1990) Fundamentals of Robotics Analysis and Control, Prentice-Hall,

Englewood Cliffs, N.J

Zhuang, X.; Wang, T & Zhang, P (1992) A Highly Robust Estimator through

Par-tially Likelihood Function Modelling and Its Application in Computer Vision,

IEEE Trans on Pattern Analysis and Machine Intelligence

West, P (2001) High Speed, Real-Time Machine Vision, CyberOptics – Imagenation, pp

1-38 Portland, Oregon

28

Visual Feedback Control of a Robot in an Unknown Environment

(Learning Control Using Neural Networks)

Xiao Nan-Feng and Saeid Nahavandi

Many methods(1)-(11) have been so far proposed to control a robot with a era to trace an object so as to complete a non-contact operation in an unknown environment e.g., in order to automate a sealing operation by a robot, Hosoda,

cam-K.(1) proposed a method to perform the sealing operation by the robot through off-line teaching beforehand This method used a CCD camera and slit lasers

to detect the sealing line taught beforehand and to correct on line the joint gles of the robot during the sealing operation

an-However, in those methods(1)-(3), only one or two image feature points of the sealing were searched per image processing period and the goal trajectory of the robot was generated using an interpolation Moreover, those methods must perform the tedious CCD camera calibration and the complicated coor-dinate transformations Furthermore, the synchronization problem between the image processing system and the robot control system, and the influences

of the disturbances caused by the joint friction and the gravity of the robot need to be solved

In this chapter, a visual feedback control method is presented for a robot to trace a curved line in an unknown environment Firstly, the necessary condi-tions are derived for one-to-one mapping from the image feature domain of the curved line to the joint angle domain of the robot, and a multilayer neural network which will be abbreviated to NN hereafter is introduced to learn the mapping Secondly, a method is proposed to generate on line the goal trajec-tory through computing the image feature parameters of the curved line Thirdly, a multilayer neural network-based on-line learning algorithm is de-veloped for the present visual feedback control Lastly, the present approach is applied to trace a curved line using a 6 DOF industrial robot with a CCD cam-

era installed in its end-effecter The main advantage of the present approach is that it does not necessitate the tedious CCD camera calibration and the com-plicated coordinate transformations.

Contact object

Robot end-effector

Workspace

camera Rigid tool Tangential direction

Workspace frame

z

C

Cameraframe

ȟ

Goal feature

Initial feature

Feature of curved line

Figure 2 Image features and mapping relation

2 A Trace Operation by a Robot

When a robot traces an object in an unknown environment, visual feedback is necessary for controlling the position and orientation of the robot in the tan-gential and normal directions of the operation environment Figure 1 shows a robot with a CCD camera installed in its end-effecter The CCD camera guides the end-effecter to trace a curved line from the initial position I to the target position G Since the CCD camera is being fixed at the end-effecter, the CCD camera and the end-effecter always move together

3 Mapping from Image Feature Domain to Joint Angle Domain

3.1 Visual feedback control

For the visual feedback control shown in Fig 1, the trace error of the robot in the image feature domain needs to be mapped to the joint angle domain of the robot That is, the end-effecter should trace the curved line according to

j

a ,aj+1 in the image domain of the features Aj,Aj+1 shown in Fig 2

Let ȟ,ȟ d∈R6×1be the image feature parameter vectors of aj,aj+1 in the image feature domain shown in Fig 2, respectively The visual feedback control shown in Fig 1 can be expressed as

where || · || is a norm, and e should be made into a minimum

From the projection relation shown in Fig 2, we know

ȟ= ϕ (p tc), (2)

where ϕ∈R6×1 is a nonlinear mapping function which realizes the projection

transformation from the workspace coordinate frame ™ O to the image feature domain shown in Fig 2

It is assumed that ptc∈R6×1is a position/orientation vector from the origin of

the CCD camera coordinate frame ™ C to the gravity center of Aj Linearizing Eq.(2) at a minute domain of p tc yields

ȟ

where Dž ȟ and Dž ptc are minute increments of ȟ and ptc , respectively, and J f=

ptc

Ǘ

∂ ∂ ∈R6×6 is a feature sensitivity matrix

Furthermore, let ș and Dž ș∈R6×1 are a joint angle vector of the robot and its

minute increment in the robot base coordinate frame ™ B If we map Dž ș from

™ B toDž p t c in ™ O using Jacobian matrix of the CCD camera J c∈R6×6, we can get

pt c

Dž =Jc· Dž ș (4) From Eqs.(3) and (4), we have

ș

Dž = (JfJc)-1 · Dž ȟ (5)

Therefore, the necessary condition for realizing the mapping expressed by

Eq.(5) is that (J f J c)-1 must exist Moreover, the number of the independent age feature parameters in the image feature domain (or the element numbers

im-of ȟ) must be equal to the degrees of freedom of the visual feedback control system

3.2 Mapping relations between image features and joint angles

Because J f and J c are respectively linearized in the minute domains of ptc and

ș, the motion of the robot is restricted to a minute joint angle domain, and Eq.(5) is not correct for large Dž ș Simultaneously, the mapping is weak to the

change of (J f J c)-1 In addition, it is very difficult to calculate (J f J c)-1 on line during the trace operation Therefore, NN is introduced to learn such mapping

Firstly, we consider the mapping from ǻptc in ™ O to ǻȟ in the image feature domain ǻȟ and ǻp tcare increments of ȟ and p tc for the end-effecter to move from Aj to Aj+1, respectively As shown in Fig 2, the mapping is depend on p tc,and the mapping can be expressed as

ǻp =ij3(ǻȟ,ȟ), (8)

where ij3( )∈R6×1 is a continuous nonlinear mapping function

Secondly, we consider the mapping from ǻ ș in ™ B to ǻp c in ™ O Let p c∈R6×1be

a position/ orientation vector of the origin of ™ C with respect to the origin of

™ O, ǻ ș and ǻp c be increments of ș and p c for the end-effecter to move from Aj

to Aj+1, respectively ǻp c is dependent on ș as shown in Fig 2, and we obtain from the forward kinematics of the CCD camera

ij ( )∈R6×1 is a continuous nonlinear mapping function

Since the relative position and orientation between the end-effecter and the CCD camera are fixed, the mapping from ǻp c to ǻp tc is also one-to-one We get

Combing Eq.(8) and Eq.(11.b) yields

ș

ǻ =ij6(ǻȟ,ȟ,ș), (12)

where ij6( )∈R6×1 is a continuous nonlinear mapping function In this paper,

NN is used to learn ij6( )

3.2 Computation of image feature parameters

For 6 DOF visual feedback control, 6 independent image feature parameters are chosen to correspond to the 6 joint angles of the robot An image feature parameter vector ( j)

1

[ ξ j , ) 2

j

ξ ,···, j T

]

) 6

ξ is defined at the window j shown in Fig

3 L and W are length and height of the window j, respectively.

᧤

1

᧥ pixel white

᧤

0

, (13.a)

the elements of )

ȟ are defined and calculated by the following equations:

(1) The gravity center coordinates

g

q g

) (

) (

, (13.b)

Imagefeature domain

u v

j

Window

) 1 (j−

ȟ

W W

j qr

g

r g

1 1

) (

1 1

) (

g

1 1

) (

(3) The lengths of the main and minor axes of the equivalent ellipse

) 02 ) 20 )

( 02 )

20

2

)(4)(

j

j j

j j

j

ξ

λλ

λλ

02 )

20

2

)(4)(

j

j j

j j

j

ξ

λλ

λλ

g

) ( 1 )

(

] [ ξ 2, (13.h)

qr r g

1 1

) ( 2 )

) ( 11 1

tan

2

1

j j j

λλ

ȟ are image feature parameter vectors in the

win-dow j and j+1 shown in Fig 3 ( 0 )( )

4 Goal Trajectory Generation Using Image Feature Parameters

In this paper, a CCD camera is used to detect the image feature parameters of

the curved line, which are used to generate on line the goal trajectory The

se-quences of trajectory generation are shown in Fig 4 Firstly, the end-effecter is

set to the central point of the window 0 in Fig 4(a) At time t=0, the first image

of the curved line is grasped and processed, and the image feature parameter vectors ( 0 ) ( 0 )

ȟ , ( 1 ) ( 0 )

ȟ and ( 2 ) ( 0 )

ȟ in the windows 0,1,2 are computed

respec-tively From time t=mT to t=2mT, the end-effecter is only moved by

ȟ At time t=mT, the second image of the curved line is

grasped and processed, the image feature parameter vector ( 2 ) ( )

− At time t=imT, (i+1)th image is grasped and processed, the

image feature parameter vector ( 2 ) ( )

im

ȟ shown in Fig 4(c) is computed From

t =imT to t=(i+1)mT, the end-effecter is only moved by

)

(im

ǻȟ = ( 1 )[( 1 ) ] ( 0 )[( 1 ) ]

m i m

ȟ

W W

L

)0() 0 (

ȟ

)0() 2 (

) 2 (

) 1 (

m

ȟ

) (

) 0 (

) (

) 2 (

im

ȟ

W W

L

) (

) 1 ( im

ȟ

) (

) 0 (

5 Neural Network-Based Learning Control System

5.1 Off-line learning algorithm

In this section, a multilayer NN is introduced to learn ij6( ) Figure 5 shows the structure of NN which includes the input level A, the hidden level B, C and the

output level D Let M, P, U, N be the neuron number of the levels A, B, C, D, respectively, g=1,2,ƛƛƛ,M, l=1,2,ƛƛƛ,P, j=1,2,ƛƛƛ,U and i=1,2,ƛƛƛ,N Therefore, NN

.

.

.

.

.1

lj

( ),

( ,

, ,

1 1

1

D i D

i C

j C

j B

l B

l A

g

A

g

D i U

j

C j CD ji D

i P

l

C j B l BC lj C

j M

g

B l A g AB

gl

B

l

x f y x

f y x

f y x

y

z y w x

z y w x

z y w

x

, (15)

where x m R, R

m

y and z m R (m=g, l, j, i; R=A, B, C, D) are the input, the output and

the bias of a neuron, respectively w gl AB, w lj BC and CD

( [ )]

( )

( [

2

T n r

ȟ ,ǻȟ (k) for the end-effecter to move along the curved line from the initial position I to the goal position G ș (k),ǻș r (k), ) ( )

ȟ and ǻȟ (k) are divided by their maximums before ting to NN, respectively CD

input-ji

w of NN is changed by

CD ji

f f

CD

ji

w

E w

j D

algo-The present learning control system based on NN is shown Fig 7 algo-The initial

()

1

(

S k

k k

k

n n

n n

n

d

ǻș ș

ș

ǻș ș

ș

, (19)

where d∈R6×1is an output of NN when ș( 0 ), ( 0 ) ( 0 )

ȟ and ǻȟ( 0 ) are used as tial inputs of NN The PID controller G c (z) is used to control the joint an-gles of the robot

ini-5.2 On-line learning algorithm

For the visual feedback control system, a multilayer neural network NNc is troduced to compensate the nonlinear dynamics of the robot The structure and definition of NNc is the same as NN, and its evaluation function is defined as

experiments While NNc is learning, the elements of e(k) will become smaller

and smaller

Figure 7 Block diagram of learning control system for a robot with visual feedback

In Fig 7, I is a 6×6 unity matrix ȁ[ș(k)]∈R6×1and J r[ș(k)]=∂ȁ[ș(k)]/∂ș(k)∈R6×6

are the forward kinemics and Jacobian matrix of the end-effecter (or the rigid tool), respectively Let p t (k)=[p t1, p t2,··· T

de-5.3 Work sequence of the image processing system

The part circled by the dot line shown in Fig 7 is an image processing system The work sequence of the image processing system is shown in Fig 8 At time

t =imT, the CCD camera grasps a 256-grade gray image of the curved line, the

image is binarizated, and the left and right terminals of the curved line are tected Afterward, the image parameters ( 0 ) ( 0 )

ȟ and ǻȟ (k) are solved at time t=kT.

5.4 Synchronization of image processing system and robot control system

Generally, the sampling period of the image processing system is much longer than that of the robot control system Because the sampling period of )

im j

] ) 1 ( [ 2

2

2

m i m

m i

im j

ȟ , (m/2 k–im<m) (24.b)

Image grabbing

), 0 (

) 0 (

ȟ

), 0 (

) 1 (

ȟ

) 0 (

) 2 (

ȟ

Binarization

Goal terminal detection

Start terminal detection

If iฺ 1, compute image feature

parameters in the window 2 ȟ (2)(im) (i=1,2,···)

),(esynchroniz)

() 1

ȟ

), (

) 0 (

k

ȟ ȟ(1)( k ),

ș(k) , (i=0,1,2,···)

If i=0, compute image feature

parameters in the windows 0,1,2

Start tracing

Stop tracing

Figure 8 Work sequence of image processing

6 DOF robot

Motoman K3S

Imageprocessing

Robotcontrol

I/O ports

Switches

6 servo drivers

CCDcameracontroller

CCD camera

Work table

Rigidtool

Operation enverimentHand

Hand controller

Parallelcommunication

Figure 9.Trace operation by a robot

6 Experiments

In order to verify the effectiveness of the present approach, the 6 DOF trial robot is used to trace a curved line In the experiment, the end-effecter (or the rigid tool) does not contact the curved line Figure 9 shows the experimen-tal setup The PC-9821 Xv21 computer realizes the joint control of the robot The Dell XPS R400 computer processes the images from the CCD camera The robot hand grasps the rigid tool, which is regarded as the end-effecter of the robot The diagonal elements of K P,K I and K D are shown in Table 1, the con-trol parameters used in the experiments are listed in Table 2, and the weight-

indus-ing matrix W is set to be an unity matrix I.

Figures 10 and 11 show the position responses (or the learning results of NN and NNc)p t1(k),p t2(k) and p t3(k) in the directions of x, y and z axes of ™ O p t1(k),p t2(k)

and p t3k) shown in Fig 10 are teaching data Figure 11 show the trace responses after the learning of NNc Figure 12 shows the learning processes of NN and

NNc as well as the trace errors E f converges on 10-9 rad2, the learning error E*

shown in Fig 12(b) is given by E*= E k N

k c

/ ] ( [N 1

0

¦−

= , where N=1000 After the 10 als (10000 iterations) using NNc, E*converges on 7.6×10-6 rad2, and the end-

tri-effecter can correctly trace the curved line Figure 12(c) shows the trace errors

of the end-effecter in x, y, z axis directions of ™ O , and the maximum error is lower than 2 mm

Table 1 Diagonal ith element of K P,K I andK D

Table 2 Parameters used in the experiment

Sampling period T =5 ms, mT=160 ms

ControlParameters ηf=0.9,ηc=0.9,˟ =1,

K=100, S=732Window size L=256, W=10

0,060,0650,070,0750,08

0 0,000001

the present approach is verified by tracing a curved line using a 6 DOF trial robot with a CCD camera installed on its end-effecter

indus-Through the above research, the following conclusions are obtained:

1 If the mapping relations between the image feature domain of the object and the joint angle domain of the robot are satisfied, NN can learn the mapping relations

2 By computing the image feature parameters of the object, the goal tory for the end-effecter to trace the object can be generated

trajec-3 The approach does not necessitate the tedious CCD camera calibration and the complicated coordinate transformation

4 Using the 2nd-order holder and the disturbance observer, the zation problem and the influences of the disturbances can be solved

synchroni-The above research is supported by the Natural Science Foundation of China (No 60375031) and the Natural Science Foundation of Guangdong (No 36552), the authors expresses their heartily thanks to the Foundations

8 References

Hosoda, K & Asada, M., (1996) Adaptive Visual Servoing Controller with

Feed-forward Compensator without Knowledge of True Jacobian, Journal

of Robot Society of Japan, (in Japanese), Vol 14, No 2, pp 313-319

Hosoda, K., Igarashi, K & Asada, M., (1997) Adaptive Hybrid Visual/Force

Servoing Control in Unknown Environment, Journal of Robot Society of pan, (in Japanese), Vol 15, No 4, pp 642-647

Ja-Weiss, L.E & Sanderson, A.C., (1987) Dynamic Sensor-Based Control of

Ro-bots with Visual Feedback, IEEE Journal of Robotics and Automation, Vol.3,

No 5, pp 404-417

Nikolaos, P & Pradeep, K., (1999) Visual Tracking of a Moving Target by a

Camera Mounted on a Robot: A Combination of Control and Vision, IEEE Journal of Robotics and Automation, Vol 9, No 1, pp 14-35

Bernardino, A & Santos-Victor J, (1999) Binocular Tracking: Integrating

Per-ception and Control, IEEE Journal Robotics and Automation, Vol 15, No 6,

pp 1080-1093

Malis, E., Chaumette, F and Boudet, S., (1999) 2-1/2-D Visual Servoing, IEEE Journal of Robotics and Automation, Vol 15, No 2, pp 238-250

Hashimoto, K., Ebine, T & Kimura, H., (1996) Visual Servoing with Hand-Eye

Manipulator-Optimal Control Approach, IEEE Journal of Robotics and Automation, Vol 12, No 5, pp 766-774

Wilson, J.W., Williams, H & Bell, G.S., (1996) Relative End-Effecter Control

Using Cartesian Position Based Visual Servoing, IEEE Trans Robotics and Automation, Vol 12, No 5, pp 684-696

Ishikawa, J., Kosuge, K & Furuta, K., Intelligent Control of Assembling Robot Using Vision Sensor, 13-18 May 1990, Cincinnati, OH, USA, Proceedings

1990 IEEE International Conference on Robotics and Automation (Cat No.90CH2876-1), Vol 3, pp 1904-1909

Yamada, T & Yabuta, T., (1991) Some Remarks on Characteristics of Direct

Neuro-Controller with Regard to Adaptive Control, Trans Soc Inst Contr Eng., (in Japanese), Vol 27, No 7, pp 784-791

Verma, B., (1997) Fast Training of Multilayer Perceptrons, IEEE Trans Neural Networks, Vol 8, No 6, pp 1314-1319

29

Joystick Teaching System for Industrial Robots

Using Fuzzy Compliance Control

Fusaomi Nagata, Keigo Watanabe and Kazuo Kiguchi

Industrial robots have been applied to several tasks, such as handling, bling, painting, deburring and so on (Ferretti et al., 2000), (Her & Kazerooni, 1991), (Liu, 1995), (Takeuchi et al., 1993), so that they have been spread to vari-ous fields of manufacturing industries However, as for the user interface of the robots, conventional teaching systems using a teaching pendant are only provided For example, in the manufacturing industry of wooden furniture, the operator has to manually input a large mount of teaching points in the case where a workpiece with curved surface is sanded by a robot sander This task

assem-is complicated and time-consuming To efficiently obtain a desired trajectory along curved surface, we have already considered a novel teaching method as-sisted by a joystick (Nagata et al., 2000), (Nagata et al., 2001) In teaching mode, the operator can directly control the orientation of the sanding tool attached to the tip of the robot arm by using the joystick In this case, since the contact force and translational trajectory are controlled automatically, the operator has only to instruct the orientation with no anxiety about overload and non-contact state However, it is not practical to acquire sequential teaching points with normal directions, adjusting the tool's orientation only with operator's eyes

When handy air-driven tools are used in robotic sanding, keeping contact with the curved surface of the workpiece along the normal direction is very impor-tant to obtain a good surface quality If the orientation of the sanding tool largely deviates from normal direction, then the kinetic friction force tends to become unstable Consequently, smooth and uniform surface quality can’t be achieved That is the reason why a novel teaching system that assists the op-erator is now being expected in the manufacturing field of furniture

In this paper, an impedance model following force control is first proposed for

an industrial robot with an open architecture servo controller The control law allows the robot to follow a desired contact force through an impedance model

in Cartesian space And, a fuzzy compliance control is also presented for an advanced joystick teaching system, which can provide the friction force acting

between the sanding tool and workpiece to the operator (Nagata et al., 2001) The joystick has a virtual spring-damper system, in which the component of stiffness is suitably varied according to the undesirable friction force, by using

a simple fuzzy reasoning method If an undesirable friction force occurs in teaching process, the joystick is controlled with low compliance Thus, the op-erator can feel the friction force thorough the variation of joystick's compliance and recover the orientation of the sanding tool We apply the joystick teaching using the fuzzy compliance control to a teaching task in which an industrial robot FS-20 with an open architecture servo controller profiles the curved sur-face of a wooden workpiece Teaching experimental results demonstrate the effectiveness and promise of the proposed teaching system

2 Impedance Model Following Force Control

More than two decades ago, two representative force control methods were proposed (Raibert, 1981), (Hogan, 1985) ; controllers using such methods have been advanced and further applied to various types of robots However, in order to realize a satisfactory robotic sanding system based on an industrial robot, deeper considerations and novel designs are needed Regarding the force control, we use the impedance model following force control that can be easily applied to industrial robots with an open architecture servo controller (Nagata et al., 2002) The desired impedance equation for Cartesian-based con-trol of a robot manipulator is designed by

I are the switch matrix diag(S1, S2, S3) and identity matrix It is assumed that

d

M , B d, K d and K f are positive definite diagonal matrices Note that if

S = I , then Eq (1) becomes an impedance control system in all directions; whereas if S is the zero matrix, it becomes a force control system in all direc-

tions If the force control is used in all direction, X =x−xdgives

f d d

d B X M K F F

M

X =− −1 + −1 − (2)