DSpace at VNU: Development of a tele-guidance system with fuzzy-based secondary controller

A tele-guidance application involving the remote control of a mobile robot over the Internet is set up as the basis for the evaluation of reliability and applicability of the fuzzy-based

Development of a Tele-guidance System with

Fuzzy-based Secondary Controller

Manh Duong PHUNG, Thanh Van Thi NGUYEN, Cong Hoang QUACH, Quang Vinh TRAN

Department of Electronics and Computer Engineering University of Engineering and Technology Vietnam National University, Hanoi

Abstract—Dealing with the uncertainties of Internet

characte-ristics is an important issue that needs being taken into account

in developing Internet-based real-time systems In this paper, we

present our approach in applying fuzzy logic to develop back-up

mechanisms for an Internet-based mobile robot to deal with

unwanted network problems such as long delays or network

interruptions A tele-guidance application involving the remote

control of a mobile robot via the Internet is set up as the context

to verify the effectiveness and applicability of the proposed

approach

Keywords—Data transmission; Internet; distributed control;

fuzzy logic; robot navigation; networked robot

I INTRODUCTION

Since the first Internet-based telerobot appeared in 1994

[3], substantial efforts have been devoted to remotely control

real-time systems over the Internet, allowing users to visit

museums, tend gardens, navigate undersea, float in blimps, or

handle protein crystals via the Internet [1]–[5] Whereas early

researches tried to answer the question of how to control a

robot through the Internet [3][6][7], recent researches focus on

how to control it in real time and dealing with the inevitable

Internet transmission delays, delay jitter and non-guaranteed

bandwidth when controlling via the Internet becomes easier

with the support of embedded Ethernet, on-chip web server,

scripting languages, socket programming, etc [8]-[10] Many

approaches have been proposed with their strengths and

limitations

A CORBA-based robotic system featured the concept of

task-level control [8] In this system, instead of step-by-step

operation over the Internet, a control command from the user is

sent to the robot at a task-level such as “give me the spoon”,

“grasp the blue bowl”, “coffee, please”, etc The robot then

analyzes the command, makes the path planning, completes the

task and returns the result to the user without requiring any

additional actions This approach reduces the influence of time

delays to the system; however, the flexibility is also reduced

In [9], a virtual environment which is proportional to the

real dimension of the laboratory is constructed in the client

side Before commands are sent to the server program, they are

processed in the virtual environment to predict the upcoming

position of the robot Basing on it, the system can tolerate a

certain amount of time delays as well as allowing users to

experience the interaction with the robot as in the real environ-ment

In another approach, Liu et al gave an attempt to deal with the key issue of Internet-based systems, the communication channel, by proposing a new transport protocol namely Trinomial [10] It is a rate-based protocol that optimizes the use

of available bandwidth and is able to adapt to the network congestion without affecting very much to the way the user teleoperates the robot This transmission mechanism is a strong solution which uses as much bandwidth as possible, providing smoothness to a bilateral teleoperation via Internet However, it introduces extra time-delay due to the fact that it sets the router buffers to the maximum load [11]

In this paper, we propose a novel approach in which a fuzzy logic controller is designed to deal with the Internet-induced uncertainties A tele-guidance application involving the remote control of a mobile robot over the Internet is set up as the basis for the evaluation of reliability and applicability of the fuzzy-based approach

II TELE-GUIDANCE SYSTEM ARCHITECTURE

The analysis to be presented in this paper will be centered around a tele-guidance system described below

A Hardware Configuration

The tele-guidance system is developed following a distributed model which consists of four stand-alone modules:

the mobile robot, the fuzzy-based secondary controller, the central server and the client computer The intercommunication between these modules is via a digital channel Fig.1 shows the hardware configuration of the tele-guidance system The central server, the secondary controller and the client are Intel Pentium IV computers with control software installed The central server has three interfaces: one to link it to the mobile robot to allow the communication of control and feedback data , one to communicate with the fuzzy-based controller and the other interface allows it to process requests from remote users

The robot is a commercial Sputnik mobile robot [12] It has basic components for motor control, sensing and navigation, including battery power, drive motors and wheels, position/speed encoders, infrared sensors, integrated sonar ranging sensors and a visual system Sensing and motor control are managed by an on-board digital signal processor (DSP) with independent motor/power and sonar controller boards for

2010 11th Int Conf Control, Automation, Robotics and Vision

Singapore, 7-10th December 2010

Figure 1: System hardware configuration

Figure 2: System software architecture

a versatile operating environment The drive system uses

high-speed, high-torque, reversible-DC motors The Sputnik mobile

robot provides three sonar sensors One is mounted to the front

of the robot and the others are mounted to its left and right side

respectively The data of sonar sensors are the basis for

fuzzy-based navigating algorithms The visual system is detachable

and mounted on the head of the Sputnik mobile robot It mainly

consists of a MCI3908 color image module with Sharp mini

color camera head LZ0P390M The image size can be up to

CIF format (353 x 288 pixels) and the operation frame rate is

up to 15 fps A Wi-Fi 802.11 wireless module is employed to

connect the Sputnik mobile robot to the Internet

B Software design

The overall system software consists of two subsystems,

namely, a client-server application and a backend fuzzy

module The front-end involves a client side applet which

presents the user with a graphical user interface (GUI)

Through the GUI, the user is able to observe the remote

environment, access the parameters of the mobile robot, and

control it in real time to explore the laboratory over the

Internet The fuzzy control software is written in MATLAB [13] This module processes ultrasonic data based on fuzzy algorithm and returns the control commands to a server program running on the central server A brief functional software structure of the platform is shown in Fig.2

1) Client-server applications: For Internet-based

applications, using client-server architecture is usually considered as a judicious selection because of its flexibility, usability, scalability, and interoperability in design and development [14] In our system, we chose a client-server architecture which consists of following necessary

subsystems:

a) Server: A server is a computer program that provides

services to other computer programs on the same computer or

on another networked computer In our system, the server handles all control requests from client and the fuzzy module

It then processes them in conjunction with network status, remote environment parameters, and forwards translated commands to the Sputnik robot The server also retrieves

Sensor Data

Control Data Sonar Data

Sputnik Mobile Robot

INTERNET

Fuzzy-based Controller TCP

Client Module

UDP

GUI

Fuzzy Module

Server Module

Data Processing WiRobotAPI Control Commands

Sensor Data Client Requests

sensor information about status of the robot and transmits it to

clients In short, the server acts as the bridge between the client

module, the fuzzy module and the Sputnik mobile robot

In the system, the server program was written in Visual

Basic 6.0 [15] and the communication between the server and

client was established through socket, an abstraction that

represents a terminal for communications between processes

across a network

b) Client: On the client side, the graphical user interface

(GUI) is responsible for receiving and interpreting human

control commands such as mouse-click events in the control

panel, and transmitting them to the server program over the

Internet The GUI also displays feedback information about the

remote side such as current robot position, ultrasonic ranging

values, and live video of remote environment (Fig.10)

c) Communication protocol: There are currently two

major transport protocols for implementing remote control

applications over the Internet: TCP and UDP [16][17] TCP is a

reliable protocol for transmitting static data such as files,

e-mails over low-bandwidth, high-error-rate networks UDP is

merely a raw protocol that does not guarantee the destination

arrival of transmitted data; however, it introduces relatively

minimized transmission delay and jitter In our system, TCP is

used in communicating over these Internet while UDP is the

protocol for local area communication

2) Fuzzy module: : The fuzzy module connects to the

server as a special client and supplies the server with following

services:

• Acquiring data from ultrasonic sensors and passing it

to the fuzzy-based controller for creating output

commands

• Sending commands to the server to operate the mobile

robot in case network congestions or unwanted

obstacles appear

With these functions, the fuzzy module plays the role of a

secondary controller to supply the system with back-up

mechanisms The implementation of the fuzzy-based controller

will be described in the next section

III IMPLEMENTATION OF FUZZY-BASED CONTROLLER

The objective of the fuzzy-based controller in our system is

twofold Firstly, it protects the robot by performing the obstacle

avoidance during the process of tele-guidance operation;

secondly, it navigates the mobile robot to a pre-defined safe

point in case of a network interruption The implementation of

the fuzzy logic-based controller is divided into four logical

steps as follow:

A Defining the problem

The system has four inputs: the distance values u1, u2, u3

of three ultrasonic sensors and the deflection angle α between

the robot-front direction and the safe point-to-robot direction

The three sensors scan a range of -600 to +600; the deflection

angle α has the range from -1800 to +1800 The outputs of the

fuzzy system are the speed of robot and the rotation angle φ

that the robot needs rotate

B Defining the linguistic variables and the membership functions

In this step, we convert the input data into suitable linguistic values which will be viewed as a label of fuzzy sets The distance values ui of ultrasonic sensors are divided in three subsets: Near (N), Medium (M) and Far (F) The deflection angle α has five subsets: Large Negative (LN), Negative (N), Zero (Z), Positive (P), and Large Positive (LP)

Figure 3: Membership functions for three distance inputs

Figure 4: Membership function for the deflection angle input

Figure 5: Membership function for the rotation angle output

Figure 6: Membership function for the speed output

The rotation angle φ is divided in two ranges corresponding

to the obstacle avoidance and the target finding task For the obstacle avoidance, the subsets include: Left Obstacle (LO), Straight Obstacle (SO), and Right Obstacle (RO) For the target finding, the subsets consist of Large Left (LL), Left (L),

Figure 7: Remotely navigating the

mobile robot around the laboratory Figure 8: Robot motion in case of a network interruption Figure 9: Robot motion in case of a long-time delay

Figure 10: Client Graphical User Interface in a laboratory environment during the tele-guidance operation Figure 11: A sequence of images showing the motion of robot

Straight (S), Right (R), and Large Right (LR) The speed output

has Low Speed (LS), Medium Speed (MS) and Fast Speed

(FS) The definition of membership functions for these fuzzy

sets is described in Fig.3-Fig.6

TABLE 1: Common fuzzy rules Rules Sensor 1 Sensor 2 Sensor 3 Target angle Rotation angle Speed

1 F F F Z Z FS

2 F F F LN LL FS

3 F F F LP LR FS

4 M M M N L MS

5 M M M P R MS

6 N F F Z RO LS

7 N F F LP LR LS

8 F F N LN LL LS

9 N N N LP LR LS

10 F N F N L LS

C Defining the fuzzy rules

Each rule in the fuzzy knowledge base corresponds to a

fuzzy relation With four inputs and two outputs, it would be

computationally expensive to produce 360 rules as well as

inefficient to divert the robot from its trajectory for each fired

rules In this system, we declared 29 rules for most common

situations and each rule is a combination of four inputs and two

outputs as the following statement: “If sensor1 is far and

sensor2 is far and sensor3 is far and the deflection angle is

zero then the rotation angle is zero and the speed is fast”

Table 1 shows some rules of our system

D Defuzzification

To get output values of speed and rotation angle, we used the centroid defuzzification method with min-max inference The formula is as follows:

∑

∑

=

) (

) (

i

i i x

x x

μ

μ ϕ

where x i is the i’th domain value, and µ(x i ) is the truth

membership value for that domain point

IV EXPERIMENTS

To verify the validity and effectiveness of proposed fuzzy-based mechanisms for the tele-guidance system, we have carried out many experiments Fig.7 shows the setup of the environment that the Sputnik mobile robot moves through The goal of the experiments is to remotely guide the Sputnik mobile robot from the starting point Oo to the objective point Od via the Internet In case of network congestions or interruptions, the robot should be navigated to the safe point Os by the fuzzy-based secondary controller

In the experiments, by using the designed client GUI (Fig.10), the user successfully navigated the Sputnik mobile robot from point Oo to point Od via the Internet (Fig.7) Fig.11 shows a sequence of the snapshots of the Sputnik mobile robot when it was remotely being guided via the Internet to move from point Oo to point P3 in the Automatic Control and

Robotics Laboratory (ACRs Lab) at the Hanoi University of

Engineering and Technology, Vietnam

To test the behavior of the system in case of a network

interruption, we unplugged the network cable which connects

the central server to the Internet during a tele-guidance process

As shown in Fig.8, the network interruption appeared when the

robot was at P2 position It continued to move to P3 before

changing its direction to react to the network problem This

matched our implementation which detected the network

interruption as the event that the server did not receive

synchronous messages from the client for 120 seconds The

robot then moved through P4, P5 and finally reach the safe point

Os

To simulate a long-time-delay event, we forced the client

and server programs not to communicate for 90 seconds

starting at the time Ts of the experiment Fig.9 shows the

motion path of the robot and fig.12 displays the received data

from ultrasonic sensors The Ts and Te are the start and end

time of the delay event The Td represents the time at which the

mobile robot is very near to an obstacle and may result in a

collision Due to the network delay, the operator, however, did

not recognize this risk and took no action The fuzzy-based

controller was then immediately set to high priority and

successfully controlled the mobile robot to avoid the obstacle

Figure 12: Ultrasonic data in case of a long-time-delay

V CONCLUSION

In this paper, we propose a novel approach to deal with the

uncertainties of Internet characteristics for Internet-based

real-time systems: the use of the fuzzy-based controller as a

secondary control The implementation of this controller is

described in details The hardware and software design of a

tele-guidance system is also represented as the context to verify

the proposed approach The experiments show that the user can

entertainingly control the mobile robot to explore the ACRs

Lab over the Internet without worrying about the network problems such as network congestions, network interruptions and long delays

ACKNOWLEDGMENT

This work was supported by Vietnam's National Foundation for Science and Technology Development (NAFOSTED)

REFERENCES

[1] K Golberg and M Mascha, “Desktop teleoperation via the World Wide Web,” Proceedings of the 1995 IEEE International Conference on Robotics and Automation, May 1995, pp.654-659

[2] R C Luo and T M Chen, “Development of a multi-behavior based mobile robot for remote supervisory control through the Internet,” IEEE/ASME Trans Mechatronics, vol 5, no 4, pp 376–385, Dec

2000

[3] Beyond Webcams: An Introduction to Online Robots, K Goldberg and

R Siegwart, Eds Cambridge, MA: MIT Press, 2002

[4] K Taylor and B Dalton, “Issues in Internet telerobotics,” in Proc Int Conf Field and Service Robotics, Canberra, Australia, 1997, pp 151–

157

[5] C Sayers, Remote Control Robotics New York: Springer Verlag, 1998 [6] http://cwis.usc.edu/dept/garden/

[7] Eric Paulos and John Canny, “Delivering real reality to the World Wide Web via telerobotics,” Proceedings of the 1996 IEEE International Conference on Robotics and Automation, April 1996, pp.1694-1699 [8] Songmin Jia and Kunikatsu Takase, Internet-Based Robotic System Using CORBA as Communication Architecture, Journal of Intelligent and Robotic Systems 34: 121–134, 2002

[9] Dawei Wang, Jianqiang Yi, Dongbin Zhao and Guosheng Yang,

“Teleoperation System of the Internet-based Omnidirectional Mobile Robot with A Mounted Manipulator,” Proceedings of the 2007 IEEE International Conference on Mechatronics and Automation, August 5 -

8, 2007

[10] Peter X Liu, Max Q.-H Meng, Polley R Liu, and Simon X Yang, “An End-to-End Transmission Architecture for the Remote Control of Robots Over IP Networks,” IEEE/ASME transactions on mechatronics, Vol 10,

No 5, October 2005

[11] Raul Wirza et al., “End-to-end congestion control protocols for remote programming of robots, using heterogeneous networks: A comparative analysis,” Journal of Robotics and Autonomous Systems 56 (2008) 865–

874

[12] http://www.drrobot.com/products/item_downloads/Sputnik_2.pdf [13] Stormy Attaway, “Matlab: A Practical Introduction to Programming and Problem Solving,” Butterworth-Heinemann Publisher, February 2009 [14] G Reese, “Database Programming with JDBC and Java,” Second edtion, O’Reilly & Associates, November 2000

[15] Steve Brown, “Visual Basic 6 Complete,” Sybex Publisher, March 1999 [16] J Postel, RFC 768: “User Datagram Protocol,” 1980

[17] J Postel, RFC 793: “Transmission Control Protocol,” DARPA Internet Program Protocol Specification, 1981