DSpace at VNU: Control of Internet-based robot systems using multi transport protocols

Control of Internet-based Robot Systems Using Multi Transport Protocols Manh Duong Phung, Thuan Hoang Tran, Thanh Van Thi Nguyen and Quang Vinh Tran Department of Electronics and Compu

Control of Internet-based Robot Systems Using Multi

Transport Protocols

Manh Duong Phung, Thuan Hoang Tran, Thanh Van Thi Nguyen and Quang Vinh Tran

Department of Electronics and Computer Engineering University of Engineering and Technology Vietnam National University, Hanoi

Abstract—This paper proposes a novel approach to implement

communication channels for Internet-based robotic systems The

exchange information between the operator and the robot is

classified into categories with specific features and requirements

Appropriate transport protocols are utilized for each set of data

The TCP is for the administrative data; the UDP is for the

control signals and the RTP is for the vision data Simulations

and experiments show that this approach strengthens advantages

of each transport protocol while maintains good performance of

the system

Keywords-Internet robot; transport protocols; mobile robot;

robot control; telerobot

I INTRODUCTION

Real-time control over the Internet is attracting more

interest based on its capability to open new types of interaction

applications such as virtual laboratory, guidance,

tele-homecare and disaster surveillance [1]-[6] It is

well-recognized that the most challenge and distinct difficulties with

these works are associated with the inevitable Internet

transmission delays, delay jitter and non-guaranteed

bandwidth There are currently two approaches to deal with

these issues

The first focuses on developing advanced remote control

algorithms and interface techniques [1]-[3] In [1], a

CORBA-based robotic system features the concept of task-level control

In this system, instead of step-by-step operating over the

Internet, a control command from the user is sent to the robot at

a task-level such as “give me the spoon”, “grasp the blue

bowl”, “coffee, please”, etc The robot then analyzes the

command, makes the path planning, completes the task and

returns the result to the user without requiring any additional

actions In [2], a virtual environment which is proportional to

the real dimension of the laboratory is constructed at the client

side Before commands are sent to the server program, they are

processed in the virtual environment to predict the upcoming

position of the robot Based on it, the system is able to tolerate

a certain amount of time delays as well as allow users to

experience the interaction with the robot as in the real

environment The idea of command filter is introduced in [3]

In the system, control commands are stored in a command

queue and outputted at each sampling time This combined

with a posture estimator enables the system to recover the lost

information of control commands caused by the Internet delay

and therefore reduce the path error between the real robot and

the virtual one Despite several advantages, this approach

avoids to cope with the key issue of control over the Internet, the communication channel, by treating the data transmission between the human operator and the remote robot as a given condition and rarely touches it

In the second approach, attempts to deal with the data communication were addressed Liu et al proposed a new transport protocol namely Trinomial [4] It is a rate-based protocol that optimizes the use of available network bandwidth and is able to adapt to the network congestion without affecting very much to the way the user teleoperates the robot An adaptive transport protocol is introduced in [5] This protocol can reconfigure itself to cater for real-time requests In [6], main transport protocols for real-time networked robots including the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), the Trinomial, the Real-Time Network Protocol (RTNP) and the Interactive Real-Time Protocol (IRTP) are simulated and compared It is concluded that each protocol has its strengths and weaknesses so that there is no single protocol can simultaneously adequate for transmitting all types of data of an online robotic system

In this paper, a multi protocol model for controlling robots over the Internet is introduced The exchange information between the robot and the operator is classified into different categories with specific features and constraints Separate transport protocols including TCP, UDP and RTP are employed to package the data and transmit them over the Internet Recovery and synchronization processes are performed at receiving ends and the data is extracted to display and execute Several simulations and experiments were carried out to evaluate the efficiency and applicability of the proposed model

The paper is organized as follows Analysis of transport protocols is described in section II The multi-protocol implementation is described in Section III Section IV introduces experiments in the real Internet environment The paper concludes with an evaluation of the system, with respect

to its strengths and weaknesses, and with suggestions of possible future developments

II TRANSPORT PROTOCOLS FOR INTERNET-BASED ROBOT

SYSTEMS

Transport protocols are protocols that operate at layer four

of the Internet layer model and provide end-to-end communication services for applications such as congestion avoidance, flow control, and multiplexing [12][13] Different

2012 International Conference on Control, Automation and Information Sciences (ICCAIS)

transport protocols such as TCP, TICP, ALCAP, and STCP

were developed for various purposes of applications [12]

Nevertheless, only protocols which are published by the

Internet Engineering Task Force (IETF) are standards for the

Internet and widely supported all over the world To ensure the

proper functioning operation, an Internet application should

only use IETF’ protocols and the most commonly ones are the

Transmission Control Protocol (TCP), the User Datagram

Protocol (UDP), and the Real-time Transport Protocol (RTP)

These protocols cover over 90% traffic of the Internet [7]-[9]

Figure 1 Characteristics of RTP, TCP and UDP in case of no network

congestion

UDP is based on the idea of sending a datagram from a device to another as fast as possible without due consideration

of the state of the network This protocol does not maintain a connection between the sender and the receiver, and it does not guarantee that the transmitted data packets will reach the destination as well as the chronological order of the data at the receiving end The main advantage of UDP is the relatively minimized transmission delay and delay jitter achieved under good network conditions It is therefore usually used for real-time data transmission

Figure 2 Characteristics of RTP, TCP and UDP in case of network

congestion

0

100

200

300

400

500

600

700

800

Time (s)

RTP UDP TCP

0

0.05

0.1

0.15

0.2

0.25

0.3

0 200 400 600 800 1000

Sequence Number

RTP TCP UDP

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 200 400 600 800 1000

Sequence Number

RTP UDP TCP

0 200 400 600 800 1000

Time (s)

TCP UDP RTP

0 0.05 0.1 0.15 0.2 0.25 0.3

0 500 1000 1500 2000

Sequence Number

TCP UDP RTP

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18

0 500 1000 1500 2000

Sequence Number

RTP UDP TCP

TCP is a more sophisticated protocol which was originally

designed for the reliable transmission of static data such as

e-mails and files over low-bandwidth, high-error-rate networks

In each transmission session, TCP establishes a virtual

connection between the sender and the receiver, performs the

acknowledgment of received data packets and implements the

retransmission mechanism when necessary TCP can also adapt

to the variation of network condition by applying congestion

control with slow start, fast recovery, fast retransmit and

window-based flow control mechanisms With these features,

TCP has been effectively used in the transmission of static

data, significantly contributing to the growth of the Internet

Nevertheless, employing the TCP for real-time data is hardly

appropriate since the timeliness of transmission outweighs the

reliability concerns so that the retransmission mechanism is

ill-suited for real-time data delivery and the strict congestion

control mechanism incurs higher delay jitter which rapidly

degrades the quality of service (QoS) in a congested network

(fig.2)

RTP is a relatively new transport protocol designed for

delivering real-time multimedia data [9][14] It bases on UDP

and has facilities for jitter compensation and detection of

out-of-sequence arrival in data, and is usually used in conjunction

with the RTP Control Protocol (RTCP) While RTP carries the

media streams (audio and video), RTCP is used to monitor

transmission statistics and information relating to the quality of

service

In order to explore the behavior of protocols in an

interactive network environment, simulations have been

conducted The widely adopted network simulation tool ns-2,

which was developed by Defense Advanced Research Projects

Agency (DARPA) through the Virtual InterNetwork Testbed

(VINT) project, was used for the simulations [11] Fig.1 shows

the simulation results of the scenario in which three sources of

traffic corresponding to RTP, UDP, and TCP are connected to

a router The router forwards the traffic to a sink through a

1.5Mbps duplex link The RTP and UDP sources send data to

the network at 0.5Mbps and the TCP source creates the traffic

based on the state of the network (fig.3)

Figure 3 Network topology in simulations

According to fig.1, all RTP, UDP and TCP flows share a

fair bandwidth of the network and introduce a common

transmission delay behavior The network jitter of the TCP

flow is however quite large in comparison to RTP and UDP

flows

Fig.2 shows the behavior of transport protocols in case of

network congestion The TCP flow cannot compete with RTP

and UDP flows to send the traffic to the network and is dropped from the network In addition, the similarity in behavior of UDP and RTP confirms that RTP is developed from the UDP

III MULTI PROTOCOL MODEL

Hardware configuration for the Internet-based robotic systems often adopts a distributed model with three modules: the robot, the network communication, and the client controller (fig.5) In order to build an appropriate control model, the communication data need to be analyzed

A Robot data analysis

Various types of information need to be exchanged between the robot and the human operator over the Internet such as credential login data, control commands, synchronous messages, and image data (fig.4) Generally, they can be grouped into three classes: the administrative data, the control signals and the vision data

The administrative data includes the access control, the user validation, and the configuration data This type of data has small packet size with the bandwidth lower than 10Kbps and is usually once-forall transmission It does not require real-time delivery but critically demands the reliability

The control signals, on the other hand, are periodic transmission They include but not limit to control commands, synchronous messages, and sensory measurements This data requires the real-time delivery and consumes the bandwidth from 1Kbps to 100Kbps

The most important and costly information is the vision data It can be used directly as the system feedback as well as the preference for recognizing algorithms The vision data requires the large packet size, periodic transmission, significant bandwidth and real-time delivery

Table.1 shows the summary of data classification and transport protocols

TABLE I D ATA CLASSIFICATION AND CORRESPONSIVE PROTOCOLS

Data Group Bandwidth

(Kbps)

Real-time Demand Feature

Transport Protocol

Administrative data (access control, user validation, configuration)

1 – 10 No One time TCP

Control signals (user commands, ultrasonic ranges, GPS signals, infrared distances)

1 – 100 Yes Periodic UDP

Live Video (camera images) 100 – 2000 Yes Periodic RTP

B Multi-protocol implementation

According to the data and transport protocol analysis, it is insufficient for a protocol to handle all the communications of the system In addition, the variety of information means different transfer modes Real-time delivery is required for all data except for the once-for-all administrative data Consequently, in the implementation, the TCP is adopted for

the communications of administrative data: A TCP connection

is opened when a teleoperation session is started and is closed

once the teleoperation is geared up The RTP protocol is

employed for the transmission of information on live scene

images The UDP is used for sensory data and robot positions

and speeds Since the human operator issues control commands

based on his/her personal judgments and decisions, the timing

of these control signals is random in nature and is controlled

solely by the human operator The transmission frequency of

control signals depends on how often the user interacts with the

mobile robot Thus, the transmission rate of control signals is

largely controlled by the human operator instead of by the

transport protocol As a result, we still utilize UDP for control

command delivery The usage of UDP is generally acceptable

because control commands are small-size packets and it should

not jeopardize inter-flow fairness if the human operator does

not issue a burst of commands when the network is heavily

congested Fig.8 describes the implementation of the

multi-protocol communication

Figure 4 Communications in an Internet-based robotic system and the

multi-protocol transmision

IV EXPERIMENTS

To verify the validity and effectiveness of the proposed

approach in control, an Internet-based robot system is

implemented in which a mobile robot is controlled over the

Internet using different transport protocols

Figure 5 Hardware configuration of an Internet-based robotic system

A Experimental setup

The Internet-based robot platform developed for experiments is shown in fig.5 [15] It consists of three components: the mobile robot, the communication channel and the client computer

The robot is a Multi-Sensor Smart Robot (MSSR) developed at our laboratory It has basic components for motor control, sensing and navigation, including battery power, drive motors and wheels, position/speed encoders, infrared sensors, integrated sonar ranging sensors, a compass sensor, a global positioning system (GPS), a laser range finder and a visual system Sensing and motor control are managed by a laptop PC placed inside the robot

The 3G mobile network is utilized as the communicating bridge between the robot and the Internet A 3G USB device with an internal mobile SIM card is used The USB is plugged into the computer inside the robot and is registered to a mobile phone service provider allowing it to have access to the Internet

The interaction devices at the user site simply consist of a personal computer and a joystick The computer is an ASUS notebook computer with 1.5GHz M-Centrino processor, 500Mb RAM and Windows XP operating system The joystick

is a 3D Logitech Extreme series with 10 bit resolution in horizontal and vertical axes and 12 functioning buttons

Figure 6 Remotely navigating the mobile robot around the laboratory

B Results

To evaluate the performance of the system, we have carried out many experiments Fig.6 shows the setup of the environment that the MSSR moves through The robot is located at the Automatic Control and Robotics Laboratory

(ACRs Lab) of the Hanoi University of Engineering and

Technology, Vietnam National University, and is controlled

over the Internet by an operator who is 12km far away The

average speed of the robot is 0.3m/s The goal of the

experiments is to remotely guide the MSSR from the starting

point Oo to the objective point Od

In the experiments, by using the proposed transport

protocol communications, the user successfully navigated the

MSSR from the point Oo to the point Od via the Internet (fig.6)

This result can be verified from the fig.7 and fig.8: an average

delay of 43ms and an average jitter of 9ms of all protocols

definitely satisfy the stable conditions of this particular setup

which can tolerate the delay and jitter up to several seconds It

is also noted from RTP in fig.8 that an average jitter of 9ms is

much better than the 15ms jitter requirement for normal video

streaming [10]

Fig.9 shows a sequence of the snapshots of the MSSR when

it was remotely being guided via the Internet to move from

point Oo to point Od in the ACRs Lab We have conducted the

experiments at different times of day: morning, noon, afternoon

and night, and tried to capture the ‘‘rush hour” of the Internet

traffic At all times, the user succeeded in navigating the

mobile robot through the ACRs Lab

Figure 7 Packet delay during the experiment

Figure 8 Delay jitter during the experiment

Figure 9 A sequence of images showing the motion of robot in a laboratory

environment during the tele-operation

We also compare between protocols by transmitting the video over the TCP (fig 10) The results show that the image quality is much lower than transmitting by RTP In addition, the dropped rate of data packets of TCP is relative high which

is 10% if the available bandwidth is 500Kbps and becomes congested if the available bandwidth goes below 200Kbps In case of UDP, the quality is acceptable but the lack of flow control mechanism makes it inappropriate for the video transmission

Figure 10 Images transmitted by TCP (a) and RTP (b)

V CONCLUSIONS

In this paper, a novel control approach for controlling Internet-based robot systems, the multi protocols, is proposed Various types of exchanged information in an online mobile robotic system are analyzed and the influence of transport protocols on the performance of system is studied Based on it, appropriate protocols are employed to transmit each class of data The result is that the operator can comfortably control a mobile robot over the Internet with good feedback and low delay

REFERENCES [1] Songmin Jia and Kunikatsu Takase, Internet-Based Robotic System Using CORBA as Communication Architecture, Journal of Intelligent and Robotic Systems 34: 121–134, 2002

[2] Dawei Wang, Jianqiang Yi, Dongbin Zhao and Guosheng Yang,

“Teleoperation System of the Internet-based Omnidirectional Mobile Robot with A Mounted Manipulator,” Proceedings of the 2007 IEEE International Conference on Mechatronics and Automation, August 5 -

8, 2007

[3] K K Han, S Kim, Y J Kim and J H Kim, “Internet Control Architecture for Internet-Based Personal Robot”, J Autonomous Robots

10, 135–147, 2001

0

20

40

60

80

100

120

140

160

0 10 20 30 40 50 60 70

Sequence number (x10 2 )

UDP RTP TCP

0

20

40

60

80

100

0 20 40 60 80 100

Sequence Number (x10 2 )

RTP TCP UDP

[4] Peter X Liu, Max Q.-H Meng, Polley R Liu, and Simon X Yang, “An

End-to-End Transmission Architecture for the Remote Control of

Robots Over IP Networks,” IEEE/ASME transactions on mechatronics,

Vol 10, No 5, October 2005

[5] Li Ping, Lu Wenjuan, Sun Zengqi, “Transport layer protocol

reconfiguration for network-based robot control system ”, Networking,

Sensing and Control, 2005 Proceedings

[6] Raul Wirz, Raul Marín, José M Claver, Manuel Ferre, Rafael Aracil,

Josep Fernández, “End-to-end congestion control protocols for remote

programming of robots using heterogeneous networks: A comparative

analysis”, J Robotics and Autonomous Systems 56, 2008

[7] J Postel, RFC 768: “User Datagram Protocol,” 1980

[8] J Postel, RFC 793: “Transmission Control Protocol,” DARPA Internet

Program Protocol Specification, 1981

[9] H Schulzrinne, S Casner, R Frederick, V Jacobson, RFC1889: “RTP:

A Transport Protocol for Real-Time Applications,” Internet Engineering

Task Force, 1996

[10] Mohamed Koubaa and Maurice Gagnaire, “A Performance Study of MPEG-4 Video Streaming in IP Networks”, European contract N° CP53

[11] The Network Simulator-ns-2 [Online]: http://www.isi.edu/nsnam/ns

[12] Behrouz Forouzan, “TCP/IP Protocol Suite “, McGraw-Hill

Science/Engineering/Math, 4th edition, 2009

[13] Kevin R Fall, W Richard Stevens, “TCP/IP Illustrated”, 2nd edition,

Addison-Wesley Professional, 2011

[14] Colin Perkins, “RTP: Audio and Video for the Internet”, Addison-Wesley Professional, 2003

[15] P M Duong, T T Hoang, N T T Van, D A Viet and T Q Vinh, “A

Novel Platform for Internet-based Mobile Robot Systems”, The 7th IEEE Conference on Industrial Electronics and Applications, Singapore, July

2012 (accepted)