DISTRIBUTED AND PARALLEL SYSTEMSCLUSTER AND GRID COMPUTING 2005 phần 1 ppsx

DISTRIBUTED AND PARALLEL SYSTEMS CLUSTER ANDGRID COMPUTING... Part I Grid Systems glogin - Interactive Connectivity for the Grid Herbert Rosmanith and Jens Volkert Parallel Program Execu

DISTRIBUTED AND PARALLEL SYSTEMS CLUSTER AND

GRID COMPUTING

THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE

DISTRIBUTED AND PARALLEL SYSTEMSCLUSTER AND

eBook ISBN: 0-387-23096-3

Print ISBN: 0-387-23094-7

Print © 2005 Springer Science + Business Media, Inc.

All rights reserved

No part of this eBook may be reproduced or transmitted in any form or by any means, electronic, mechanical, recording, or otherwise, without written consent from the Publisher

Created in the United States of America

Boston

©200 5 Springer Science + Business Media, Inc.

Visit Springer's eBookstore at: http://www.ebooks.kluweronline.com

and the Springer Global Website Online at: http://www.springeronline.com

Part I Grid Systems

glogin - Interactive Connectivity for the Grid

Herbert Rosmanith and Jens Volkert

Parallel Program Execution Support in the JGrid System

Szabolcs Pota, Gergely Sipos, Zoltan Juhasz and Peter Kacsuk

VL-E: Approaches to Design a Grid-Based Virtual Laboratory

Vladimir Korkhov, Adam Belloum and L.O Hertzberger

Scheduling and Resource Brokering within the Grid Visualization Kernel

Paul Heinzlreiter, Jens Volkert

Part II Cluster Technology

Message Passing vs Virtual Shared Memory, a Performance Comparison

Wilfried N Gansterer and Joachim Zottl

MPI-I/O with a Shared File Pointer Using a Parallel Virtual File System

Yuichi Tsujita

An Approach Toward MPI Applications in Wireless Networks

Elsa M Macías, Alvaro Suárez, and Vaidy Sunderam

Deploying Applications in Multi-SAN SMP Clusters

Albano Alves, António Pina, José Exposto and José Rufino

13

21

29 3

39

47

55

63

Part III Programming Tools

Monitoring and Program Analysis Activities with DeWiz

Rene Kobler, Christian Schaubschläger, Bernhard Aichinger,

Dieter Kranzlmüller, and Jens Volkert

Integration of Formal Verification and Debugging Methods in

P-GRADE Environment

Róbert Lovas, Bertalan Vécsei

Tools for Scalable Parallel Program Analysis - Vampir NG and DeWiz

Holger Brunst, Dieter Kranzlmüller, Wolfgang E Nagel

Process Migration In Clusters and Cluster Grids

József Kovács

Part IV P-GRADE

Graphical Design of Parallel Programs With Control Based on Global

Application States Using an Extended P-GRADE Systems

M Tudruj, J Borkowski and D Kopanski

Parallelization of a Quantum Scattering Code using P-GRADE

Ákos Bencsura and György Lendvay

Traffic Simulation in P-Grade as a Grid Service

T Delaitre, A Goyeneche, T Kiss, G Terstyanszky, N Weingarten,

P Maselino, A Gourgoulis, and S C Winter.

Development of a Grid Enabled Chemistry Application

István Lagzi, Róbert Lovas, Tamás Turányi

Part V Applications

Supporting Native Applications in WebCom-G

John P Morrison, Sunil John and David A Power

Grid Solution for E-Marketplaces Integrated with Logistics

L Bruckner and T Kiss

Incremental Placement of Nodes in a Large-Scale Adaptive Distributed

vii Component Based Flight Simulation in DIS Systems

Krzysztof Mieloszyk, Bogdan Wiszniewski

A Concurrent Implementation of Simulated Annealing and Its Application

to the VRPTW Optimization Problem

Agnieszka Debudaj-Grabysz and Zbigniew J Czech

201

This page intentionally left blank

DAPSYS (Austrian-Hungarian Workshop on Distributed and Parallel tems) is an international conference series with biannual events dedicated toall aspects of distributed and parallel computing DAPSYS started under a dif-ferent name in 1992 (Sopron, Hungary) as a regional meeting of Austrian andHungarian researchers focusing on transputer-related parallel computing; a hotresearch topic of that time A second workshop followed in 1994 (Budapest,Hungary) As transputers became history, the scope of the workshop widened

Sys-to include parallel and distributed systems in general and the DAPSYS in

1996 (Miskolc, Hungary) reflected the results of these changes Since then,DAPSYS has become an established international event attracting more andmore participants every second year After the successful DAPSYS’98 (Bu-dapest) and DAPSYS 2000 (Balatonfüred), DAPSYS 2002 finally crossed theborder and visited Linz, Austria

The fifth DAPSYS workshop is organised in Budapest, the capital of gary, by the MTA SZTAKI Computer and Automation Research Institute As

Hun-in 2000 and 2002, we have the privilege agaHun-in to organise and host DAPSYStogether with the EuroPVM/ MPI conference While EuroPVM/MPI is dedi-cated to the latest developments of the PVM and MPI message passing envi-ronments, DAPSYS focuses on general aspects of distributed and parallel sys-tems The participants of the two events will share invited talks, tutorials andsocial events fostering communication and collaboration among researchers

We hope the beautiful scenery and rich cultural atmosphere of Budapest willmake it an even more enjoyable event

Invited speakers of DAPSYS and EuroPVM/MPI 2004 are Al Geist, JackDongarra, Gábor Dózsa, William Gropp, Balázs Kónya, Domenico Laforenza,Rusty Lusk and Jens Volkert A number of tutorials extend the regular program

of the conference providing an opportunity to catch up with latest

develop-ments: Using MPI-2: A Problem-Based Approach (William Gropp and Ewing Lusk), Interactive Applications on the Grid - the CrossGrid Tutorial (Tomasz Szepieniec, Marcin Radecki and Katarzyna Rycerz), Production Grid systems

and their programming (Péter Kacsuk, Balázs Kónya, Péter Stefán).

The DAPSYS 2004 Call For Papers attracted 35 submissions from 15 tries On average we had 3.45 reviews per paper The 23 accepted papers cover

coun-a brocoun-ad rcoun-ange of resecoun-arch topics coun-and coun-appecoun-ar in six conference sessions: GridSystems, Cluster Technology, Programming Tools, P-GRADE, Applicationsand Algorithms

The organisation of DAPSYS could not be done without the help of manypeople We would like to thank the members of the Programme Committeeand the additional reviewers for their work in refereeing the submitted papers

x DISTRIBUTED AND PARALLEL SYSTEMS

and ensuring the high quality of DAPSYS 2004 The local organisation wasmanaged by Judit Ajpek from CongressTeam 2000 and Agnes Jancso fromMTA SZTAKI Our thanks is due to the sponsors of the DAPSYS/EuroPVMjoint event: IBM (platinum), Intel (gold) and NEC (silver)

Finally, we are grateful to Susan Lagerstrom-Fife and Sharon Palleschi fromKluwer Academic Publishers for their endless patience and valuable support inproducing this volume, and David Nicol for providing the WIMPE conferencemanagement system for conducting the paper submission and evaluation

D IETER K RANZLMÜLLER

P ÉTER K ACSUK

Z OLTÁN J UHÁSZ

Program Committee

M Baker (Univ of Portsmouth, UK)

L Böszörményi (University Klagenfurt, Austria)

M Bubak (CYFRONET, Poland)

Y Cotronis (University of Athens, Greece)

J Cunha (Universita Nova de Lisboa, Portugal)

B Di Martino (Seconda Universita’ di Napoli, Italy)

J Dongarra (Univ of Tennessee, USA)

G Dozsa (MTA SZTAKI, Hungary)

T Fahringer (Univ Innsbruck, Austria)

A Ferscha (Johannes Kepler University Linz, Austria)

A Frohner (CERN, Switzerland)

M Gerndt (Tech Univ of Munich, Germany)

A Goscinski (Daekin University, Australia)

G Haring (University of Vienna, Austria)

L Hluchy (II SAS, Slovakia)

Z Juhász (University of Veszprem, Hungary)

P Kacsuk (MTA SZTAKI, Hungary)

K Kondorosi (Technical University of Budapest, Hungary)

B Kónya (Univ of Lund, Sweden)

H Kosch (University Klagenfurt, Austria)

G Kotsis (University of Vienna, Austria)

D Kranzlmüller (Johannes Kepler University Linz, Austria)

D Laforenza (CNUCE-CNR, Italy)

E Laure (CERN, Switzerland)

T Margalef (UAB, Spain)

L Matyska (Univ of Brno, Czech Rep)

Zs Németh (MTA SZTAKI, Hungary)

T Priol (INRIA, France)

W Schreiner (University of Linz, Austria)

F Spies (University de Franche-Comte, France)

P Stefán (NIIFI, Hungary)

V Sunderam (Emory University, USA)

I Szeberényi (Tech Univ of Budapest, Hungary)

G Terstyánszky (Westminster University, UK)

M Tudruj (IPI PAN / PJWSTK, Poland)

F Vajda (MTA SZTAKI, Hungary)

J Volkert (Johannes Kepler University Linz, Austria)

S Winter (Westminster University, UK)

R Wismüller (Technische UniversitäT München, Germany)

This page intentionally left blank

GRID SYSTEMS

This page intentionally left blank

GLOGIN - INTERACTIVE CONNECTIVITY

FOR THE GRID*

Herbert Rosmanith and Jens Volkert

GUP, Joh Kepler University Linz

Altenbergerstr 69, A-4040 Linz, Austria/Europe

through-livered for post-mortem analysis The glogin tool provides a novel approach for

grid applications, where interactive connections are required With the solution

implemented in glogin, users are able to utilize the grid for interactive

applica-tions much in the same way as on standard workstaapplica-tions This opens a series of new possibilities for next generation grid software.

grid computing, interactivity

Grid environments are todays most promising computing infrastructures forcomputational science [FoKe99], which offer batch processing over networkedresources However, even in a grid environment, it may sometimes be neces-sary to log into a grid node Working on a node with an interactive command-shell is much more comfortable for many tasks For example, one might want

to check the log files of a job Without an interactive shell, it would be sary to submit another job for the same result This is much more impracticalthan interactive access to the system

neces-Today, the administrators of such grid nodes accommodate this by givingtheir users UNIX accounts This has some disadvantages Firstly, user ad-ministration also has to be done on the UNIX level This is an unnecessaryadditional expense, since – from the grid point of view – we are already able

to identify the users by examining their certificates Secondly, access to shell

*This work is partially supported by the EU CrossGrid project, “Development of Grid Environment for Interactive Applications”, under contract IST-2001-32243.

4 DISTRIBUTED AND PARALLEL SYSTEMS

functionality like telnet or even secure shell [Ylon96], may be blocked by wall administrators This leads to configurations where users are given ac-counts on multiple machines (one without the administrative restrictions of aprohibitive network configuration) only to be able to bounce off to the finalgrid node No need to say, that this is a very uncomfortable situation for boththe users and the administrators

fire-The above mentioned problem is addressed in this paper by focusing on thefollowing question: Is there a way to somehow connect to the grid node? Theresulting solution as described below is based on the following idea: in order

to submit jobs, one has to be able to at least contact the gatekeeper Why don’t

we use this connection for the interactive command-shell we desire? The way

to do this is described in this paper and has been implemented as the prototype

tool glogin1

As we work with our shell, we will recognise that we have got “true activity” in the grid Keystrokes are sent to the grid-node only limited by thespeed of the network Based on this approach, we might now ask how we cancontrol any interactive grid-application, not just shells

inter-This paper is organised as follows: Section 2 provides an overview of theapproach: it shows how to overcome the limitations of the Globus-gatekeeperand get interactive connections In Section 3, the details of how to establish

a secure interactive connection and how to run interactive commands (such asshells and others) are shown Section 4 compares related work in this area,before an outlook on future work concludes this paper

Limitations of Globus-Gatekeeper

As of today, various implementations of grid-middleware exist However,

glogin has been developed for the Globus-Toolkit [GTK], an open source

soft-ware toolkit used for building grids GT2 is the basic system used in severalgrid-projects, including the EU CrossGrid project [Cros01]

A central part of GT is the Globus-gatekeeper which was designed for abatch-job-entry system As such, it does not allow for bidirectional communi-cation as required by an interactive shell Looking at the Globus programmingAPI, we have to understand that the connection to the Globus-gatekeeper al-lows transportation of data in one direction only This is done by the GlobusGASS server, a https-listener (Globus transfers all data by means of http/s),which is set up as part of the application, reads data from the gatekeeper anddelivers it to the standard output file descriptor A possibility for transportingdata in the opposite direction using the currently established gatekeeper–GASSserver connection is not available

glogin - Interactive Connectivity for the Grid 5

In addition, there is another batch-job-attribute of the Globus-gatekeeperwhich turns out to be preventing the implementation of an interactive shell

It has been observed that data sent from the grid is stored into the so called

“GASS cache” There seem to be two different polling intervals at which it

is emptied: If a program terminates fast enough, the GASS cache is emptied

at program termination time, otherwise, the GASS cache is emptied every 10seconds, which means that the data in the cache will be stored past programtermination for 10 seconds at worst As of Globus-2.4, there is no API call toforce emptying the cache Thus, if one needs an interactive shell, a differentapproach has to be used

An example demonstrates this situation Assuming we have a shell scriptnamed “count.sh”, which outputs an incremented number every second:

If we start this job via the Globus-gatekeeper, we will see nothing for thefirst 10 seconds, then, all at once, the numbers from 0 to 9 will be displayed,followed by a another 10 second pause, after which the numbers from 10 to 19will be displayed and so on until we terminate the job

Getting Interactive Connections

The solution is as follows: since the connection between the GASS serverand Globus-gatekeeper can only be used for job-submission, a separate con-nection has to be created Once the remote program has been started on thegrid-node, it has to take care of communication itself2 Figure 1 shows thesteps performed when creating a separate connection

(1)

(2)

(3)

(4)

the requesting client contacts the gatekeeper

the gatekeeper starts the requested service on the same node via fork()the requested service creates a listener socket

the requesting client directly contacts the requested service

A direct connection without the Globus-gatekeeper’s interference between theclient and the service has now been established Interactive data exchange be-tween the peers can now take place Since both peers make use of the Globus-software, they can establish a secure connection easily

We have to be aware that this approach only works with the fork at thegatekeeper machine At the moment, the requested service is required to run

6 DISTRIBUTED AND PARALLEL SYSTEMS

Figure 1 Setting up a separate connection

on the same machine the gatekeeper is on It is currently not possible thatthe requested service is started at some node “behind” the gatekeeper Sincethe “worker nodes” can be located in a private network [Rekh96], connectionestablishment procedure would have to be reversed However, if we limit our-selves to passing traffic from the (private) worker-nodes to the requesting clientvia the gatekeeper, we could use traffic forwarding as described below

For ease of implementation and for ease of use, glogin is both the client and the service In (1), glogin contacts the Globus-gatekeeper by using the Globus job

submission API and requests that a copy of itself is started in (2) on the

grid-node glogin has an option to differentiate between client and service mode.

By specifying -r, glogin is instructed to act as the remote part of the connection.

How does the client know where to contact the service?

With “contact”, we mean a connection In (3), the service creates a listener and waits for a connection coming from the client in (4) Therefore ithas to somehow communicate its own port-number where it can be reached to