Integrated Research in GRID Computing- P10 pps

New Grid Monitoring Infrastructures 169 system and in fact needs knowledge about the semantics of the monitored data we have proposed an external module to Mercury called Event Monitor

New Grid Monitoring Infrastructures 169 system and in fact needs knowledge about the semantics of the monitored data

we have proposed an external module to Mercury called Event Monitor (EM)

In a nutshell, EM implements more sophisticated push mechanisms as it is highlighted in Fig 4 Event Monitors allow clients dynamic management and control of event-like metrics as very useful information providers for clients or management systems We see many real scenarios in which an external client wants to have access to metrics described in the previous section (regardless of their type) and additionally, often due to performance reasons, does not want

to constantly monitor their values

Internet

FIREWALL

•••[ client or

I grid middleware

\ RMS I [ MM j

T Gateway

\ (public IP)

Figure 4 Event Monitors as external Mercury modules for event-like monitoring of resources

and applications

Nowadays, policy-driven change and configuration management that can dynamically adjust the size, configuration, and allocation of resources are be-coming extremely important issues In many real use cases, a resource manage-ment system may want to take an action according to predefined managemanage-ment rules or conditions For example, when application progress reaches a certain level, the process memory usage becomes too high or dedicated disc quota is exceeded Event Monitor was developed to facilitate such scenarios Its main functionality is to allow an external client to register a metric in Event Monitor for receiving appropriate notifications when certain conditions are met Strictly speaking, clients can setup an appropriate frequency (a default one has been set to 5 seconds) of Event Monitor requests to LM They can also use a pre-defined standard relational operator (greater than, less than, etc.) and different values of metrics to define various rules and conditions Example EM rules for fine-grained enforcement of resource usage or application control are presented below:

• Example application oriented rules in Event Monitor:

a p p p r i v j o b i d L O A D ( t i d ) > 0 8

170 INTEGRATED RESEARCH IN GRID COMPUTING

app.priv.jobid.MEMORY(tid) > 100MB

app.priv.jobid.PROGRESS(tid) > 90

• Example host oriented rules in Event Monitor:

host.loadavgS(host) > 0.5

host.net.total.error(host, interface) > 0

When the condition is fulfilled Event Monitor can generate an event-like message and forward it to interested clients subscribed at the Mercury Main Monitor component - MM Note that any metric, host or application specific, that returns a numerical value or a data type that can be evaluated to a simple numerical value (e.g a record or an array) can be monitored this way

In fact, four basic steps must be taken in order to add or remove a new rule/condition to Event Monitor First of all, the client must discover a metric in Mercury using its basic features Then it needs to specify both a relation operator and a value in order to register a rule in Event Monitor After successfully

registering the rule in Event Monitor, a unique identifier (called Event ID) is

assigned to the monitored metric To start the actual monitoring, the commit control of Event Monitor on the same host has to be executed Eventually, the client needs to subscribe to listen to the metric (with no target host specified)

through Main Monitor and wait for the event with the assigned Event ID to

occur

6 Example Adaptive Multi-criteria Resource

Management Strategic

The efficient management of jobs before their submission to remote do-mains often turns out to be very difficult to achieve It has been proved that more adaptive methods, e.g rescheduling, which take advantage of a migra-tion mechanism may provide a good way of improving performance[6] [7] [8] Depending on the goal that is to be achieved using the rescheduling method, the decision to perform a migration can be made on the basis of a number of events For example the rescheduling process in the GrADS project consists

of two modes: migrate on request (if application performance degradation is unacceptable) and opportunistic migration (if resources were freed by recently completed jobs)[6] A performance oriented migration framework for the Grid, described in[8], attempts to improve the response times for individual applica-tions Another tool that uses adaptive scheduling and execution on Grids is the GridWay framework[7] In the same work, the migration techniques have been classified into the application-initiated and grid-initiated migration The former category contains the migration initiated by application performance degrada-tion and the change of applicadegrada-tion requirements or preferences (self-migradegrada-tion)

New Grid Monitoring Infrastructures 111

The grid-initiated migration may be triggered by the discovery of a new, better resource (opportunistic migration), a resource failure (failover migration), or

a decision of the administrator or the local resource management system Re-cently, we have demonstrated that checkpointing, migration and rescheduling methods could shorten queue waiting times in the Grid Resource Management System (GRMS) and, consequently, decrease the application response times[9]

We have explored a migration that was performed due to the insufficient amount

of free resources required by incoming jobs Application-level checkpointing has been used in order to provide full portability in the heterogeneous Grid environment In our tests, the amount of free physical memory has been used

to determine whether there are enough available resources to submit the pend-ing job Nevertheless, the algorithm is generic, so we have easily incorporated other measurements and new Mercury monitoring capabilities described in pre-vious two sections Based on new sensor-oriented features provided by Event Monitor we are planning to develop a set of tailor-made resource management strategies in GRMS to facilitate the management of distributed environments

?• Preliminary Results and Future Work

We have performed our experiments in a real testbed connecting two clus-ters over the Internet located in different domains The first one consists of 4 machines (Linux 2-CPU Xeon 2,6GHz), and second consists of 12 machines (Linux 2-CPU Pentium 2,2 GHz) The average network latency time between these two clusters was about 70ms

_0 4

a 3

a

c

I 1

•o 0

Average additional CPU load generated by Mercury and Event Monitor

- i

Event Monitor triggers

Mercury metric calls (LM)

Mercury metric calls (MM)

Figure 5 Performance costs of Mercury and Event Monitor

In order to test capabilities as well as performance costs of Mercury and Event Monitors running on testbed machines we have developed a set of exam-ple MPI applications and client tools As it is presented in Fig 5 all control, monitoring and event-based routines do not come at any significant perfor-mance Additional CPU load generated during 1000 client requests per minute did not exceed ca 3% and in fact was hard to observe on monitored hosts

172 INTEGRATED RESEARCH IN GRID COMPUTING

Additional memory usage of Mercury and Event Monitor was changing from

2 to 4 MB on each host

Average response time of example

application metrics

10 11 12

MPI processes -checkpoint —-progress —whereami

Average local and remote response time

of example host metrics

I 0,4

C 0,2

o S- 0.1

host.load

REMOTE LOCAL

host, mem free

Figure 6

Monitor

Response times of basic monitoring operations performed on Mercury and Event

In our tests we have been constantly querying Mercury locally from many client tools and the average response time of all host metrics monitored on various hosts was stable and equaled approximately 18 ms Remote response times as we expected were longer due to network delays (70ms) The next figure shows us results of application oriented metrics which have been added

in various testing MPI applications The important outcome is that the response time (less than 1 second) did not increase significantly when more MPI processes were used, what is important especially to adopt monitoring capabilities for large scale experiments running on much bigger clusters

All these performance tests have proved efficiency, scalability and low in-trusiveness of both Mercury and Event Monitor and encouraged us for further research and development Currently, as it was mentioned in Sect 5, Event Monitor works as an external application as far as Mercury's viewpoint is con-cerned but this does not restrict its functionality However, in the future it may become more tightly integrated with the Mercury system (e.g as a Mercury module) due to performance and maintenance reasons To facilitate integration

of Mercury and Event Monitor with external clients or grid middleware ser-vices, in particular GRMS, we have also developed the JEvent-monitor-client package in Java which provides a higher level interface as a simple wrapper based on the low-level metric/control calls provided by Mercury API Addi-tionally, to help application developers we have developed easy-to-use libraries which connect applications to Mercury and allow them to take advantage of mentioned monitoring capabilities

New Grid Monitoring Infrastructures 173

Acknowledgments

Most of presented work has been done in the scope of CoreGrid project This

project is founded by EU and aims at strengthening and advancing scientific

and technological excellence in the area of Grid and Peer-to-Peer technologies

References

[1] http://www.gridlab.org

[2] http://glite.web.cern.ch/glite/

[3] http://www.globus.org

[4] http://www.gridlab.org/grms/

[5] G Gombas and Z Balaton "A Flexible Multi-level Grid Monitoring Architecture", In

Proceedings of 1st European Across Grids Conference, Santiago de Compostela, Spain,

2003 Volume 2970 of Lecture Notes in Computer Science, p 214-221

[6] K Cooper et al., "New Grid Scheduling and Rescheduling Methods in the GrADS Project",

In Proceedings of Workshop for Next Generation Software (held in conjunction with the

IEEE International Parallel and Distributed Processing Symposium 2004), Santa Fe, New

Mexico, April 2004

[7] E Huedo, R Montero and I Llorente, "The GridWay Framework for Adaptive Scheduling

and Execution on Grids", In Proceedings of AGridM Workshop (in conjunction with the

12th PACT Conference, New Orleans (USA)), Nova Science, October 2003

[8] S Vadhiyar and J Dongarra, "A Performance Oriented Migration Framework For The

Grid", In Proceedings of CCGrid, IEEE Computing Clusters and the Grid, CCGrid 2003,

Tokyo, Japan, May 12-15, 2003

[9] "Improving Grid Level Throughput Using Job Migration and Rescheduling Techniques

in GRMS Scientific Programming", Krzysztof Kurowski, Bogdan Ludwiczak, Jaroslaw

Nabrzyski, Ariel Oleksiak, Juliusz Pukacki, lOS Press Amsterdam The Netherlands 12:4

(2004) 263-273

[10] M Gerndt et al., "Performance Tools for the Grid: State of the Art and Future",

Re-search Report Series, Lehrstuhl fuer Rechnertechnik und Rechnerorganisation

(LRR-TUM) Technische Universitaet Muenchen, Vol 30, Shaker Verlag, ISBN 3-8322-2413-0,

2004

[11] Serafeim Zanikolas and Rizos Sakellariou, "A Taxonomy of Grid Monitoring Systems

in Future Generation Computer Systems", volume 21, p 163-188, 2005, Elsevier, ISSN

0167-739X

TOWARDS SEMANTICS-BASED

RESOURCE DISCOVERY FOR THE GRID*

William Groleau^

Institut National des Sciences Appliquees de Lyon (INSA),

Lyon, France

william.groleau@insa-lyon.fr

Vladimir Vlassov

Royal Institute of Technology (KTH),

Stockholm, Sweden

vlad@it.l<th.se

Konstantin Popov

Swedish Institute of Computer Science (SICS),

Kista, Sweden

kost@sics.se

Abstract We present our experience and evaluation of some of the state-of-the-art software

tools and algorithms available for building a system for Grid service provision and discovery using agents, ontologies and semantic markups We believe that semantic information will be used in every large-scale Grid resource discovery, and the Grid should capitalize on existing research and development in the area

We built a prototype of an agent-based system for resource provision and selection that allows locating services that semantically match the client requirements Services are described using the Web service ontology (OWL-S) We present our prototype built on the JADE agent framework and an off-the-shelf OWL-S toolkit

We also present preliminary evaluation results, which in particular indicate a need for an incremental classification algorithm supporting incremental extension of

a knowledge base with many unrelated or weakly-related ontologies

Keywords: Grid computing, resource discovery, Web service ontology, semantics

*This research work is carried out under the FP6 Network of Excellence CoreGRID funded by the European Commission (Contract IST-2002-004265)

^The work was done when the author was with the KTH, Stockholm, Sweden

176 INTEGRATED RESEARCH IN GRID COMPUTING

1 Introduction

The Grid is envisioned as an open, ubiquitous infrastructure that allows treating all kinds of computer-related services in a standard, uniform way Grid services need to have concise descriptions that can be used for service location and composition The Grid is to be become large, decentralized and heteroge-neous These properties of the Grid imply that service location, composition and inter-service communication needs to be sufficiently flexible since services being composed are generally developed independently of each other [4,3], and probably do not match perfectly This problem should be addressed by using semantic, self-explanatory information for Grid service description and inter-service communication [3], which capitalizes on the research and development

in the fields of multi-agent systems and, more recently, web services [1]

We believe that basic ontology- and semantic information handling will be

an important part of every Grid resource discovery, and eventually - service composition service [2, 6, 20] W3C contributes the basic standards and tools,

in particular the Resource Description Framework (RDF), Web Ontology Lan-guage (OWL) and Web service ontology (OWL-S) [21] RDF is a data model for entities and relations between them OWL extends RDF and can be used

to explicitly represent the meaning of entities in vocabularies and the relations between those entities OWL-S defines a standard ontology for description of Web services Because of the close relationship between web- and Grid ser-vices, and in particular - the proposed convergence of these technologies in the more recent Web Service Resource Framework (WSRF), RDF, OWL and OWL-S serve as the starting point for the "Semantic Grid" research

In this paper we present our practical experience and evaluation of the state-of-the-art semantic-web tools and algorithms We built an agent-based resource provision and selection system that allows locating available services that se-mantically match the client requirements Services are described using the Web service ontology (OWL-S), and the system matches descriptions of ex-isting services with service descriptions provided by clients We extend our previous work [12] by deploying semantic reasoning on service descriptions

We attempted to implement and evaluate matching of both descriptions of ser-vices from the functional point of view (service "profiles" in the OWL-S ter-minology), and descriptions of service structure (service "models"), but due to technical reasons succeeded so far only with the first

The remainder of the paper is structured as follows Section 2 presents some background information about semantic description of Grid services and matchmaking of services The architecture of the agent-based system for Grid service provision and selection is presented in Section 3 Section 4 describes implementation of the system prototype, whereas Section 5 discusses evaluation

of the prototype Finally, our conclusions and future work are given in Section 6

Towards Semantics-Based Resource Discovery for the Grid 111

2 Background

2,1 Semantic Description of Grid Services

The Resource Description Framework (RDF) is the foundation for OWL

and OWL-S RDF is a language for representing information about resources

(metadata) on the Web RDF provides a common framework for expressing

this information such that it can be exchanged without loss 'Things" in RDF

are identified using Web identifiers (URIs) and described in terms of simple

properties and property values RDF provides for encoding binary relations

between a subject and an object Relations are "things" on their own, and can

be described accordingly There is an XML encoding of RDF

RDF Schema can be used to define the vocabularies for RDF statements

RDF Schema provides the facilities needed to describe application-specific

classes and properties, and to indicate how these classes and properties can

to be used together RDF Schema can be seen as a type system for RDF

RDF Schema allows to define class hierarchies, and declare properties that

characterize classes Class properties can be also sub-typed, and restricted with

respect to the domain of their subjects and the range of their objects RDF

Schema also contains facilities to describe collections of entities, and to state

information about other RDF statements

OWL [13] is a semantic markup language used to describe ontologies in

terms of classes that represent concepts or/and collection of individuals,

indi-viduals (instances of classes), and properties OWL goes beyond RDF Schema,

and provides means to express relations between classes such as '^disjoint",

car-dinality constraints, equality, richer typing of properties etc There are three

versions of OWL: "Lite", "DL", and "Full"; the first two provide

computation-ally complete reasoning In this work we need the following OWL elements:

• owl: Class defines a concept in the ontology (e.g

<owl:Class rdf:ID="Winery7>)\

• rdfs.'subClassOf relates a more specific class to a more general class;

• rdfs:equivalentClass defines a class as equivalent to another class

OWL-S [14] defines a standard ontology for Web services It comprises

three main parts: the profile, the model and the grounding The service profile

presents "what the service does" with necessary functional information: input,

output, preconditions, and the effect of the service The service model describes

"how the service works", that is all the processes the service is composed of, how

these processes are executed, and under which conditions they are executed

The process model can hence be seen as a tree, where the leaves are the atomic

processes, the interior nodes are the composite processes, and the root node is

the process that starts execution of the service

178 INTEGRATED RESEARCH IN GRID COMPUTING

<ions:LangInput rfd:ID="InputLanguage">

<process:parameterType rdf:resource=

*'http://www.mindswap.org/2004/owl-s/l 1/BabelFishTranslator" />

</ion s: Langlnput>

Figure 1 Definition of an OWL-S service parameter

An example definition of an OWL-S service input parameter is shown in

Fig-ure 1 In this example, the concept attached to the parameter InputLanguage is

SupportedLanguage, found in the ontology

http://www.mindswap.org/2004/owl-s/l.l/BabelFishTranslator.owl The class of the parameter is Langlnput, which has been defined as a subclass of Input (predefined in the OWL-S ontology) in the namespace ions

Few basic OWL-S elements need to be considered by matchmakers:

• profile:Profile defines the service profile that includes a textual

descrip-tion of the service, references to the model, etc., and a declaradescrip-tion of the parameters:

~ profile: has Input / profile :hasOutput

• process:Input / process:Output defines the parameters previously

de-clared in the profile, and mostly contains the following elements:

- process:parameterType which defines the type of the parameter

Note that inputs can be defined by process:input ox process:output or by any

subclass of input or output, as in our example Figure L Moreover a profile can

also be defined by a subclass of pro file: Pro file

2.2 Matching Services

Matchmaking is a common notion in multi-agent systems It denotes the process of identifying agents with similar capabilities [11] Matchmaking for

Web Services is based on the notion of similar services [16] since it is unre-alistic to expect services to be exactly identical The matchmaking algorithms

proposed in [19, 8,16] calculate a degree of resemblance between two services Services can be matched by either their OWL-S profiles or OWL-S mod-els [17] In this work we consider only matching service profiles leaving match-ing of service models to our future work Matchmatch-ing service profiles can include matching (1) service functionalities and (2) functional attributes The latter is exemplified by the ATLAS matchmaker [17] We focus on matching service functionalities as, in our view, it is more important than matching functional attributes The idea of matching capabilities of services described in OWL-S

Towards Semantics-Based Resource Discovery for the Grid 179

using the profiles has been approached first in [16] and refined in [19, 8] We use

the latter extension in our work as it allows more precise matchmaking by

tak-ing into account more elements of OWL-S profiles Other solutions such as the

ATLAS matchmaker [17], are more focused in matching functional attributes

and do not appear to be as complete as the one we use

Our profile matchmaker compares inputs and outputs of request and

adver-tisement service descriptions, and includes matching of the profile types A

service profile can be defined as an instance of a subclass of the class Profile,

and included in a concept hierarchy (the OWL-S ServiceCategory element is not

used in our prototype) When two parameters are being matched, the relation

between the concepts linked to the parameters is evaluated (sub/super-class,

equivalent or disjoint) This relation is called "concept match" In the

exam-ple in Figure 1, SupportedLanguage would be the concept to match Next, the

relation existing between the parameter property classes is evaluated

(sub/super-property, equivalent, disjoint or unclassified) This relation is called ^'property

match" In the example in Figure 1, Langlnput would be the property to match

The final matching score assigned for two parameters is the combination of

the scores obtained in the concept and property matches, as shown in Table 1

Finally, the matching algorithm computes aggregated scores for outputs and

inputs, as shown below for outputs:

min(max(scoreMatch{outputAdv^ outputReq)

\outputAdv E AdvOutputs)

\outputReq G ReqOutputs) scoreMatch is the combination score of the "concept match" and "property

match" results (see Table 1); AdvOutputs is the list of all outputs parameters

of the provided service; reqOutputs is the list of all outputs parameters of the

requested service (requested outputs) The algorithm identifies outputs in the

provided service that match outputs of the requested service with the maximal

score, and finally determines the pair of outputs with the worst maximal score

For instance, the score will be sub-class if all outputs of the advertised service

perfectly match the requested outputs, except for one output which is a sub-class

of its corresponding output in the requested service (if we neglect the ^'property

match" score) A similar aggregated score is computed also for inputs

The final comparison score for two services is the weighted sum of outputs-,

inputs- and profile matching scores Typically, outputs are considered most

important ([16]) and receive the largest weight The profile matchmaker returns

all matching services sorted by the final scores

When a requestor does not want to disclose to providers too much information

about the requested service, the requestor can specify only the service category