Tài liệu Grid Computing P29 pptx

It then examines some lessons learned from the process of developing an implementation of a family of GridServices that conforms to the Open Grid Services Architecture OGSA [2] and the G

Unicore and the Open Grid

Services Architecture

David Snelling

Fujitsu Laboratories of Europe, Hayes, Middlesex, United Kingdom

29.1 INTRODUCTION

This chapter describes a GridService demonstrator built around the Unicore Grid envi-ronment, its architectural design and implementation [1] It then examines some lessons learned from the process of developing an implementation of a family of GridServices that conforms to the Open Grid Services Architecture (OGSA) [2] and the Grid Service Specification [3]

The goals of this project were two fold Primarily, it is only through implementation that complexities such as those that arise in OGSA can be fully understood and analyzed Secondly, Unicore (www.unicore.org) is a complete production Grid environment that seemed to conform extremely well to the OGSA The author wanted to test this hypothesis and see how difficult it would be to construct an OGSA-hosting environment on top of the Unicore infrastructure

By way of a brief summary, both goals were met easily and with satisfaction The OGSA model is remarkably clean and the abstractions well matched to what is needed

at the implementation level The lessons learned reveal where cracks exist and, in many

Grid Computing – Making the Global Infrastructure a Reality. Edited by F Berman, A Hey and G Fox

 2003 John Wiley & Sons, Ltd ISBN: 0-470-85319-0

cases, support the changes currently under discussion in the Open Grid Services Infras-tructure (OGSI) working group of the Global Grid Forum (GGF) The reader should note that this chapter and the implementation on which it is based are targeted at the first draft (2/15/2002) of the Grid service specification [3] By the time of going to the press, many

of the issues discussed here and elsewhere will have become part of the next version With respect to the second goal, the implementation of a complete OGSA GridService environment that could be used to deploy task specific GridServices was significantly easier than expected initially The combination of the Unicore Grid infrastructure with

a flexible Web Services tooling environment (GLUE from www.themindelectric.com) proved to be effective and easy to use

29.1.1 Caveats

The implementation discussed here conforms to the OGSA model, but not to the speci-fications as such The specification is still under development and requires extensions to the Web Services Definition Language (WSDL) [4] It will be some time before tooling platforms are available that support these extensions Therefore, our approach has been

to develop the architecture described by OGSA, but to work within the constraints of existing Web Services The OGSA abstractions (serviceTypes in particular) are exposed

in Java implementation, and then merged into a ‘flat’ WSDL representation, as the var-ious portTypes are aggregated into a single Web service This restriction has very little impact on our ability to meet the goals of this project In fact, the extent to which this project was possible within the Web Services framework raises questions about the need for extensions, see Section 29.3.1 below

29.2 IMPLEMENTATION

This section outlines the overall architecture and implementation of this GridService demonstrator On top of a ‘hosting environment’, the demonstrator supports a generalized, distributed, application-steering GridService This section also includes a summary of this application-steering GridService

29.2.1 Infrastructure and architecture

In ‘The Physiology of the Grid’ [2], Foster et al., describe a hosting environment

as follows:

In practice, Grid services are instantiated within a specific execution environment or hosting environment A particular hosting environment defines not only implementa-tion programming model, programming language, development tools, and debugging tools, but also how an implementation of a Grid service meets its obligation with respect to Grid service semantics.

The platform component of this demonstrator fits easily into this definition; however,

the overall environment is more complex than this definition indicates, see Figure 29.1

Hosting environment GridService platform

Client

GLUE server

resource B

Unicore server A Computing

resource A http(s) +SOAP

UPL

SSL

Figure 29.1 Architecture of Unicore GridService hosting environment.

Implicit in the above definition, a hosting environment is based on a single programming model/language; it implies that the GridService client interacts with a single ‘program’ Although true of this demonstrator’s platform component (i.e the client interacts with a Java program constructed as described below), this simplification belies the complexity of the Unicore infrastructure that interacts with the platform and the distributed computing

resources The Unicore hosting environment supports applications written in a variety of

programming languages and run on many different computer architectures Nonetheless, the notion of Unicore deployed in this way is well described as a hosting environment This hosting environment is based on two primary building blocks, the Web Services platform GLUE from The Mind Electric, (www.themindelectric.com) and the Unicore server implementation developed by Fujitsu Laboratories of Europe for Unicore-related projects (www.unicore.org) Both environments are freely available for downloading along with the source code of this demonstrator.1 All components are written in Java Only a small part of the Unicore environment requires Perl

29.2.1.1 Client

The user accesses several GridServices from a single client application, linked with the client side classes from GLUE The client included in the demonstration download is

a crude, command line client supporting all the GridServices provided, for example, Factory, Registry, HandleMap, a NotificationService, and various JobGridServices More realistic and user friendly clients could be written using any Web Service environment, for example, NET

The client connects to the hosting environment’s platform component, via http or https, with Simple Object Access Protocol (SOAP) bindings, as specified in the WSDL

1 Note the demonstrator download does not include the Unicore or GLUE infrastructure components To experiment with this demonstrator, the GLUE Web Services environment is required and must be obtained from The Mind Electric A ‘DummyGrid-Service’ is included in the demonstrator that does not require a complete Unicore Grid This demonstrator will interact with

sent from the platform For https, a pair of certificates and the associated key stores are required for the client and server The use of an https binding automatically activates a password-based authorization step In the http mode, no user authentication is needed With the simple http binding, the ‘console’ supported by GLUE can be started (see GLUE documentation for details) The console introspects the WSDL and generates dynamic Web pages for most of the GridService operations The exceptions are those that return structured data types, which the simple console interface cannot introspect; however, no fundamental barriers exist to automatic tooling within this GridService environment

29.2.1.2 Platform

On the server side, the GLUE Web Services server is integrated with a servlet pack-age that supports the Unicore protocols From the servlet, Unicore Abstract Job Objects (AJOs) can be consigned to Unicore sites, monitored, controlled, and the results retrieved Within a Unicore Grid, this servlet can be used to construct lightweight clients In this

implementation, the GridService package implements the basic OGSA portTypes and a collection of JobGridServices The servlet package manages the interaction between the GridService package and the Unicore Grid This servlet also manages Unicore security,

see section 29.2.1.4

29.2.1.3 Unicore server

From the servlet, Unicore sites are contacted via the Unicore Protocols Layer (UPL) The Unicore Server receives AJOs from Unicore clients, the GridService platform, and other Unicore Servers As the various workflow tasks of the AJO are processed, they are converted from their abstract representation in the AJO to platform-specific forms by Unicore mechanisms All aspects of the Unicore Grid, including the AJO, are of course hidden from the OGSA user Further details of the Unicore side are not central to this chapter See Unicore download documentation and Reference [5] for more information

29.2.1.4 Security framework

At the time of writing, the Grid Service Specification [3] only indicated that security was a binding time property As a result, there are two separate levels to the security framework, one at the https binding between the client and the GridService platform and another between the platform and the rest of the Unicore hosting environment

Https is used to connect the client to the platform component of the GridService The use of https triggers the need for simple authentication using user names and passwords.2 Once the https connection is established, a Unicore certificate, loaded by the servlet when

it starts up, is used to authenticate and authorize work by all users of the GridService In

2 The user name/password step is redundant, given that the authentication can be taken from the https connection However,

we chose to leave it in place, since the normal practice for an https connection assumes the use of a generic client certificate

this case, the GridService is acting like an Application Service Provider (ASP) The user authenticates the ASP, using user name and password over https, and the ASP authenti-cates the supercomputer centres, using Unicore authentication over UPL In this demon-strator, the computer centres do not know the identity of the end user, only that of the ASP Other security and trust models are clearly possible as detailed below:

1 Globus style proxy credentials could be generated by the Web Services client and sent

to the platform This proxy certificate could then be used by Unicore to identify the user to the computer centres

2 The servlet can be run on the client side In this case, Unicore authentication and authorization are used throughout the process The Web Services authentication can

be omitted since this connection is local This approach has little to offer byway of practical GridServices beyond providing a GridService client to existing Unicore users

3 The Unicore sites could accept authentication according to the ASP model as described above, but the Unicore sites can be configured to authorize on the basis of the login used at the Web Services point of entry

4 As OGSA security plans evolve, other approaches will be considered

29.2.2 Supported Grid Services

The basic set of portTypes, as outlined in Reference [3], has been implemented By aggre-gating and extending these, a collection of practical GridServices has been implemented

29.2.2.1 Basic GridService portTypes

All the basic portTypes are coded as Java interfaces The WSDL representation of each

is generated automatically by the GLUE environment However, GLUE has not been extended to support the serviceType extension; hence, the aggregation of portTypes into services takes place in the Java implementation and the complete Web service description for each GridService is created automatically from the Java implementation

Given that the Grid Service Specification indicates that all GridServices should support the GridService portType, all other portTypes are constructed using interface extensions from the GridService interface The basic GridService interface hierarchy is therefore

as follows:

interface com.fujitsu.arcon.gridservice.IGridService

interface com.fujitsu.arcon.gridservice.IGridServiceFactory

interface com.fujitsu.arcon.gridservice.IHandleMap

interface com.fujitsu.arcon.gridservice.IJobGridService

interface com.fujitsu.arcon.gridservice.ISteeringGridService

interface com.fujitsu.arcon.gridservice.INotificationService

interface com.fujitsu.arcon.gridservice.INotificationSink

interface com.fujitsu.arcon.gridservice.INotificationSource

interface com.fujitsu.arcon.gridservice.IRegistry

29.2.2.2 Grid Service implementations

The implementation hierarchy parallels the interface hierarchy; there is an implementation

of the GridService portType that all other portTypes extend Although a GridService that implemented only the GridService portType could be created and used, it performs no useful function and therefore, no factory has been written for it The following GridSer-vices can be created (or are created automatically) and accessed from the demonstrator’s client application

RegistryAndMapper : The HandleMap and Registry portTypes are implemented as a

sin-gle GridService This GridService is created automatically when the service starts up as

a function of the hosting environment GridServices created by the hosting environment are registered automatically with both the HandleMap and Registry when they are cre-ated Also, there is a Registry portType for registering new, external GridServices with the Registry

GridServiceFactory : The factory is started automatically, but the client must enquire of

the Registry to find the Grid Service Handle (GSH) of the factory and then contact the HandleMap to obtain a Grid Service Reference (GSR) with which to bind to the factory This process could be streamlined; however, the goal was to expose the full functioning structure of the OGSA The factory can construct a variety of JobGridServices and a NotificationService, as outlined below

NotificationService: Given a Grid Service Handle, this service will wait for any notified

state change in the target GridService The NotificationService implements the Notifica-tionSink portType and requires that the target GridService implements the Notification-Source portType

DummyJobGridService: This service mimics the functions of the JobGridService, but

does not require access to a Unicore enabled site, nor does it support the Notification-Source portType

JobGridService: This GridService provides support for running a ‘black box’ application

at a Unicore site The serviceData describing these applications is instance specific, but made available as part of the factory’s serviceData Each Unicore site may support many different applications, and the factory may query the Unicore sites for this serviceData Note that, in this implementation, the factory’s version of this serviceData element is hard coded, so that the DummyJobGridService can function without reference to a Unicore site The dynamic information from the Unicore site is available in the GridServiceDescription

of the JobGridService after it has been created

Each ‘black box’ application accepts an input file and writes an output file The names

of these files are part of the GridServiceDescription, and the users may specify file names

of their choosing The GridService manages the renaming automatically The application

may also access other resources at the site, but these are configured by the site and invisible to the user (effectively inside the ‘black box’) The input file may come from any Unicore site or the client and the output file may be sent to any Unicore site or returned to the client

SteeringGridService: This GridService directly extends the JobGridService with

opera-tions that allow the user to fetch a sample file back to the client and write a control file

to be read by the running application The assumption is that the application occasionally inspects the control file and modifies its behaviour accordingly The sample file may be written explicitly as a result of fields in the control file or according to some other policy, for example, every ten iterations The client, included with the download of this demon-strator, assumes that the sample file is in GIF format and starts a viewer to display the contents The same flexibility with respect to file names exists for the sample and control files as it does for the input and output files

29.3 LESSONS

This section briefly outlines the author’s thoughts on some issues that arose out of the process of implementing this OGSA interface to Unicore Although arrived at indepen-dently, many of these ideas are not unique Following the release of the first draft of the Grid Service Specification, a significant and active community of informal reviewers was formed Many of the ideas and perspectives listed here can also be seen in the proceedings

of this community, see Reference [6]

29.3.1 Are GridService extensions to WSDL really needed?

The author certainly feels that some of the proposed extensions are desirable, but from both the server and client-development points of view, many of the proposed extensions seem unnecessary This project was carried out with only standard WSDL facilities on both the client and server sides and has resulted in a working demonstrator that could

be deployed in a variety of contexts However, several aspects of the OGSA model are missing

29.3.1.1 ServiceType

The serviceType extension provides a mechanism for aggregating and naming a collection

of portTypes Multiple serviceTypes can be combined to create new serviceTypes, and multiple implementations of a given serviceType can be constructed In this implemen-tation, Java interfaces encapsulate the abstractions of the OGSA portTypes and therefore capture the serviceType structure, albeit in Java rather than (extended) WSDL For each serviceType, the WSDL emitted by the platform is a single portType containing all the operations of the constituent portTypes (represented by Java interfaces) used to form the

serviceType The serviceType structure, retained at the Java level, is effectively flattened into a single portType at the WSDL level For example, the RegistryAndHandleMap ser-viceType would contain three portTypes, that is, GridService, HandleMap, and Registry The single WSDL portType constructed by the platform is semantically equivalent to what would be expressed in the extended serviceType representation, but the underlying structure is lost

The author’s belief is that a serviceType extension that supports serviceType inheritance should be a minimum extension to WSDL Such an inheritance mechanism should support aggregation, versioning, extension (in the sense of adding additional operations through new portTypes), and implementation naming, see Section 29.3.1.2

29.3.1.2 ServiceImplementation

The related extension, serviceImplementation, provides a mechanism for naming a par-ticular implementation of a serviceType as part of the creation of a service However, serviceImplementation extends the WSDL element service, which includes endpoint and binding information The author favours separating the naming aspect from binding and endpoint aspects Allowing serviceTypes to be extended, as recommended earlier, pro-vides an effective naming mechanism for particular implementations, while the standard WSDL service element provides the binding and endpoint details The author believes that serviceImplementation can be dropped from the specification

29.3.1.3 ServiceData

Because serviceData can be obtained dynamically, through the findServiceData opera-tion in the required GridService portType, there is no requirement to expose service-Data as an extension to the WSDL representation of a serviceType The typical pat-tern adopted in this implementation is to include some (usually static) serviceData ele-ments for one GridService as run-time serviceData eleele-ments in the Grid Service used to find (or create) the given GridService In this way, serviceData that is needed to cre-ate a Grid Service, for example, is available from the factory as part of its run-time serviceData

Although not required, there are good reasons for making some serviceData available through the WSDL In particular, it allows serviceData to remain associated with the serviceType to which it refers Clearly not all serviceData elements can be defined when the WSDL is generated, however, wherever possible there are clear advantages of having

it available For example, if the list of applications supported by a given JobGridService were available as part of the WSDL, the client could decide if this service supported the desired application before creating the service It is possible to obtain this information from the serviceData of the factory, as in our demonstrator, but this separates this important information from the GridService

Furthermore, serviceData is an ideal framework to capture increasing detail about a GridService as it evolves from interface, to implementation, to created instance, to running instance, to completed instance, and finally to destroyed GridService In some cases

the content changes as the state of the service changes; in other cases new serviceData elements are added At the transition from implementation to instance, serviceData allows site-specific details to be provided by the Unicore hosting environment In particular, the list of supported applications at a given site is included in the factory’s serviceData Also, details about the file names required by the application are provided to the client as the GridService is created

However, there are several places in the architecture where care must be taken to keep serviceData consistent with other structures in the model, in particular, the GSR There are three different sources for a GSR The client may have a copy of the GSR that was obtained from the factory when the GridService was created, one may be obtained from

a HandleMap, or the client may query the serviceData of the GridService itself In most cases the consistency of the GSR is not important, but with serviceData being generally available from several sources, consistency can become an issue

In general, serviceData is an essential part of the OGSA framework The specification should make it clear what serviceData elements are required and when in the GridService life cycle they are relevant Allowing serviceData as part of the WSDL of a serviceType

is certainly desirable, but all serviceData should remain available dynamically from some source, be it the GridService itself or a registry or a factory

29.3.1.4 Compatibility assertions

There are many issues surrounding compatibility assertions Since this demonstrator is the first GridService of any type (that the author had access to), it could not have been compatible with another implementation During the analysis and design phases for this effort, many questions arose These were also echoed in the OGSI mailing list and are detailed as follows:

Who asserts the assertions?

How do we believe the assertion?

Does an assertion mean more semantically that ‘This one is like that one.’?

Is compatibility asserted at the operation, port, service, or implementation level? Are compatibility assertions object-oriented programming by the back door? How are compatibility assertions composed?

What is the meaning of composition of compatibility assertions?

Recent discussions in the OGSI Working Group indicate that compatibility assertions will be removed from the specification and discussion postponed to some later time The author concurs with this strategy

In the meantime, it is possible to experiment with compatibility issues in the context of GridServices As part of GridService development, inclusion of a compatibility portType would provide information on the compatibility of one version to all earlier versions There

is no complex trust framework required, as the service provider is making assertions about his own implementation only Similarly, as part of a registry or other discovery services,

a compatibility portType could check that two distinct implementations were compatible

Such a service could be supported and run by a VO, or a corporate consortium, covering their own members and implementations

29.3.1.5 PrimaryKey

In this implementation no use could be found for the primaryKey portType Initially, during development, it seemed to make sense as an internal handle, but it never made sense as an externally visible concept As development proceeded, its internal role also vanished The primaryKey portType should be removed from the specification

29.3.2 Can a single GridService support multiple portTypes?

One aspect of GridServices in general, and of this implementation in particular, is that multiple portTypes can be combined into a single serviceType In this demonstrator, the Registry and the HandleMap are combined into a single serviceType However, a single service possessing two distinct functions creates some complexities in the development

of a clean client demonstrator, as well as raising questions of identity and equality within the GridService context

The bootstrap process for the client is based on knowing the address of an ‘interesting’ Registry The Registry advertises a running factory for creating application steering Grid-Services Following the procedure set out in the Grid Service Specification, the Registry’s

or GridService’s GSH yields the handle for the home HandleMap These two GSHs (the Registry and the HandleMap) and their mapped GSRs are different, but point to the same instance and therefore have the same serviceData One of the serviceData elements is the GSH of the GridService The GSH must appear exactly once in the serviceData and, in this case, can have two possible values In the demonstrator, the serviceData contains the GSH of the Registry In general, the issue of identity needs further discussion and some resolution needs to be arrived at in the context of GridServices

29.3.3 Is the ‘Push Only’ notification framework sufficient?

The OGSA notification framework is based around a subscribe/deliver model However, the underlying assumptions in Unicore are based on periodic polling for changes in state This was largely motivated by the need to deal with clients sitting behind firewalls, the norm with the majority of Unicore-related partners.3

For this demonstrator, a NotificationService provides an operation that waits for a state change in a selected GridService and then returns that state to the client, that is,

a ‘pull’-based model To implement ‘push’-based notification back to the client, at least partial server-side functionality would need to be provided to the client, creating the same firewall issues that motivated the ‘pull’-based model in Unicore

This NotificationService (a NotificationSink) subscribes to state change notification with the nominated JobGridService (a NotificationSource), using the OGSA ‘push’-based

3 Incidentally, this is the only significant point where the architecture of OGSA and Unicore differ; otherwise a very direct