grid computing esis analysis of its impact on business and the shift to a services infrastructure

GRID COMPUTING: ESI’s ANALYSIS OF ITS IMPACT ON BUSINESS AND THE SHIFT TO A SERVICES INFRASTRUCTURE Robert Cohen, Ph.D., Fellow Economic Strategy Institute bcohen@bway.net National Ac

GRID COMPUTING:

ESI’s ANALYSIS OF ITS IMPACT

ON BUSINESS AND THE SHIFT TO

A SERVICES INFRASTRUCTURE

Robert Cohen, Ph.D.,

Fellow Economic Strategy Institute bcohen@bway.net

National Academy of Sciences

GUIRR

February 24, 2004

APPLICATIONS INFRASTRUCTURE AWARE

INTELLIGENT LAYER BETWEEN APPLICATIONS

AND COMPUTE/STORAGE RESOURCES

• PRIORITIZING & AUTOMATING THE USE OF

COMPUTE/STORAGE RESOURCES AND

• AUTOMATING COMPLEX BUSINESS PROCESSES

• LET CORPORATIONS MANAGE CRITICAL STANDARDS

ACROSS THE FIRM: CREDIT RISK, SIX SIGMA

• SUPPORT COLLABORATIVE PRODUCT DEVELOPMENT IN

THE FIRM & WITH PARTNERS

• ANALYTICAL CAPABILITIES OF A NEW ORDER REDUCE

FINANCIAL RISK, PROVIDE A TYPE OF “SIX SIGMA” FOR SERVICE FIRMS

• CREATE WAY TO SUPPORT SERVICE-BASED PRODUCTS

break out from the HPC marketplace and capture the attention

of commercial users

shift from hardware in the HPC market to a more software and services based model in the commercial market

 THE MANAGEMENT AND SERVICES SIDE OF GRID

IS FAR GREATER THAN DIRECT COMPUTER

SERVICES

CHAINS, FINANCIAL PERFORMANCE

What is a Grid? - Ian Foster, “What is the

Grid? A Three Point Checklist”

 1 Coordinates resources that are not subject to centralized control …

– integrates and coordinates resources and users that live within different control domains— for example, the user’s desktop vs central computing;

– It addresses the issues of security, policy, payment, membership, and so forth that arise in these settings Otherwise, we are dealing with a local management system.

 2 Uses standard, open, general-purpose protocols and

interfaces

– built from multi-purpose protocols and interfaces that address such fundamental issues as authentication, authorization, resource discovery, and resource access.

– these protocols and interfaces must be standard and open Otherwise, we are dealing with

an application specific system.

 3 Delivers nontrivial qualities of service

– its constituent resources are to be used in a coordinated fashion to deliver various qualities

of service, relating to response time, throughput, availability, and security, and/or allocation of multiple resource types to meet complex user demands

co-– the utility of the combined system is significantly greater than that of the sum of its parts.

0 5000 10000 15000 20000 25000 30000

funding Grid work,

DOE adds support

GT 1.0.0 Released

Early Application Successes Reported

NSF & European Commission Initiate Many New Grid Projects

Anatomy of the Grid

Commercial Interest in Grids

Physiology of the Grid

Paper Released

GT 2.0 Released

Does not include downloads from:

NMI, UK eScience, EU Datagrid,

IBM, Platform, etc.

Data Management Security ManagementExecution Information Services CommonRuntime

Web Services Components

Non-WS Components

Pre-WS Authentication

Grid Resource Allocation Mgmt (Pre-WS GRAM)

Monitoring

& Discovery System (MDS2)

C Common Libraries

G

T

2

WS Authentication

Authorization

Reliable File Transfer

OGSA-DAI [Tech Preview]

Grid Resource Allocation Mgmt (WS GRAM)

Monitoring

& Discovery System (MDS4)

C WS Core

Community Scheduler Framework [contribution]

Services Oriented Infrastructure

as the Next Phase in the

Evolution of Grids

Infrastructure is Globus that is “completely” Web Service-ized

The SOI links to applications using Grid Services, Applications are linked by Web Services.

Source: Rich Miller, Univa

GRID: FLEXIBLE, PERFORMANCE ACCESS TO ALL

HIGH-SIGNIFICANT RESOURCES

On-demand creation of powerful virtual computing systems

Source: Dietmar Erwin, “UNICORE and EUROGRID: Grid Computing in Europe,” EUROGRID and UNICORE

GRIDS LINK GLOBAL

OPERATIONS:

COORDINATING SERVICES

GRIDS IN THE CORPORATION:

COMPUTE RESOURCES, STORAGE, CONTROL

Wolfgang Gentzsch, Director Grid Computing, Sun Microsystems, “Sun Grid Computing: Vision, Strategy, Products,”

13

Globus as Service-Oriented Infrastructure

User Application

User Application

MDS-Tool Tool

Service Oriented Computing – An Ecosystem

that Manages Change as a Competitive

Weapon

© Copyright 2004 CSFB

Policies

Auctions, Commodity Markets, Rule Based, etc

Virtual Resource Market

Virtual Resource Market

Business Logic, Workload Management

SOAService Oriented Architecture

Applications

Demand

Performance Management

Provisioning Charge-Backs

Process/Data Migration, Clustering, SSI, CFS

Physical Resources

SOIService Oriented Infrastructure

Governance

ECONOMIC RATIONALE OF ANALYSIS

 GRIDS ARE TECHNOLOGICAL INNOVATIONS THAT LOWER THE OPERATING COSTS OF ADOPTER FIRMS.

• BY REDUCING COSTS, ADOPERS BECOME MORE EFFICIENT THEY BRING PRODUCTS TO MARKET AT LOWER COST THIS FREES UP

CAPITAL FOR NEW INVESTMENT BY THE EARLY ADOPTER

INDUSTRIES

• WHEN EARLY ADOPTER FIRMS PRODUCE CHEAPER GOODS THAN THEIR COMPETITORS, THEY ARE ABLE TO SELL MORE AND

INCREASE PROFITS

– THEY GROW FASTER AND “CROWD OUT” LESS DYNAMIC SECTORS.

– THEY ATTRACT MORE INVESTMENT THAN SLOWER GROWING SECTORS

 EARLY ADOPTERS GAIN KEY ADVANTAGES THAT ARE

REINFORCED BY ADDING MORE OPERATING EFFICIENCIES.

– THEY REDESIGN THE WAY THEY DEVELOP PRODUCTS AND GET THEM TO

MARKET FASTER.

– AS A RESULT, THEY BECOME EVEN MORE DYNAMIC A “VIRTUOUS

CYCLE.”

MAIN INDUSTRY FINDINGS

INCREASE THEIR SALES DRAMATICALLY DUE TO GRIDS AND WEB SERVICES

SERVICES

• THE LARGEST GAINS WILL BE IN COMPUTERS (36%, WITH 19% OF THE

GAINS DUE TO GRIDS), AUTOS (19%, WITH 12% OF THE GAINS DUE TO GRIDS) AND PHARMACEUTICALS (13%, WITH 7% OF THE GAINS DUE TO GRIDS).

• THE GREATEST PRODUCTIVITY GAINS WILL BE IN FINANCIAL SERVICES

(26%) AUTOS AND PHARMACEUTICALS (25%) AND HOSPITALS (24%) AND AEROSPACE (20%)

THIS WILL RESULT IN GREATER SPENDING ON BROADBAND ACCESS

• THE AUTO INDUSTRY WILL INCREASE THIS SPENDING FROM AN EXPECTED

$0.5 BILLION TO ABOUT $2.3 BILLION ( 1996 DOLLARS).

Percentage Change in US Industries' Sales due to the Adoption

of Grids and Web Services, 2002-2010

of Web Services

Changes due to the Adoption

of Grids and Web Services

20GRID ADOPTION -

PHARMAS, AEROSPACE

21GRID ADOPTION - HEALTH

CARE

GRIDS RAISE PRODUCTIVITY IN

AUTOS AND FINANCE

Labor productivity - Autos

Comparing Adoption of Grids (Blue) and Baseline Forecast to 2010

245

187

130

1990 1995 2000 2005 2010

baseline Grid Adoption

Labor productivity - Financial Services

Comparing Adoption of Grids (Blue) and Baseline Forecast to 2010

87.9

65.9

43.9

1990 1995 2000 2005 2010 baseline Grid Adoption

AEROSPACE AND HOSPITALS HAVE LOWER

Cost of Deploying Grids, Software Costs Only,

THESE ESTIMATES ARE ABOUT 60% MORE THAN THE INSIGHT ESTIMATES FOR 2008 (SOFTWARE ONLY) AND ABOUT 5 TO 6 TIMES THE INSIGHT ESTIMATES IF ALL COSTS INCLUDING

COMMUNICATIONS COSTS ARE INCLUDED

THE SPENDING BY THE FINANCIAL SERVICES SECTOR DRIVES MUCH OF THE SPENDING TO DEPLOY GRIDS THROUGH 2010 WE ARE RECHECKING THIS ASSUMPTION TO SEE IF IT NEEDS TO BE

MODIFIED.

IDC: $12 Bn GRID SPENDING IN

‘07, INSIGHT: $5 BN IN ‘08

Miriam Vializ-Briggs and Dr Daron Green,

“IBM Grid Computing Business Strategy

Update,” April 28-29, 2004

Insight Research Corporation, “GRID COMPUTING A VERTICAL MARKET PERSPECTIVE 2003-2008,” June 2003 http://www.insight-corp.com “By 2008, financial applications and professional business services will account for the highest grid expenditures.”

HEALTH CAREAEROSPACEPHARMACEUTICALSFINANCIAL SVCSCOMPUTERSSEMICONDUCTORS

IDC FIGURES FOR WEB SERVICES SPENDING FOR ALL US INDUSTRIES ARE 2007: $21 BILLION; 2008: $28 BILLION THE IDC FIGURES ARE NEARLY DOUBLE THE ESI ESTIMATES, LARGELY BECAUSE IDC

ASSUMES A MORE RAPID DEPLOYMENT OF WEB SERVICES THE IDC STUDY MAY ALSO ASSUME

THAT MORE INDUSTRIES, SUCH AS THE RETAIL SECTOR, WILL BE EARLY ADOPTERS OF WEB

SERVICES WE HAVE NOT INCLUDED THAT SECTOR IN OUR ANALYSIS.

28

GM, FORD AND THE

SERVICE GRID

 ADVANTAGE: GOING BEYOND ENTERPRISE GRIDS TO

CONNECT TO SUPPLIERS & CUSTOMERS

• GM, FORD USED BIG CLUSTERS, GRIDS FOR CRASH

SIMULATIONS, COMPLEX DESIGN NEEDS

• SERVICE GRID WILL GET SOFTWARE FIXES TO

CUSTOMERS WITH “HIGHLY ELECTRONIC” CARS &

INFORMATION TO DEALERS

• NEW ISSUE: CORRUPTION OF SOFTWARE FOR ENGINE

CONTROLS; IMPOSSIBLE TO FIX WITHOUT SERVICE

GRID

• SERVICE GRID SENDS OUT FIX AFTER THE GLITCH IS

ANALYZED DEALERS, CUSTOMERS GET FIX OVER

HIGH SPEED CONNECTIONS FOR THE SERVICE GRID

BANK OF AMERICA, JPMORGANCHASE AND RISK

ANALYSIS

 ADVANTAGE: TREMENDOUS COST SAVINGS ACROSS A BANK

CLUSTERS AND GRIDS EMPLOYED TO ESTIMATE A BANK’S RISK EXPOSURE EVERY DAY LOWERING RISK RESERVES CAN SAVE

BILLIONS

MODELS FOR TRADING GROUPS

REDUCE THEIR CAPITAL ALLOCATION FOR RISK

MATHEMATICAL MODELS TO DO SOPHISTICATED RISK ANALYSIS

ACROSS ALL PARTS OF BANKS THIS WOULD NOT BE POSSIBLE

WITHOUT VERY LARGE CLUSTERS & PARALLEL PROCESSING

SOPHISTICATED COMPUTER SYSTEMS

CADENCE/ SYNOPSYS: GREATER INSIGHT INTO THE DETAILS OF

CHIP DEVELOPMENT

 KEY COMPETITIVE ADVANTAGE: VAST COMPUTE POWER

CREATES ABILITY TO EXAMINE DESIGN AT FORMERLY

IMPOSSIBLE LEVELS

LOCATIONS IN A SINGLE DESIGN TEAM REDUCES TIME TO MARKET, EXTENDS ANALYTIC CAPABILITIES TO ALL PARTS OF FIRM

 ADDITIONAL INNOVATIONS & SAVINGS:

EFFICIENT BATCH PROCESSING OF COMPLEX JOBS; ROI: 1 MONTH

AND USE INFORMATION, COST OF DESIGNER TIME

OF A NEW SOFTWARE-SERVICE PRODUCT

INDUSTRY LEVEL CHANGES DUE TO

ADOPTION OF GRIDS AND WEB

GRIDS AND BANDWIDTH

DEMAND

OF COMPUTE RESOURCES AND STORAGE - COST GAINS

NETWORK, HIGHER LEVEL OF CONTROL AND

MANAGEMENT

VIRTUAL NETWORKS & HIGHER LEVELS OF

VIRTUALIZATION OF COMPUTE RESOURCES & STORAGE

SERVICES BEYOND VPNs; ARE THESE JUST VIRTUAL LAN SERVICES WITH ETHERNET/LIGHTWAVE CONNECTIVITY?

• GRID progressively turns all Enterprise ICT resources virtual

• Traditional Enterprise relationships with Telcos and IT companies

replaced

• BT provides network-centric application delivery services to Enterprise

– Distributable Applications delivered across “Pool” of virtual ICT resources

– Non-distributable applications through dedicated resources from “Pool”

– Vertically integrated delivery includes network connectivity

– Resource management of Virtual Network is the key success factor

GRID “Computing” – The BT

ON-DEMAND COMPUTING

 ON-DEMAND COMPUTING IS ONE OF THE GREATEST BENEFITS OF GRIDS

 IN ON-DEMAND COMPUTING, A FIRM DOES NOT BUILD ITS OWN

GRID THAT INCLUDES IN-HOUSE COMPUTERS AND DATA STORAGE

 A GRID SERVICES FIRM PROVIDES THE COMPUTE POWER AND DATA STORAGE CAPABILITIES TO SUPPORT A FIRM’S GRID AND BUNDLES THEM WITH NETWORK CONNECTIONS THROUGH A SERVICE

PROVIDER(S)

• RELYING ON OUTSOURCING, MEANS THAT A FIRM NO LONGER NEEDS

TO PURCHASE ALL THE COMPUTERS AND STORAGE REQUIRED TO

ESTABLISH ITS OWN GRID AND TO PAY FOR COMMUNICATIONS

SERVICES TO LINK ALL PARTS OF ITS CORPORATE GRID

 LARGE, INTEGRATED FIRMS BELIEVE THERE ARE CLEAR COST

SAVINGS IN USING O ON-DEMAND COMPUTING VS PURCHASING EQUIPMENT AND BUILDING THEIR OWN GRID

36

BANDWIDTH GROWTH PROJECTIONS FOR INDIVIDUAL FIRMS, 2003-2006

AND BEYOND

Firm Bandwidth Growth

per year, 2003-4 Bandwidth Growth per year, 2005-6 Bandwidth Growth per year, after 2006 Large pharma 15% 30% 30-35%

Large pharma 40% 60% 75%

Large semiconductor 300% over period 10-15% >10-15%

Large auto 10% 40% 30%

The Use of Grids and Web Services Results in a

Substantial Increase in Bandwidth Demand: The examples are derived from two separate studies

Forecast Bandwidth Monthly Demand: Baseline

AT&T/RHK vs Grid Forecast

evaluate how grids affected the demand for broadband access These charts provide some illustrations from the research.

The economic model provides us

with a way to see how much

spending for communications

services will shift from its assumed

pattern through 2010 In many of

the early adopter industries we

studied, spending on

communications services more than

doubled by 2010 We believe that

this is likely to reflect new spending

on broadband access 0

0.5 1 1.5 2 2.5

Billions of Dollars

0.0500.01000.01500.02000.02500.03000.0

Millions of

Dollars

Auto Industry Spending on Communications Services -

Baseline Spending and the Impact of Grids and Web

Services

0 10

CONCLUSIONS

 COLLABORATION USING GRIDS CAN LEVERAGE THE BENEFITS OF

COMPUTE POWER AND USE EXCESS COMPUTE CAPACITY THAT IS

AVAILABLE NOW MOST CXOs ARE NOT AWARE OF THIS CHANGE RFID HAS GOTTEN MORE PRESS THAN GRIDS

 GRIDS CREATE THE POTENTIAL FOR NEW BUSINESS MODELS THEY

WILL FACILITATE SMALLER AND MEDIUM SIZED FIRMS (SUPPLIERS)

WORKING EFFECTIVELY WITH LARGER FIRMS

 LARGE EFFICIENCY GAINS WILL MAKE EARLY ADOPTER FIRMS MORE DYNAMIC THAN OTHER FIRMS IN THE ECONOMY

 A MAJOR BREAKTHROUGH WILL OCCUR AS FIRMS USE GRIDS TO

CHANGE THE WAY THEY DO BUSINESS - CHANGING BASIC BUSINESS PROCESSES, SUCH AS PRODUCT DESIGN, INVESTMENT POLICIES

WITH WITH

SUCH CHANGES, PRODUCTIVITY GAINS WILL BE DRAMATIC, BUT

COMPETITIVE CHANGES WILL BE EVEN GREATER FIRMS THAT USE

GRIDS WILL CREATE NEW BUSINESS MODELS