autonomic computing concepts, infrastructure, and applications

Library of Congress Cataloging‑in‑Publication Data Autonomic computing : concepts, infrastructure, and applications / editors: Manish Parashar and Salim Hariri.. Part III of the handboo

Computing

Concepts, Infrastructure, and Applications

TEAM LinG

TEAM LinG

10 9 8 7 6 5 4 3 2 1

International Standard Book Number‑10: 0‑8493‑9367‑1 (Hardcover)

International Standard Book Number‑13: 978‑0‑8493‑9367‑9 (Hardcover)

This book contains information obtained from authentic and highly regarded sources Reprinted

material is quoted with permission, and sources are indicated A wide variety of references are

listed Reasonable efforts have been made to publish reliable data and information, but the author

and the publisher cannot assume responsibility for the validity of all materials or for the conse‑

quences of their use

No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any

electronic, mechanical, or other means, now known or hereafter invented, including photocopying,

microfilming, and recording, or in any information storage or retrieval system, without written

permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www.

copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc (CCC)

222 Rosewood Drive, Danvers, MA 01923, 978‑750‑8400 CCC is a not‑for‑profit organization that

provides licenses and registration for a variety of users For organizations that have been granted a

photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and

are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging‑in‑Publication Data

Autonomic computing : concepts, infrastructure, and applications / editor(s):

Manish Parashar and Salim Hariri.

p cm.

Includes bibliographical references and index.

ISBN‑13: 978‑0‑8493‑9367‑9 (alk paper) ISBN‑10: 1‑4200‑0935‑4 (alk paper)

1 Autonomic computing I Parashar, Manish, 1967‑ II Hariri, Salim III

P1: Binaya Dash

Dedication

To Gowrie and Anushka - Manish Parashar

To Sonia, Lana and George - Salim Hariri

to as autonomic computing.

The Autonomic Computing Paradigm has been inspired by the human

auto-nomic nervous system Its overarching goal is to realize computer and ware systems and applications that can manage themselves in accordancewith high-level guidance from humans Autonomic systems are character-ized by their self-* properties including self-configuration, self-healing, self-optimization, and self-protection Meeting the grand challenges of autonomiccomputing requires scientific and technological advances in a wide variety

soft-of fields, as well as new programming paradigms and ssoft-oftware and systemarchitectures that support the effective integration of the constituent tech-

nologies The goal of this handbook, titled “Autonomic Computing: Concepts,

Infrastructure and Applications,” is to give a comprehensive overview of the

state-of-the-art of this emerging research area, which many believe to be thenext paradigm for designing and implementing future computing systemsand services

Overview of the Handbook:The handbook is organized into four parts Part I

of the handbook focuses on the “The Autonomic Computing Paradigm” andincludes chapters that present the vision, underlying concepts, challenges andrequirements, and proposed architectures In Chapter 1, Alan Ganek provides

an overview of autonomic computing and its origins, evolution, and tion This chapter defines autonomic computing, i.e., creating systems that

direc-all systems could, or indeed, should be constructed as autonomic systems, inChapter 2, David W Bustard and Roy Sterritt present a requirements engineer-ing perspective of autonomic computing and examine the implications of therequirements of autonomic systems from a software engineering perspective.

In Chapter 3, Robbert van Renesse and Kenneth P Birman argue that nomic systems cannot be built simply by composing autonomic components,but that an autonomic system-wide monitoring and control infrastructure isrequired as well In Chapter 4, Manish Parashar investigates the challenges

auto-of emerging wide-area Grid environments and applications, and motivatesself-management as a means for addressing these challenges This chapterpresents autonomic Grid computing solutions and sample autonomic Gridapplications Finally, in Chapter 5, John Sweitzer and Christine Draper present

an architecture for autonomic computing and detail its four key aspects, i.e.,process definition, resource definition, technical reference architecture, andapplication patterns The chapter highlights the fundamental functions of the

“autonomic manager” and “touchpoint” building blocks of the architecture,and summarizes an initial set of application patterns commonly found inautonomic computing systems

Parts II and III of this handbook focus on achieving self-* properties inautonomic systems and applications Part II presents different approachesand infrastructures for enabling autonomic behaviors In Chapter 6, Tom DeWolf and Tom Holvoet present a taxonomy of self-* properties for decen-tralized autonomic computing systems, and use the taxonomy to guide thedesign and verification of self-* properties In Chapter 7, Richard Anthony

et al investigate the use of emergent properties for constructing autonomicsystems and realizing self-* properties In Chapter 8, Sherif Abdelwahed andNagarajan Kandasamy describes a more formal control theoretic approachusing model-based control and optimization strategies to design autonomiccomputing systems that continually optimize their performance in response

to changing workload demands and operating conditions Since some nomic applications cannot be built from scratch, it is necessary to transpar-ently introduce autonomic behaviors into existing composite systems with-out modifying the existing code In Chapter 9, S Masoud Sadjadi and Philip

auto-K McKinley address this requirement and describe transparent shaping toenable dynamic adaptations of existing applications Chapter 10 addressesthe design of self-* systems in which adaptive behavior can be specified as

a set of externalized adaptation strategies In this chapter, Peter Steenkisteand An-Cheng Huang present an architecture that separates service-specificknowledge, represented as a service recipe, from generic functionality,and supports automatic service-specific optimizations of a broad class ofservices

Part III of the handbook presents core enabling systems, technologies, andservices that support the realization of self-* properties in autonomic systems

P1: Binaya Dash

application and system behaviors to be dynamically specified at runtime

In Chapter 12, Thomas Heinis et al describe a self-configuring compositionengine for Grid and Web services that achieves self-configuring, self-tuning,and self-healing behaviors in the presence of varying workloads Dynamiccollaboration among self-managing resources is a key requirement for system-level self-management In Chapter 13, David Chess et al establish a set ofbehaviors, interfaces, and patterns of interaction within the Unity system tosupport such dynamic collaborations In Chapter 14, Karsten Schwan et al.describe the AutoFlow project designed to meet the critical performancerequirements of distributed information flow applications In Chapter 15,Robert Adams et al propose an approach for the management of large scaledistributed services based on scalable publish-subscribe event systems, scal-able WS-based deployment, and model-based management The autonomicsystems and prototypes described in this handbook have been typically im-plemented through middleware or through OS modifications As an alter-nate, in Chapter 16, Lenitra Durham et al investigate autonomic computingsupport at the hardware/physical layer and describe platform support forautonomics

Part IV, the final part of this handbook, focuses on specific realizations ofself-* properties in autonomic systems and applications Chapter 17, the firstchapter in this part, studies how autonomic computing techniques can beused to dynamically allocate servers to application environments in a waythat maximizes a global utility function In this chapter, Daniel A Menasce

et al present a system that exhibits self-* properties and successfully cates servers when workload changes and/or when servers fail Chapter 18examines how a managed execution environment can be leveraged to sup-port runtime system adaptations In this chapter, Rean Griffith et al describethe Kheiron adaptation framework that dynamically attaches/detaches anengine capable of performing reconfigurations and repairs on a target systemwhile it executes Kheiron remains transparent to the application and doesnot require recompilation of the application or specially compiled versions

reallo-of the runtime In Chapter 19, Arjav Chakravarti et al describe the OrganicGrid, which is a biologically inspired and fully decentralized approach tothe organization of computation Organic Grid is based on the autonomousscheduling of strongly mobile agents on a peer-to-peer network Efficientand robust data streaming services are a critical requirement of emergingGrid applications, which are based on seamless interactions and couplingbetween geographically distributed application components In Chapter 20,Viraj Bhat et al present the design and implementation of a self-managingdata-streaming service based on online control and optimization strategies

In Chapter 21, Bithika Khargharia et al discuss the construction of an nomic data center using autonomic clusters, servers and device components,and demonstrate autonomic power and performance management for a three

auto-to detect abnormal behaviors and faults In Chapter 23, Guangzhi Qu andSalim Hariri address the self-management and self-protection of networks,and presents methodologies for effectively detecting network attacks in realtime These methodologies configure network and system resources and ser-vices to proactively recover from network attacks and prevent the attacksfrom propagating in the network.

Once again, the goal of this handbook is to provide readers with an overview

of the emerging discipline of autonomic computing We do hope that it willlead to insights into the underlying concepts and issues, current approachesand research efforts, and outstanding challenges of the field, and will inspirefurther research in this promising area

Acknowledgements:This book has been made possible due to the efforts andcontributions of many individuals First and foremost, we would like to ac-knowledge all the contributors for their tremendous efforts in putting togetherexcellent chapters that are comprehensive, informative, and timely We wouldlike to thank the reviewers for their excellent comments and suggestions

We would also like to thank Nora Konopka, Jessica Vakili, and the team atTaylor & Francis Group LLC–CRC Press for patiently helping us put this booktogether Finally, we would like to acknowledge the support of our familiesand would like to dedicate this book to them

Distributed Applications”, by Richard Anthony which appeared in FirstInternational Conference on Autonomic Computing (ICAC), New York, USA,May 2004 IEEE

Based on “Self-Optimization in Computer Systems via Online Control: plication to Power Management,” by authors N Kandasamy, S Abdelwahed,and J P Hayes in the Proceedings of the IEEE Conf AutonomicComputing (ICAC), pp 54–61, 2004, C[2004] IEEE

Ap-Based on “Online Control for Self-Management in Computing Systems,”

by authors S Abdelwahed, N Kandasamy, and S Neema in the Proceeding ofthe 10th IEEE Real-Time and Embedded Tech & Application Symp (RTAS),

pp 368–375, 2004, c[2004] IEEE

Based on “Building Self-adapting Services Using Service-specific edge”, by An-Cheng Huang and Peter Steenkiste, which appeared in Four-teenth IEEE International Symposium on High-Performance DistributedComputing (HPDC-14), Research Triangle Park, NC, July 24–27, 2005, C 2005IEEE; and based on “Building Self-configuring Services Using Service-specific Knowledge,” by An-Cheng Huang and Peter Steenkiste, whichappeared in 13th IEEE Symposium on High-Performance DistributedComputing (HPDC’04), IEEE, June 2004, Honolulu, Hawaii, pages 45–54, C

Knowl-2004 IEEE

Based on “Accord: A programming framework for autonomic tions”, by H Liu and M Parashar, which appeared in IEEE Transactions

applica-on Systems, Man and Cybernetics, Special Issue applica-on Engineering Autapplica-onomicSystems, Editors: R Sterritt and T Bapty, IEEE Press, 2005

Based on “Design and Evaluation of an Autonomic Workflow Engine”,

by Thomas Heinis, Cesare Pautasso and Gustavo Alonso which appeared

in the proceedings of the Second International Conference on AutonomicComputing, 2005 (pp 27–38)

Based on “Implementing Diverse Messaging Models with Self-ManagingProperties using IFLOW”, by Vibhore Kumar, Zhongtang Cai, Brain F Cooper,Greg Eisenhauer, Karsten Schwan, Mohamed Mansour, BalasubramaniamSeshasayee, Patrick Widener which appeared in IEEE InternationalConference on Autonomic Computing, 2006, ICAC’06

Based on Lenitra M Durham, Milan Milenkovic, and Phil Cayton,

“Plat-form Support for Autonomic Computing: a Research Vehicle,” in Proceedings

of the Third International Conference on Autonomic Computing, pp 293–294, June

13–16, 2006, C 2006 IEEE

Based on Paper title “Resource Allocation for autonomic data centersusing analytic performance models” which appeared in the Proceedings ofthe 2005 IEEE International Conference on Autonomic Computing (ICAC’05),Seattle, Washington, June 13–16, 2005

Based on their chapter in “Self-organizing Scheduling on the Organic Grid”.Based on “Autonomic Power and Performance Management for Com-puting Systems”, by Bithika Khargharia, Salim Hariri and Mazin S Yousilwhich appeared in “IEEE International Conference on Autonomic Computing,

Professor, Department of Electrical and Computer Engineering

University of Arizona, Tucson, AZ, USA

P1: Binaya Dash

About the Editors

Dr Manish Parasharis Professor of Electrical and Computer Engineering atRutgers University, where he also is director of the Applied Software SystemsLaboratory He has received the Rutgers Board of Trustees Award for Excel-lence in Research (2004-2005), NSF CAREER Award (1999), and the EnricoFermi Scholarship from Argonne National Laboratory (1996) His researchinterests include autonomic computing, parallel and distributed computing(including peer-to-peer and Grid computing), scientific computing, and soft-ware engineering

Manish is a senior member of IEEE and of the executive committee of theIEEE Computer Society Technical Committee on Parallel Processing (TCPP),part of the IEEE Computer Society Distinguished Visitor Program (2004-2006),and a member of ACM He is the co-founder of the IEEE International Confer-ence on Autonomic Computing (ICAC) and has served as General Co-Chair

of ICAC 2004, 2005, and 2006 He is actively involved in the organization ofconferences and workshops, and has served as general and program chairsfor several conferences/workshops He also serves on various steering com-mittees and journal editorial boards

Manish has co-authored over 200 technical papers in international journalsand conferences, has co-authored/edited over 15 books and proceedings,and has contributed to several others, all in the broad area of computationalscience and applied parallel and distributed computing

Manish received a BE degree in Electronics and Telecommunications fromBombay University, India and MS and Ph.D degrees in Computer Engineer-ing from Syracuse University For more information, please visit http://www.caip.rutgers.edu/∼parashar/

Dr Salim Hariri is Professor of Electrical and Computer Engineering atThe University of Arizona He is the director of the Center for AdvancedTeleSysMatics (CAT): Next Generation Network Centric Systems CAT is atechnology transfer center to facilitate the interactions and collaborationsamong researchers in academia, industry as well as government researchlabs and is a vehicle for academic research to be transitioned to industry.Salim is the Editor-In-Chief for the Cluster Computing Journal (Springer,http://www.springer.com/journal/10586) that presents research techniquesand results in the area of high speed networks, parallel and distributed com-

Computing (HPDC) and the co-founder of the IEEE International ence on Autonomic Computing His current research focuses on autonomiccomputing, self protection and self-healing of networked systems and ser-vices, and high performance distributed computing He has co-authored over

Confer-200 journal and conference research papers, and is the co-author/editor ofthree books, Tools and Environments for Parallel and Distributed Computing(Wiley, 2004), Virtual Computing: Concept, Design and Evaluation (Kluwer,2001), and Active Middleware Services (Kluwer, 2000)

Salim received his B.S from Damascus University, M.S degree in ComputerEngineering from The Ohio State University in 1982 and Ph.D in ComputerEngineering from University of Southern California in 1986 For further in-formation, please visit http://www.ece.arizona.edu/∼hpdc

Georgia Institute of Technology

Atlanta, GA, U.S.A

Georgia Institute of Technology

Atlanta, GA, U.S.A

Department of Computer Science

Louisiana State University

Baton Rouge, LA, U.S.A

Zhongtang Cai

College of Computing

Georgia Institute of Technology

Atlanta, GA, U.S.A

The MathWorks, Inc

Natick, MA, U.S.A

Haifeng Chen

Robust and Secure System Group

NEC Laboratories America

Georgia Institute of Technology

Atlanta, GA, U.S.A

Alan Ganek

IBMAutonomic ComputingSomers, NY, U.S.A

Ada Gavrilovska

College of ComputingGeorgia Institute of TechnologyAtlanta, GA, U.S.A

IBM Thomas J WatsonResearch CenterHawthorne, NY, U.S.A

Salim Hariri

Department of Electrical andComputer EngineeringUniversity of ArizonaTucson, AZ, U.S.A

P1: Binaya Dash

An-Cheng Huang

Department of Computer Science

Carnegie Mellon University

Pittsburgh, PA, U.S.A

Robust and Secure System Group

NEC Laboratories America

Hua Liu

XeroxWebster, NY, U.S.A

Jay Lofstead

College of ComputingGeorgia Institute of TechnologyAtlanta, GA, U.S.A

Mohamed Mansour

College of ComputingGeorgia Institute of TechnologyAtlanta, GA, U.S.A

Georgia Institute of Technology

Atlanta, GA, U.S.A

Balasubramanian Seshasayee

College of Computing

Georgia Institute of Technology

Atlanta, GA, U.S.A

Peter Steenkiste

Department of Computer

Science and Electrical

and Computer Engineering

Roy Sterritt

School of Computing andMathematics

University of UlsterJordanstown campusNewtownabbey, Co AntrimNorthern Ireland

John Sweitzer

IBMU.S.A

Vanish Talwar

HP LabsPalo Alto, CA, U.S.A

Cristian Ungureanu

Robust and Secure System GroupNEC Laboratories AmericaPrinceton, NJ, U.S.A

Giuseppe Valetto

Department of Computer ScienceColumbia University

New York, NY, U.S.A

Robbert van Renesse

Department of Computer ScienceCornell University

Ithaca, NY, U.S.A

Ian Whalley

Distributed ComputingDepartment

IBM Thomas J WatsonResearch CenterHawthorne, NY, U.S.A

Steve R White

Distributed ComputingDepartment

IBM Thomas J WatsonResearch Center

P1: Binaya Dash

Patrick Widener

Department of Computer Science

University of New Mexico

Albuquerque, NM, U.S.A

Matt Wolf

College of Computing

Georgia Institute of Technology

Atlanta, GA, U.S.A

Kenji Yoshihira

Robust and Secure System GroupNEC Laboratories AmericaPrinceton, NJ, U.S.A

P1: Binaya Dash

Contents

1 Overview of Autonomic Computing: Origins, Evolution,

Direction 3 Alan Ganek

2 A Requirements Engineering Perspective on Autonomic

Systems Development .19 David W Bustard and Roy Sterritt

3 Autonomic Computing: A System-Wide Perspective 35 Robbert van Renesse and Kenneth P Birman

and Infrastructure 49 Manish Parashar

5 Architecture Overview for Autonomic Computing 71 John W Sweitzer and Christine Draper

6 A Taxonomy for Self-∗Properties in Decentralized

Autonomic Computing .101 Tom De Wolf and Tom Holvoet

7 Exploiting Emergence in Autonomic Systems 121 Richard Anthony, Alun Butler, and Mohammed Ibrahim

Management in Computing Systems 149 Sherif Abdelwahed and Nagarajan Kandasamy

10 Recipe-Based Service Configuration and Adaptation 189 Peter Steenkiste and An-Cheng Huang

Part III Achieving Self-* Properties — Enabling Systems,

Applications 211 Hua Liu and Manish Parashar

12 A Self-Configuring Service Composition Engine 237 Thomas Heinis, Cesare Pautasso, and Gustavo Alonso

13 Dynamic Collaboration in Autonomic Computing 253 David M Chess, James E Hanson, Jeffrey O Kephart, Ian Whalley,

and Steve R White

14 AutoFlow: Autonomic Information Flows for Critical

Information Systems 275 Karsten Schwan, Brian F Cooper, Greg Eisenhauer, Ada Gavrilovska,

Matt Wolf, Hasan Abbasi, Sandip Agarwala, Zhongtang Cai, Vibhore Kumar, Jay Lofstead, Mohamed Mansour, Balasubramanian Seshasayee, and Patrick Widener

Large-Scale, Distributed Systems 305 Robert Adams, Paul Brett, Subu Iyer, Dejan Milojicic, Sandro Rafaeli, and Vanish Talwar

16 Platform Support for Autonomic Computing: A Research

Vehicle 329 Lenitra Durham, Milan Milenkovic, Phil Cayton, and Mazin Yousif

17 Dynamic Server Allocation for Autonomic Service Centers in the Presence of Failures 353 Daniel A Menasc´e and Mohamed N Bennani

18 Effecting Runtime Reconfiguration in Managed Execution

Environments 369 Rean Griffith, Giuseppe Valetto, and Gail Kaiser

of Internet Data 435 Bithika Khargharia and Salim Hariri

22 Trace Analysis for Fault Detection in Application Servers 471 Guofei Jiang, Haifeng Chen, Cristian Ungureanu, and Kenji Yoshihira

23 Anomaly-Based Self Protection against Network Attacks 493 Guangzhi Qu and Salim Hariri

Index 523

P1: Binaya Dash

Part I The Autonomic Computing

Paradigm

“Civilization advances by extending the number of important operationswhich we can perform without thinking about them.” Said by mathemati-cian and philosopher Alfred North Whitehead almost a century ago, thisstatement today embodies both the influence and importance of computertechnology.

In the past two decades alone, the information technology (IT) industry hasbeen a driving force of progress Through the use of distributed networks,Web-based services, handheld devices, and cellular phones, companies ofall sizes and across all industries are delivering sophisticated services thatfundamentally change the tenor of daily life, from how we shop to how webank to how we communicate And in the process it’s dramatically improvingbusiness productivity Consider that call centers can now respond to customer

inquiries in seconds Banks can approve mortgage loans in minutes Phonecompanies can activate new phone service in just one hour.

Yet, while business productivity is soaring, these advances are creatingsignificant management challenges for IT staffs The sophistication of ser-vices has inspired a new breed of composite applications that span multipleresources—Web servers, application servers, integration middleware, legacysystems—and thus become increasingly difficult to manage At the same time,escalating demand, growing volumes of data, and multi-national services aredriving the proliferation of technologies and platforms, and creating IT envi-ronments that require tens of thousands of servers, millions of lines of code,petabytes of storage, multitudes of database subsystems, and intricate globalnetworks composed of millions of components

With physically more resources to manage and increasingly elaborateinterdependencies among the various IT components, it is becoming moredifficult for IT staff to deploy, manage, and maintain IT infrastructures Theimplications are far-reaching, affecting operational costs, organizational flex-ibility, staff productivity, service availability, and business security In fact,

up to 80 percent of an average company’s IT budget is spent on maintainingexisting applications.1And an increasing number of companies today reportthat their IT staffs spend much of their time locating, isolating, and repairingproblems

The rapid pace of change—unpredictable demand, growing corporate ernance prompted by both regulatory requirements and an increasingly liti-gious world, and an escalating number of access points (cellular phones,PDAs, PCs)—makes these problems even more acute In an IBM study of 456organizations worldwide, only 13 percent of CEOs surveyed believed thattheir organizations could be rated as “very responsive” to change.2

gov-With so much time required to manage core business processes, IT staffshave little time left to identify and address areas of potential growth Toenable companies to focus on the application of technology to new businessopportunities and innovation, the IT industry must address this complexity.That’s where autonomic computing comes in

The term autonomic computing was coined in 2001 by Paul Horn, seniorvice president of research for IBM According to Horn, the industry’s focus

on creating smaller, less expensive, and more powerful systems was fueling

http://www.ibm.com/news/us/2004/02/241.html (accessed Dec 6, 2005).

P1: Binaya Dash

the problem of complexity Left unchecked, he said, this complexity wouldultimately prevent companies from “moving to the next era of computing”and, therefore, the next era of business.3 In response, he issued a “GrandChallenge” to the IT industry to focus on the development of autonomicsystems that could manage themselves

In much the same way as the autonomic nervous system regulates andprotects our bodies without our conscious involvement, Horn envisionedautonomic IT infrastructures that could sense, analyze, and respond tosituations automatically, alleviating the need for IT professionals to performtedious systems management tasks and enabling them to focus on applying

IT to solve business problems For example, rather than having to worry aboutwhat database parameters were needed to optimize data delivery for a cus-tomer service application, IT administrators could instead spend their timeextending that application to provide customers even greater conveniences.For years, science fiction writers have imagined a world in which androidsand sentient systems could make decisions independent of human input Thisrepresents neither the spirit nor the goal of autonomic computing Althoughartificial intelligence plays an important role in the field of autonomiccomputing—as some of the research highlighted later in this chapter shows—autonomic computing isn’t focused on eliminating the human from theequation Its goal is to help eliminate mundane, repetitive IT tasks so that

IT professionals can apply technology to drive business objectives and setpolicies that guide decision making At its core, the field of autonomic com-puting works to effectively balance people, processes, and technology.When first introduced, the concept of autonomic computing was some-times confused with the notion of automation However, autonomic comput-ing goes beyond traditional automation, which leverages complex “if/thenscripts” written by programmers to dictate specific behavior, such as restart-ing a server or sending alerts to IT administrators Rather, autonomic com-puting focuses on enabling systems to adjust and adapt automatically based

on business policies It addresses the process of how IT infrastructures aredesigned, managed, and maintained And it calls for standardizing, integrat-ing, and managing the communication among heterogeneous IT components

to help simplify processes In a self-managing autonomic environment, a tem can sense a rapidly growing volume of customer requests, analyze theinfrastructure to determine how best to process these requests, and then makethe necessary changes—from adjusting database parameters to provisioningservers to rerouting network traffic—so that all requests are handled in atimely manner in accordance with established business policies

sys-Although the target of autonomic computing is improving the ability of IT processes, it has great implications on a company’s ability totransition to on demand business, where business processes can be rapidly

Dec 6, 2005).

adapted and adjusted as customer demand or market requirements dictate.Without autonomic computing, it would be nearly impossible for the ITinfrastructure to provide the level of flexibility, resiliency, and responsive-ness necessary to allow a company to shift strategies to realize on demandgoals.

Autonomic computing represents a fundamental shift in managing IT ginning in the 1970s, reliability, availability, and serviceability (RAS) technol-ogy worked to help improve performance and availability at the resourcelevel This technology—primarily developed for mainframe computers—created redundancies within the hardware systems themselves and used em-bedded scripts that could bypass failing components, enabling IT staffs toeasily install spare parts while the system was running, or even detect andmake use of new processors automatically By the late 1980s, as distributedsystems began taking hold, IT vendors created management solutions thatcentralized and automated monitoring and management at the systems level.Examples of such management systems include the use of a single consolefor managing a particular set of resources or business services, and the au-tomation of problem recovery using scripts for well-known and well-definedproblems

Be-Although these approaches helped streamline many discrete IT activities,they were not based on interoperable standards and, hence, they lacked theability to integrate business requirements in a holistic manner across all re-sources in the infrastructure and across IT processes themselves Moving tothis next era of IT management requires more than the work of a single ITvendor It requires a collaborative effort by a critical mass of vendors to create

an industry-standard management architecture that enables heterogeneousresources working in concert to take action based on business policy

Autonomic computing proposes the development of intelligent, open, andself-managing systems; but what does this really mean? At its core, a self-managing system must:4

Dec 6, 2005).

P1: Binaya Dash

• Have knowledge—not only of its components, status, capacity, etc.,but also of the context of its activity and those of other resourceswithin the infrastructure

• Be able to sense and analyze environmental conditions This cludes both the ability to proactively take the pulse of individualcomponents and services, looking for ways to improve its functions,and the ability to notice change and understand the implications ofthat change

in-• Be able to plan for and affect change by altering its own state,effecting changes in other components of the environment, or chore-ographing IT processes based on business policy

These characteristics are being applied today in four fundamental areas ofself-management to drive significant operational improvements where tradi-tional manual-based processes are neither efficient nor effective These fourareas are:5

• Self-configuring capabilities that enable the system to adapt to

un-predictable conditions by automatically changing its configuration,such as adding or removing new components or resources, or in-stalling software changes without disrupting service

• Self-healing capabilities that can prevent and recover from failure by

automatically discovering, diagnosing, circumventing, and ering from issues that might cause service disruptions

recov-• Self-optimizing capabilities that enable the system to continuously

tune itself—proactively to improve on existing processes and actively in response to environmental conditions

re-• Self-protecting capabilities that can detect, identify, and defend against

viruses, unauthorized access, and denial-of-service attacks protection also could include the ability for the system to protectitself from physical harm, such as the motion detection capabilities

Self-of today’s laptops that can temporarily park their disk drive heads

if they sense that they are being dropped

These characteristics are defined in terms of the overall system behavior.However, the system is composed of thousands—or even tens of thousands—

of components from numerous vendors that interact in complex ways on ous levels It is not feasible to change all the constituent technologies in concert

vari-to achieve auvari-tonomic systems Rather, an approach that delivers significantvalue while evolving to greater and greater autonomic characteristics is far

more pragmatic and promising This evolution is guided by the Autonomic

5Kephart, J.O and Chess, D.M., The vision of autonomic computing, Computer, Jan 2003 http://

(accessed Dec 6, 2005).

Entry Entry Entry Entry Entry

Complete Adoption

Closed Loop With Business Priorities

Closed Loop Analysis Instrument

& Monitor Manual

Multiple of same type C

Multiple of same type C

B

Single Instance Single B

Instance

Multiple of different types Multiple of D

different types D

A

Sub component Sub - A

Complete Adoption

Service Flows

FIGURE 1.1

Autonomic computing adoption model (Adapted from IBM, An architectural blueprint for autonomic computing http://www-03.ibm.com/autonomic/pdfs/AC%20Blueprint%20White

%20Paper%20V7.pdf With permission.)

Computing Adoption Model, which analyzes the computing environment in

three dimensions as depicted in Figure 1.1:

1 Increasing the scope of autonomic control, from a sub-componentlevel to the entire business system, in which self-managing functionsencompass all components supporting a business solution (depicted

in Figure 1.1 along the y-axis)

2 Increasing autonomic functionality within one or more components,moving from a position where tasks are completed manually to anautonomic level where components of the infrastructure work inconcert to dynamically manage processes based on business policy

At each step within this progression, shown along the x-axis inFigure 1.1, increasing levels of automation, correlation, analysis,and adaptation are implemented

3 Coordinating and integrating functions across IT processes such asincident, change, performance, and security management to enable

P1: Binaya Dash

the system to autonomically address a service requirement thattraverses multiple IT processes For example, providing an em-ployee access to new services or applications requires the comple-tion of tasks not only within configuration management (to installthe new software), but also within security management (to verifyuser rights), release management (to manage version control), andchange management (to authorize deployment of services) as well

In Figure 1.1, this dimension is illustrated as the z-axis of the tion model

adop-This model allows both IT vendors and customers to approach the duction of autonomic computing capabilities in a flexible way For example,one could evaluate the value of introducing a high level of autonomic be-havior to a small scope of IT resources in the context of a specific IT process

intro-In another case, one could consider a less aggressive level of self-managingbehavior with respect to a broader set of components or processes The IT pro-cess dimension enables a focus on the tasks and best practices for managing

IT resources that are discussed next

As the role of IT has evolved from cost center to profit center, organizationshave begun to apply traditional business process management techniques toimprove the control, order, and governance of IT infrastructure operations

This has led to the emergence of the IT Infrastructure Library (ITIL) best

prac-tices as a commonly used reference for designing the approach and nance for managing IT Forrester Research Group estimates that ITIL adoptionwill increase from 13 percent in 2004 to 40 percent in 2006 to 80 percent by

gover-2008.6

Originally created in the 1980s by the U.K Central Computer and munications Agency—now the Office of Government Commerce—ITILmethodologies help govern the processes for effective and efficient IToperations in both service support (including incident management, prob-lem management, change management, configuration management, and re-lease management) and service delivery (including availability management,capacity management, continuity management, and financial managementfor IT services).7

Telecom-However, it’s important to note that ITIL recommendations are simplywritten guidelines—outlined in more than 2,000 pages of text IT service

2672/050418itilpitfalls/ (accessed Dec 6, 2005).

management (ITSM) is a new focus for technology management that takesthese processes and converts them into workflows with definable, repeatabletasks, linked directly to available technology components and products, tohelp companies implement ITIL best practices.

Although a number of IT process tasks, such as software distribution, ventory, and resource monitoring, can be automated today using traditionalsystems management tools, ITSM extends the process model to the end-to-end process definitions outlined in ITIL

in-Self-managing autonomic technologies are critical tools utilized in ITSM toachieve the levels of efficiency and effectiveness envisioned by ITIL Thesetechnologies provide the core capabilities to:

• Automate and integrate portions of a process flow

• Provide coordination across different IT processes

• Align processes with business policy and create an enterprise viewthat resolves conflicts

• Provide a knowledge source to enable process managers to stand the state of the system and how to respond to situations.For example, as described earlier, granting an employee access to a newapplication can require the coordination of at least four IT processes (changemanagement, configuration management, security management, and releasemanagement) as well as the myriad discrete tasks within each process.Through autonomic computing, an autonomic manager can analyze therequest, plan how to choreograph the various tasks in and across each process,and then execute each task in the correct order, automating those tasksappropriately

under-1.4.1 Relating Autonomic Computing to Grid, Service-Oriented

Architecture, and Virtualization Technologies

Autonomic computing represents an entirely new paradigm that both tributes to and benefits from a number of other technologies, such as gridcomputing, service-oriented architecture (SOA), and virtualization

con-1.4.1.1 Grid Computing

Grid computing, which enables organizations to access and integrate erogeneous distributed resources, forms an important conduit that enablesintelligent provisioning of resources regardless of their supplier or location.Via grid technologies, independent system owners from different institutionscan establish connectivity with one another and introduce protocols that allowresource sharing This enables an expansion of systems to groups of systems,greatly extending the capabilities available to any user in the group With this

het-P1: Binaya Dash

Overview of Autonomic Computing: Origins, Evolution, Direction 11

expanded environment, however, comes the threat of additional complexity

as systems participate in grids with many other systems

Autonomic computing provides the architecture that enables systems, cluding grids, to be flexible, dynamic, and adaptable It provides technologies

in-to offset the inherently increased complexity as grids expand the domain ofcomputing For example, self-managing autonomic systems can help opti-mize where processes are performed and manage workloads across the grid

to ensure that resources are most efficiently utilized Likewise, they can enablethe provisioning of software and configuration specifications so that serverscan be allocated or deallocated on the fly Conversely, grid technologies pro-vide enhancements to facilitate distributed computing that can be lever-aged to implement autonomic behavior across distributed, heterogeneousresources

1.4.1.2 Service-Oriented Architecture (SOA)

Through the use of standards-based interfaces and connections, SOA enables

IT organizations to create application building blocks that can be “wired”together as needed For example, a financial services organization may create acredit-checking capability based on an SOA that can be easily coupled with itsmortgage loan application (to approve mortgages), its credit card application(to determine a customer’s credit limit), and its customer service application(to determine the appropriate interest rate for each customer)

Autonomic computing leverages this same Web-based services model tofacilitate communication among heterogeneous components At the sametime, autonomic computing helps simplify the modeling, assembly, deploy-ment, and management of the discrete processes that are composed togetherusing an SOA to help IT staffs more effectively and efficiently create therequired building blocks for service delivery

1.4.1.3 Virtualization

Similarly, virtualization technology and autonomic computing share anequally symbiotic relationship Virtualization technology allows an institu-tion to pool its computing resources—servers, networks, storage devices—into a single environment with a common interface so that they can be allo-cated as needed and managed more efficiently However, within a virtualizedenvironment, how do IT staffs stay attuned to available capacity? How dothey rapidly move workloads as performance wanes? How do they ensurethat data from business-critical applications is moved to storage devices withthe highest quality of service?

Through the virtualization of resources, autonomic computing can manage

IT resources on a more granular level, dynamically provisioning softwareand managing workloads at multiple levels At the same time, autonomiccomputing provides critical “sense and respond” capabilities to reduce thecomplexity of managing virtualized resources

1.5 Current State of Autonomic Computing

Since Paul Horn challenged the industry in 2001, autonomic computing hasemerged as a strategic initiative for computer science and the IT industry,and progress has been made on a number of fronts to help make the vision

of autonomic computing a reality in the data center today

The most visible sign to IT staffs has been the incorporation of managing autonomic capabilities into individual products Today, at everylevel of the infrastructure, vendors are embedding autonomic capabilitieswithin their products Chips can now sense change and alter the configuration

self-of circuitry to enhance processor performance or avoid potential problems.Databases can automatically tune themselves as workload fluctuates andoptimize performance as data organization changes Networking compo-nents can intelligently route traffic Blade servers can automatically populatenew blades with the required software as they’re plugged in The list goes on.However, as described earlier, this is only one aspect of a self-managingautonomic infrastructure Regardless of the autonomic capabilities built intoindividual products, unless these components can communicate and effectchange in an orchestrated fashion across the infrastructure and across IT pro-cesses, the promise of autonomic computing cannot be realized

Considerable progress has been made in this area as well, including ating an open architecture that provides a common language and design forautonomic computing; developing industry standards that enable commu-nication among heterogeneous components; and producing reference imple-mentations (software development kits, toolkits, proofs-of-concept, etc.) forapplying these standards

cre-1.5.1 Autonomic Computing Architecture

Traditionally, IT operations have been organized based on individual silos—separated by both component type and platform type For example, a par-ticular administrator might be concerned with managing only databases, oronly application servers, or only Linux
servers The autonomic computingarchitecture formalizes a reference framework that identifies common func-tions across these silos and sets forth the building blocks required to achieveautonomic computing These building blocks include:8

• A task manager that enables IT personnel to perform management

functions through a consistent user interface

• An autonomic manager that can automate common functions

and management activities using an autonomic control loop (see

pdfs/AC%20Blueprint%20White%20Paper%20V7.pdf (accessed Dec 6, 2005).

Figure 1.2) Through this control loop, autonomic managers itor resource details, analyze those details, plan adjustments, andexecute the planned adjustments—using both information from hu-mans (administrators) as well as rules and policies both defined (byhumans) and learned by the system.

mon-• A knowledge source that provides information about the managed

resources and data required to manage them, such as business and

IT policies

• An enterprise service bus that leverages Web standards to drive

com-munications among components throughout the environment

• A touchpoint that provides a standardized interface for managed

resources—servers, databases, storage devices, etc.—enabling nomic managers to sense and effect behavior within these resources

auto-The notion of a consistent view of the configuration of the computingenvironment—from the vantage point of the many technology components

as well as from the perspective of the IT processes—is central to the tration of IT processes so that they work in concert ITIL describes the idea

orches-of a configuration management database (CMDB) as a technology that tracks the

configurable elements of a system as well as the relationships among them.The CMDB concept, in conjunction with configuration and change manage-ment processes, is emerging as a fundamental capability in the evolution ofautonomic systems.9Several IT vendors are working on implementations

com/software/tivoli/resource-center/overall/eb-itil-it-serv-mgmt.jsp (accessed Dec 6, 2005).

1.5.2 Standards

Open communication among heterogeneous systems is vital to enabling tonomic computing How can systems sense and effect behavior across dis-parate resources without such capabilities? The autonomic computing archi-tecture laid the foundation for industry participants to identify the mecha-nisms by which this communication would take place This resulted in theidentification of existing standards and the development of new standardsthat are needed to enable the open exchange of management data that can beprocessed programmatically

au-The outcome of this work has been significant Already, several tant standards have been ratified and many more are being developed Theneed to enable individual components to communicate with each other andallow autonomic managers to execute management functions (monitoring,provisioning, configuring, etc.) across disparate resources has resulted in the

impor-development of the Web Services Distributed Management (WSDM) standard

ratified by the Organization for the Advancement of Structured InformationStandards (OASIS), which now provides a standard for a common interfacefor managed resources.10

The need to correlate and aggregate log file information for use by tonomic managers in problem determination and self-healing functions has

au-resulted in the WSDM event format (WEF) standard, which is part of the OASIS

WSDM standard just mentioned.11Traditionally, IT administrators must porethrough hundreds of log files, all generated in different formats, and corre-late that information manually Using the WEF specification, resources cancreate events and log records in a standard format (either natively or throughconversion) that can be correlated programmatically by autonomic man-agers Already, this format is helping to relieve some of the manual workassociated with problem determination One company that now convertsselected log data into WEF has realized a 40 percent reduction in time spent onproblem determination, eliminated critical runtime errors, improved overallquality and availability of its IT services, and minimized delays in customerproduction schedules.12 And that’s simply through the use of WEF by ITprofessionals who are still manually handling problem determination andresolution functions Ultimately, the development and implementation ofadditional autonomic computing standards will enable the software to man-age these processes as well

Another important standard is the Solution Deployment Descriptor (SDD)

specification that is currently under review by the OASIS SDD TechnicalCommittee This specification will provide the framework for standard

P1: Binaya Dash

Overview of Autonomic Computing: Origins, Evolution, Direction 15

descriptors that enable individual software entities to identify their dencies and their relationships so that a system can understand how the piecesfit together and how change will affect them.13This information can be used

depen-to validate installations (such as conducting the necessary dependency checksand downloading required patches or software prior to installation), supportdynamic changes, and enable high availability

These are clearly the first steps in creating a uniform approach that enablesthe implementation of self-managing autonomic behavior across heteroge-neous systems Other standards currently in development include:

• The specification of symptoms (patterns of events) in a common

format for use by autonomic managers in problem determinationand self-healing activities

• A consistent format for capturing IT policies so that autonomic

systems can ensure that appropriate actions are performed inaccordance with corporate policies

• A consistent method for managing change requests

• A standard approach for obtaining configuration information fromheterogeneous systems so that information can be consolidated in

a single configuration management database

1.5.3 Reference Implementations

Through the use of software development kits, technology toolkits, of-concept, and other reference implementations, the application of thesestandards is well underway Individual vendors have work in progress toenable their products to function as part of an autonomic ecosystem and

proofs-IT departments are working to facilitate problem determination and uration management For example, one software vendor—working to de-crease the deployment time for new solutions and free up staff assigned to

config-packaging solution componentry—separated its solution into installable units

that could be aggregated at the customer site Using SDD technology, thecompany’s sales and services organizations could build solutions at the cus-tomer sites based on each customer’s specific business need By doing so, theorganization realized a 30 percent reduction in solution deployment timeand experienced a 50 percent gain in productivity of the packaging staff.14

Key to this success was the availability of open source technology that ables industry-wide collaboration on the development of new software Forexample, IBM’s delivery of open source self-managing autonomic technolo-gies based on industry standards—technologies for solution deployment,

coverpages.org/SDD-Announce2005.html (accessed Dec 6, 2005).