John wiley sons mastering the sap business information warehouse fly

Library of Congress Cataloging-in-Publication Data: Mastering the SAP business information warehouse / Kevin McDonald .... Mastering the SAP Business Information Warehouse provides a dee

TE AM

Team-Fly®

Wiley Publishing, Inc.

Kevin McDonald Andreas Wilmsmeier

David C Dixon W.H Inmon

Mastering the SAP Business Information

Warehouse

Information Warehouse

Wiley Publishing, Inc.

Kevin McDonald Andreas Wilmsmeier

David C Dixon W.H Inmon

Mastering the SAP Business Information

Warehouse

Publisher: Robert Ipsen

Executive Editor: Robert M Elliott

Assistant Editor: Emilie Herman

Managing Editor: John Atkins

New Media Editor: Brian Snapp

Text Design & Composition: John Wiley Composition Services

Designations used by companies to distinguish their products are often claimed as trademarks In all instances where John Wiley & Sons, Inc., is aware of a claim, the product names appear in initial capi-

com-plete information regarding trademarks and registration.

SAP, the SAP logo, mySAP, SAP R/2, SAP R/3, SAP BW, SAP CRM, SAP GUI, SAP APO, ABAP, BAPI, mySAP.com, mySAP BI, mySAP SEM, mySAP SCM, mySAP BI, and mySAP Enterprise Portals are trademarks of SAP Aktiengesellschaft, Systems, Applications and Products in Data Processing, Neu- rottstrasse 16, 69190 Walldorf, Germany The publisher gratefully acknowledges SAP’s kind permission

to use its trademark in this publication SAP AG is not the publisher of this book and is not responsible for it under any aspect of press law.

This book is printed on acid-free paper ∞

Copyright © 2002 by Kevin McDonald, Andreas Wilmsmeier, David C Dixon, and W.H Inmon

All rights reserved.

Published by Wiley Publishing, Inc.

Published simultaneously in Canada.

Wiley also publishes its books in a variety of electronic formats Some content that appears in print may not be available in electronic books.

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form

or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee

to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4744 Requests to the Publisher for permission should be addressed to the Permissions Depart- ment, John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, (212) 850-6011, fax (212) 850-6008, email: PERMREQ@WILEY.COM.

This publication is designed to provide accurate and authoritative information in regard to the subject matter covered It is sold with the understanding that the publisher is not engaged in professional ser- vices If professional advice or other expert assistance is required, the services of a competent profes- sional person should be sought.

Library of Congress Cataloging-in-Publication Data:

Mastering the SAP business information warehouse / Kevin McDonald

[et al.].

p cm.

Includes bibliographical references and index.

ISBN 0-471-21971-1

1 SAP Business information warehouse 2 Data warehousing 3.

Management information systems 4 Business—Computer programs I.

McDonald, Kevin,

1969-HF5548.4.B875 M37 2002

650’.0285’5785—dc21

2002008736 Printed in the United States of America.

10 9 8 7 6 5 4 3 2 1

sea-Richard M Dunning Chair, American SAP Users Group and BW-BI & Analytics Interest Group

“Kevin McDonald and his team of experts know more about SAP’s Business gence offerings than anyone else—including SAP This book demonstrates their exper-tise clearly and precisely It is a must-have for anyone initiating a BW implementation.”

Intelli-Claudia Imhoff Co-author of Corporate Information Factory, 2nd Edition; Building the

Customer-Centric Enterprise; and Exploration Warehousing (Wiley)

President, Intelligent Solutions, Inc.

“SAP BW is today an essential part of any SAP implementation, as it is becoming the

leading Business Intelligence suite in the world Mastering the SAP Business Information

Warehouse provides a deep understanding of BW’s core capabilities, and serves as a

great head start in the development of new collaborative and predictive solutions foryour enterprise.”

Shai Agassi Executive Board Member, Collaborative Solutions Group, SAP AG

“This book is an excellent guide to traversing SAP terminology and provides hensive coverage of the Business Intelligence solution, which is critical for every SAPcustomer.”

compre-Mark A Smith President, Full Circle Strategies and x-CMO of SAP Portals

“Mastering the SAP Business Information Warehouse provides a comprehensive,

cradle-to-grave review of the most important aspects of the design, development, and mentation of the SAP BW The authors have combined years of data warehousing andSAP BW experience into an effective mix of theory and practical applications This is anexcellent handbook for project members involved in the implementation of SAP busi-ness intelligence solutions such as SAP BW or mySAP.”

imple-Catherine M Roze Author of SAP BW Certification: A Business Information Warehouse Study Guide (Wiley)

IBM and SAP BW Certified Consultant

“It would be hard to imagine a more complete guide to SAP BW for administrators,programmers, and users The authors know the territory well, and have produced avaluable reference work But more than this, the authors are able to weave in the story

of the history and evolution of BW from Hasso Plattner’s mandate to create a ing server’ to BW’s current state as a platform for business intelligence and analyticapplications.”

‘report-Henry Morris

VP for Applications and Information Access, IDC

“Along the lines of the information supply chain, this book lays out the options theSAP Business Information Warehouse provides to build collaborative, closed-loopAnalytic Applications based upon the Business Content building blocks.”

Dr Wolfgang Martin META Group Research Fellow

vi Advance Praise for Mastering the SAP Business Information Warehouse

Rita, Theresa, and Arne Karl, Noriko, and Danjiro

TE AM

Team-Fly®

Foreword xvii

Chapter 1 The Origins of Business Intelligence 1

Early Data Processing Applications 1

What Is a Data Warehouse? 4

ix

Chapter 2 The SAP Business Intelligence Solution 19

Chapter 3 SAP Business Information Warehouse Architecture 35

SAP BW Architectural Components 36

Scheduling 38Monitoring 38

OLAP Engine 50

x Contents

BEx Formatted Reporting 53

InfoProviders 59MultiProviders 60InfoAreas 60

Mapping the Corporate Information Factory to

The Architectural Roots of SAP BW 75

Security 79Administration 81

Summary 87

Chapter 4 Information Modeling 89

Information Modeling Prerequisites 90

Understanding the Business Content 103Developing an Information Model 106

Information Flow Modeling 146

SAP BW Interfaces for Information Flow Modeling 151Summary 152

Chapter 5 Understanding Business Content 153

What Is Business Content? 154Business Content Progression 157

Business Content and Analytic Applications

Using Business Content 164

Myths 165Usability 166Challenges 168

The Future of Business Content 196

Disintermediation 197

Summary 198

xii Contents

Chapter 6 ETL Services 201

Extraction 205

Architecture 216

Transformation 237

InfoObject Transfer Routines 255

Loading 271

Information Analysis and Access Services 310

OLAP Engine 314

Summary 331

xiv Contents

Chapter 8 Analytic Applications 333

Analytic Application Architecture 334Customer Relationship Analytics 337

Strategic Enterprise Management 371

Chapter 9 Administration in the SAP Business Information

Warehouse 395

Process-Oriented Administration 396

Dependencies 398

Security 424

Transports 431

Activation 434Upgrades 436

MultiProviders 448

Standards and Internet Resources 492

xvi Contents

Today’s challenging business environment calls for an enterprise data warehouseapproach that integrates and standardizes information from within the enterprise andfrom outside the enterprise, and it serves as an information hub for internal and exter-nal information consumers In 1997 SAP started developing its own enterprise datawarehouse solution, the SAP Business Information Warehouse Five years later, the

Intelli-gence solution

SAP Business Information Warehouse now plays a central role in nearly every tion brought to market by SAP SAP BW enables customers to accurately forecast andstrategically analyze information for better customer service, optimized business oper-ations, and improved corporate performance By applying analytics solutions and

cus-tomers may realize maximum benefits as quickly as possible Whether the softwarecomponent is CRM, APO, or SEM, they rely on SAP BW as an information hub andplatform for analytics

The writing and publishing of this book reflects the success of and the growinginterest in the SAP Business Information Warehouse Thousands of customers arealready using the software in productive applications—sourcing data from SAP

databases

understanding the SAP Business Information Warehouse Mastering the SAP Business

Information Warehouse links theoretical data warehouse concepts to customer

require-ments, and offers a guide for implementing the solutions It speaks to data ing specialists as well as those that have implemented ERP The authors of this book allhave longterm experience in data warehousing, reporting, and analytic applications

warehous-xvii

Their perspective on SAP BW comes from years of implementing the product andworking with our development teams on enhancing the offering.

Integral to the success of any data warehouse is the availability of resources andguides that describe how to successfully deploy professional solutions You need thisinformation from people who have been in the trenches, who have implemented suc-cessful data warehouses, and who can speak from experience, not simply theory Thisbook provides you with the best of three worlds: an understanding of data warehous-ing, application of these concepts to the SAP BW, and the authors’ own expertise indeploying the solutions

So sit back, enjoy this book from cover to cover, and use it as a reference guide foryour SAP BW implementation

Dr Heinz Haefner Vice President SAP BW Development

SAP AG, Walldorf

xviii Foreword

Team-Fly®

First, we would like to thank Bob Elliott and Emilie Herman of Wiley for their ance and patience through the authoring process and for providing us the opportunity

has helped create a common voice and a consistency throughout the chapters that wemay not have been able to accomplish on our own We would also like to thank ourcoauthor Bill Inmon, who joined the authoring team shortly after the project started

He provided the needed stimulus to take the project through to completion

Writing a book about a software component that has hundreds of people dedicated

to evolving the product as quickly as possible presented an interesting challenge.There were many individuals and teams at SAP AG, SAP Portals, SAP America, andSAP International that provided invaluable feedback and support without which thisbook may never have happened The list below does not come close to acknowledgingall the people who supported us in our lives, careers, and on this project

A special thank you goes to Klaus Kreplin, Dr Werner Sinzig, and Lothar Kallweitfor their guidance and mentoring through the years The SAP BW™ development andproduct management teams, namely Heinz Häfner, Lothar Henkes, Claudia Weller,Gunther Rothermel, and from the marketing department, Sabine Eggl, provided greatsupport in straightening out the SAP BW architecture sections in the book We’d alsolike to thank Mark A Smith for his eleventh-hour briefing on the SAP Business Intelli-gence Solution and his insight over the years as we have often debated the future of theindustry Naeem Hashmi and Claudia Imhoff both lent an empathetic ear and helpedcontemplate key decisions, notably to write or not to write a book

Countless discussions on information modeling, data extraction, and staging withRainer Höltke and Christian Dressler from the SAP BW development team as well asJürgen Habermeier have evolved our thinking on these subjects The SAP BW RegionalImplementation Groups (RIGs) have rounded out our thoughts in several areas related

to system administration Much appreciation goes specifically to Joachim Mette andhis keen scrutiny during review and Rudolph Hennecke for his meaningful ideas and

xix

insightful thoughts They have somehow found the time to discuss product plans,review our outline, and provide invaluable feedback Thanks to Lonnie Luehmann ofNike for his meticulous attention to detail and helpful comments.

Many thanks go to Jürgen Hagedorn for his advice and candid commentary on thefuture of analytic applications, Marcus Dill for his feedback and enthusiastic support,and Armin Elbert for his contributions to our book discussions Also, thank you toAlbrecht Ricken of Quandian and Colin Bailey for lending their expertise and sugges-tions for improving the analytic applications chapter in the book Stefan Sigg, GuidoSchröder, Falko Schneider, and Andreas Wesselmann, thank you all for your input,candid feedback, and understanding Your development on the Business Explorer isdeserving of a book of its own

Since the release of SAP BW in 1998, Klaus Majenz has always provided invaluableinsight into planning for and managing performance He, Thomas Zurek, Alex Peter,Thomas Becker, and Uwe Heinz supported us throughout the authoring process

A very special thank you to Julia McDonald for her patience and understandingthroughout yet another project, to a good friend and colleague, Bryan Katis, for themany years of knowledge sharing, collaboration, and valued support, and to Rita,Theresa, and Arne Holverscheid for their support and patience throughout the author-ing process The COMPENDIT team, specifically Stefan Krauskopf, MohammadMazhar, Bahram Assadollahzadeh, Jens Koerner, Rob Valcich, and those team mem-bers who help us research 3.0 topics, thank you all

A final thanks to you the readers and organizations that have opted to implementSAP BW as part of a business intelligence solution

xx Acknowledgments

Kevin McDonald is a cofounder and the CEO of DIT, Inc., a leading business intelligence consulting services

COMPEN-firm that was named in Entrepreneur magazine’s “Hot 100”

listing of America’s fastest-growing new businesses He hasinstructed, implemented, and designed mission-critical client/server transaction processing systems and enterprise decision-support systems for dozens of clients Prior to cofoundingCOMPENDIT in 1999, Kevin was the Director of New Dimen-sion Technology and a corporate spokesperson for SAP,where he had worked since 1993 in both America and Ger-many He was Program Manager during the successful market launch of SAP BusinessInformation Warehouse (SAP BW), and he authored the SAP BW product map that wasfirst used to define the scope and development direction for the software component Kevin started his career at Baxter Healthcare, where he held positions in both IT andFinance functions He has authored numerous articles about SAP’s Business Intelli-gence Solution for The Data Warehouse Institute’s newsletter, and he has made pre-sentations on business intelligence at DCI’s Data Warehouse World, HP World, ERPWorld, TDWI conferences, ASUG, SAP TechEd, SAP Sapphire, Informix User Confer-ence, Decision Processing 98 and 99, and Informatica World Kevin may be contacted

at kevin.mcdonald@compendit.com

xxi

Andreas Wilmsmeier is a managing director of DIT Deutschland Andreas has been a member of the initialSAP BW core development team, where he has been respon-sible for designing and implementing parts of the StagingEngine (e.g., the Staging BAPI) Andreas has been consultingSAP BW clients since the initial first customer shipment ofSAP BW 1.2A in early 1998 and has continued to contribute tothe development of SAP BW by providing feedback from thefield.

COMPEN-After receiving his diploma in computer science and ness economics, Andreas started his career in developing data warehouse and Internetsolutions Prior to joining COMPENDIT, Andreas developed and managed the Busi-ness Intelligence line of business at a German consulting company

busi-His knowledge of data warehousing, data mining, and knowledge management hasbeen showcased at numerous international conferences, including SAP Sapphire, SAPTechEd, ASUG, Cebit in Hanover, Germany, and Systems in Munich, Germany.Andreas has authored articles in the SAP Technical Journal, now featured on intelli-

gentERP.com and the German language E/3 Magazine Andreas may be contacted at

andreas.wilmsmeier@compendit.com

COM-PENDIT, Inc David started his career in 1995 as a financialsand controlling (FI/CO) consultant with SAP, specializing inall of the SAP reporting and analysis applications and tools.Prior to joining COMPENDIT as a founding team member,David was a Platinum Consultant with SAP

David has worked with SAP’s SEM development team onnumerous occasions in support of Business Consolidationsand Business Planning and Simulation He has extensiveproject experience in implementing complicated global solu-tions for Fortune 100 companies David has presented at various SAP BW forums such

as SAP TechEd and ASUG He may be contacted at david.dixon@compendit.com

and is co-creator of the “corporate information factory.” Hehas over 28 years of experience in database technology man-agement and data warehouse design He is known globallyfor his seminars on developing data warehouses and hasbeen a keynote speaker for every major computing associa-tion and many industry conferences, seminars, and tradeshows As an author, Bill has written about a variety of topics

on building, usage, and maintenance of the data warehouseand the corporate information factory More than 500 of his

articles have been published in major computer journals such as Datamation,

Computer-World, and Byte magazine Bill is currently a columnist with Data Management Review,

and has been since its inception He has published 39 books

xxii About the Authors

Bill founded and took public a Silicon Valley company, Prism Solutions, in 1991.Prism Solutions became Ardent Software, which was recently acquired by Informix,and subsequently renamed Ascential Software Most recently (1999) Bill decided topublish his vast data warehousing information resources on his Web site atwww.billinmon.com The Web site has now grown to support millions of visitors amonth Bill consults with a large number of Fortune 1000 clients, offering data ware-house design and database management services Recently Bill collaborated withClaudia Imhoff and others to develop the corporate information factory.

The enterprise resource planning (ERP) and data warehousing industries have evolved

in parallel over the past decade Now these two trends are finally converging More

as a transaction-processing solution These organizations may have started their jects in isolated divisions, but over the past several years, larger portions of the busi-ness are being run on ERP software

pro-While we have seen only a few IT shops that rely exclusively on SAP software, thepercentage of online transaction processing that is conducted via SAP software hassteadily increased We see many organizations adopting IT philosophies that have SAPsoftware as the default solution It is not uncommon to hear a CFO comment, “You bet-ter have a very good reason not to use the SAP software we already paid for.” Theseorganizations have moved beyond automating and integrating business processes andnow wish to optimize their business performance, reduce the slack in their supplychains, and realize the potential value of their customer relationships

Parallel to the ERP and business process reengineering evolution was the evolution

of informational processing, now commonly referred to as business intelligence Theexplosive growth in data captured by organizations, in part due to the rapid adoption

of the Internet, has made available an increasing amount of business information This,combined with the increased pace in the way business is conducted, has created sig-nificant demand for efficient decision-making processes The data warehouse was con-ceived to enable such processes

SAP has brought to market software that has created a tremendous opportunity fororganizations to lay a common foundation for both the transaction-processing appli-cation and the decision-processing applications Organizations that have implementedSAP as an ERP solution may find themselves able to bypass their competition byquickly deploying closed-loop analytic applications and Enterprise Portals that are notonly technically integrated with their transaction processing systems but also inte-grated from a business perspective The SAP Business Intelligence Solution, which

xxv

includes an Enterprise Portal™ with Knowledge Management and SAP BusinessInformation Warehouse (SAP BW™), not only establishes a new benchmark for busi-ness intelligence solutions but also creates, when implemented in conjunction with the

We have often stated that the decision to use SAP BW is less a matter of volume ofdata that is found in SAP software versus that found in non-SAP systems than of thevolume and potency of the meta data found in an organization Regardless of the roleSAP BW is to play in your organization, there are several steps you may take to ensurethat you are getting the most out of your SAP data while leveraging the processes,people, and products you already employ

Why did we write this book? First, many books on SAP focus on step-by-stepinstructions for accomplishing a given configuration task and spoon-feed readers withchecklists, transaction codes, and code samples Our goal was to bridge the gapbetween these low-level books and the high-level books that focus on data warehousingarchitecture but do not necessarily explain how the SAP BW software may be used torealize such architecture Furthermore, a goal was to expose the various implementa-tion options available in the SAP BW component and to explore the fundamental archi-tecture and concepts in order to enable readers to understand and use these options

We have included how-to guides on the accompanying Web site that address some of

the options in a step-by-step manner.

Our hope is that this book will inspire readers to implement these options in order

to strategically analyze information and accurately forecast optimized operations,

product, the SAP BW 3.0 version is approaching a richness in functionality that willsoon extend beyond the capacity of any one person in the world, not even among itscreators, to know in detail every option available in the software and the potential con-sequences that implementing one option may have on another The product’s maturity,combined with the fact that the SAP BW component is now part of nearly every solu-tion sold by SAP, compelled us to share our knowledge from years of developing andimplementing the SAP BW and put our experience in writing for the data warehousingindustry

We have taken the approach that technology is there to serve business and haveattempted to counterbalance technical sections with commentary on how a particularoption may be used to drive business value

As the lines between transaction processing and decision processing blur and theunification of these system and Web services becomes dynamic, and with the adoption

of Enterprise Portals, organizations implementing SAP will search for ways to get the

most out of their SAP data Mastering SAP Business Information Warehouse looks at

options for designing, building, deploying, populating, accessing, analyzing, ing, and administering the data and information in the SAP BW component This book

present-is our contribution to accelerating the search actionable information

xxvi Introduction

Who Should Read This Book

In our experience, most SAP BW implementations are deployed in organizations thathave implemented SAP as an ERP solution As such, we assume that you, the SAP BWproject team member, are familiar with SAP, although you may not be as familiar withdata warehousing and business intelligence We have included a list of other resourcesthat offer more in-depth background on these subjects on the accompanying Web sitefor this book (www.wiley.com/compbooks/mcdonald)

Business and IT professionals of large organizations who are considering menting any of the mySAP Solutions will also find this book useful, as SAP BW isincluded as a component in nearly every SAP business solution that is sold

imple-How This Book Is Organized

This book may be thought of as consisting of four parts, meant to reflect the process anorganization goes through during an implementation of the software

We begin with an introduction to business intelligence and the SAP BW architecture,which is meant to provide a backdrop for readers that may come from more of an SAPERP implementation background than a data warehousing background Chapter 1 is anintroduction to business intelligence and how enterprises tackled challenges such as:

From these needs arose the idea of combining both traditional data with documents

to offer organizations a collaborative platform for analyzing information and ing business performance Today this is called business intelligence

optimiz-While data warehousing and analytic applications are not new, the toolsets available

to realize them are—relatively speaking We have entered into a time where technicalintegration is a worry of the past and semantic integration is at the forefront The recentintroduction of Enterprise Portals has enabled dynamic closed-loop analytical process-ing In Chapter 1, we examine SAP’s flagship enterprise resource planning package.You will quickly see what SAP realized around 1996: ERP systems are not designed foranalytical processing We explain the challenges of reporting and analyzing data in theERP system

Readers who are familiar with data warehousing, the evolution of SAP, and mation processing may wish to start reading the book at Chapter 2, where we discussthe SAP BW component and how it fits into the SAP Business Intelligence Solution that

infor-includes Enterprise Portal There we define the major architectural components of theSAP BI Solution and set the stage for the services found in SAP BW From the SAP pro-duction data extractors to the analysis of information via a Web browser, readers willstart to understand the breadth and depth of functionality in the SAP BW in Chapter 3.

We also map the SAP BW to the corporate information factory (CIF) You will quicklysee the characteristics SAP BW has in common with non-SAP data warehousing plat-forms, as well as the unique features found in SAP BW

The second part focuses on the business content and the options available to mation modelers as they work to deliver value from the data stored in SAP BW Chap-ter 4 explains the information model and how this collection of meta data objects,which describes business processes, business objects, information containers, and theirmutual relationships, as well as the scope, granularity, and semantics of the informa-tion available in the data warehouse system, is an important part of a proper deploy-ment of SAP BW We also comment on the design of an information flow model, or theflow of information from the originating systems through a possibly multistaged datawarehouse system landscape to an information consumer that might be an end user, ananalyst, or another system utilizing the information provided to control a specific deci-sion or business process (closed-loop approach)

infor-Chapter 5 defines business content and its application We use the analogy of ing blocks to help describe Business Content, in the sense that Business Contentincludes the extraction for data sources, transformation of that data, storage in aschema, and the queries and applications that access and present the information.These building blocks are foundational to analytic applications Business Content metadata can also be mapped to architectural components of the CIF You will see how Busi-ness Content has grown horizontally and vertically in scope The usability of BusinessContent is assessed and the challenges to its growth critiqued We end the chapter bypresenting three subject area scenarios in Business Content linking the informationmodels to information modeling concepts introduced in Chapter 4

build-The third section focuses on the services available in the SAP BW used to realizesuch an information model (Chapters 6 to 8) These include:

Chapter 6 leads readers through identifying the SAP sources of data, extracting datafrom these sources, applying the transformations required, and storing the trans-formed data in a way that best supports reporting and analysis In other words, thischapter presents the functionality provided by the ETL services layer of the SAP BWarchitecture This is often the most time-consuming part of building a data warehouse

solution In the CIF framework this is referred to as sourcing and manufacturing of data

and information The options described in Chapter 6 will enable readers to take aninformation model and instantiate it in the SAP BW software

In Chapter 6, we describe how to integrate and transform data so it may be stored inthe various SAP BW storage constructs such as ODS Objects, InfoCubes, and MasterData Chapter 7 picks up the information logistics process where Chapter 6 left off and

xxviii Introduction

Team-Fly®

highlights the main services provided in SAP BW that retrieve data, turn it into ingful business information, and deliver it to an information consumer The chapter hasbeen organized in two main sections: SAP BW Information access and analysis servicesand SAP BW presentation services The Business Explorer suite of tools, including BEx Designer, BEx Analyzer, BEx Formatted Reporting, BEx Mobile, and BEx WebApplication Designer, are described We also have included a section on the applicationprogramming interfaces (APIs) options, with which custom applications or third-partytools may interface.

mean-The SAP BW is a platform for building analytic applications mean-The architecture of ananalytic application is detailed in Chapter 8 Readers will see three different examples

of analytic applications and how they each interact with SAP BW The three analyticapplications covered are customer relationship analytics supported by mySAP Cus-tomer Relationship Management (mySAP CRM), supply chain analytics supported bymySAP Supply Chain Management (mySAP SCM), and Corporate Performance Mea-surement supported by mySAP Financials

The last section focuses on the administration and performance options for the software component (Chapters 9 and 10) In this section of the book, SAP BW adminis-tration tasks—both process-oriented tasks and system-oriented tasks—are described.Process-oriented tasks consist of application processes such as scheduling, monitoring,and troubleshooting of data loads, as well as archiving System-oriented tasks consist

of security measures, transports, and upgrades There are many different applicationprocesses besides data loading, such as index maintenance, building aggregates, andbatch scheduling of reporting jobs All these application processes can have complexdependencies The SAP BW technology process chains are covered in detail

SAP BW security is explained from a design perspective, detailing the decisions tomake when building authorizations, such as making them user-based versus role-based or object-centric versus data-centric We continue the administration section bydescribing the options in the SAP BW transport system with specific attention on thetransportation of meta data from a development system to quality assurance and pro-duction We end Chapter 9 looking at the considerations for a multilayered applicationenvironment when performing an upgrade

From an end user’s perspective, the data warehouse is only as good as the last query.Performance should be carefully planned and given constant attention However,because of the discontinuous, unpredictable user behavior characteristic of an infor-mation consumer, this may prove to be a challenging task In Chapter 10 we describethe performance management process We have divided this discussion into two parts:performance planning and performance management

During the system development process, performance planning is essential mance planning lays the foundation for overall system performance It involvesreviewing information models; designing an appropriate information logistics modeland system landscape; implementing efficient transformations; defining parallel, collision-free data loads and data maintenance process chains; and managing userexpectations

Perfor-Performance management, on the other hand, is part of production system istration It entails monitoring all processes and resources in the system We describehow the system may be tuned by defining aggregates, adjusting operating system

admin-parameters, determining database management system settings, and configuringhardware Like many of the options that we describe in the book, performance plan-ning and performance management deal with trade-offs The trade-offs in this case arebetween disk and memory space, flexibility, loading time, and retrieval time.

Throughout the book we have included images, lists, notes, and tips to you to helpyou implement your own solutions This book is not a step-by-step list of configurationsettings, and it is not intended to be a substitute for hands-on learning You do notbecome a black belt in karate by reading a book The same is the case with SAP BW Weencourage our readers to log in to a test system, configure the services described in thisbook, and assess the trade-offs

What’s on the Web Site

The accompanying Web site for this book can be found at www.wiley.com/compbooks/mcdonald It contains:

SAP BW system behave in a particular manner

authors and the business drivers that lead to the use of SAP BW The case studies are intended to be an executive overview of how the SAP BW is used

to solve specific business issues

From Here

“The mind is not a vessel to be filled, yet a spark to be lighted.” It is our hope that ers of this book will discover the options that are available in the SAP BW softwarecomponent and uncover a new means to drive business performance through theimplementation of the SAP Business Intelligence Solution We hope you enjoy the book

read-as we open with Chapter 1 and an introduction to business intelligence

xxx Introduction

The origins of business intelligence may be traced back to the first data processingapplications, which were simple applications such as accounts payable and receivable.These applications ran on sequential technology, such as magnetic and paper tapes.Using sequential media for storage meant the entire file had to be accessed, even if only

a fraction of the file was needed Oxide often stripped off of magnetic tapes, and entirefiles were lost These issues led to the need for a new way to analyze information

In this chapter we discuss how enterprises tackled the challenges of extracting datafrom online transaction processing systems, dealing with poor data quality, and struc-turing the data We describe what a data warehouse is from its data model, tablestructure, granularity, and support of historical data, and how it fits into a broaderintellectual concept called the corporate information factory, discussed later in thechapter

Early Data Processing Applications

With sequential storage, data was organized onto what was called a master file, whichheld central information that was useful to many applications Punch cards, magnetictapes, and reports were generated from the applications, but were soon replaced bydisk storage With disk storage, data could be accessed directly and efficiently Proces-sors grew more powerful and versatile, and the speed and costs of processing droppeddramatically As data was stored onto disk, master files mutated into databases A

1

The Origins of Business

Intelligence

database is a centralized collection of data that resides on disk storage and is available

for processing by any application that needs it

With disk storage, transactions could be processed directly and online against thecentral databases that were built This meant that the transaction could execute in ashort amount of time—from 2 to 3 seconds Soon, online transaction processingresulted in online applications Online applications were tied together by the centralonline databases that they ran against

Online applications focused on high availability and good response time The onlinetransaction application soon became central to running the business, as the onlinetransaction became an essential part of direct interaction with the customer However,the custom development of online transaction applications led to several challenges:

An additional problem with many online applications was lack of integrationamong them Each online application was developed according to its own specifica-tion, and a different set of requirements shaped each one There was no commonunderstanding of what was:

Enter Extract Files

The first reaction to the challenge of not having corporate data was to create what was

called an extract file An extract file would be created by a database from one

applica-tion and shipped to another applicaapplica-tion, so it seemed that data could be shared andcorporate data could be created Extracts became very popular, and soon there were a

2 Chapter 1

lot of them Every new extraction exacerbated the problems of the spiderweb Addingextractions made matters worse, not better The problems of the spiderweb included:

the element had a value of 25 In another place the element had a value of 67 In

still another place the element had a value of 135 No one really knew what the

right value was

was being shuffled from one place to the next, and in doing so, the burden of

massive amounts of data being repeated over and over began to add up to

significant amounts of storage and processing power

one day, the value of a unit of data may change five or six times The extract

processing simply was not capable of keeping up with the speed with which

data changed

with no coordination or integration with outside silos The organization found

itself making decisions that were contrary to the interest of other parts of the

organization

applications were difficult to change in any case But wrapping lines of

extrac-tion around the online applicaextrac-tions glued those applicaextrac-tions into a permanent

position

coordina-tion requirements on the environment, which ensured that accurate data was

impossible to obtain, and so forth

Of particular interest is the lack of historical data Online applications value currentdata How much is a bank account balance right now? Where is a shipment right now?What is the status of an insurance claim right now? Online applications optimize the

“right now” aspect of information processing As soon as data became dated, it was carded Lots of historical data clogged the arteries of efficient online processing There-fore, online data and processing required that older data be jettisoned as soon as possible.But there is real value in historical data With historical data, organizations can start

dis-to see the forest and the trees With hisdis-torical data, organizations can start dis-to

under-stand their customer base, because customers are creatures of habit

Because there was no corporate integrated data or historical data, data was difficult

to access Even if accessed, data was not trustworthy, so it is no wonder organizationsbegan to grow frustrated with their ability to find and process information Depart-ment after department would say, “I know the data is somewhere in my corporation; if

I could only get at it.”

The frustrations of the end user with data locked in the spiderweb environmentresulted in the realization that there were different kinds of data There was an essen-

tial difference between operational data and informational data Table 1.1 outlines those

differences

Table 1.1 Characteristics of Operational versus Informational Systems

OPERATIONAL INFORMATIONAL/DSS

Supports small uniform transactions Supports mixed workload

Yields 2- to 3-second response time Yields 30- to 24-hour response timeData designed for optimal storage Data designed for optimal access

Data designed around functional usage Data designed around subject areasReferential integrity is useful Referential integrity is not useful

High availability is normal High availability is nice to have

A fundamental split exists between operational information and informational

informa-tion Operational information is used to support the daily operations of a business.

Informational information is commonly called decision support systems (DSS)

informa-tion The foundation for DSS processing became an architectural structure known as

the data warehouse A data warehouse is a physically distinct place from the online

operational application In the following sections we describe the data warehouse andhow it enables informational information

What Is a Data Warehouse?

Since the beginning of movement toward data warehousing, data warehouses havebeen defined as being:

orga-nization Classic examples include subject area databases for customer, material,vendor, and transaction

another

placed in the warehouse, it is not subject to change This contrasts with a record

of data in an online environment, which is indeed very much subject to change

4 Chapter 1

Time-variant. A record is accurate only as of some moment in time In some

cases the moment in time is a single moment In other cases it is a span of time

But in any case, the values of data found in a data warehouse are accurate and

relevant only to some moment in time

Created for the purpose of management decisions.

The preceding definition has remained unchanged since the inception of the datawarehouse In addition, the data warehouse provides:

it is outside of SAP And the elements of a data warehouse are as valid for SAP as forthe non-SAP environment

The data warehouse evolves from these requirements and supports the process ofmoving data from source systems, transforming, and cleansing the data so that it may

be stored in an integrated data model at an atomic level of granularity There are manyfactors that influence the design of a data warehouse and the structure that datarecords are stored We discuss some of these factors in the next sections

The Data Model

The design of the data warehouse begins with a data model At the highest level, the

data model is known as an entity relationship diagram (ERD) The ERD represents the

abstraction of the granular data found in the data warehouse Note that for the poses of data warehouse design the ERD represents only granular data, not deriveddata This distinction is important because it greatly limits the size and complexity ofthe data model There are, of course, other data models outside of the data warehouseenvironment that do attempt to take into account derived data and atomic data

pur-The ERD consists of entities and relationships Each entity represents a major subjectarea of the corporation Typical subject areas are customer, product, transaction, and

vendor Each entity is further defined at a lower level of data modeling called the data

item set (DIS) The DIS specifies a lower level of detail than the entity does,

encom-passing such things as keys and attributes, as well as the structure of those things The

DIS is further broken down into a low level of design called the physical design At the

physical level of design the physical characteristics of the data are created

N OT E For a more detailed description of the methodology required to build a

data warehouse, visit the Web site www.billinmon.com.

Figure 1.1 The data warehouse is designed from the data model.

The data warehouse is now specified and defined to the database management tem that will house it Other physical aspects of database design such as partitioning,loading, indexing, storage media, and timestamping are determined here as well Fig-ure 1.1 shows the design of the data warehouse from the different components of thedata model

sys-Different Physical Tables

The data warehouse is made up of interrelated tables or physical databases Within thedata warehouse, there are different physical tables that represent different subject areas

or even subsets of subject areas One table relates to another by means of a shared key

or foreign key relationship The data warehouse has five subject areas:

mid-level data model—

the data item set (DIS)

low-level data model—

the physical design

datawarehouse

6 Chapter 1

Integration and Transformation Processing

One of the most important and most difficult aspects of data warehouse developmentand population is that the movement and conversion of data from theoperational/legacy source environment It is estimated that for the first iteration ofdevelopment, at least 75 percent of the resources required for development will beexpended here During extraction, data is pulled from the legacy environment andmoved into the data warehouse environment This data is pulled from a variety ofsources, such as mainframe order entry systems, proprietary shop flow control sys-tems, and custom-built payroll systems

But data is not merely moved from the legacy environment to the data warehouseenvironment Instead, data undergoes a thorough transformation as it is moved,including:

In the early days of data warehousing there was no way to create the interface grams between the legacy environment and the data warehouse other than to writethem by hand But with recent technology, extract/transfer/load (ETL) softwareautomatically creates the interfaces needed to bring the data into the world of datawarehouse Figure 1.2 shows the place of ETL between the source systems and thedata warehouse

pro-One of the real benefits of ETL processing is that data enters the ETL process in anapplication mode and exits in an integrated corporate mode The processing thatoccurs inside ETL software allows the data to be integrated

SAP BW addresses the ETL process by including a so-called Staging Engine to form simple transformations and conversions In Chapter 6 we detail the functiondelivered within the SAP BW related to ETL, as well as discuss the usage of specializedETL tools that are certified with SAP BW