Auerbach oracle internals tips tricks and techniques for DBAs jul 2001 ISBN 084931139x

Converting from one file access method to another e.g., converting from an indexed or flat file structure into a DBMS Converting from one coding structure or format to another e.g.,from

Oracle Internals: Tips, Tricks, and Techniques for DBAs

by Donald K.

Burleson (ed)

ISBN:084931139X

Auerbach Publications © 2002(878 pages)

Provides a valuable compilation of

articles,pointers and practices from the Oracle Internal newsletter.

Chapter 69 - Using the DBMS_LOCK Built-In Package

List of Appendices

List of Exhibits

Team-Fly

Back Cover

If you are a typical Oracle professional, you don't have the luxury of time to keep up with new technology and read all the new manuals to understand each new

Demonstrates how Java can be used to create

portable and robust Oracle applications

Contains invlauable tips and tricks regarding how to use the different dialects of UNIX to communicate with the database

Provides information from Oracle tuning experts on maximizing the performance of SQL statements

within Oracle's library cache and ensuring that your Oracle database runs at optimal levels

Covers the technical nuances of the EBU and RMAN products

Oracle Internals-Tips, Tricks, and Techniques for DBAs

Donald K Burleson, Editor

Chapter 44, 'Oracle 8i Buffer Cache: New Features,' copyright ©2000 byJohn Beresniewicz andSavant Corporation Printed with permission

Reasonable efforts have been made to publish reliable data and

information, but the author and the publisher cannot assume

responsibility for the validity of all materials or for the consequences oftheir use

Neither this book nor any part may be reproduced or transmitted in anyform or by any means, electronic or mechanical, including photocopying,microfilming, and recording, or by any information storage or retrievalsystem, without prior permission in writing from the publisher

personal use, or the personal or internal use of specific clients, may begranted by CRC Press LLC, provided that $1.50 per page photocopied ispaid directly to Copyright Clearance Center, 222 Rosewood Drive,

Danvers, MA 01923 USA The fee code for users of the TransactionalReporting Service is ISBN 0-8493-1139-X/02/$0.00+$1.50 The fee issubject to change without notice For organizations that have been

granted a photocopy license by the CCC, a separate system of paymenthas been arranged

The consent of CRC Press LLC does not extend to copying for generaldistribution, for promotion, for creating new works, or for resale Specificpermission must be obtained in writing from CRC Press LLC for suchcopying

Direct all inquiries to CRC Press LLC, 2000 N.W Corporate Blvd., BocaRaton, Florida 33431

Lecturer, Department of Computer Science, University of West Florida, Pensacola, Florida

Charles Banyay

Manager, Deloitte Consulting, Toronto, Ontario, Canada

John Beresniewicz

Technical Product Manager, Precise Software Solutions, Montgomery Village, Maryland

Bradley D Brown

Chairman and Chief Architect, The Ultimate Software Consultants, Lakewood, Colorado

Jeffery Feldman

Manager, Deloitte Consulting, Toronto, Ontario, Canada

Howard Fosdick

Independent Database Administrator, Chicago, Illinois

CISA, CDE, CGFM, MSBA, Audit Advisor and Faculty Member,

Computer Information Systems Department, California State Polytechnic University, Pomona, California

Jonathan Gennick

Oracle Database Administrator, Writer, and Editor, O'Reilly & Associates, Cambridge, Massachusetts

Founder, CEO, and President, Dot to Dot Communications, Denver, Colorado

Robin Schumacher

Vice President, Product Management, Embarcadero Technologies, Inc., New Albany, Indiana

Gary E Sharpe

President and Chief Executive Officer, Terascape Software, Needham, Massachusetts

Serg Shestakov

Oracle Database Administrator, St Petersburg, Russia

Anunaya Shrivastava

Financial and Manufacturing Applications, Computech Answers, Detroit, Michigan

David C Sisk

Poulenc, Research Triangle Park, North Carolina

Database Administrator and Internal Technology Consultant, Rhone-Michael J.D Sutton

Adm.A., CMC, ISP, MIT, Business Process and Document Management Services Group Director, Rockland, Ottawa, Ontario, Canada

Biju Thomas

Oracle Database Administrator, Renaissance Worldwide, Inc., Fort

Worth, Texas

Mark B Wallace

Since the inception of Oracle Internals we have been very fortunate to

have some of the world's leading Oracle gurus contribute highly technicalmaterial to our publication

Today's Oracle professionals are now challenged more than ever before

to keep pace with the Oracle technology Oracle has evolved from a

simple relational database into one of the most complex E commerceplatforms ever devised It is not enough for today's Oracle professional tojust understand the Oracle database Rather, the Oracle professionalmust also understand the components of Web server technology, XML,Oracle security, Oracle and Java, and a host of other areas that are veryimportant so that Oracle administrators can do their jobs properly

Just as the Oracle professional is challenged to keep pace with

technology, Oracle Internals is challenged with finding top-notch technical

articles that can be used to assist Oracle professionals in maximizing theutilization of their knowledge to incorporate of all of Oracle's new

features

This book is the culmination of the best articles from Oracle Internals

over the past two years It is our hope that you will find these in-depthstudies of Oracle to be both informative and useful.

Techniques

The first area we present is one regarding Oracle applications tips andtechniques As is known, Oracle supports a host of different front ends,all the way from Visual Basic, to PowerBuilder, Web HTML, and Oracle'snative development tools There is a great deal of demand from the

marketplace for people who have expertise in the development of Oracleapplications and can provide tips for the rapid and successful

development of custom applications This section also discusses theOracle Applications product platforms and how Oracle managers can useOracle Apps to fully implement the various Oracle Applications modules.The area of Oracle applications is one of the broadest within the Oraclefield

Rather than concentrating on a specific product, Oracle Internals

provides solicited information regarding Oracle Forms, Oracle Developer,and other applications tools that directly impact the performance andconfiguration of the Oracle database

The chapters in this section derive from a wealth of sources - from HervéDeschamps, a senior Oracle corporate technical manager, to John

Palinski, a noted author and expert on Oracle developer tools

Chapter 6: Oracle Designer 2.1.2.API: Enforcing Column NamingStandards

Chapter 1: A Practical Example of Data Conversion

Charles Banyay

Conversion - the word is enough to dim the enthusiasm of most systemsdevelopers The word instills fear in some, trepidation and loathing inothers Regardless of the nature of the project with which she or he isinvolved, if there is any conversion effort involved, the reaction is thesame Exclude it from project scope! Let someone else do it! Althoughsome might suspect that there may be some religious connotation here,and rightly so, the topic of this article is not converting from one religion

to another Nor is the topic software conversion, although this would becloser to the mark This article deals with the various forms of the

implementations involve some form of conversion When the softwarechanges, the data model or the data itself often changes with it

For some reason, conversions have come to be associated with the

mundane, boring, and tiresome aspects of systems implementation Mostdevelopers would consider conversion efforts as boring, tiresome, anddevoid of interesting challenges, when compared to the implementation

of state-of-the-art technology

This is a misconception in many instances Conversion efforts can be aschallenging as any state-of- the-art technology They can exercise themost creative abilities of technology professionals An entire book

chapter probably could be devoted to discussing the possible reasonsbehind the general lack of enthusiasm for the conversion effort This

chapter, however, will focus on examining the following:

Different types of conversion efforts that one encounters during

The Taxonomy of the conversion effort

Common pitfalls that can have rather detrimental effects on theoverall effort if one is not aware of them and does not take thenecessary precautions beforehand

Converting from one file access method to another (e.g.,

converting from an indexed or flat file structure into a DBMS)

Converting from one coding structure or format to another (e.g.,from EBCDIC to ASCII)

Converting application software such as upgrading versions of anapplication or replacing one application with another (e.g.,

replacing an outmoded payroll application with a state-of-the- artpay benefits system)

One of the most common pitfalls of conversions is to combine into oneconversion effort a change of too many variables, for example, changinghardware, operating system(s), file access method(s), and applicationsoftware all at once Sometimes this cannot be avoided Ideally, however,

as few as possible of the variables should be changed at once With onlyone variable changing, error detection and correction is the simplest Anyproblem can be attributed to the change of the single variable and thuscan be rectified by analyzing the single variable With combinations andpermutations, the effort increases exponentially

Unfortunately, as often happens in life, the ideal state is the exception Ingeneral, it is a rare conversion that does not have some combination ofthe above variables changing at once The taxonomy of each, however,can be explored individually, as can most of the pitfalls Some

combinations will have unique pitfalls simply due to the combination of

changes in variables.

In general, the simplest conversion is upgrading hardware, assuming thatall of the other variables remain constant, that is, operating systems, fileaccess method, coding structure and format, and application software.This can best be illustrated in the PC world PCs have been continuouslyupgraded with relative ease from one configuration to another for the past

10 years As long as the operating system does not change, the upgrade

in hardware usually involves nothing more than copying the files from onehard disk to another This migration of files is usually accomplished withthe assistance of some standard utilities Using utilities rather than

custom-developed software lowers the amount of effort involved in

ensuring that the files have migrated successfully Most utilities providefairly good audit trails for this purpose Even files on the same floppiescan be used in a 286, 386, 486, or Pentium machine Data on floppies donot require any conversion

In environments other than personal computers, the same simplicity ofconversion generally holds true Upgrading from one main frame

configuration to another is relatively easy Changing configurations of aminicomputer, such as from one AS/400 to a more powerful configuration

of the same or from one HP/3000 to a more powerful HP/3000, does notusually require significant effort These kinds of conversions generally areimperceptible to the users and are done without much involvement fromthe user community There usually is no requirement for any user testing

or programmer testing This cannot be said for the more complex

conversions such as changes in the operating system

Another

Changes to the operating system are generally more complicated from aconversion perspective than changes to hardware The complexity,

however, is usually more pronounced at the application software ratherthan the data level There is considerable insulation by the operatingsystem of the application software and associated data from the

hardware In general, there is little to insulate the application softwarefrom the operating system Object-oriented approaches are slowly

changing this fact, but for now it is safe to say that a change in operatingsystem requires a more complex conversion effort than a change in

hardware

For individuals who primarily have limited their involvement with

technology to the WINTEL world, conversion complexity due to changes

in operating system may come as a surprise In the WINTEL world, onegenerally can change from DOS to Windows 3.x to Windows 95 (or

higher) with few or limited problems In fact, most users do this on a

regular basis This may imply that changes in an operating system are assimple as changes in hardware This is a misconception The people atMicrosoft® and to a limited extent at Intel have spent innumerable hours

to ensure that there exists a degree of compatibility between these

operating systems that does not exist in any other environment

Even in the WINTEL world this compatibility is breaking down As themove to NT accelerates, this is becoming evident Users moving to NThave discovered that many of their favorite software programs are notfunctioning as they would like them to, or the programs are not

functioning at all

Although some form of conversion effort is usually involved when

operating systems are changed, the changes in operating system moredefinitely impact the application software than the data The impact onany of the data is usually from indirect sources, such as from a change inone of the other variables such as data format or file access method.Different operating systems may support only different data coding

structures or different file access methods.

It is not often that one changes a file access method while leaving theoperating system and the application system the same The general

reasons for doing this would be suspect unless the current file accessmethod was being abandoned by whomever was providing support

Another valid reason for changing the file access method may be if apackaged application system vendor released a new version of its

application This new version may offer a new data architecture such as

an RDBMS There may be valid reasons, such as better-reporting

capability using third-party tools, for upgrading to this new version withthe RDBMS For whatever reason, a change in file access method

usually requires some form of change in data architecture

A simple illustration of this change in the underlying data architecturewould be in converting a flat file sequential access method to an indexedfile access method Some form of indexing would have to be designedinto the file structure resulting in a change in the underlying data

architecture A more complex example would be in changing from a

sequential access method to a database management system

This change, at a minimum, would involve some degree of data

normalization and a break-up of the single segment or record structure ofthe file The resultant change in data architecture would be quite

substantive This type of conversion generally is not simple and requires

a comprehensive conversion utility In the case where it is a packagedapplication being upgraded, the vendor would probably provide the

conversion utility In the case where a custom-developed application isbeing converted, the conversion utility would probably have to be custom-developed as well

In either case, the tasks are straightforward All of the data must be

converted Every record must have a corresponding entry or entries insome table or tables Each field in the source file needs to be transferred

to the target database Field conversion is not required There is only alimited degree of selection involved The conversion utility is run againstthe source data to create the target data store Often, there are a number

of intermediate steps Different tables or segments of the database may

minimizing the number of variables that can change at once

There are a number of approaches to ensuring that the resultant datastore has the required integrity These approaches are identical to theones used to ensure the integrity of data that is converted due to a

change in the application software

The extent of the effort depends on the degree of reliability that is

required The effort has an obvious cost The lack of data integrity alsohas a cost A financial system requires a high degree of data integrity Itcan be argued that the data controlling the operation of a nuclear powerstation requires an even higher degree of integrity

Another

Changing or upgrading applications always requires converting data fromthe old to the new application These conversions are generally the mostcomplex and require the most effort

One of the first steps in the conversion process is to decide which is thedriving application What is most important in the conversion process? Is

it being exhaustive in converting the data in the old application or

ensuring that the new application has the required fields that it needs tooperate effectively? This may not be intuitively obvious This is not toimply that the decision as to which data to convert is at the whim of theperson designing the conversion programs

There is always a base amount of data that must be converted Many oldapplications, however, accumulate various codes and indicators over theyears that either lose meaning over time or are particular to that

application and are not required in a new application This situation ismore particular to operational applications such as payroll, materialsmanagement, etc When converting data in an operational application,the emphasis is on converting the minimum amount of current data forthe new application to fulfill its role and be able to operate The data

requirements of the new application drive the conversion design

Recordkeeping applications, on the other hand, such as document

management systems and pension administration systems need to retainalmost all of the information within the current database These

applications generally hold a tremendous amount of history that needs to

be retained Recordkeeping applications as a rule require that the

emphasis be on being exhaustive in converting all of the informationwithin the current database The data requirements of the old applicationdrive the conversion design

Generally speaking, converting operational applications is considerablyeasier than converting recordkeeping applications Populating fields

necessary for the operation of a particular piece of software can be done

in various ways New information required for the effective operation of

be collected from other repositories This is generally the most time-consuming and complex way of meeting the data requirements of thenew application On the one extreme of the conversion continuum is thepossibility of disregarding the old application completely and satisfyingthe data requirements of the new application by collecting the data fromoriginal sources This approach is particularly useful when the data

integrity of the old application is very suspect

New information can also be provided as defaults based on other data,which are available from the old application For example, in classifyingemployees for payroll purposes, give each employee the same

classification based on the department where they work In some

instances new information can be fudged if the new data is not critical tothe output required For example, if source medium for an invoice is arequired field in a new accounts payable application and it is not a currentbusiness requirement to keep source medium, then it could be assumedthat all invoices are on paper and the information fudged with that

indicator

Being exhaustive and ensuring that all of the data in an old application isconverted to a new application is, as a rule, more complex than meetingthe data requirements of a new application The complexity is not just inthe conversion The old application must be analyzed much more

thoroughly to ensure that all of the data is understood and put into propercontext The converted data must be screened much more thoroughly toensure that everything has been converted appropriately and is in theproper context within the new application In addition, there are still thedata requirements of the new application to consider

Converting historical information often requires shoehorning existing datainto fields that were not designed for that data Very often, field

conversions are required For various reasons there may be an array ofinformation in the old application for which there is only one field in thenew application Pension administration systems are notorious for this.For example, it is not uncommon to have numerous pension enrollmentdates depending on the prior plans of which an individual was a member.The new application, especially if it is not sophisticated, may provide only

Acquisitions, mergers, and changes in union agreements and

government legislation can wreak havoc with historical recordkeepingsystems These then result in a conversion nightmare when one of theseapplications needs to be converted to a new application system A verycommon experience is that the conversion routines often approach thecomplexity of artificial intelligence applications These are the

conversions that tax the abilities of even the most experienced

developers These conversions are also the ones that are potentially themost interesting and challenging to complete

Once the driving application is determined, the next decision, which isbasic to any conversion, is whether an automated conversion is the mosteffective way of transferring the data to the new application In certaininstances, an automated conversion may not be possible For example, ifthe source data architecture or the data format is not known and cannot

be determined, and there is no export utility provided by the application,then it would be very difficult to develop an automated conversion utility

In certain instances, it is simply not cost-effective to develop an

automated conversion utility If the volume of source data is relatively lowand the complexity of the data requires conversion routines approachingthe complexity of artificial intelligence routines, then a manual conversioneffort may be more cost effective

The next conversion decision that must be made is how to get the datainto the new application For some reason, many application system

designers never think of the initial population of their application with therelevant data It is as if this is supposed to occur by magic There are fourbasic ways of populating the new application In order of relative

complexity, these are:

1 Using a bulk load facility if one is provided by the target

application

2 Generating input transactions into the new application if theapplication is transaction based and the format of the

transactions is known

3 Real-time data entry through keystroke emulation

4 Creating the target database so that it is external to the

application

Bulk load facilities are often provided by packaged application systemvendors If a bulk load facility is not provided, then the vendor often

provides the necessary APIs so that a bulk load facility can be

developed Bulk load facilities are the most effective tools with which topopulate a new application The bulk load facility generally provides thenecessary native edit and validation routines required by the application,while providing the necessary audit capabilities with which to determinethe degree of success of the conversion

If a bulk load facility is not provided and cannot be developed from

vendor-provided APIs, then the next best thing is to generate the

transactions that ordinarily would be used to enter data into the system

In this way, the data is cleansed by the application-provided routines, andone is ensured that the resultant data has the required integrity from theapplication perspective and is appropriately converted This approachgenerally requires multiple conversion routines, possibly one per

transaction type, and multiple iterations of the conversion as the

transactions are loaded

If neither of the previous methods for converting the data is available,then one can explore using keystroke emulation as a method of enteringthe data There are numerous keystroke emulation or screen scrapingutilities available that can assist in this endeavor The trick here is to

generate flat files from the source application and then to assemble

screens of information that ordinarily are used by the application for dataentry The application is in essence fooled into behaving as if a client wascommunicating with it for data entry

There are some technical limitations or challenges with this approach.With large volumes of information, multiple clients with multiple clientsessions may have to be established This is dependent on the efficiency

of the client application The slower the client application and the higherthe volume of data, the greater the number of clients who need to