Student Guide - Oracle® GoldenGate Fundamentals

• Minimal overhead and impact on your source systems and overall architecture to capture and move real-time data • No requirement for batch windows • Transactional data integrity helps i

Oracle® GoldenGate

Fundamentals Student Guide Version 10.4

October 2009

Oracle GoldenGate Fundamentals Student Guide, version 10.4

Copyright © 1995, 2009 Oracle and/or its affiliates All rights reserved

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws Except as expressly permitted in your license agreement

or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited

The information contained herein is subject to change without notice and is not warranted to be error-free If you find any errors, please report them to us in writing

If this software or related documentation is delivered to the U.S Government or anyone licensing it on behalf of the U.S Government, the following notice is applicable:

U.S GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007) Oracle USA, Inc., 500 Oracle Parkway, Redwood City, CA 94065

This software is developed for general use in a variety of information management applications It is not developed

or intended for use in any inherently dangerous applications, including applications which may create a risk of personal injury If you use this software in dangerous applications, then you shall be responsible to take all

appropriate fail-safe, backup, redundancy, and other measures to ensure the safe use of this software Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software in dangerous applications

Oracle is a registered trademark of Oracle Corporation and/or its affiliates Other names may be trademarks of their respective owners

This software and documentation may provide access to or information on content, products, and services from third parties Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services

ABOUT GOLDENGATE – COMPANY AND SOLUTIONS 5

TECHNOLOGY OVERVIEW 18

ARCHITECTURE 22

CONFIGURING ORACLE GOLDENGATE 28

S TEP 1 P REPARE THE E NVIRONMENT 29

G OLDEN G ATE C OMMAND I NTERFACE 46

S TEP 2 C HANGE C APTURE 51

S TEP 3 I NITIAL L OAD 64

S TEP 4 C HANGE D ELIVERY 71

EXTRACT TRAILS AND FILES 76

G OLDEN G ATE D ATA F ORMAT 79

A LTERNATIVE F ORMATS 85

V IEWING IN L OGDUMP 92

R EVERSING THE T RAIL S EQUENCE 104

PARAMETERS 107

GLOBALS P ARAMETERS 109

M ANAGER P ARAMETERS 110

E XTRACT P ARAMETERS 113

R EPLICAT P ARAMETERS 122

DATA MAPPING AND TRANSFORMATION 131

D ATA S ELECTION AND F ILTERING 131

C OLUMN M APPING 138

F UNCTIONS 141

SQLEXEC 149

M ACROS 158

U SER T OKENS 163

U SER E XITS 170

O RACLE S EQUENCES 177

CONFIGURATION OPTIONS 178

BATCHSQL 179

C OMPRESSION 182

P ROCESS S TARTUP AND TCP/IP E RRORS 219

R EPORTING AND S TATISTICS 221

M ONITIORING O RACLE G OLDEN G ATE 229

T ROUBLESHOOTING 235

T ECHNICAL S UPPORT 244

About GoldenGate – Company and Solutions

Real-Time Access Availability: the degree to which information can be instantly accessed.

Real-Time Information Integration: the process of combining data from different sources to provide a unified view.

Mission-Critical Systems

Oracle GoldenGate provides solutions that enable your mission-critical systems to have continuous availability and access to real-time data

We offer a robust yet easy platform for moving real-time transactional data between operational and analytical systems to enable both:

- High Availability solutions, and

- Real Time Integration solutions

• Real-Time Access meaning your critical data is accessible and available whenever you need it, 24x7

At the same time…

• Real-Time Information meaning that the data available is as current as possible – not 24 hours old, not even 4 hours old

Oracle GoldenGate Success

Company Strength and Service

Rapid Growth in Strategic Partners

500+ customers 4000+ solutions implemented… in 35 countries Established, Loyal

Customer Base

GoldenGate Software established in 1995

Acquired by Oracle in 2009

Global sales and support

Our partnerships are rapidly increasing with major technology players, including database and IT infrastructure, packaged applications, business intelligence, and service providers

And because our software platform supports a variety of solution use cases… Our more than 500 customers are using our technology for over 4000 solutions around the world What we typically find that once an initial solution is implemented and the benefits achieved, our customers then find additional areas across the enterprise where we can further drive advantages for them

Transactional Data Management (TDM)

Oracle GoldenGate provides low-impact capture, routing, transformation, and delivery of database transactions across heterogeneous environments in real time

Our focus is on transactional data management (TDM) – which means delivering a platform in which that data can be best utilized in real-time enterprise wide

Oracle GoldenGate captures, routes, transforms, and delivers transactional data in real time – and it works across heterogeneous environments with very low impact and preserved transaction integrity

Our Key Capabilities in which we architect the product are:

• We move data essentially in “real time” – with sub-second speed

• Works in heterogeneous environments – across different database and hardware types

• Transactional – we are “transaction aware” and apply read-consistent changed data to maintain its referential integrity between source and target systems

We further Differentiate ourselves from other technologies with:

- High performance with low impact – we can move large volumes of data very efficiently while maintaining very low lag times/latency

- Our flexibility – we meet a wide range of customer solution and integration needs, thanks to our open, modular architecture

- Our reliability – our architecture is extremely resilient against potential interruptions; no single point of failure or dependencies, and easy to recover

•Zero-Downtime Upgrades and Migrations

Eliminate downtime for upgrades and migrations

•Live Reporting

Feeding a reporting database so that you don’t burden your source production systems

•Operational Business Intelligence (BI)

Real-time data feeds to operational data stores or data warehouses, directly or via ETL tools

•Transactional data integration

Real-time data feeds to messaging systems for business activity monitoring (BAM), business process monitoring (BPM) and complex event processing (CEP)

Uses event-driven architecture (EDA) and service-oriented architecture (SOA)

Oracle GoldenGate Solutions

High Availability & Disaster Tolerance

Real-Time Data Integration

 Real-Time Data Warehousing

 Live Reporting

 Transactional Data Integration

Oracle GoldenGate provides two primary solution areas:

High Availability/Disaster Tolerance and Real-Time Data Integration

Within High Availability and Disaster Tolerance, we offer:

• Active-Active solutions for continuous availability and transaction load distribution between two or more active systems

•Zero-Downtime Operations that eliminates downtime for planned outages involving upgrades, migrations, and ongoing maintenance

•Live Standby for an immediate fail-over solution that can later re-synchronize with your primary source

Within Real-Time Data Integration, we offer:

• Real-Time Data Warehousing which gives you real-time data feeds to data

warehouses or operational data stores

•Transactional Data Integration for distributing data in real-time between

transaction processing systems

•Live Reporting is for feeding a reporting database so that you don’t burden your source production systems

Oracle GoldenGate for High Availability & Disaster Tolerance

High Availability & Disaster Tolerance

For High Availability and Disaster Tolerance solutions, it’s about “real-time” or CONTINUOUS access to your data via your critical applications

The benefits that Oracle GoldenGate drives here include:

-Improved uptime and availability (helping you reach aggressive service level agreements/SLAs)

-Higher Performance for your production systems – help to eliminate scalability or response time delays that can give users the impression of an availability or access issue

-Faster Recovery and Minimized Data Loss – so you can achieve higher Recovery Time Objectives (RTOs) and Recovery Point Objectives (RPOs)

- and an overall lower Total Cost of Ownership by putting your standby systems to work for other solutions!

High Availability: Live Standby

Benefits:

 Eliminate unplanned downtime

 Reduce data loss and isolate corrupt data

 Re-synchronize backup and primary systems

 Remove distance constraints

 Automate switchovers

 Improve ROI with active standby available for reporting

Live Standby: helps eliminate unplanned outages to enable continuous availability, with no geographic distance constraints

Oracle GoldenGate moves changed data from primary database to a standby in seconds so that end users have a reliable failover system with up to date data that they can immediately switchover

sub-There is no database recovery process required because changed data is queued

outside of the database in persisted Trail files, and data loss risk is minimized Oracle GoldenGate also isolates corrupt data during movement to make sure the secondary system is reliable when it is needed

The customer’s Return on Investment can be further increased by using the live standby system for reporting or testing – Oracle GoldenGate allows the standby

database to be open, so it does not have to sit idle and can be put to work!

High Availability: Active-Active

Benefits:

 Achieve continuous availability

 Enable transaction load distribution (with built-in conflict resolution)

 Improve performance

 Lower TCO

Active-Active: Oracle GoldenGate enables bidirectional data movement between two

or more databases that actively support an application, with no geographic distance constraints

The active-active solution allows data updates and changes (“write” activity) to occur

on two or more active databases supporting live applications Oracle GoldenGate synchronizes the two active databases by replicating the data between each at a logical level, and allows load distribution to improve system performance In the case

of an outage of one system, there is no downtime for the end user because the other active system continues with operations

Because Oracle GoldenGate is an asynchronous solution, conflict management is required to ensure data accuracy in the event that the same row is changed in two or more databases at (or about) the same time Oracle GoldenGate provides capabilities

to detect and resolve conflicts as well

A variety of active-active scenarios can be supported – depending on the desired implementations We have strong experience in active-active solutions for both High Availability as well as Zero-Downtime upgrades and migration projects

Zero-Downtime Upgrades and Migrations

Benefits:

 Eliminate “planned downtime” during hardware, database, OS and/or application upgrades and migrations

 Minimize risk with fail-back contingency

 Improve success with phased user migrations

Zero Downtime Operations: is for eliminating planned outages during database, application or server upgrades, migrations, and/or maintenance

Oracle GoldenGate captures all the changed data in the primary system while the new system is initiated and prepared Once the second or the new system is upgraded or migrated Oracle GoldenGate applies all the changed data to the new system Oracle GoldenGate then keeps the two environments synched with our real-time replication

Often with such projects, there is always concern about what will happen once you switchover to the new environment Oracle GoldenGate alleviates many of those risks with our fail-back capabilities after switchover Oracle GoldenGate captures the changes that happen in the new system so that the old system is kept up to date in case there is a need for fail back to the old environment

And currently in Oracle or HP Nonstop environments, our Oracle GoldenGate

Veridata product verifies that the data is consistent in both systems before and even after switchover

Oracle GoldenGate for Real-Time Data Integration

Real-Time Data Integration

 Live Reporting

 Operational Business Intelligence

 Transactional Data Integration

The benefits that Oracle GoldenGate drives here include:

• Fresher, real-time data available for use and decision-making – remove latency as a technical constraint

• Minimal overhead and impact on your source systems and overall architecture to capture and move real-time data

• No requirement for batch windows

• Transactional data integrity helps improve overall data quality

• Ease of integration – Oracle GoldenGate easily fits into existing and desire

architecture, and is overall easy to maintain over long term

Data Integration: Live Reporting

Benefits:

 Use real-time data for better, faster decision making

 Remove reporting overhead on source system

 Reduce cost-to-scale as user demands and data volumes grow

 Leverage cost-effective systems for reporting needs

Oracle GoldenGate's Live Reporting enables both real-time reporting capabilities while improving the performance of operational source systems

Oracle GoldenGate feeds real-time data from the source to a secondary reporting-only database such as an operational data store (ODS) This allows reporting activity to be off-loaded from the production database This secondary database can be a different database and/or platform from the production database, to lower the total cost of ownership and allow organizations to leverage emerging open source technologies The solution also helps increase scalability as user demands and data volumes grow

Operational Business Intelligence

Benefits:

 Use real-time data for better, faster decision making

 Eliminate batch window dependency

 Reduce overhead on source system

 Maintain referential integrity for data quality

 Leverage its flexibility for transformations and integration with ETL

For Real-Time Data Warehousing The Oracle GoldenGate Real-Time Data

Warehousing solution enables continuous, real-time data feeds for data warehouses or operational data stores, to improve business intelligence Our log-based changed data capture has very minimal impact on the source, no batch windows and moves the data

in sub-seconds Each transaction’s commit boundaries are maintained for data

integrity

Oracle GoldenGate’s architecture also improves data recoverability in case there is an outage during the data movement This is an important requirement as data latency decreases in feeding the analytical environment Oracle GoldenGate’s trail files that store the changed data are persisted, so if needed they can be reapplied to the target and also source system without having to capture the data again

Transformations or co-existing with ETL:

Oracle GoldenGate out-of-the box can support a number of common data

transformations often required for data integration However, where complex

transformations are needed Oracle GoldenGate can be used to augment an existing ETL solution in several ways:

1) First, Oracle GoldenGate can deliver transactional data to staging tables in real time, which then would be used by the ETL to extract from and perform

transformations and then load user tables This method works best when the ETL product is optimized to perform the transformations within the target database This

is an “ELT” model

2) Second method: Oracle GoldenGate provides the data to the ETL engine as flat files and in micro-batches The latency depends on the ETL product and business

requirements but we typically deliver every few minutes to an hour

3) Third method: Oracle GoldenGate publishes changed data to a messaging system and the ETL solution (that can subscribes to the queue or topic) receives it in real-time

In each of these architectures combining real-time change data capture with ETL decreases data latency to real time or near real-time and eliminates the batch window dependency

non-enable event driven architecture (EDA) and to support service oriented architecture (SOA)

Technology Overview

How Oracle GoldenGate Works: Modular “Building Blocks”

Source Database(s)

Target Database(s)

Network (TCP/IP)

Route: Data is compressed, encrypted for routing to targets.

Capture: Committed changes are captured (and can be filtered) as they occur by reading the transaction logs.

Capture Trail files: Stages and queues data for routing.

Source Trail

Delivery: Applies data with transaction integrity, transforming the data as required.

Target Trail Delivery

Bi-directional

Source Trail Target

Trail

Oracle GoldenGate consists of decoupled modules that are combined to create the best possible solution for your business requirements

On the source system(s):

•Oracle GoldenGate’s Capture (Extract) process reads data transactions as they occur,

by reading the native transaction log, typically the redo log Oracle GoldenGate only moves changed, committed transactional data, which is only a % of all transactions – therefore operating with extremely high performance and very low impact on the data infrastructure

• Filtering can be performed at the source or target - at table, column and/or row level

• Transformations can be applied at the capture or delivery stages

• Advanced queuing (trail files):

To move transactional data efficiently and accurately across systems, Oracle GoldenGate converts the captured data into a Oracle GoldenGate data format in

“trail” files With both source and target trail files, Oracle GoldenGate’s unique architecture eliminates any single point of failure and ensures data integrity is maintained – even in the event of a system error or outage

Routing:

•Data is sent via TCP/IP to the target systems Data compression and encryption are supported Thousands of transactions can be moved per second, without distance limitations

On the target system(s):

•A Server Collector process (not shown) reassembles the transactional data into a target trail

•The Delivery (Replicat) process applies transactional data to the designated target systems using native SQL calls

Bi-directional:

•In bi-directional configurations/solutions, this process runs the same in reverse, to concurrently synchronize data between the source and target systems

Manager processes (not shown) perform administrative functions at each node

Oracle GoldenGate Supports Applications Running On…

Databases O/S and Platforms

 All listed above

 MySQL, HP Neoview, Netezza, and any ODBC compatible databases

 ETL products

 JMS message queues

Windows 2000, 2003, XP Linux

Sun Solaris

HP NonStop HP-UX

HP TRU64

HP OpenVMS IBM AIX IBM z/OS

Oracle GoldenGate is ideal for heterogeneous environments – not just supporting different versions of the same database or operation system/hardware, but replicating and integrating data across vendor systems

We support log-based Capture of changed data from nearly all major database

Oracle GoldenGate Advantages

Movement

 Speed – Subsecond Latency

 Volume – Thousands of TPS

 Log-based Capture

 Native, Local Apply

 Efficient IO and Bandwidth Usage

 Dynamic Rollback

 Incremental TDM

 Initial Data Load

 GUI-based Monitoring and Configuration

 Proactive Alerts

 Encryption

 Real-Time Deferred or Batch

 Event Markers

Integration

 Heterogeneous Data Sources

Oracle GoldenGate Director

 Manages, defines, configures, and reports on Oracle GoldenGate components

 Key features:

 Centralized management of Oracle GoldenGate modules

 Rich-client and Web-based interfaces

 Alert notifications and integration with 3rd-party monitoring products

 Real-time feedback

 Zero-impact implementation

Oracle GoldenGate Director is a centralized server-based graphical enterprise

application that offers an intuitive way to define, configure, manage, and report on Oracle GoldenGate processes

Oracle GoldenGate Director is a value added module to centralize management and improve productivity

Oracle GoldenGate Director supports all platforms and databases supported by Oracle

GoldenGate

Oracle GoldenGate Veridata

 A high-speed, low impact data comparison solution

 Identifies and reports data discrepancies between two databases without interrupting those systems or the business processes they support

 Supports Oracle, Teradata, SQL Server, NonStop SQL/MP and Enscribe

 Supports homogeneous and heterogeneous compares

 Benefits:

 Reduce financial/legal risk exposure

 Speed and simplify IT work in comparing data sources

 No disruption to business systems

 Improved failover to backup systems

 Confident decision-making and reporting

Oracle GoldenGate Veridata is a high-speed data comparison solution that identifies and reports data discrepancies between databases without interrupting ongoing

business processes Using Oracle GoldenGate Veridata, companies can audit and verify large volumes of data across a variety of business applications with certainty, and maintain reliable data synchronization Oracle GoldenGate Veridata reduces the amount of time and resources required to compare data, it minimizes the impact of human errors, and it ensures that potential problems can be instantly identified and addressed

Key Veridata Features:

•Compares large data volumes with high speed and efficiency

•Allows both data sources to be online by handling in-flight transactions

•Performs selective, parallel comparison

•Offers intuitive Web interface and personalized views

•Enables the comparison of databases that are different database versions or on

different operating systems

•(HP Nonstop only) Supports the comparison of only the data changed since the initial comparison (delta processing)

Why would you need Veridata?

Data discrepancies arise even without malicious intent due to infrastructure

problems, application errors, operator mistakes, configuration errors, or unexpected user behavior With vigilant verification procedures using Oracle GoldenGate

Veridata, companies can eliminate data inconsistencies across different business applications and avoid potential operational, financial, or regulatory risks

Architecture

Oracle GoldenGate Data Capture and Delivery

Oracle GoldenGate Transactional Data Management:

 Primarily used for change data capture and delivery from database transaction logs

 Can optionally be used for initial load directly from database tables

 Especially useful for synchronizing heterogeneous databases

 Database-specific methods may be preferable for homogeneous configurations

Change Data Capture & Delivery

Transaction Log

Collector Trail Replicat

Source Database

Target Database

Network (TCP/IP)

On the source system:

An Extract process captures transactional changes from transaction logs

The Extract process sends data across a TCP/IP network to the target system

On the target system:

A Server Collector process reassembles and writes the data to a GoldenGate trail

A Replicat process reads the trail and applies it to the target database This can be

concurrent with the data capture or performed later

Manager processes on both systems control activities such as starting, monitoring and restarting processes; allocating data storage; and reporting errors and events

Change Data Capture & Delivery using a Data Pump

Transaction Log

Target Database

Data Pump Local

Trail

Network (TCP/IP)

On the source system:

An Extract process captures transactional changes from the database transaction log

The Extract process writes the data to a local GoldenGate trail This preserves the captured data if the network or target trail fails

A second Extract process (called a Data Pump) sends the data across the network to

the target system

On the target system:

A Server Collector process reassembles and writes the data to a GoldenGate trail

A Replicat process reads the trail and applies it to the target database This can be

concurrent with the data capture or performed later

restarting processes; allocating data storage; and reporting errors and events

Initial Load

Source Database

Network (TCP/IP)

Server Collector Files

Or DB Bulk Load Utility

GoldenGate initial load methods:

 Direct Load (Extract sends data directly to Replicat to apply using SQL)

 Direct Bulk Load (Replicat uses Oracle SQL*Loader API)

 File to Replicat (Extract writes to a file that Replicat applies using SQL)

 File to database utility (Extract writes to a file formatted for a DB bulk load utility)

On the source system:

An Extract process captures source data directly from tables

The Extract process sends data in large blocks across a TCP/IP network to the target system

On the target system, one of the following scenarios:

1 Direct Load Replicat reads the data stream and concurrently applies the data to

the target database using SQL

2 Direct Bulk Load (Oracle) Replicat can apply the data using the Oracle

SQL*Loader API to improve performance

3 File to Replicat Server Collector reassembles and writes the data to Extract files

Replicat applies the data to the target database using SQL

4 File to database utility Server Collector reassembles and writes the data to files

formatted for a bulk loader, which applies the data to the target database

restarting processes; allocating data storage; and reporting errors and events

Online versus Batch

 Change data capture & delivery can be run either continuously ( online )

or as a special run ( batch run ) to capture changes for a specific period of time.

 Initial load is always a special run ( batch run ).

transaction in log

End of last committed transaction written to trail

and write position of a process Checkpoints ensure that data changes marked for synchronization are extracted, and they prevent redundant extractions They provide fault tolerance by preventing the loss of data should the system, the network, or a GoldenGate process need to be restarted

Checkpointing - Replicat

 Best practice is to create a checkpoint table in the target database

 Checkpoints are maintained in both the checkpoint table (if it exists) and

uncommitted transaction

Parameters, Process Groups and Commands

 GoldenGate processes are configured by ASCII parameter files.

 A process group consists of:

 An Extract or Replicat process

 Associated parameter file

 Associated checkpoint file

 Any other files associated with that process

Each process group on a system must have a unique group name.

 Processes are added and started using the GoldenGate Software Command Interface (GGSCI) with the group name

 GGSCI commands also add trails, check process status, etc.

Solutions and Architecture – Discussion Points

1 How is Oracle GoldenGate different from simply replicating database operations?

2 What are some use cases for Oracle GoldenGate software?

3 What is the purpose of checkpointing?

1 Log-based change data capture, decoupled from database architecture Real-time, heterogeneous and transactional

2 (a) High availability – live standby, active-active, zero down-time upgrades and

Configuring Oracle GoldenGate

Configuring Oracle GoldenGate

Oracle GoldenGate can be deployed quickly and easily in four steps:

1 Prepare the Environment

2 Change Capture

3 Initial Load

4 Change Delivery

Note: You can run log-based change capture after the initial data load if you set the

extract begin time to the start of the longest running transaction committed during the initial data load

Configuring Oracle GoldenGate

3 Initial Load (various methods)

1 Prepare the Environment

Target Database Transaction

Log

Source Database

2 Change Capture

Trail Local

Trail

Data Pump

4 Change Delivery

Replicat

Oracle GoldenGate can be deployed quickly and easily in four steps:

• Prepare the environment, e.g

• Install Oracle GoldenGate software on source and target

• Enable transaction logging

• (Heterogeneous source/target) Generate source definitions so Replicat can process trail data

• Configure and start change capture to GoldenGate trail files (Extract processes – primary and data pump)

• Perform initial load to synchronize databases by database-specific or GoldenGate methods

• Configure and start change delivery (Replicat process)

Step 1 Prepare the Environment

Step 1 Prepare the Environment

3 Initial Load (various methods)

Log

Source Database

Local Trail

Data Pump

Step 1 Prepare the Environment

Set up each system:

 Install Oracle GoldenGate software on source and target

 Configure and start GoldenGate Manager on source and target

 If heterogeneous source/target, generate source definitions and copy to target

 Prepare the database For example:

 Ensure database access by GoldenGate

 Enable transaction logging

Installing Oracle GoldenGate installs all of the components required to run and manage GoldenGate processing, and it installs the GoldenGate utilities

Manager must be running on each system before Extract or Replicat can be started, and must remain running while those processes are running so that resource

management functions are performed

The source definitions file contains the definitions of the source tables and is required

on the target system in hetereogeneous configurations Replicat refers to the file to when transforming data from the source to the target

To reconstruct an update operation, GoldenGate needs more information than Oracle and SQL Server transaction logs provide by default Adding supplemental log data forces the logging of the full before and after image for updates

Install Oracle GoldenGate Software

Prepare Environment: Installation – Access the Media Pack

 Access the product media pack (software and documentation) at

edelivery.oracle.com

 Identify the proper release of GoldenGate for your source and target environments

 Database and version

 Operating system and version

A GoldenGate instance is a single installation of GoldenGate

Prepare Environment: Installation - Windows

 Download zip file to C:\GGS

 Unzip zip file into C:\GGS folder

 Configure a Windows Service Name for Manager process in a GLOBALS parameter file (required only if multiple Managers on the server)

 C:\GGS> INSTALL ADDSERVICE ADDEVENTS

 GGSCI> CREATE SUBDIRS

For Windows: Do not install Oracle GoldenGate into a folder that contains spaces in

its name, for example “GoldenGate Software.” The application references path

names, and the operating system does not support path names that contain spaces, whether or not they are within quotes

Prepare Environment: Installation – Windows INSTALL Program

 On Windows, an INSTALL program performs the following functions:

 Installs GoldenGate event messages into the system registry

 Installs the Manager as a Windows service

 Syntax:

INSTALL <item> [<item> …]

 Example:

C:\GGS> INSTALL ADDEVENTS ADDSERVICE

Note: The uninstall command is:

INSTALL DELETESERVICE DELETEEVENTS

Items (all optional)

ADDEVENTS Adds the GoldenGate events to the registry so that event messages

appear in the Windows Event Log

this way, Manager will stop when the user logs out By using install, you can install Manager as a Windows service so that it can be operated independently of user

connections and can be configured to start either manually or when the system starts You can configure the Manager service to run as the Local System account or as a specific named account The configuration of a service can be changed by using the

Services applet of the Windows Control Panel and changing the service Properties

DELETESERVICE Removes the GoldenGate Manager service

AUTOSTART Specifies that the service be started at system boot time (the default)

MANUALSTART Specifies that the service be started only at user request (with GGSCI

or the Control Panel Services applet)

USER Specifies a user name to logon as when executing Manager If specified, user name should include the domain name, a backward slash, and the user name

PASSWORD Specifies the user’s password for logon purposes This can be changed using the Control Panel Services applet

Prepare Environment: Installation - Multiple Manager Services

 GoldenGate supports running multiple Manager services on Windows

 For two or more GoldenGate instances, or

 GoldenGate with a Veridata C Agent (which uses a Manager)

 Each Manager service must be assigned a unique name

 Before installing the service, you can specify the name

 Create a GLOBALS parameter file for each Manager

 Specify the one-word name of the service using the MGRSERVNAME <name> parameter

 INSTALL ADDSERVICE

 Reads the GLOBALS MGRSERVNAME for the service name

 If no GLOBALS setting, uses default service name GGSMGR

A GLOBALS file stores parameters that relate to the GoldenGate instance as a whole,

as opposed to runtime parameters for a specific process This file is referenced when installing the Windows service, so that the correct name is registered

Prepare Environment: Installation - UNIX, Linux or z/OS

 Download gz file to /ggs

 gzip –d {filename}.tar.gz

 tar -xvof {filename}.tar

 GGSCI> CREATE SUBDIRS

For UNIX, z/OS, or Linux: Use the gzip and tar options appropriate for your

system If you are installing GoldenGate into a cluster environment, make certain that the GoldenGate binaries and files are installed on a file system that is available

to all cluster nodes After installing GoldenGate, make certain to configure the

GoldenGate Manager process within the cluster application, as directed by the

vendor’s documentation, so that GoldenGate will fail over properly with the other applications The Manager process is the master control program for all GoldenGate operations

A GoldenGate instance is a single installation of GoldenGate

Prepare Environment: Installation – NonStop SQL/MX

 For a SQL/MX source, install Oracle GoldenGate on OSS running

on the NonStop source system:

 Download gz file to /ggs

 gzip –d {filename}.tar.gz

 tar -xvf {filename}.tar

 GGSCI> CREATE SUBDIRS

 Run the ggmxinstall script

 For a SQL/MX target, install Oracle GoldenGate

 Either on OSS running on the NonStop target system (as described above)

 Or on an intermediate Windows system(as described earlier)

The command to run it is:

OSS> ggmxinstall /G/<Guardian vol>/<Guardian subvol>

where: <Guardian vol>/<Guardian subvol> is the destination NonStop volume and subvolume in OSS format

Prepare Environment: Installation - GoldenGate Directories

Directory Contents dirchk GoldenGate checkpoint files dirdat GoldenGate trail and extract files dirdef Data definitions produced by DEFGEN and used to

translate heterogeneous data dirpcs Process status files dirprm Parameter files dirrpt Process report files dirsql SQL scripts dirtmp Temporary storage for transactions that exceed

allocated memory

dirchk

Contains the checkpoint files created by Extract and Replicat processes, which store current read and write positions to support data accuracy and fault tolerance Written

in internal GoldenGate format Do not edit these files

The file name format is <group name><sequence number>.<ext> where <sequence

number> is a sequential number appended to aged files and <ext> is either cpe for Extract checkpoint files or cpr for Replicat checkpoint files Examples: ext1.cpe, rep1.cpr

dirdef

The default location for data definitions files created by the DEFGEN utility to contain source or target data definitions used in a heterogeneous synchronization

environment Written in external ASCII

File name format is a user-defined name specified in the DEFGEN parameter file These files may be edited to add definitions for newly created tables If you are unsure

of how to edit a definitions file, contact technical support Example: defs.dat

dirpcs

Default location for status files File name format is <group>.<extension> where

<group> is the name of the group and <extension> is either pce (Extract), pcr

(Replicat), or pcm (Manager)

These files are only created while a process is running The file shows the program name, the process name, the port, and process ID that is running Do not edit these files Examples: mgr.pcm, ext.pce

dirprm

The default location for GoldenGate parameter files created by GoldenGate users to store run-time parameters for GoldenGate process groups or utilities Written in external ASCII format

File name format is <group name/user-defined name>.prm or mgr.prm These files may be edited to change GoldenGate parameter values They can be edited directly from a text editor or by using the EDIT PARAMS command in GGSCI Examples:

defgen.prm, finance.prm

dirrpt

The default location for process report files created by Extract, Replicat, and Manager processes to report statistical information relating to a processing run Written in external ASCII format

File name format is <group name><sequence number>.rpt where <sequence number>

is a sequential number appended to aged files Do not edit these files Examples: fin2.rpt, mgr4.rpt

Oracle GoldenGate Documentation

Prepare Environment: Oracle GoldenGate Documentation

 Quick Install Guide

 Installation and Setup Guides (by database)

 Administration Guide

 Reference Guide

 Troubleshooting and Tuning Guide

Note: You can download the documentation from

http://www.oracle.com/technology/documentation/index.html

Windows and UNIX platforms:

● Oracle GoldenGate Quick Install Guide: Describes the structure of the media pack

and where to find installation instructions

● Oracle GoldenGate Installation and Setup Guides: There is an installation guide

and setup guide for each database that is supported by Oracle GoldenGate These include database-specific configuration information

● Oracle GoldenGate Administration Guide: Introduces Oracle GoldenGate

components and explains how to plan for, configure, and implement Oracle GoldenGate

on the Windows and UNIX platforms

● Oracle GoldenGate Reference Guide: Provides detailed information about Oracle

GoldenGate parameters, commands, and functions for the Windows and UNIX

platforms

● Oracle GoldenGate Troubleshooting and Tuning Guide: Provides suggestions for

improving the performance of Oracle GoldenGate in different situations, and provides solutions to common problems

Configure and Start Manager

Prepare Environment: Manager - Overview

 Performs system management and monitoring tasks

 Starting GoldenGate processes

 Starting dynamic Server Collector, Replicat, or GGSCI processes

 Error and lag reporting

 GoldenGate trail management

 Parameter file

 mgr.prm file in GGS /dirprm directory

 Event information written to ggserr.log file

The Manager process performs system management and monitoring tasks on

Windows and Unix, including the following

Starting Server Collector processes to collect data from remote Extract processes
Threshold reporting (for example, when Extract falls behind GGSLOG)

Purging trails

Purging of GGSLOG or GGSLOG_HISTORY data

Manager Parameters

Enter Manager parameters in the dirprm/mgr.prm file, under the GoldenGate

installation directory If no mgr.prm file exists, default management parameters are used

Error and Informational Reporting

Manager reports critical and informational events to the ggserr.log file in the

GoldenGate installation directory

Prepare Environment: Manager - Configuration

 Create the parameter file using GGSCI

GGSCI> EDIT PARAM MGR

 Start the Manager using GGSCI

PURGEOLDEXTRACTS /ggs/dirdat/aa*, USECHECKPOINTS PURGEOLDEXTRACTS /ggs/dirdat/bb*, &

USECHECKPOINTS, MINKEEPDAYS 5

AUTOSTART ER * AUTORESTART EXTRACT *, WAITMINUTES 2, RETRIES 5

LAGREPORTHOURS 1 LAGINFOMINUTES 3 LAGCRITICALMINUTES 5

This parameter file has the Manager listening on PORT 7809 Ports 8001, 8002, and those in the range 9500 to 9520 will be assigned to the dynamic processes started by Manager

This manager process will recycle GoldenGate trails that match the file name of /ggs/dirdat/aa* and /ggs/dirdat/bb* It will only recycle the trail once all Extracts

and Replicats have a checkpoint beyond the file (USECHECKPOINTS), however bb* trails will not be purged until there has been no activity for 5 days

The manager will automatically start any Extract and Replicat process at startup and will attempt to restart any Extract process that abends after waiting 2 minutes, but only up to 5 attempts

The manager will report lag information every hour, but only for processes that have

3 and 5 minutes of latency The message will be flagged informational for lags of 3 minutes and critical for any process that has a lag greater than 5 minutes

Generate Source Definitions (Heterogeneous Source/Target)

Prepare Environment: Source Definitions - Overview

 The problem

 Understanding source and target layouts across disparate systems and databases

 The solution – the DEFGEN utility program

 DEFGEN produces a file containing layout definitions of the source files and tables

 This source definition file is used to interpret layouts for data stored in GoldenGate trails

- At start up Replicat reads the definition file specified with the SOURCEDEFS parameter

- Server Collector uses the –d argument to specify which definition file

The Solution - The DEFGEN Utility Program

The DEFGEN utility program produces a file containing a definition of the layouts of the source files and tables The output definitions are saved in an edit file and

transferred to all target systems in text format Replicat and Collector read in the definitions at process startup and use the information to interpret the data from the GoldenGate trails

Note: The user should never modify the DEFGEN output

Prepare Environment: Source Definitions – Run DEFGEN

 DEFGEN is initiated from the command prompt:

defgen paramfile <paramfile> [ reportfile <reportfile> ]

 Definitions are saved to the file specified in the parameter file

 This file needs to be transferred to the target system as a text file

Prepare Environment: Source Definitions - Sample DEFGEN Parameters

DEFSFILE /ggs/dirdef/source.def, PURGE SOURCEDB mydb, USERID ggs, PASSWORD ggs TABLE SALES.ACCOUNT;

TABLE SALES.PRODUCT;

Parameter Specifies DEFSFILE The output definitions file location and name

SOURCEDB The database name (if needed)

USERID The user ID and password (if needed) to access the

database TABLE The table(s) to be defined