The two main sections of this chapter focus on the process and tools forperformance tuning: • Performance tuning process: The Applications DBA must identify and document the cause of per
Trang 1Performing a Network Test from the Application
Oracle Applications also provides a Network Trace Form that the user mayexecute This form is accessed by logging in to the application and choosingthe Application ➤ Network Test menu option Click the Run Test button, andthe Form will return information about network latency and bandwidth Thenumber of iterations and trails can be customized for each execution of thetest A sample of the output is shown in Figure 3-6
Figure 3-6.Forms network test
This information can be used by your company’s or customer’s ing team to determine whether networking bottlenecks exist
network-Additional Monitoring and Troubleshooting Topics
This section will cover some miscellaneous monitoring and troubleshootingissues:
• Monitoring profile changes, which may assist with troubleshooting figuration issues
con-• Monitoring and validating system backups
• Troubleshooting JInitiator issues such as deleting the cache and usingthe Java console
Trang 2Monitoring Profile Changes
Application profile settings allow for users and system administrators to tomize the functionality of the E-Business Suite As a result, altering theseprofile settings can sometimes have an adverse effect on the behavior of theapplication The Applications DBA should make a practice of monitoring anychanges to the profile settings
cus-Checking profile settings is especially important when performingmajor patching efforts, because a patch may overwrite profile options Newprofile settings may also be created during the patching process In addition
to patching, the most common reason for profile changes is user error Forexample, a system administrator may change a profile setting without realiz-ing the full impact on the system
The following script can be used to alert you to profile options changedwithin a specified threshold:
#Script used to monitor for application profile changes
#Threshold is the number of days to query for profile changes
#For example, if you set it to 7, all profile changes that
#have occurred in the past 7 days will be displayed
THRESHOLD=$1
LOGFILE=/tmp/profile_changes_$ORACLE_SID.txt
sqlplus -s apps/apps << EOF
set heading off
spool $LOGFILE
select '$ORACLE_SID - Profile Changes Past '||
'Threshold of $THRESHOLD days - '||count(1)
from fnd_profile_option_values
where last_update_date > (sysdate-$THRESHOLD)
having count(1) > $THRESHOLD
union
select 'no rows'
from fnd_profile_option_values
where last_update_date <= (sysdate-$THRESHOLD)
having count(1) <= $THRESHOLD;
Trang 3OAM may also be used to display profile changes made in the Oracleapplication This requires manually viewing the screen From the ApplicationDashboard, in the Configuration Changes portion of the screen (shown inFigure 3-7), it is possible to see the number of profiles that have beenchanged in the last 24 hours To display profile options sorted by the LastUpdated date, click the number displayed next to Site Level Profile Options
Figure 3-7.Viewing configuration changes from the OAM Dashboard
Monitoring System Backups
The Applications DBA should set up monitoring for the database backups
If a failure occurs during the backup, the DBA should be notified so that thesituation can be resolved in a timely manner This monitoring processshould be in place regardless of the backup methodology
Some systems benefit from exporting certain objects or schemas inaddition to the regular backups These exports can be monitored for errormessages within their log files Since backups are largely environment spe-cific, we will not attempt to address how monitoring should occur, only that
it should occur
In addition to monitoring backups, periodic restores from backupsshould be performed in order to confirm backup validity Also verify thatthe time required to restore the instance falls within your service level agree-ment This testing process should be part of your company’s disaster
recovery plan
Resolving JInitiator Issues
When users encounter problems running Forms applications on their stations, the source of the problem may be JInitiator Troubleshootingtechniques for JInitiator include the following:
work-• Clearing the JAR cache
• Using the Java console
Trang 4Clearing the JInitiator Cache
Most JInitiator problems can be resolved by clearing the JAR cache for theprogram This can be accomplished by selecting the Cache tab on theJInitiator Control Panel, shown in Figure 3-8, and clicking the Clear JARCache button
Figure 3-8.Clearing the JAR Cache
From this screen, you can also alter the location where the cache isstored and the default size of the cache This can be useful if there are clientissues such as the user not having write permission to the directory wherethe cache is stored
Using the Java Console
Java console display settings can be altered using the JInitiator ControlPanel This program may be started from Start ➤ Control Panel menu option
on the client PC There should be a JInitiator icon for each version installed
on the computer Select the appropriate icon to start the program’s controlpanel
For troubleshooting purposes there is a Show Java Console checkbox onthe Basic tab of this control panel Select this option and click the Apply but-ton to allow for the Java console to be displayed on startup This will providethe client with some useful troubleshooting information
C H A P T E R 3 ■M O N I TO R I N G A N D T R O U B L E S H O OT I N G 93
Trang 5■ Tip If clearing the JInitiator cache and monitoring the Java console do not provideresolution to client Forms issues, it may be necessary to reinstall the JInitiator.
Monitoring and Troubleshooting Best
Practices
When it comes to monitoring the Oracle E-Business Suite, the best advice is
to be proactive It is much easier to react to issues before they result in ous problems or outages than to wait for problems to arise
seri-The key to proactive monitoring is to understand how Oracle tions is used in your environment Without knowing what is normal behavior,
Applica-it is difficult to capture the unexpected Gain an understanding of the day activities for your environment, and monitor for exceptions to thesetrends As your environment and system change, modify your proactivemonitoring to fit new trends
day-to-The best tool for troubleshooting your environment and systems is anunderstanding of the underlying configuration and how the Oracle E-BusinessSuite works Without detailed knowledge of the configuration and function-ality, it is difficult to gauge where to begin looking for solutions Do not be apassive DBA: strive to be an active DBA Learn the components of the appli-cation so that you are empowered to respond quickly when it is broken
Trang 6Performance Tuning
This chapter will focus on performance tuning techniques for many of thecomponents of the Oracle E-Business Suite In the course of reading thischapter, you will become familiar with various tools that will enable you todiagnose and fix performance degradation problems
The two main sections of this chapter focus on the process and tools forperformance tuning:
• Performance tuning process: The Applications DBA must identify and
document the cause of performance degradation, and then develop anaction plan for implementing a solution This section will outline thesteps in this process
• Tools for resolving performance issues: This section will discuss several
tools that can be used to tune the database and the server and providetips for tuning Forms, Apache, JServ, and Concurrent Manager Informa-tion for client tuning, generating and analyzing trace files, and some of
the new features of Oracle10g for SQL statement tuning, will also be
covered
There are numerous books dedicated to the subject of performancetuning This chapter is not meant to provide intricate details for in-depthperformance tuning; however, it will provide an overview of the informationrequired for identifying and resolving performance issues
■ Note Tuning may also be performed on SQL statements and the networking and I/Ocomponents of Oracle Applications; however, an extensive discussion of these sorts oftuning is beyond the scope of this guide
C H A P T E R 4
95
Trang 7Performance Tuning Process
Performance issues may occur on any of the tiers, infrastructure, or modulesthat make up the Oracle Applications environment Because of the complex-ity of the application, it is necessary to gather information from the usercommunity to determine which component of the application is experienc-ing the degradation in performance This section provides you with amethod for identifying and documenting performance issues and theirresolution
The first phase is to identify the cause of the degradation by collectinginformation to help you understand the underlying issue The second phase
is to develop an action plan in order to provide resolution
Identifying Performance Issues
Traditional methodologies for performance tuning begin with tuning theapplication code and SQL Since Oracle E-Business Suite is a packaged appli-cation, however, the first step in the traditional methodology is often difficult
to implement There are times when code performs so poorly that assistancefrom Oracle Support and Development is required to fix the underlying code.This situation will usually surface as a requirement while the Oracle Applica-tions DBA works to identify the underlying performance issue
To begin resolving performance problems, it is imperative that thenature of the degradation is understood The strategy presented here fordiagnosing the cause of performance degradation starts with generalquestions and proceeds to questions specific to different pieces of the appli-cation: the client tier, the network, a module of the application, and theentire application It is advisable to document performance issues and theanswers to these questions in order to facilitate resolution to the problem.The Applications DBA may pose the following questions in order to col-lect the required information If the answer to a question is no, proceed tothe next question
Question: Can the performance issue be systematically reproduced?
If the answer is yes: Document the process Proceed to next question
Question: Is the performance issue observed in only one instance?
If the answer is yes: Determine the difference between the instance
where it performs poorly versus the instance where performance is mal It is possible that the difference is a configuration parameter or arecently applied patch
Trang 8nor-Question: Are all users of the application that are experiencing the
per-formance degradation located within the same network segment?
If the answer is yes: The performance issue could be a networking issue.
Suggest testing the module on a PC that is located on a different ment of the network It may be necessary to involve the networkingteam to capture sniffer tracing on the network segment that is suspected
seg-to be performing poorly and seg-to assist in identifying the cause of theunderlying network performance problems
Question: Is the performance degradation limited to a particular
window of time?
If the answer is yes: The performance issue could be the result of a
scheduled job on the database or server This job may be causingresource contention Check all scheduled activity at the application,database, and server level Monitor the database and server during thetime period of performance issues for high resource processes If aresource-intensive job is found, reschedule it for a better time or acquiremore resources to support the required processing
Question: Is the entire application experiencing performance
degradation?
If the answer is yes: Begin monitoring the database and server to
iden-tify the underlying performance issue Tools for monitoring databaseand server performance are described later in this chapter Additionalmethods for troubleshooting are outlined in Chapter 3 of this guide
Question: Is the performance degradation limited to one module?
If the answer is yes: Ask the user community to provide the name of the
module that is performing poorly Then proceed to work with the usercommunity to open an SR and generate a user trace to provide to OracleSupport Additional details regarding generating trace files are providedlater in this chapter
Question: Is the performance degradation limited to one user?
If the answer is yes: If only one user is experiencing performance
degra-dation, the degradation is most likely isolated to the client Suggest thatthe user experiencing the performance issue try the task on a differentworkstation If the user successfully executes the application from adifferent PC, additional monitoring needs to take place on the user’sworkstation It is possible that additional memory or CPU resources arerequired, or operating system or browser patching is required Addi-tional tips for diagnosing client issues are discussed later in this chapter
C H A P T E R 4 ■P E R F O R M A N C E T U N I N G 97
Trang 9This list of questions may be enhanced and revised to suit your ment Update the questions regularly, and keep them in a central location.
environ-Be certain that the people providing the first line of support for Oracle cations are familiar with the questions developed by your organization Ifthese questions have already been addressed when performance issues arebrought to the Applications DBA, the time required to resolve the problemcan be reduced
Appli-Additional information can be found in MetaLink Note 69565.1, titled
“A Holistic Approach to Performance Tuning Oracle Applications Systems.”Using this note and the preceding sample questions, you may begin todevelop your own template for performance tuning
Developing an Action Plan
When a performance problem is reported, the Applications DBA should beable to respond with an action plan for attempting to resolve the issue Oncethe issue has been narrowed down to a possible cause, steps for resolving theissue should be outlined Next, a more detailed tuning effort should takeplace, focusing on the possible source of problem The tools discussed in thefollowing section will assist you in determining the steps required for a moredetailed tuning effort
After a resolution to the problem has been identified, update the actionplan with the steps taken to resolve the issue This information can be pro-vided to management and the user community for feedback during thetuning process This document also becomes a tool for future tuning efforts
If the same problem is encountered again, a potential solution has alreadybeen documented
Tools for Resolving Performance Issues
Tools and methods for identifying and resolving performance issues may becategorized at the database, server, application tier, and user levels TheApplications DBA will need to be familiar with all of these tools The toolsavailable will vary, depending upon the versions of the database and appli-cation installed
Trang 10The following topics will be discussed in this section:
• Database tuning: Tools for tuning the Oracle9i and Oracle10g databases
are described in this section For Oracle9i, details for executing and analyzing Statspack reports will be given For Oracle10g, the use of the
manageability infrastructure, including Active Session History (ASH),Automatic Workload Repository (AWR), and Automatic DatabaseDiagnostics Monitor (ADDM) will be covered
• Server tuning: This section will discuss using UNIX commands,
includ-ing top, sar, vmstat, and ps, to generate performance-related server data
An overview of how to analyze the server data will also be provided
• Application tuning: This section outlines how to analyze performance
issues for the following application tiers: Forms, Apache, JServ, and current Manager Included in this section are configuration guidelinesfor improving performance
Con-• User tuning: The use of the Client System Analyzer, an Oracle
Applica-tions tool for assessing the client tier, will be covered, and tips formanually analyzing the client will be included
• Trace files: The process for generating trace files for Forms and Self
Service applications will be detailed Steps for using tkprof and trcanlzr
to analyze trace files will also be covered
• Additional performance considerations: Several application profile
settings that affect performance will be outlined
• Common performance issues: Common performance problems will be
outlined
Tuning the Database
The best tools for identifying database performance issues are Statspack,
provided with Oracle9i, or Automatic Workload Repository (AWR), provided with Oracle10g It is possible to perform tuning tasks by querying the Oracle
database dictionary, which contains performance-related data, but suchtasks are beyond the scope of this guide We will rely on using the standardOracle tools, such as Statspack, Automatic Workload Repository, and Auto-matic Database Diagnostic Monitoring for collecting system performancedata Tool availability is dependent upon the version of the database
C H A P T E R 4 ■P E R F O R M A N C E T U N I N G 99
Trang 11Using Statspack for Oracle9i
Statspack is an Oracle-provided utility comprising PL/SQL code and scriptsthat are used for collecting and storing performance data The perfstat userowns the Statspack database objects, including packages and tables Pleasereview information specific to your database version for installation of theStatspack utility
■ Tip When defining the perfstatuser, be sure to set the default tablespace to atablespace other than SYSTEM, such as the TOOLStablespace Also, set the temporarytablespace for the perfstatuser to a temporary tablespace defined in the databaseinstance, such as TEMP
When using Oracle Applications, additional scripts are available to
enhance the Statspack utility MetaLink Note 153507.1 describes 11i
Statspack specifics in detail One additional script of interest, spmodfn.sql,may be executed to include the source module name of queries in the output
of a Statspack report
Once Statspack has been installed, a snapshot of the database may betaken, to collect performance information Two snapshots are required togenerate a performance collection window Reports can then be generated toanalyze database performance for the time period between two snapshots
■ Tip When generating snapshot reports, the two snapshots should not include a periodwhen the database was shut down; if it does, the report will be invalid
The SQL*Plus command in the following example may be executed asthe perfstat user to obtain a default snapshot:
sql>exec statspack.snapshot
Some parameters for thresholds should be changed from their defaultwhen executing Statspack snapshots for Oracle Applications These are listed
in Table 4-1
Trang 12Table 4-1.Statspack Thresholds and Descriptions
Parameter Name Description
i_snap_level Snapshot level; level 0 is the most general collection,
and 10 is the most detailedi_executioins_th SQL threshold; number of times the statement was
executedi_parse_calls_th SQL threshold; number of parse calls the statement
madei_disk_reads_th SQL threshold; number of disk reads the statement
madei_buffer_gets_th SQL threshold; number of buffer gets the statement
madei_sharable_mem_th SQL threshold; amount of memory required to
execute the statementi_version_count_th SQL threshold; number of versions (children cursors)
used by the statementi_all_init When set to TRUE, Statspack will capture all non-
default Oracle initialization parameters
The following example shows the recommended thresholds for shots taken on Oracle Applications:
Trang 13It is advisable to collect snapshots during periods of normal activity inthe system to provide a baseline of comparison for snapshots that are takenduring periods when performance degradation is being experienced In order
to compare information, snapshots should be of the same duration and beexecuted during the same level and type of activity Enterprise Manager (EM),dbms_jobs, or crontab may be used to schedule snapshots on a regular basis.The use of these methods for scheduling activities will be described in Chap-ter 6 of this guide The spauto.sql script in the $ORACLE_HOME/rdbms/admindirectory will schedule snapshots to execute every hour
■ Tip Executing a snapshot at the start and end of a payroll run is a good example ofwhen regularly scheduled snapshots should be scheduled
Once snapshots have been captured, the sprepins.sql script should beexecuted as the perfstat user in order to generate the performance report.The database ID and beginning and ending snapshot numbers are requiredinput for the sprepins.sql report to be generated The following exampleshows the command to generate a Statspack report:
-3015003723 1 VIS VIS vis1
Enter value for dbid:
After entering the DB ID and instance number, the script will display alist of available snapshot IDs The script will prompt for a beginning and anending snapshot ID to be used for the report
Analyzing Oracle9i Statspack Reports
Once the database performance data has been generated with Statspack,the Applications DBA will need to review the report The report will containmuch useful information, such as instance efficiency percentages, top timedevents, and top SQL, ordered by several criteria, such as physical reads, logi-cal reads, and executions, among others An example of a Statspack report isshown in Figure 4-1