White Paper Online Performance Configuration Guidelines For Peopletools 8.45, 8.46 and 8.47
White Paper Online Performance Configuration Guidelines For Peopletools 8.45, 8.46 and 8.47
White Paper Online Performance Configuration Guidelines For Peopletools 8.45, 8.46 and 8.47
This white paper is a practical guide for technical users, installers, system administrators, and
programmers who implement, maintain, or develop applications for your PeopleSoft system. In this white
paper, we discuss guidelines on how to diagnose a PeopleSoft Online Transaction environment, including
PeopleSoft Internet Architecture and Portal configuration. Configuration of Batch processes is not
covered in this document.
Much of the information contained in this document originated within the PeopleSoft Global Support
Center and is therefore based on "real-life" problems encountered in the field. Although every conceivable
problem that one could encounter with Tuxedo, the PeopleSoft Application Server, or your web server is
not addressed in this document, the issues that appear in this document are the problems that prove to be
the most common or troublesome.
This white paper consists provides guidance across the major PIA components: Application Server, Web
Server, Web Browser, and OS Kernel.
Keep in mind that PeopleSoft updates this document as needed so that it reflects the most current
feedback we receive from the field. Therefore, the structure, headings, content, and length of this
document are likely to vary with each posted version. To see if the document has been updated since you
last downloaded it, compare the date of your version to the date of the version posted on Oracle Metalink
or Customer Connection.
This paper is not a general introduction to environment tuning and we assume that our readers are
experienced IT professionals, with a good understanding of PeopleSoft’s Internet Architecture.To take full
advantage of the information covered in this document, we recommend that you have a basic
understanding of system administration, basic Internet architecture, relational database concepts/SQL, and
how to use PeopleSoft applications.
This document is not intended to replace the documentation delivered with the PeopleTools 8 or 8.4x
PeopleBooks. We recommend that before you read this document, you read the PIA related information in
the PeopleTools PeopleBooks to ensure that you have a well-rounded understanding of our PIA
technology. Note: Much of the information in this document eventually gets incorporated into subsequent
versions of the PeopleBooks.
Many of the fundamental concepts related to PIA are discussed in the following PeopleSoft PeopleBooks:
Additionally, we recommend that you read the BEA documentations (in HTML format) delivered with the
BEA CD-ROM to gain a thorough understanding of the BEA products that PeopleSoft uses -- Tuxedo,
Jolt, and WebLogic Server, as well as documentations from IBM if you plan to deploy WebSphere. Refer
to your PeopleSoft Installation and Administration book for directions on accessing the delivered BEA
and IBM documentations.
This material has not been submitted to any formal PeopleSoft test and is published AS IS. It has not been
the subject of rigorous review. Oracle assumes no responsibility for its accuracy or completeness. The use
of this information or the implementation of any of these techniques is a customer responsibility and
depends upon the customer's ability to evaluate and integrate them into their operational environments.
TOC/Navigation Title
This white paper contains the following information.
Lack of memory resources on the application is the most common cause of serious online performance
issues.If you are experiencing more than 4 seconds of response time for every (as opposed to just a few
transactions) web page refresh, it is very likely a problem with application server memory.These problems
are most likely caused by "memory swapping". These are the same problems you experience when trying
to run Microsoft PowerPoint, Word and Excel on a PC with 12 MB of memory.Here are some things you
can do to see if you need more memory for your application server.
1. Make sure don't have too many PSAPPSRV processes booted. During peak usage times, you
should see some small amount of queuing for the psappsrv processes. If you have too many
processes, this will affect performance. The extra PSAPPSRV processes will not only take up
memory space and CPU time from the OS, the main issue is that each PSAPPSRV will not be
'sufficiently' cached. I.e. Each PSAPPSRV can only cache a portion of the user panels. It will take
much longer time to fully cache each PSAPPSRV. To determine the queue length of the specific
domain, do the following steps:
o Run psadmin
o Go to “PeopleSoft Domain Administration” for the specific domain name
o Select "TUXEDO command line"
o Run the command ‘pq’
o The “# Queued” column shows the queue length for that application server process
2. Here is a guideline for the memory footprint for the PSAPPSERV:
o CRM and HRMS use about 100-150 MB per app server process. CRM gets 50 users per
process, HRMS gets less
o Supply chain uses 300-500 MB per process. You get about 10-20 users per process
o Financials use 150-300 MB per process. You get about 20-30 users per process
So, for example, if you have 1GB of memory available for a financials application server, boot no
more than 4 PSAPPSRV processes.
3. Make sure you include all test/dev/prod domains on a computer in your calculation.A common
problem is having 4 large domains on one computer with insufficient memory.
4. Check your memory utilization on the appserver when you experience performance
problem.Check for swapping by using the memory monitoring procedures outlined in the
following sections.
5. If you are swapping, you need to either add more memory or reduce the number of domains or
PSAPPSRV process on the computer.
Use NT TaskManager to monitor all the PeopleSoft processes. PeopleSoft processes have a process name
prefix with ‘ps’.
Start TaskManager and select ‘View’ from the menu bar, and then choose ‘’Select Columns’. Pick the
Memory Usage and Virtual Memory Size columns.
Sort the process by name, and the two memory columns will tell you approximately the amount of
memory each PeopleSoft process is using. "Memory Usage" is not the entire memory consumption of the
process, but only the amount of physical RAM that process is using. However, with NT, an application
may be using much more Virtual Memory (i.e., the paging file) than physical memory, so if you're
gauging memory at all, you have to have both these fields selected.
The memory usage columns will include the shared libraries loaded into each process; therefore, if you
add up all the processes, you will be ‘double counting’ the shared libraries portion. Since PeopleSoft
shared a lot of the common libraries, the summation of all PS processes is a quick estimator, not an
absolute measurement of memory usage.
The total “Memory Usage” and the “VM Size” of the combined PeopleSoft processes should not be
higher than 70% of the Real memory of the server.
Add the Object Counter, under ‘Memory’, and then the ‘Page/sec’.
Pages/sec is the number of pages read from the disk or written to the disk to resolve memory references to
pages that were not in memory at the time of the reference.This is the sum of Pages Input/sec and Pages
Output/sec.This counter includes paging traffic on behalf of the system Cache to access file data for
applications.This value also includes the pages to/from non-cached mapped memory files. This is the
primary counter to observe if you are concerned about excessive memory pressure (that is, thrashing), and
the excessive paging that may result.
When the Page/sec goes above 100 hard pages per second, there is a swapping problem. The number of
PSAPPSERV processes should be reduced or more memory needs to be added to the server.
To assist end-user better understand the memory consumption and CPU usage of the PeopleSoft
Application Domain, we have created several UNIX vendor specific shell scripts to monitor the above
parameters. The monitoring shell scripts can be obtained from Customer Connection web site
(https://www.peoplesoft.com/corp/en/login.jsp)
Login to the Customer Connection, select Products + Industries on the right. Choose Enterprise Product
Lines, select Enterprise Tools and Technology, select Enterprise PeopleTools, select PeopleTools 8.4
underneath. In the middle of the page is the Red Paper section, which you will find the latest revision of
this Red Paper and the Shell Scripts for download.
---
Resident memory refers to the real physical memory currently required by the process for its operation.
Virtual memory refers to the process virtual address size, this include memory that has been paged out to
the physical disk. If the Virtual memory continues to increase, the “RecycleCount” of the AppServer
should be lowered.
The output of ps_chk_domain.sh divides the portion of PSAPPSRV processes from the entire Domain so
that the end-user (or system administrator) can get a better understanding with the distribution of the
memory resources.
It is recommended that the total Resident memory for the entire ‘PS Processes’ should not exceed 70% of
the total real memory available on the server.
To check whether paging is a problem run vmstat and check out the pi column to see if you are doing
much paging in and po for paging out. You can also run sar -q to check paging;or vmstat -s and take
multiple samples to see if the page ins and outs to paging space are the majority of the total paging.
This configurable parameter dictates the number of services after which AppServer will automatically
restart.
Background
The AppServer processes will use physical runtime memory to cache Panel objects in order to speed up
user response time instead of fetching the Panel objects from the database server every time. However, as
the number of requests serviced by the AppServer increases, the amount of physical memory occupied by
the AppServer processes also increase. When the amount of memory occupied by the AppServer becomes
too large (relative to the real memory available at the time), paging to file system will occur and paging
will impact user experience.
In order to effectively manage the memory footprint of the AppServer, keeping the “Recycle Count” at a
realistic level is important. When the AppServer reaches the specified “Recycle Count” value, the
AppServer will terminate and restart itself. When the AppServer terminated, the occupied memory will be
released.
Recommendation
The ‘recycle count’ should be adjusted so that no memory swapping is introduced because of a high
‘recycle count’ value. (See session above for how to determine memory swapping.)
To minimize the cost of re-caching the AppServer, it is important to enable ‘File Cache’ on the AppServer.
To enable Server Caching, set the EnableServerCaching=2, (the default value is 1 for 8.42 and below, 2
for 8.43 and above).
-----------------------------------------------------------------------
; EnableServerCaching -
; 0 Server File caching disabled
; 1 Server File caching limited to most used classes
; 2 Server File caching for all types
EnableServerCaching=2
-----------------------------------------------------------------------
PeopleTools 8.4x has a shared cache feature in which the various psappsrv processes in an app server
domain use a single set of cache files rather than separate cache files for each process. All managed
objects are included in the shared cache files so that objects are not loaded from the database.
Note:
You have to pre-load all the necessary cache objects before you can enable the
Shared Cache feature.
PeopleSoft installation supplied with an Application Engine program called LOADCACHE to assist
customer in generating this pre-loaded shared cache.
• Save disk space since all AppServer processes use the same common directory to read the file
cache contents
• Speed up operation since all the managed objects are cached into the common directory ahead of
time, instead of retrieving from the database tables on demand
1. Make sure that the database that the application server runs against produces clean SYSAUDIT
runs. If SYSAUDIT is not clean, the LOADCACHE program may fail.
2. Make sure the user who is executing the program has the “Load Application Server Cache”
component defined to its Permission List in User Profile. Find or add the Utilities page for the
users permission list and add the component Load Application Server Cache.
3. (For 8.42 and below only) Check your psprcs.cfg file (Process Scheduler configuration file).
psprcs.cfg is where you specify the type of objects to cache using the EnableServerCaching
parameter. For PeopleTools 8.4x set it to 2. The LOADCACHE reads this setting and caches
metadata according to the value specified in the Process Scheduler configuration. Note. Do not
enable shared caching for Process Scheduler. Make sure ServerCacheMode=0
4. [Cache Settings]
5. ;============================================
6. ; Settings for Tools that use Cache
7. ;============================================
8. CacheBaseDir=%PS_SERVDIR%\CACHE
9. ;-----------------------------------------------------------------------
10.; EnableServerCaching -
11.;0 Server File caching disabled
12.;1 Server File caching limited to most used classes
13.;2 Server File caching for all types
14.EnableServerCaching=2
15.;-----------------------------------------------------------------------
16.; CacheBaseDir = the base cache directory
17.;CacheBaseDir=%PS_SERVDIR%\CACHE
18.;-----------------------------------------------------------------------
19.; ServerCacheMode
20.;0 One cache directory per App Server Process
21.;1 Shared Cache
22.ServerCacheMode=0
23. In PeopleTools 8.40 or above: Select PeopleTools, Utilities, Administration, Load Application
Server Cache.
24. Enter the appropriate Run Control ID (You may have to add a new value here). The Load
Application Server Cache page appears.
25. In the Output Directory specify directory where you want the cached metadata to be written:
Unix example> /ds1/home/testora/pt814c1/PT814U25/appserv
NT example> c:\temp\
26. Select the correct process scheduler and click Run. Navigate to…PeopleTools>Process
Monitor.Wait for the process to become “Success”. The first time you run the process may take 4-
5 hours.
Note: Run the program with Run Location set to “Server”.
27. Shut down your application server domain.
28. Enable shared caching with the ServerCacheMode parameter (ServerCacheMode=1) in
psappsrv.cfg of the domain, and reconfigure the domain so that the changes are reflected.
29.[Cache Settings]
30.;============================================
31.; Settings for Tools that use Cache
32.;============================================
33.CacheBaseDir=%PS_SERVDIR%\CACHE
34.;-----------------------------------------------------------------------
35.; EnableServerCaching -
36.;0 Server File caching disabled
37.;1 Server File caching limited to most used classes
38.;2 Server File caching for all types
39.EnableServerCaching=2
40.;-----------------------------------------------------------------------
41.; CacheBaseDir = the base cache directory
42.;CacheBaseDir=%PS_SERVDIR%\CACHE
43.;-----------------------------------------------------------------------
44.; ServerCacheMode
45.;0 One cache directory per App Server Process
46.;1 Shared Cache
47.ServerCacheMode=1
48. Create the <PS_HOME>\<DomainName>\cache\share directory for the appropriate domain.
49. Navigate back to your Process Scheduler. You will now have some cache generated under a new
'stage' directory.
Unix example /ds1/home/testora/pt814c1/PT814U25/appserv/CACHE/stage
NT example c:\temp\CACHE\stage
50. You should see a bunch of files with *.dat and *.key extensions inside the stage directory. Copy
the contents of the stage directory into the share directory on your appserver domain.
51. Reboot your application server domain.
Note:
For Shared Cache users: Shared Cache is intended to be used in a very disciplined
environment. If you chose the use Shared Cache, you should not try to make any
application changes that will affect the Meta Data objects, such as Panel definition,
Menu structure or even Permission List. It is because once you chose to use Shared
Cache, the cache files are all marked as READ ONLY. The AppServer processes
cannot updated the cache files for any delta, and each access to the changed Meta
Data will result in a database access which is very costly.
The preferred way to use LoadCache files is to copy the entire cache file set to the individual AppServer
process (i.e. seeding the cache directory with the pre-built LoadCache fileset). The multiple copies of
Cache file in each directory will consume disk space, but it does allow process to update the cache file
content. This flexibility will allow you to make application changes and not suffer from performance lost.
PSAPPSRV Instances
The number of PSAPPSRV instances should be kept to a low number. PSAPPSRV need to cache
PeopleSoft meta-objects in order to be effective. Too many PSAPPSRV process instances will make it
difficult to fully cache all of them. This is even more difficult for a Win2000/NT Tuxedo domain because
the Bulletin-board process will try to use the same PSAPPSRV process id (not round-robin’ing like in
UNIX).
In general, it is a good idea to allow queuing to happen during peak working hours. Spawning extra
PSAPPSRV instance to handle high-volume workload is not recommended. Spawning is a relatively new
feature in Tuxedo, and is not necessary for running the PeopleSoft application servers. There is actually a
high cost incurred when each new PSAPPSRV process is started up. Each process has to establish its
cache, which takes time, CPU, and memory. Instead, the "min" setting should be used to indicate how
many PSAPPSRV processes are required for proper throughput.
To disable spawning, set the Min Instances the same as Max Instances.
The Java virtual machine heap space is the memory region where the Java objects (both live and dead)
resided. When the Java heap runs out of space, the Java Garbage Collector will be invoked to deallocate
unreferenced objects and free up more space for the program to continue its operation. The JVM cannot
service user requests during garbage collections.Many customers have their JVM heap size set to a
minimum heap size of 64MB and maximum size of 256MB.Setting the JVM heap size to a larger
minimum value (preferably equal to the maximum value) avoids the performance hit incurred by
dynamically growing the JVM and improves predictability; it also lessens the frequency of JVM garbage
collection. To set the heap size for PeopleSoft Internet Architecture open opmn.xml in
<ORACLE_HOME>\opmn\conf for editing. Locate the process-type node for your PIA installation and
add
to the value attribute of the java-options node. The –verbosegc switch allows you to monitor the amount
of heap usage and the time Oracle AS took for garbage collections so you can make further adjustments if
necessary.The garbage collection details will be written to the log file
<ORACLE_HOME>\opmn\logs\OC4J~<PIA Install Name>~default_island~<jvm instance number>.
Depending on the expected interval between your users’ requests, it may be beneficial to change the
Apache KeepAlive Timeout from 15 to 30 seconds and set the Maximum KeepAlive Requests to zero.
Open <ORACLE_HOME>\Apache\Apache\conf\httpd.conf for editing. Locate the KeepAlive,
MaxKeepAliveRequests, and KeepAliveTimeout lines and set them to ‘On’, ‘0’, and ‘30’
respectively.Since KeepAlive controls how long OHS will keep the connection between the client and
OHS, it is possible that if there are a large number of concurrent users, increasing KeepAlive from 15 to
30 will negatively impact performance because there will be fewer available connections to service the
requests. You should only change it if you see excessive CPU consumption of the OHS process (look for
apache.exe in your Task Manager in Windows, or do ps –ef | grep httpd in UNIX) and that you
expect a stable number of users making requests at interval larger than the default setting of 15
seconds.When you increase KeepAlive, you may need to increase MaxClients or ThreadsPerChild (see
the next section) to ensure incoming requests are not starving for connections.
For UNIX, the equivalent setting is MaxClients. In most cases the default value of 150 is sufficient.
For Windows, if you decide to increase ThreadsPerChild, you should set the Oc4jCacheSize to the same
value.This setting controls the number of connections between the Oracle HTTP Server and OC4J and
setting it to the same value as ThreadsPerChild should give the optimal performance.To change this value,
open <ORACLE_HOME>\Apache\Apache\conf\mod_oc4j.conf and search for Oc4jCacheSize.If the
entry is not there, add the following line above the first Oc4jMount entry (this assumes you have
increased ThreadsPerChild to 150):
Oc4jCacheSize 150
Open mod_oc4j.conf in <Apache_home>\conf for editing. Locate the appropriate <virtual host> section
and comment out the line beginning with TransferLog .
JDK 1.4.2 is installed automatically when you install Oracle AS. To check the JVM versions certified with
Oracle AS, please login to metalink.oracle.com, click the “Certify and Availability” menu and click “View
Certifications By Product”.
WebLogic
PeopleSoft Platform Database lists the minimum required service pack for WebLogic Server.
%WLS_HOME%/logs/log.txt specifically indicates service pack installed. For PeopleTools 8.40 and 8.41
Service Pack 1 should be used, for PeopleTools 8.42 Service Pack 2 should be used, and for PeopleTools
8.43 to 8.47 Service Pack 4 should be used.
Under normal circumstances, WebLogic should be using the "Posix Performance Pack", and this will use
the native OS's socket implementation. When the "Performance Pack" is not loaded properly, generic
socket implementation will be used, which is not efficient and there could be performance issue or socket
stability problems.
To verify that your WebLogic Server is loading the Performance Pack correctly, check your WebLogic
output message and search for the reference to "NT/Posix Performance Pack".
<Server ListenPort="80" Name="PIA" Notes="The PIA server is the default server for
PeopleSoft 8.40 Internet Architecture."
TransactionLogFilePrefix="C:\Apps\bea\wlserver6.1/config/peoplesoft/logs/"
XMLEntityCache="XMLCacheMBean" XMLRegistry="PeopleSoft IG XML
Registry">
<ExecuteQueue Name="default" ThreadCount="N" />
...
</Server>
where N should be based on the “peak” concurrent HTTP usage at any given point of day, ie, the number
of HTTP actions (such as posting a HTTP request or downloading a HTTP response) that can be executed
concurrently. This should not include user whom are login but not actively performing any HTTP action.
To preserve system resources, administrator can fine tune the number of ExecuteThread to be a
percentage of the estimated peak value, such as N = 90% of Peak usage.One way to obtain a more
accurate Peak value is to monitor the thread usage via WebLogic console.
Note:
For Unix platforms, when you increase the ThreadCount, you are allowing more
socket connections to be established. You will need to increase the number of file
descriptors (maxfiles and maxfiles_lim) accordingly. To check the current file
descriptor value, use the following command:
You also need to raise the number of threads limit per process (max_thread_proc) in UNIX. As for
Windows these are implicitly limited by other system resources such as memory, and there is no explicit
parameter that controls them.
Confirm JRE
Confirm from BEA's platform page that the installed JRE is certified (may require OS patches).
http://e-docs.bea.com/wls/certifications/certifications/index.html
Note: For Linux it is necessary to set an environmental variable to work around a JRE bug.
$ export J2SE_PREEMPTCLOSE=1
The rationale behind is the current Linux is still using the old UNIX network/thread semantics, and the
close() is not preemptive as in Solaris and AIX. This applies to JRE1.2.2, 1.3 and 1.3.1.
Many customers have their JVM heap size set to a minimum heap size of 64MB and maximum size of
256MB.Setting the JVM heap size to a larger minimum value (preferably equals the maximum value)
avoids the performance hit incurred by dynamically growing the JVM and improves predictability; it also
lessens the frequency for the JVM's garbage collection.
See JVM Heap Size section below to learn how to change the Heap size for specific web server.
A file descriptor is required for every file that is opened, every *.class file read in by WebLogic, every jolt
connection PIA/Portal make to the appserver, every connection that has to open back to a client, plus any
other socket based communication that was occurring on that machine.
To raise the file descriptors for UNIX, use the following command:
ulimit –n 4096
In Windows NT/2000 there is not an explicit parameter for the number of file descriptors. It is implicitly
limited by hardware resources, mainly system memory.
Socket based applications that are opening and closing hundreds or thousands of sockets need to have
sockets they have marked as closed, truly closed. Once a process closes a socket, it is really only marked
as closed until the OS, based on a cleanup/flush timeout, makes that socket available again. The default
value for this is 11 minutes, lower this value to 1 minute.
See the Reducing TCP Wait Time section to learn how to change the TCP Wait Time for a specific OS.
The Java virtual machine heap space is the memory region where the Java objects (both live and dead)
resided. When the Java heap runs out of space, the Java Garbage Collector will be invoked to deallocate
the dead objects and free up more space for the program to continue its operation. The JVM cannot
service user requests during garbage collections.
To monitor the amount of heap usage and the time WebLogic took for the Garbage Collection, you can
add the verbosegc switch to the setEnv.cmd script file. You have to start the WebLogic from the command
line -- startPIA.cmd (instead of an NT service) to see the GC output. In setEnv.cmd,
To keep the performance degradation from Garbage Collection at a minimum, users should use the
command line option -noclassgc. This will inhibit a thread that would normally clear out unused classes
(thus saving the load incurred by that thread).
The goals of tuning your heap size are twofold: minimize the amount of time that you spend doing GC
while maximizing the amount of clients that you can handle at a given time.
In a production environment the servlets are not modified so checking and reloading would only incur
unnecessary work, and thus ServletReloadCheckSecs should be set to “–1” for each of the components.
(If the field is not present it defaults to 0 – always reload, which is undesirable. In that case the field
should be introduced with value –1.)
WebSphere
Note:
PeopleTools 8.4 supports native Http Server within WebSphere at port 9080 and
9443, for http and https, respectively. Use of IBM Http Server, which listens at port
80 or 443 for http or https, is optional
Note:
Before you begin the installation, review the PeopleSoft Platforms Database to make
sure you are installing WebSphere into a supported environment.
For detailed installation instructions, especially the required e-fixes needed for WebSphere please refer to
the “WebSphere Install” document in Customer Connection.
Note:
Install WebSphere eFixes.The user can install eFixes in any order. To remove eFixes
rollback efixes in the reverse order (that you applied).
where N should be based on the “peak” concurrent HTTP usage at any given point of day. The number of
HTTP actions (such as posting a HTTP request or downloading a HTTP response) that can be executed
concurrently. This should not include user whom are login but not actively performing any HTTP action.
For WebSphere 5, the tags have changed so please consult the WebSphere documentation on the correct
attribute to modify.
To preserve system resources, administrator can fine tune the number of Thread to be a percentage of the
estimated peak value, such as N = 90% of Peak usage
Confirm JRE
Note:
When you install WebSphere, IBM’s JRE 1.3 is installed automatically. For
WebSphere 5, JRE 1.4.1 is installed
Set JVM Heap Size to 256MB and Monitor JVM Garbage Collection
(Refer to the WebLogic section on how to decide the JVM heap size.)
In %WAS_HOME%/config/server-cfg.xml, locate the following lines:
<jvmSettings xmi:id="JavaVirtualMachine_1"
classpath="${WAS_ROOT}/lib/bootstrap.jar;${WAS_ROOT}/properties;${WAS_ROOT}/installed
Apps/peoplesoft/PORTAL/WEB-
INF/lib/entappletbase.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEB-
INF/lib/entapplethttp.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEB-
INF/lib/entappletp10.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEB-
INF/lib/entappletp12.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEB-
INF/lib/entappletp5.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEB-
INF/lib/entappletp7.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEB-
INF/lib/entappletssl.jar" bootClasspath="" verboseModeClass="false"
verboseModeGarbageCollection="false" verboseModeJNI="false" initialHeapSize="256"
maximumHeapSize="256" runHProf="false" hprofArguments="" debugMode="false"
debugArgs="" genericCommandLineArgs="com.ibm.ws.runtime.StandardServer"
disableJIT="false"></jvmSettings>
With IBM websphere 5, you can go to administration console (http://:9090/admin/) to make setting
changes. Login with blank user id. On the left Navigation , click on Servers -> Application Servers. Click
on your server name (default is server1). Click on Process Definition under Additional Properties. Click
on Java Virtual Machine under additional Properties. Here you can change and all the parameters you
need verbosegc etc.
Other JVM options, if needed (e.g. -noglassgc), should be inserted in the server-cfg.xml file rather than
the startup script (startserver). For example,
(In the PSINTERLINKS/WEB-INF directory such file may not exist. In that case create the file “ibm-
web-ext.xmi” with the following content:
<webappext:WebAppExtension xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI"
xmlns:webappext="webappext.xmi" xmlns:webapplication="webapplication.xmi"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmi:id="WebAppExtension_1"
reloadInterval="0" reloadingEnabled="false" fileServingEnabled="true"
directoryBrowsingEnabled="true" serveServletsByClassnameEnabled="false">
<defaultErrorPage xsi:nil="true"/>
<additionalClassPath xsi:nil="true"/>
<webApp href="WEB-INF/web.xml#WebApp_1"/>
<extendedServlets xmi:id="ServletExtension_1">
<extendedServlet href="WEB-INF/web.xml#Servlet_1"/>
</extendedServlets>
</webappext:WebAppExtension>
<applicationext:ApplicationExtension xmi:version="2.0"
xmlns:xmi="http://www.omg.org/XMI"
xmlns:applicationext="applicationext.xmi" xmlns:application="application.xmi"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmi:id="Application_ID_Ext" reloadInterval="0">
<application href="META-INF/application.xml#Application_ID"/>
</applicationext:ApplicationExtension>
Here is the instructions on how to collect JOLT request timing traces for WebSphere:
To use the Resource Analyzer with the Advanced Single Server Edition of WebSphere, do the following:
• Graphics objects
• Style Sheets
• Java Scripts
• Some HTML pages (such as PIA Login page)
The following web browser settings are the recommended configurations. These configurations are
usually the default settings of the respective browsers. Please verify to ensure correctness.
Netscape Browser
Most newer browsers since 1998/1999 have been equipped to support the HTTP 1.1 standard known as
"content-encoding." Essentially the browser indicates to the server that it can accept "content encoding"
and if the server is capable it will then compress the data and transmit it. The browser decompresses it and
then renders the page.
This is an important performance feature for the Web Browsers that are using the slower bandwidth
medium, such as dialup connections or 56K lines.
Here is the configuration recipe to verify if Internet Explorer is configured to use the HTTP 1.1 protocol:
Additional Configurations
Browser Compression
If compressResponse is set to true, compression in the communication between the web server and the
browser is enabled. Setting it to false turns off the compression. Gzip and Compress are supported.
Default is true.
If compressCacheFiles is set to true, compression of cache files from web server to the browser is
enabled.Only those cache files that have their mime-types specified in the list of compressMimeTypes
will be compressed. Gzip and Compress are supported. Default is set to false because there were some
versions of Internet Explorer with certain settings that didn't like javascript and css in compressed format.
However, that is a rare scenario therefore in general it should be set to true.
The setting compressMimeTypes specifies a comma-delimited list of the MIME-type objects that should
be sent in compressed form to the browser. Note that this is applicable only when compressCacheFiles is
set to true.Gzip and Compress are supported. Default: application/x-
javascript,text/javascript,text/css,text/html.
Note:
The main purpose of compression is to reduce the amount of data to be transmitted,
but that comes with a price – the extra CPU processing time. In cases where high-
speed links are used and the gain in transmission time does not justify the loss in
CPU processing time, compression should be turned off.
Note:
For PeopleTools up to 8.41 the compression settings would cause an error for certain
update of Internet Explorer 6.0. In particular, the PIA menu may disappear when a
user opens a PIA page. So it is advised to turn them off. This is fixed in PeopleTools
8.42 onwards.
Note:
Some query entries are truncated when IE tries to open queries in Mircosoft Excel. A
workaround (for 8.43 and above) is to turn off query compression via an optional
parameter compressQuery=false in configuration.properties, and the parameter
introduced is to turn off JUST queries.
Homepage Caching
Caching improves system performance by reducing service requests from the web server to the
application server. If PortalCacheObjects is set to "true" the portal servlet (psp) will cache the following
objects in web server memory:
• Portal Registry
• Node (Remote and Local)
• Content Reference
• Template (Static only)
If PortalCacheObjects is set to true, object changes won't take effect until the objects become stale, and
are refreshed (see PortalObjectStaleInterval setting below), or the web server is restarted. Default setting:
true
PortalCacheStaleInterval is the amount of time, in seconds, before portal cache is considered stale, and
updated with the latest copy from the application server.In other words, this is the amount of time before
changes to cached objects take effect. This setting applies to the same objects as PortalCacheObjects.
Default setting: 86200 (24 hours)
The portal automatically throws away all cache entries in memory after CachePurgeAllHitCount requests.
Note:This setting applies for all web sites on this web server.It should be the same value, within all
configuration.properties files on this web server.Setting this value to -1 disables hitcount purging. Default
setting: 1000
The portal automatically throws away all stale cache entries after CachePurgeStaleInterval seconds.This
setting also applies for all web sites on this web server. Setting this value to -1 disables stale cache
purging. Default setting: 300
The portal will cache proxied javascripts to improve performance if portalUseCachedProxiedJS is set to
"true.
Target content is cached in memory when the TargetContent tag in the template specifies that the target
should be cached. Only static content should be displayed in a template with a cached target tag. The
TargetContent tag should look like this:
The CacheTargetContent setting should be "true" to allow caching of target content.Setting this to "false"
disables all target content caching in the portal servlet, even if the target tag specifies cached content.
The following table summarizes the properties and their default values.
Homepages can also be cached on each user's browser. This means that the browser does not access the
web server after the homepage is initially retrieved. You can turn this feature on or off, and also adjust the
specific time interval that must pass before the web server is accessed again to get a "fresh" homepage. In
any case, if a user clicks the browser's "refresh" button, the homepage is accessed from the web server
again, overwriting the homepage cached on the browser.
Caching the Homepage should be beneficial for either production or development environment. It is
recommended to have Homepage cache turned on.
The following table lists default values of the parameters in PIA configuration.properties:
Note:
The file browserprops.xml specifies which web browser could be used with
homepage caching. Please verify to make sure your choice of browser is enabled for
caching.
As with homepages, navigation pages can be cached on each user's browser. This will improve the users’
response time in traversing between cached menu pages. The Portal Administrator can set systemwide
options for navigation cache by using the “Define Personalizations” page and modifying the value for
METAXP. It is recommended to set the METAXP to a high value. This will keep the menu pages inside
the browser cache. If your menu pages are not going to change frequently, set METAXP to 10080 (one
week) or more.
A user can override the value of METAXP for his/her own browsing sessions: after logging on, go to My
Personalizations, then hit the Personalize Option button for General Options. Change the METAXP value
by entering a new value for Time page held in cache and then click OK.
Navigation Caching should not be used during development time because any newly added menus will
not be reflected in the cache. Navigation cache should only be used in production systems.
HTTP KeepAlive
HTTP KeepAlive is intended to maintain a persistent socket connection between the Web Browser and the
Web Server so that no new connections are required when the HTML pages refer to other HTML objects.
However, keeping the socket connection persistent will occupy a socket pair between the Browser and the
Web Server. When the keepalive timing is set too long, the following problems are created:
Keeping too many socket handles around will reduce the number of sockets available for new connection
use.
It is generally advised that in 8.4x, keepalive should be turned on, which is the default for WebLogic,
WebSphere, and Oracle Application Server.
The default TCP Wait Time for both NT and most UNIX operating systems are 4 minutes, which is too
long for PIA usage and tends to leave too many socket connections staying in the TIME_WAIT state.
By shortening the TCP Wait time, the socket can be recycled more efficiently
For Windows NT
Use the registry editor, regedit, and create a REG_DWORD named TcpTimedWaitDelay under
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TcpIp\Parameters
For AIX
/usr/sbin/no –o tcp_timewait =1
The tcp_timewait option is used to configure how long connections are kept in the timewait state. It is
given in 15-second intervals, and the default is 1. Using the default or set it to 2 for slower network is
recommended.
Note:
Be careful when you use this command. The no command performs no range checks,
therefore it accepts all values for the variables. If used incorrectly, the no command
can cause your system to become inoperable.
This command only operates on the currently running kernel. The command must be run again after each
startup or after the network has been configured.
As root do:
where 60000 is for 60 secs, the minimum recommended value. You must put this command in one of the
rc2.d scripts to get it to be automatically set at boot time.
For HP/UX 11, you can place the ndd settings inside the file /etc/rc.config.d/nddconf without the need of
the startup scripts.
IBM has recommended the following environment variables setup to improve the Threading model with
PeopleSoft’s PIA.
Set up the following environment variables for the PeopleSoft Application Server user, database user and
the Web Server user:
For Korn Shell user, please place the following lines in the .profile:
export AIXTHREAD_SCOPE=S
export AIXTHREAD_MNRATIO=1:1
export AIXTHREAD_COND_DEBUG=OFF
export AIXTHREAD_GUARDPAGES=4
export AIXTHREAD_MUTEX_DEBUG=OFF
export AIXTHREAD_RWLOCK_DEBUG=OFF
For C Shell user, please place the following lines in the .cshrc file:
set AIXTHREAD_SCOPE = S
set AIXTHREAD_MNRATIO = 1:1
set AIXTHREAD_COND_DEBUG = OFF
set AIXTHREAD_GUARDPAGES = 4
set AIXTHREAD_MUTEX_DEBUG = OFF
set AIXTHREAD_RWLOCK_DEBUG = OFF
Timeout Settings
1. Timeout during a PIA transaction (Intra-Transactional) – When the timeout expires, the transaction
will fail and need to be resubmitted.
2. Timeout between PIA transaction (Inter-Transactional) – When the timeout expires, the user has
been idling for too long, and the resources associated with the user will be freed. Need to relogin
to re-establish the state.
3. Timeout that "reserves" a resource until the expiration time before putting it back into the pool for
recycling. (e.g. tcp_timewait)
It should be noted that for the Intra-transactional timeouts, their values should be “staged”. In other
words, the end-to-end timeout value should always be greater than that of its intermediate leg(s). With this
in mind we can look at our timeout values, from bottom up:
Type=Intra-transaction, and it includes the time it takes at the DB server. According to the PIA
Answer Book:
Service Timeout
Specifies the number of seconds a PSAPPSRV waits for a service request, such as MgrGetObj or
PprLoad to complete, before timing out. Service Timeouts are recorded in the TUXLOG and
APPSRV.LOG. In the event of a timeout, PSSAPSRV terminates itself and Tuxedo automatically
restarts this process.
In other words it has to be large enough to accommodate the longest acceptable requests and
queries It is recommended to set x=1200 (20 minutes).
• PIA: tuxedo_send_timeout = w sec (default 50) ; tuexdo_receive_timeout = x sec (default 600) (in
pstools.properties)
The receive timeout is also intra-transactional, and it has to be bigger than appserver service
timeout.
The receive timeout indicates the maximum number of seconds that the servlet waits for a
response from the application server. If you increase your application server service timeouts, such
as the Service Timeout setting for PSAPPSRV, then increase the tuxedo_receive_timeout
parameter to be greater than the Service Timeout values that appear in the PSAPPSRV.CFG
configuration file on the application server.
This is inter-transactional, as one has to re-login when the servlet expires (technically this is a
security setting).
The meta refresh tag in seconds. It should be less than or equal to the session.timeout for the
servlet. For WebLogic 6.1 it should be less than or equal to <session-timeout> as discussed below.
This is inter-transactional. See description (default is 60, but in most cases this can be reduced to
10 to conserve resources):
Specifies the amount of time, in minutes, that a client connection can remain idle (no work
requested) before Tuxedo will terminate a client connection. Client disconnects are transparent to a
client, and a user just needs to click the mouse to cause a reconnection.
<session-config><session-timeout>x</session-timeout></session-config>
This is inter-transactional and specifies the number of minutes (WebLogic) to wait before
invalidating an unused session.
Note that setting this value too high would tie up webserver resources, especially when users close
the browser instead of logging out gracefully. Setting it to be the same as (or a bit higher than) the
JOLT cleanup timeout is generally a good idea.
• HTTP timeout:
A common problem:
PIA: Time out Errors on PeopleSoft pages on PIA even when people are using the page.
Solution:
1. Increase the timeout values in servlet session timeout AND webserver session timeout.
In configuration.properties:
In web.xml:
<session-config>
<session-timeout>60</session-timeout>
</session-config>
"meta-tag" session timeout to be 3600 (This will allow users to use the page for 60 minutes with
no time out errors)
tuxedo_receive_timeout to 1500
For Oracle database user who has database logging turn on may see large volume of archive log created.
The Archive Log (or Redo log) was a side effect created by the Application Messaging Dispatchers’ select
statement, which is invoked periodically based on the value of the “Scan Interval”.
Application Messaging has been enhanced to minimize the side effect on Oracle logging.
Problem: These PSSUBDSP_dflt errors were occurring upon starting the App server. There are hundreds
of entries like this. While these errors occurred, no access to the PS system was possible in 3-tier mode. 2-
tier mode was fine.
Second Error
Resolution:
This problem only appears when the database is shutdown while the appserver is running.The nightly
backup was inadvertently doing this.We tested shutting down everything in the correct order and this
problem does not occur.The PSSUBHND instances are too busy erring out to handle even one
subscription.
The only way to fix this is to shut everything down (except for the OS) and restart it in the correct
order.For some reason, you can shut down the appserver and process scheduler (not the DB) and re-start
them.This does not fix the problem, though this might be a combination of the fact there is an under-
configured PSSUBHND.
Kernel Configurations
For a large environment with many domains the following kernel parameters will provide a good deal of
resources. Kernel parameters this size is not uncommon in our customer environments.
For Solaris
MSGMAP=2048
MSGMAX=65536
MSGMNB=65536
MSGMNI=1024
MSGSEG=32767
MSGTQL=1024
SEMMAP=512
SEMMNI=512
SEMMNU=4096
SEMUME=10
SEMMNS=4096
SEMMSL=8
For HP-UX
dbc_max_pct=10
dbc_min_pct=8
default_disk_ir=1
fs_async=1
max_thread_proc=2048
maxfiles=2048
maxfiles_lim=4096
maxssiz=200802304 (191.5 MB)
maxswapchunks=2048
maxuprc=512
maxusers=2000
msgmap=2048
msgmax=65535
msgmnb=65535
msgmni=1024
msgseg=32767
msgtql=2046
semmap=512
semmni=512
semmns=4096
semmnu=4096
For Tru64
ipc:
msg_max = 262144
msg_mnb = 262144
msg_mni = 16384
msg_map = 20000
msg_tql = 4096
shm_max = 4294967295
shm_min = 1
shm_mni = 300
shm_seg = 100
sem_mni = 4096
sem_mnu = 1048
sem_mns = 819200
sem_msl = 1000
sem_opm = 400
sem_ume = 1048
sem_vmx = 32767
sem_aem = 16384
ssm_threshold=8388608
num_of_sems = 1024
max_kernel_ports = 22487
inet:
tcpnodelack = 0
tcbhashnum = 16
tcbhashsize = 8192
ipqmaxlen = 1024
ipqmaxlen = 2048
ipqs = 1
ipqs = 32
tcp_mssdflt = 1460
tcp_msl = 60
pmtu_enabled = 0
tcp_sendspace = 61440
tcp_recvspace = 61440
udp_ttl=60
vfs:
bufcache = 1
fifo_do_adaptive = 0
socket:
somaxconn = 65535
sominconn = 65535
proc:
This is needed if large numbers of Cobol programs are to be compiled. Many applications, for example
Oracle and the Tru64 Java Fast VM, need per_proc_data_size to be set correctly. However, this number
should not be larger than the physical memory available on the system, otherwise swapping will occur.
<H2Call Telephony Interface (CTI)
1. reaper_interval
2. # How often (in sec) to look for and delete expired events.
ns_param reaper_interval 600
Default used to be 3600 (1 hour) which caused a steady increase in RENSERVER memory.
3. default_kn_expires
4. # Default to use if an event is received without a kn_expires header.
5. # Use "infinity" or something like "+3600" for 1 hour; must be at least "+5"
ns_param default_kn_expires "+10"
Default used to be 3600 (1 hour). Again, this caused a steady increase in RENSERVER memory
6. permission_max_idle
7. # How long (in sec) the permission can stay in cache if the tunnel is closed
ns_param permission_max_idle 300
Default used to be 60. This increases the time the PSTOKEN of the agent is expired at the
RENSERVER.
8. mtu_size
New parameter to pad the network TCP packets with additional data to reduce latency in TCP
ACK packets.
1. interval_topic_reaper
2. # Topic reaper interval in milliseconds
interval_topic_reaper = 3000000
Default value 30000 (30 seconds). This change is to keep the session alive for extended period of
time.
3. heartbeats_to_miss
heartbeats_to_miss = 10
New parameter specifies the number of heartbeats to miss before the session is set for deletion. It
used to be 2 hard coded in the code.
4. mtu_size
5. #for psmcapi
mtu_size=1300
New parameter to pad the network TCP packets with additional data to reduce latency in TCP
ACK packets.
-Xms512m -Xmx768m
Default –Xms256m –Xmx512m. Increase to above value to handle higher call rate (more than 10
cps)
If you have an AE job that runs longer than 10 sec, PSAE is equivalent to PSAESRV. PSAE has
the added advantage of being recycled at the end of each AE job, cleaning up any outstanding SQL
cursors to the database that may have been left behind. Since PSAE recycles after each use, it does
not has any possible memory leakage problem that may occupied the precious system memory. In
short, PSAE is a cleaner workhorse.
To shutdown PSAESRV, when you configure the Process Scheduler, you can change the default of
the PSAESRV instance to 0.
Change Record
Date Description
07/01/2002 Created document for PeopleTools 8.4 (inherited from 8.1 Performance
RedPaper, version 8)
12/20/2002 Revised.
Oracle Corporation
Author and Date
Samuel To, Michael Turner, Michael Simons, Ravi Selvaraj, Indraneel Ganguli, February 2006
Copyright Information
Copyright © 2004, 2005, 2006 Oracle. All rights reserved.
Disclaimer
This document in any form, software or printed matter, contains proprietary information that is the
exclusive property of Oracle. Your access to and use of this confidential material is subject to the
terms and conditions of your Oracle Software License and Service Agreement, which has been
executed and with which you agree to comply. This document and information contained herein
may not be disclosed, copied, reproduced or distributed to anyone outside Oracle without prior
written consent of Oracle. This document is not part of your license agreement nor can it be
incorporated into any contractual agreement with Oracle or its subsidiaries or affiliates.
This document is for informational purposes only and is intended solely to assist you in planning
for the implementation and upgrade of the product features described. It is not a commitment to
deliver any material, code, or functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or functionality described in this
document remains at the sole discretion of Oracle.
Due to the nature of the product architecture, it may not be possible to safely include all features
described in this document without risking significant destabilization of the code.
Trademark Information
Oracle, JD Edwards, and PeopleSoft are registered trademarks of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective owners.