Best TFA practices
- To cleanup TFA BDB you need to uninstall/install TFA
- Always use at least 4h for diag collection period ( 4h is default )
- Check TFA repository for free space
- Add your OSWatcher files by running : tfactl directory add /home/OSW/oswbb/archive
- Always provide graphical output for OSWatcher data for each node
- Sample: # java -jar oswbba.jar -i ./new_archive -P grac1 -6 -7 -8 -B Oct 23 10:00:00 2013 -E Oct 23 12:00:00 2013
- Check that zip file for all nodes are created – rerun trace collection command if not
# ls /u01/app/grid/tfa/repository/collection_Wed_Oct_23_12_26_11_CEST_2013_node_all/*.zip
/u01/app/grid/tfa/repository/collection_Wed_Oct_23_12_26_11_CEST_2013_node_all/grac1.tfa_Wed_Oct_23_12_26_11_CEST_2013.zip
/u01/app/grid/tfa/repository/collection_Wed_Oct_23_12_26_11_CEST_2013_node_all/grac2.tfa_Wed_Oct_23_12_26_11_CEST_2013.zip
/u01/app/grid/tfa/repository/collection_Wed_Oct_23_12_26_11_CEST_2013_node_all/grac3.tfa_Wed_Oct_23_12_26_11_CEST_2013.zip
- Check above zip files are holding major trace files ( rdbms alert.log / CRS alert.log / ocssd.log )
# unzip -l grac1.tfa_Wed_Oct_23_12_26_11_CEST_2013.zip | egrep 'alert|ocssd.log'
2019952 10-23-2013 12:26 grac1//u01/app/11203/grid/log/grac1/cssd/ocssd.log
10802 10-23-2013 12:27 grac1/rdbms/grace2/grac21/trace/alert_grac21.log
353 10-23-2013 12:27 grac1//u01/app/11203/grid/log/grac1/alertgrac1.log
Reinstall TFA ( > 11.2.0.4 ) on a specific node
Check status
[root@grac42 ~]# $GRID_HOME/bin/tfactl print status
.----------------------------------------------------------------------------------------------.
| Host | Status of TFA | PID | Port | Version | Build ID | Inventory Status |
+--------+---------------+-------+------+------------+----------------------+------------------+
| grac42 | RUNNING | 1015 | 5000 | 12.1.2.0.0 | 12120020140619094932 | COMPLETE |
| grac41 | RUNNING | 31453 | 5000 | 12.1.2.0.0 | 12120020140619094932 | COMPLETE |
'--------+---------------+-------+------+------------+----------------------+------------------'
Uninstal TFA on grac42
[root@grac42 ~]# $GRID_HOME/bin/tfactl uninstall
TFA will be Uninstalled on Node grac42:
Removing TFA from grac42 only
Please remove TFA locally on any other configured nodes
.
[root@grac41 Desktop]# $GRID_HOME/bin/tfactl print status
.----------------------------------------------------------------------------------------------.
| Host | Status of TFA | PID | Port | Version | Build ID | Inventory Status |
+--------+---------------+-------+------+------------+----------------------+------------------+
| grac41 | RUNNING | 31453 | 5000 | 12.1.2.0.0 | 12120020140619094932 | COMPLETE |
'--------+---------------+-------+------+------------+----------------------+------------------'
--> TFA removed on node grac42
Re-install TFA
[root@grac42 ~]# $GRID_HOME/crs/install/tfa_setup.sh -silent -crshome /u01/app/11204/grid
Starting TFA installation
Using JAVA_HOME : /u01/app/11204/grid/jdk/jre
..
Installing TFA on grac42:
HOST: grac42 TFA_HOME: /u01/app/11204/grid/tfa/grac42/tfa_home
.--------------------------------------------------------------------------.
| Host | Status of TFA | PID | Port | Version | Build ID |
+--------+---------------+------+------+------------+----------------------+
| grac42 | RUNNING | 8457 | 5000 | 12.1.2.0.0 | 12120020140619094932 |
'--------+---------------+------+------+------------+----------------------'
Verify TFA status
root@grac42 ~]# $GRID_HOME/bin/tfactl print status
.----------------------------------------------------------------------------------------------.
| Host | Status of TFA | PID | Port | Version | Build ID | Inventory Status |
+--------+---------------+-------+------+------------+----------------------+------------------+
| grac42 | RUNNING | 8457 | 5000 | 12.1.2.0.0 | 12120020140619094932 | COMPLETE |
| grac41 | RUNNING | 31453 | 5000 | 12.1.2.0.0 | 12120020140619094932 | COMPLETE |
'--------+---------------+-------+------+------------+----------------------+------------------'
--> TFA reinstalled on node grac42
Installation of TFA Collector on top of 11.2.0.3
# ./installTFALite.sh
Starting TFA installation
Enter a location for installing TFA (/tfa will be appended if not supplied) [/home/oracle/RAC/TFA]:
/u01/app/grid
Enter a Java Home that contains Java 1.5 or later :
/usr/lib/jvm/jre-1.6.0-openjdk.x86_64
Would you like to do a [L]ocal only or [C]lusterwide installation ? [L|l|C|c] [C] : C
TFA Will be Installed on the Following Nodes:
++++++++++++++++++++++++++++++++++++++++++++
Install Nodes
=============
grac1
grac2
grac3
Do you wish to make changes to the Node List ? [Y/y/N/n] [N] N
Installing TFA on grac1
HOST: grac1 TFA_HOME: /u01/app/grid/tfa/grac1/tfa_home
Installing TFA on grac2
HOST: grac2 TFA_HOME: /u01/app/grid/tfa/grac2/tfa_home
Installing TFA on grac3
HOST: grac3 TFA_HOME: /u01/app/grid/tfa/grac3/tfa_home
Host grac2 is part of TFA cluster
Host grac3 is part of TFA cluster
.-----------------------------------------------.
| Host | Status of TFA | PID | Port | Version |
+-------+---------------+------+------+---------+
| grac1 | RUNNING | 9241 | 5000 | 2.5.1.5 |
| grac2 | RUNNING | 6462 | 5000 | 2.5.1.5 |
| grac3 | RUNNING | 6555 | 5000 | 2.5.1.5 |
'-------+---------------+------+------+---------'
Summary of TFA Installation:
.--------------------------------------------------------.
| grac1 |
+---------------------+----------------------------------+
| Parameter | Value |
+---------------------+----------------------------------+
| Install location | /u01/app/grid/tfa/grac1/tfa_home |
| Repository location | /u01/app/grid/tfa/repository |
| Repository usage | 0 MB out of 2629 MB |
'---------------------+----------------------------------'
.--------------------------------------------------------.
| grac2 |
+---------------------+----------------------------------+
| Parameter | Value |
+---------------------+----------------------------------+
| Install location | /u01/app/grid/tfa/grac2/tfa_home |
| Repository location | /u01/app/grid/tfa/repository |
| Repository usage | 0 MB out of 2629 MB |
'---------------------+----------------------------------'
.--------------------------------------------------------.
| grac3 |
+---------------------+----------------------------------+
| Parameter | Value |
+---------------------+----------------------------------+
| Install location | /u01/app/grid/tfa/grac3/tfa_home |
| Repository location | /u01/app/grid/tfa/repository |
| Repository usage | 0 MB out of 2629 MB |
'---------------------+----------------------------------'
Removed ssh equivalency setup on grac3
Removed ssh equivalency setup on grac2
TFA is successfully installed..
Start , stop and shutdown TFA
Stop TFA
# /etc/init.d/init.tfa stop
Stopping TFA
TFA is running - Will wait 5 seconds (up to 3 times)
TFA is running - Will wait 5 seconds (up to 3 times)
TFA is running - Will wait 5 seconds (up to 3 times)
TFAmain Force Stopped Successfully
. . .
Successfully stopped TFA..
Start TFA
# /etc/init.d/init.tfa start
Starting TFA..
start: Job is already running: oracle-tfa
Waiting up to 100 seconds for TFA to be started..
. . . . .
.
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands
Other useful commands
Restart TFA
# /etc/init.d/init.tfa restart
Stop TFAMain process and removes related inittab entries
# /etc/init.d/init.tfa shutdown
Verify TFA status
Verify TFA runtime status
# /u01/app/grid/tfa/bin/tfactl print status
.-----------------------------------------------.
| Host | Status of TFA | PID | Port | Version |
+-------+---------------+------+------+---------+
| grac1 | RUNNING | 9241 | 5000 | 2.5.1.5 |
| grac2 | RUNNING | 6462 | 5000 | 2.5.1.5 |
| grac3 | RUNNING | 6555 | 5000 | 2.5.1.5 |
'-------+---------------+------+------+---------'
Show current TFA configuration
# /u01/app/grid/tfa/bin/tfactl print config
.---------------------------------------------------.
| Configuration Parameter | Value |
+-----------------------------------------+---------+
| TFA version | 2.5.1.5 |
| Automatic diagnostic collection | OFF |
| Trimming of files during diagcollection | ON |
| Repository current size (MB) in grac1 | 6 |
| Repository maximum size (MB) in grac1 | 2629 |
| Trace level | 1 |
'-----------------------------------------+---------'
Use tfactl to check for out standing actions ( for example during diacollect )
# $GRID_HOME/tfa/bin/tfactl print actions
.------------------------------------------------------------------------.
| HOST | TIME | ACTION | STATUS | COMMENTS |
+--------+--------------+----------------+---------+---------------------+
| grac42 | Mar 04 08:56 | Collect traces | RUNNING | Collection details: |
| | | & zip | | |
| | | | | Zip file: |
| | | | | tfa_Tue_Mar_4_08_56 |
| | | | | _25_CET_2014.zip |
| | | | | Tag: |
| | | | | collection_Tue_Mar_ |
| | | | | 4_08_56_25_CET_2014 |
| | | | | _node_all |
+--------+--------------+----------------+---------+---------------------+
| grac42 | Mar 04 08:56 | Run inventory | RUNNING | RDBMS |
| | | | | all:ASM:CRS:DBWLM:A |
| | | | | CFS:CRS:ASM:OS:INST |
| | | | | ALL:TNS:CHMOS |
+--------+--------------+----------------+---------+---------------------+
| grac41 | Mar 04 08:56 | Run inventory | RUNNING | -c:RDBMS |
| | | | | all:ASM:CRS:DBWLM:A |
| | | | | CFS:CRS:ASM:OS:INST |
| | | | | ALL:TNS:CHMOS |
+--------+--------------+----------------+---------+---------------------+
| grac41 | Mar 04 08:56 | Collect traces | RUNNING | Collection details: |
| | | & zip | | |
| | | | | Zip file: |
| | | | | tfa_Tue_Mar_4_08_56 |
| | | | | _25_CET_2014.zip |
| | | | | Tag: |
| | | | | collection_Tue_Mar_ |
| | | | | 4_08_56_25_CET_2014 |
| | | | | _node_all |
+--------+--------------+----------------+---------+---------------------+
| grac43 | Mar 04 08:56 | Collect traces | RUNNING | Collection details: |
| | | & zip | | |
| | | | | Zip file: |
| | | | | tfa_Tue_Mar_4_08_56 |
| | | | | _25_CET_2014.zip |
| | | | | Tag: |
| | | | | collection_Tue_Mar_ |
| | | | | 4_08_56_25_CET_2014 |
| | | | | _node_all |
+--------+--------------+----------------+---------+---------------------+
| grac43 | Mar 04 08:56 | Run inventory | RUNNING | RDBMS |
| | | | | all:ASM:CRS:DBWLM:A |
| | | | | CFS:CRS:ASM:OS:INST |
| | | | | ALL:TNS:CHMOS |
'--------+--------------+----------------+---------+---------------------'
Use tfactl to check for Errors and Startups
Check for errors:
# /u01/app/grid/tfa/bin/tfactl print errors
++++++ Error Start +++++
Event Id : GRACE232lppjmk62mrsu7si7ltbau98j
File Name : /u01/app/oracle/diag/rdbms/grace2/GRACE21/trace/alert_GRACE21.log
Error Code : ORA-1109
Error Description : ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
Error Time : Mon Jul 22 12:37:06 CEST 2013
Startup Time : Mon Jul 22 12:36:39 CEST 2013
Trace File Name : NONE
++++++ Error End +++++
..
Check for database startups:
# /u01/app/grid/tfa/bin/tfactl print startups
++++++ Startup Start +++++
Event Id : GRACE27rpp25v1c83sago9ohf1s6gn8s
File Name : /u01/app/oracle/diag/rdbms/grace2/GRACE21/trace/alert_GRACE21.log
Startup Time : Mon Jul 22 12:32:34 CEST 2013
Dummy : FALSE
++++++ Startup End +++++
++++++ Startup Start +++++
Event Id : GRACE2nahm9evlmrtssl7etl36u7n25f
File Name : /u01/app/oracle/diag/rdbms/grace2/GRACE21/trace/alert_GRACE21.log
Startup Time : Mon Jul 22 12:36:39 CEST 2013
Dummy : FALSE
++++++ Startup End +++++
Collect tracefiles using TFA
# /u01/app/grid/tfa/bin/tfactl diagcollect -all -since 1h
Collecting data for all components using above parameters...
Running an inventory clusterwide ...
Run inventory completed locally ...
Collection name tfa_Thu_Sep_26_11_58_47_CEST_2013.zip
Sending diagcollect request to host : grac2
Sending diagcollect request to host : grac3
Getting list of files satisfying time range [Thu Sep 26 10:59:30 CEST 2013, Thu Sep 26 11:59:30 CEST 2013]
grac1: Zipping File: /u01/app/oracle/diag/rdbms/grace2/GRACE2_1/trace/GRACE2_1_dbrm_4375.trc
...
grac1: Zipping File: /u01/app/oracle/diag/rdbms/race2/RACE21/trace/alert_RACE21.log
Trimming file : /u01/app/oracle/diag/rdbms/race2/RACE21/trace/alert_RACE21.log with original file size : 255kB
Collecting extra files...
Total Number of Files checked : 2181
Total Size of all Files Checked : 1.7GB
Number of files containing required range : 41
Total Size of Files containing required range : 106MB
Number of files trimmed : 16
Total Size of data prior to zip : 43MB
Saved 77MB by trimming files
Zip file size : 1.9MB
Total time taken : 101s
Completed collection of zip files.
Logs are collected to:
/u01/app/grid/tfa/repository/collection_Thu_Sep_26_11_58_47_CEST_2013_node_all/grac1.tfa_Thu_Sep_26_11_58_47_CEST_2013.zip
/u01/app/grid/tfa/repository/collection_Thu_Sep_26_11_58_47_CEST_2013_node_all/grac2.tfa_Thu_Sep_26_11_58_47_CEST_2013.zip
/u01/app/grid/tfa/repository/collection_Thu_Sep_26_11_58_47_CEST_2013_node_all/grac3.tfa_Thu_Sep_26_11_58_47_CEST_2013.zip
Collect trace by a specific time
# /u01/app/grid/tfa/bin/tfactl diagcollect -all -from "Oct/18/2013 00:00:00" -to "Oct/18/2013 06:00:00"
Collecting data for all components using above parameters...
Scanning files from Oct/18/2013 00:00:00 to Oct/18/2013 06:00:00
Running an inventory clusterwide ...
Collecting diagnostic data for a specific day
# $GRID_HOME/bin/tfactl diagcollect -all -for "Mar/22/2014"
Collecting data for all components using above parameters...
Collecting data for all nodes
Scanning files for Mar/22/2014
Repository Location in grac41 : /u01/app/grid/tfa/repository
2014/03/22 14:03:35 CET : Running an inventory clusterwide ...
2014/03/22 14:03:36 CET : Collection Name : tfa_Sat_Mar_22_14_03_29_CET_2014.zip
2014/03/22 14:03:43 CET : Sending diagcollect request to host : grac42
2014/03/22 14:03:43 CET : Sending diagcollect request to host : grac43
....
Logs are collected to:
/u01/app/grid/tfa/repository/collection_Sat_Mar_22_14_03_29_CET_2014_node_all/grac41.tfa_Sat_Mar_22_14_03_29_CET_2014.zip
/u01/app/grid/tfa/repository/collection_Sat_Mar_22_14_03_29_CET_2014_node_all/grac42.tfa_Sat_Mar_22_14_03_29_CET_2014.zip
/u01/app/grid/tfa/repository/collection_Sat_Mar_22_14_03_29_CET_2014_node_all/grac43.tfa_Sat_Mar_22_14_03_29_CET_2014.zip
Add and remove directory
- TFA directories may change due to switching from adming managed to policy managed
- You run add database / add instance
- Keep your TFA repository in sync with any changes
Remove directories:
# /u01/app/grid/tfa/bin/tfactl directory remove /u01/app/oracle/diag/rdbms/grace2/GRACE2_1/trace
# /u01/app/grid/tfa/bin/tfactl directory remove /u01/app/oracle/diag/rdbms/grace2/GRACE21/trace
Add a directory:
# /u01/app/grid/tfa/bin/tfactl directory add /u01/app/oracle/diag/rdbms/grace2/grac21/trace
Add OSWatcher archive for TFA processing
Check TFA archive location:
$ ps -ef | grep OSWatcher | grep -v grep
root 11018 1 0 14:51 ? 00:00:00 /bin/sh ./OSWatcher.sh 10 60 gzip /home/OSW/oswbb/archive
root 12028 11018 0 14:51 ? 00:00:00 /bin/sh ./OSWatcherFM.sh 60 /home/OSW/oswbb/archive
--> /home/OSW/oswbb/archive is OSWatcher tracefile location
Run now on each instance:
# /u01/app/grid/tfa/bin/tfactl directory add /home/OSW/oswbb/archive
Add CHM directory
- Note you should not need to add CHM directory as this directory should be already availabe
- Check with $GRID_HOME/tfa/bin/tfactl print directories | grep crf
First check whether a crf directory is already available
$ $GRID_HOME/tfa/bin/tfactl print directories | grep crf
| /u01/app/11204/grid/log/grac41/crf | CRS | public | root |
If your get above output - stop here as the directory is already listed
Find CHM repository directory
Note : ologgerd is only running on a single node - Read following link to find this node.
$ ps -ef | grep ologgerd | grep -v grep
root 16996 1 0 01:06 ? 00:04:19 /u01/app/11203/grid/bin/ologgerd -m grac2 -r -d /u01/app/11203/grid/crf/db/grac1
Add directory to TFA
# /u01/app/grid/tfa//bin/tfactl directory add /u01/app/11203/grid/crf/db/grac1
Failed to add directory to TFA. Unable to determine parameters for directory: /u01/app/11203/grid/crf/db/grac1
Please enter component for this Directory [RDBMS|CRS|ASM|INSTALL|OS|CFGTOOLS] : OS
Running Inventory ...
.----------------------------------------------------------------------------------------------------------------------------.
| Trace Directory | Component | Permission | Added By |
+-------------------------------------------------------------------+--------------------------------+------------+----------+
| /etc/oracle | CRS | public | root |
+-------------------------------------------------------------------+--------------------------------+------------+----------+
| /home/OSW/oswbb/archive | OS | public | root |
+-------------------------------------------------------------------+--------------------------------+------------+----------+
| /u01/app/11203/grid/cfgtoollogs/opatch | INSTALL | public | root |
+-------------------------------------------------------------------+--------------------------------+------------+----------+
| /u01/app/11203/grid/crf/db/grac1 | OS | public | root |
+-------------------------------------------------------------------+--------------------------------+------------+----------+
Verify TFA repository space
# /u01/app/grid/tfa/bin/tfactl print repository
.-----------------------------------------------------.
| grac1 |
+----------------------+------------------------------+
| Repository Parameter | Value |
+----------------------+------------------------------+
| Location | /u01/app/grid/tfa/repository |
| Maximum Size (MB) | 2092 |
| Current Size (MB) | 156 |
| Status | OPEN |
'----------------------+------------------------------'
.-----------------------------------------------------.
| grac3 |
+----------------------+------------------------------+
| Repository Parameter | Value |
+----------------------+------------------------------+
| Location | /u01/app/grid/tfa/repository |
| Maximum Size (MB) | 2273 |
| Current Size (MB) | 67 |
| Status | OPEN |
'----------------------+------------------------------'
.-----------------------------------------------------.
| grac2 |
+----------------------+------------------------------+
| Repository Parameter | Value |
+----------------------+------------------------------+
| Location | /u01/app/grid/tfa/repository |
| Maximum Size (MB) | 1981 |
| Current Size (MB) | 60 |
| Status | OPEN |
'----------------------+------------------------------'
Setting TFA tracelevel and review trace file
# /u01/app/grid/tfa/bin/tfactl set tracelevel=4 -c
Running on Host : grac1
Running Check through Java CLI
Opening Port file /u01/app/grid/tfa/grac1/tfa_home/internal/port.txtOpening Parameter file /u01/app/grid/tfa/grac1/tfa_home/tfa_setup.txtWe got : CheckOK
...
'-----------------------------------------+---------'
| Configuration Parameter | Value |
+-----------------------------------------+---------+
| TFA version | 2.5.1.5 |
| Automatic diagnostic collection | OFF |
| Trimming of files during diagcollection | ON |
| Repository current size (MB) in grac1 | 131 |
| Repository maximum size (MB) in grac1 | 2629 |
| Trace level | 4 |
'-----------------------------------------+---------'
#### Done ####
Tracefile name and location
# pwd
/u01/app/grid/tfa/grac3/tfa_home/log
# ls -rlt
-rw-r--r-- 1 root root 3641 Oct 23 10:26 syserrorout
-rw-r--r-- 1 root root 134679 Oct 23 10:28 diagcollect.log
-rw-r--r-- 1 root root 25108979 Oct 23 10:32 tfa.10.22.2013-19.54.37.log
With level 4 tracing we see following message for a tracefile added ZIP file
10.23.2013-10.26.30 -- MESSAGE : ZipTracesForDatesThread Transferring from /u01/app/oracle/diag/rdbms/grace2/grac23/trace/alert_grac23.log to /u01/app/grid/tfa/repository/temp_1382516788278/collections/alert_grac23.log
Uninstall TFA
# cd /u01/app/grid/tfa/grac1/tfa_home/bin
# ./uninstalltfa.sh
Stopping TFA in grac1...
Shutting down TFA
oracle-tfa stop/waiting
Killing TFA running with pid 15860
Successfully shutdown TFA..
Stopping TFA in grac2 and removing /u01/app/grid/tfa/grac2/tfa_home...
Removing TFA from grac2...
Stopping TFA in grac2...
Shutting down TFA
oracle-tfa stop/waiting
Killing TFA running with pid 3251
Successfully shutdown TFA..
Deleting TFA support files on grac2:
Removing /etc/init.d/init.tfa...
Removing /u01/app/grid/tfa/bin...
Removing /u01/app/grid/tfa/grac2...
Stopping TFA in grac3 and removing /u01/app/grid/tfa/grac3/tfa_home...
Removing TFA from grac3...
Stopping TFA in grac3...
Shutting down TFA
oracle-tfa stop/waiting
Killing TFA running with pid 1615
Successfully shutdown TFA..
Deleting TFA support files on grac3:
Removing /etc/init.d/init.tfa...
Removing /u01/app/grid/tfa/bin...
Removing /u01/app/grid/tfa/grac3...
Deleting TFA support files on grac1:
Removing /etc/init.d/init.tfa...
Removing /u01/app/grid/tfa/bin...
Removing /u01/app/grid/tfa/grac1...
Upgrade TFA 2.5 to 3.1
- If you install SupportBundle_v1_3 you will get TFA 3.1
- 11.2.0.4 has installed TFA 2.5.1.5 per default
- Reinstall TFA 11.2.0.4 in a 11.2.0.4 GRID_HOME after a downgrade
/u01/app/grid/11.2.0.4/crs/install/tfa_setup.sh -silent -crshome /u01/app/grid/11.2.0.4
Extract zip file from SupportBundle_v1_3 and run
# ./installTFALite
Starting TFA installation
TFA is already installed. Patching /u01/app/11204/grid/tfa/grac41/tfa_home...
TFA will be Patched on:
grac41
grac42
grac43
Do you want to continue with patching TFA? [Y|N] [Y]:
Checking for ssh equivalency in grac42
grac42 is configured for ssh user equivalency for root user
Checking for ssh equivalency in grac43
grac43 is configured for ssh user equivalency for root user
Auto patching is enabled in grac41
Key Stores are already updated in grac42
Key Stores are already updated in grac43
Shutting down TFA for Patching...
Shutting down TFA
oracle-tfa stop/waiting
. . . . .
Killing TFA running with pid 18946
. . .
Successfully shutdown TFA..
Renaming /u01/app/11204/grid/tfa/grac41/tfa_home/jar to /u01/app/11204/grid/tfa/grac41/tfa_home/jlib
Adding INSTALL_TYPE = GI to tfa_setup.txt
Copying /u01/app/11204/grid/tfa/grac41/tfa_home/output/ to /u01/app/grid/tfa/grac41/
The current version of Berkeley DB is 4.0.103
Copying je-4.1.27.jar to /u01/app/11204/grid/tfa/grac41/tfa_home/jlib/
Copying je-5.0.84.jar to /u01/app/11204/grid/tfa/grac41/tfa_home/jlib/
Running DbPreUpgrade_4_1 utility
Output of upgrade : Pre-upgrade succeeded
Creating ZIP: /u01/app/11204/grid/tfa/grac41/tfa_home/internal/tfapatch.zip
Running commands to fix init.tfa and tfactl in localhost
Starting TFA in grac41...
Starting TFA..
oracle-tfa start/running, process 9222
Waiting up to 100 seconds for TFA to be started..
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands
Removing /u01/app/11204/grid/tfa/grac41/tfa_home/jlib/je-4.0.103.jar
Enabling Access for Non-root Users on grac41...
Adding default users and groups to TFA Access list...
Using SSH to patch TFA to remote nodes:
Applying Patch on grac42:
...
Applying Patch on grac43:
..
When running tfactl diagcollect like you will get following Inventory Status message
# $GRID_HOME/tfa/bin/tfactl diagcollect -all -since 1h
# $GRID_HOME/tfa/bin/tfactl print status
.-----------------------------------------------------------------------------------------.
| Host | Status of TFA | PID | Port | Version | Build ID | Inventory Status |
+--------+---------------+-------+------+---------+--------------------+------------------+
| grac42 | RUNNING | 3951 | 5000 | 3.1 | 310020140205043544 | RUNNING |
| grac41 | RUNNING | 4201 | 5000 | 3.1 | 310020140205043544 | RUNNING |
| grac43 | RUNNING | 22284 | 5000 | 3.1 | 310020140205043544 | RUNNING |
'--------+---------------+-------+------+---------+--------------------+------------------'
- Note it will take some time until Inventory status goes to completed
- You can monitory outstanding actions by running: # $GRID_HOME/tfa/bin/tfactl print actions
Check TFA for integration of CHM and OSWatcher Output
Script check_tools.sh
#!/bin/sh
#
# OSWatcher location: /u01/app/11204/grid/oswbb -> search string oswbb
# CHM location: /u01/app/11204/grid/crf -> search string crf
#
OSWatcher_loc_string="oswbb"
CHM_loc_string="crf"
echo "-> OSWatcher status grac41:"
sudo ssh grac41 "ps -elf | grep OSWatcher | grep -v grep"
echo "-> OSWatcher status grac42:"
ssh grac42 "ps -elf | grep OSWatcher | grep -v grep"
echo "-> OSWatcher status grac43:"
ssh grac43 "ps -elf | grep OSWatcher | grep -v grep"
echo "-> CHM Mater/Replica info"
ssh grac41 "$GRID_HOME/bin/oclumon manage -get MASTER REPLICA"
echo "-> CHM state "
ssh grac41 "$GRID_HOME/bin/crsctl status res ora.crf -init | grep STATE"
ssh grac42 "$GRID_HOME/bin/crsctl status res ora.crf -init | grep STATE"
ssh grac43 "$GRID_HOME/bin/crsctl status res ora.crf -init | grep STATE"
echo "-> TFA print status and open actions "
ssh grac41 $GRID_HOME/tfa/bin/tfactl print status
ssh grac41 $GRID_HOME/tfa/bin/tfactl print actions
echo "-> TFA directories for OSWatcher/CHM grac41"
echo "-> TFA directory search string: $OSWatcher_loc_string|$CHM_loc_string"
ssh grac41 $GRID_HOME/tfa/bin/tfactl print directories | egrep "$OSWatcher_loc_string|$CHM_loc_string"
echo "-> TFA directories for OSWatcher/CHM grac42"
ssh grac42 $GRID_HOME/tfa/bin/tfactl print directories | egrep "$OSWatcher_loc_string|$CHM_loc_string"
echo "-> TFA directories for OSWatcher/CHM grac43"
ssh grac43 $GRID_HOME/tfa/bin/tfactl print directories | egrep "$OSWatcher_loc_string|$CHM_loc_string
Output from check_tools.sh script running on a 11.2.0.4 RAC system
[grid@grac41 ~]$ ./check_tools.sh
-> OSWatcher status grac41:
[sudo] password for grid:
0 S root 15448 1 0 80 0 - 26670 wait 13:36 ? 00:00:00 /bin/sh ./OSWatcher.sh 48 10 gzip /u01/app/11204/grid/oswbb/archive
0 S root 16274 15448 0 80 0 - 26535 wait 13:36 ? 00:00:00 /bin/sh ./OSWatcherFM.sh 10 /u01/app/11204/grid/oswbb/archive
-> OSWatcher status grac42:
0 S root 15230 1 0 80 0 - 26669 wait 13:38 ? 00:00:00 /bin/sh ./OSWatcher.sh 48 10 gzip /u01/app/11204/grid/oswbb/archive
0 S root 15970 15230 0 80 0 - 26535 wait 13:38 ? 00:00:00 /bin/sh ./OSWatcherFM.sh 10 /u01/app/11204/grid/oswbb/archive
-> OSWatcher status grac43:
0 S root 21210 1 0 80 0 - 26668 wait 13:23 ? 00:00:01 /bin/sh ./OSWatcher.sh 48 10 gzip /u01/app/11204/grid/oswbb/archive
0 S root 21860 21210 0 80 0 - 26535 wait 13:24 ? 00:00:00 /bin/sh ./OSWatcherFM.sh 10 /u01/app/11204/grid/oswbb/archive
-> CHM Mater/Replica info
Master = grac43
Replica = grac42
Done
-> CHM state
STATE=ONLINE on grac41
STATE=ONLINE on grac42
STATE=ONLINE on grac43
-> TFA print status and open actions
.-----------------------------------------------------------------------------------------.
| Host | Status of TFA | PID | Port | Version | Build ID | Inventory Status |
+--------+---------------+-------+------+---------+--------------------+------------------+
| grac41 | RUNNING | 4201 | 5000 | 3.1 | 310020140205043544 | COMPLETE |
| grac42 | RUNNING | 3951 | 5000 | 3.1 | 310020140205043544 | COMPLETE |
| grac43 | RUNNING | 22284 | 5000 | 3.1 | 310020140205043544 | COMPLETE |
'--------+---------------+-------+------+---------+--------------------+------------------'
.------------------------------------------.
| HOST | TIME | ACTION | STATUS | COMMENTS |
+------+------+--------+--------+----------+
'------+------+--------+--------+----------'
-> TFA directories for OSWatcher/CHM grac41
-> TFA directory search string: oswbb|crf
| /u01/app/11204/grid/log/grac41/crf | CRS | public | root |
| /u01/app/11204/grid/log/grac41/crf | CRS | public | root |
| /u01/app/11204/grid/oswbb/archive | OS | private | root |
-> TFA directories for OSWatcher/CHM grac42
| /u01/app/11204/grid/log/grac42/crf | CRS | public | root |
| /u01/app/11204/grid/log/grac42/crf | CRS | public | root |
| /u01/app/11204/grid/oswbb/archive | OS | private | root |
-> TFA directories for OSWatcher/CHM grac43
| /u01/app/11204/grid/log/grac43/crf | CRS | public | root |
| /u01/app/11204/grid/log/grac43/crf | CRS | public | root |
| /u01/app/11204/grid/oswbb/archive | OS | private | root |