Table of Contents
Debug Clusterware startup problems with DTRACE
- Download DTRACE Init script : check_rac.d
Why to use DTRACE ?
- Using DTRACE scripts can save you many hours of debugging
- DTRACE scritps can be used to capture intermittent errors
- DTRACE can easily configured to track very specific things like: [ How many bytes where written by the 5.th I/O for file XXXX after it gets newly created ]
- DTRACE can provide very detail information about system call usage and timing
- DTRACE is the next level of debugging on Linux much better than strace
- Note : Not all reported errors by DTRACE are significant and prevents CW from booting
- Best practise is to test the start on a working node to figure out what is relevant or not
If you change a lot of things for testing and if you have many RAC instances running
where lot of people are testing DTRACE will be become you best friend !
Version used and OS error codes handled by DTRACE script
- OEL 6.6 with Linux hract21.example.com 3.8.13-55.1.2.el6uek.x86_64
- Oracle 12.1.0.2
Installed Dtrace packages: [root@hract21 Desktop]# rpm -qa | grep dtrace dtrace-utils-0.4.5-2.el6.x86_64 dtrace-utils-devel-0.4.5-2.el6.x86_64 libdtrace-ctf-0.4.1-1.x86_64 dtrace-modules-3.8.13-44.1.1.el6uek-0.4.3-4.el6.x86_64 libdtrace-ctf-devel-0.4.1-1.x86_64 dtrace-modules-headers-0.4.3-4.el6.x86_64 dtrace-modules-3.8.13-55.1.2.el6uek-0.4.3-4.el6.x86_64 Error Codes tracked by dtrace script: /usr/include/asm-generic/errno-base.h #define ENOENT 2 /* No such file or directory */ #define ENXIO 6 /* No such device or address */ #define EAGAIN 11 /* Try again */ #define EACCES 13 /* Permission denied */ /usr/include/asm-generic/errno.h #define EADDRINUSE 98 /* Address already in use */ #define EADDRNOTAVAIL 99 /* Cannot assign requested address */ #define ECONNREFUSED 111 /* Connection refused */ #define EHOSTUNREACH 113 /* No route to host */ #define EINPROGRESS 115 /* Operation now in progress */
Many thx
This is very helpful
I really like looking through an article that can make people think.
Also, thank you for permitting me to comment!