Error : CRS-4602: Failed 27 to add voting file 9e50b0768b874fc7bf173cd55ce106a3
# $GRID_HOME/bin/crsctl replace votedisk +OCR
CRS-4602: Failed 27 to add voting file 9e50b0768b874fc7bf173cd55ce106a3.
CRS-4602: Failed 27 to add voting file 285788c4af634f99bfdd9edbf30dacc0.
CRS-4602: Failed 27 to add voting file 55ace1807eca4f0dbf0545faa57122ac.
Failure 27 with Cluster Synchronization Services while deleting voting disk.
Failure 27 with Cluster Synchronization Services while deleting voting disk.
Failure 27 with Cluster Synchronization Services while deleting voting disk.
Failed to replace voting disk group with +OCR.
CRS-4000: Command Replace failed, or completed with errors.
Reading Traces:
alertgrac41.log
[cssd(3456)]CRS-1650:Configuration change 1381848175:0 rejected by node 2 grac42; details at (:CSSNM000066:)
in /u01/app/11204/grid/log/grac41/cssd/ocssd.log
2013-10-15 16:42:56.068:
[cssd(3456)]CRS-1604:CSSD voting file is offline: /dev/asmdisk6_udev_sdg1; details at (:CSSNM00069:)
in /u01/app/11204/grid/log/grac41/cssd/ocssd.log.
2013-10-15 16:42:56.079:
Note: Always check your asm_diskstring when getting CRS-4602 errors !
grac41/cssd/ocssd.log:
2013-10-15 15:00:00.191: [ CSSD][1646704384](:CSSNM00069:)clssnmvDeconfigureVFs: CSS voting file /dev/asmdisk6_udev_sdg1 is not usable.
2013-10-15 15:00:00.191: [ CSSD][1646704384]clssnmvDiskStateChange: state from pending to deconfigured disk /dev/asmdisk6_udev_sdg1
2013-10-15 15:00:00.191: [ CSSD][1646704384](:CSSNM00069:)clssnmvDeconfigureVFs: CSS voting file /dev/asmdisk7_udev_sdh1 is not usable.
2013-10-15 15:00:00.191: [ CSSD][1646704384]clssnmvDiskStateChange: state from pending to deconfigured disk /dev/asmdisk7_udev_sdh1
2013-10-15 15:00:00.191: [ CSSD][1646704384](:CSSNM00069:)clssnmvDeconfigureVFs: CSS voting file /dev/asmdisk5_udev_sdf1 is not usable.
2013-10-15 15:00:00.191: [ CSSD][1646704384]clssnmvDiskStateChange: state from pending to deconfigured disk /dev/asmdisk5_udev_sdf1
2013-10-15 15:00:00.191: [ CSSD][1646704384]clssnmCompleteGMReq: Completed request type 1 with status 6
2013-10-15 15:00:00.192: [ CSSD][1636685568]clssnmvVoteDiskValidation: Voting disk(/dev/asmdisk7_udev_sdh1) cluster GUID mismatch
2013-10-15 15:00:00.192: [ CSSD][1636685568]clssnmvWorkerThread: disk /dev/asmdisk7_udev_sdh1 corrupted
2013-10-15 15:00:00.192: [ CSSD][1636685568]clssnmvDiskAvailabilityChange: voting file /dev/asmdisk7_udev_sdh1 now offline
2013-10-15 15:00:00.192: [ CSSD][1641441024]clssnmvVoteDiskValidation: Voting disk(/dev/asmdisk6_udev_sdg1) cluster GUID mismatch
--> Here we know that node 2 has rejected the configurations change and it seems that /dev/asmdisk6_udev_sdg1, /dev/asmdisk7 and
/dev/asmdisk5_udev_sdf1 are not usable
Debug steps - Go to grac42 and check ASM status
$ $GRID_HOME/bin/asmcmd lsdsk
Connected to an idle instance. <-- No disks reported
$ $GRID_HOME/bin/asmcmd lsdg
Connected to an idle instance.
ASMCMD-8102: no connection to Oracle ASM; command requires Oracle ASM to run
$ ls /dev/asm*
/dev/asmdisk1_udev_sdb1 /dev/asmdisk2_udev_sdc1 /dev/asmdisk3_udev_sdd1 /dev/asmdisk4_udev_sde1
--> ASM instance not up and running - because diskgroup OCR and can not moumt disks dev/asmdisk5_udev_sdf1, ...
because OS has not discovered these disks
Fix:
Configure udev rules for disks /dev/asmdisk6_udev_sdg1, /dev/asmdisk7 and /dev/asmdisk5_udev_sdf1 on grac32
Pages: Page 1, Page 2, Page 3, Page 4
Hi, Nicely described, almost the same activity i intend to perform.
i have to have one confirmation:
will this all cause any downtime????
No these operations are not expected to cause any downtime.
But as as always in a RAC and high production env you should test
and verify any change on a test cluster first.