Table of Contents
OCR diskgroup content
- Storage for ASM SPFile
- Storage for Voting Disk
- Storage for OCR repository
Destroy all ASM disk headers from our OCR diskgroup
[root@grac41 Desktop]# dd if=/dev/zero of=/dev/asm_ocr_11204_2G_disk3 bs=1024 count=1024 1024+0 records in 1024+0 records out 1048576 bytes (1.0 MB) copied, 0.00483026 s, 217 MB/s [root@grac41 Desktop]# dd if=/dev/zero of=/dev/asm_ocr_11204_2G_disk1 bs=1024 count=1024 [root@grac41 Desktop]# dd if=/dev/zero of=/dev/asm_ocr_11204_2G_disk3 bs=1024 count=1024 CW status after restart [grid@grac41 ~]$ crsi NAME TARGET STATE SERVER STATE_DETAILS ------------------------- ---------- ---------- ------------ ------------------ ora.asm ONLINE OFFLINE ora.cluster_interconnect.haip ONLINE OFFLINE ora.crf ONLINE ONLINE grac41 ora.crsd ONLINE OFFLINE ora.cssd ONLINE OFFLINE STARTING ora.cssdmonitor ONLINE ONLINE grac41 ora.ctssd ONLINE OFFLINE ora.diskmon OFFLINE OFFLINE ora.drivers.acfs ONLINE OFFLINE ora.evmd ONLINE OFFLINE ora.gipcd ONLINE ONLINE grac41 ora.gpnpd ONLINE ONLINE grac41 ora.mdnsd ONLINE ONLINE grac41 --> Cssd doesn't start as none of our voting disk are accessible anymore ! CW alert log reports : [cssd(22327)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11204/grid/log/grac41/cssd/ocssd.log Stop CRS and start crs in exclusiv mode: # crsctl stop crs [-f] Verify that all CW processes were stopped [root@grac41 Desktop]# ps -elf | grep d.bin 0 S root 6934 28649 0 80 0 - 25826 pipe_w 11:37 pts/5 00:00:00 grep d.bin --> Note you may need to kill remaining CW processes Start CW stack in exclusive mode without CRS A new option '-nocrs' has been introduced with 11.2.0.2, which prevents the start of the ora.crsd resource. [root@grac41 Desktop]# $GRID_HOME/bin/crsctl start crs -excl -nocrs Connect to ASM instance an check ASM disk status [grid@grac41 ~]$ sqlplus / as sysasm SQL> select dg.name dg_name, dg.state dg_state, dg.type, d.DISK_NUMBER dsk_no, d.MOUNT_STATUS, d.HEADER_STATUS, d.MODE_STATUS, d.STATE, d. PATH, d.FAILGROUP FROM V$ASM_DISK d, v$asm_diskgroup dg where dg.group_number(+)=d.group_number order by dg_name, dsk_no; DG_NAME DG_STATE TYPE DSK_NO MOUNT_S HEADER_STATU MODE_ST STATE PATH FAILGROUP ---------- ---------- ------ ------- ------- ------------ ------- -------- ------------------------------ -------------- 1 CLOSED CANDIDATE ONLINE NORMAL /dev/asm_ocr_11204_2G_disk2 2 CLOSED CANDIDATE ONLINE NORMAL /dev/asm_ocr_11204_2G_disk1 3 CLOSED CANDIDATE ONLINE NORMAL /dev/asm_ocr_11204_2G_disk3 --> As we have erased the ASM disk header with d the header status should be shown as CANDIDATE status
Recreate diskgroup , restore backup, configure ASM SPFile and Voting disks
SQL> CREATE DISKGROUP OCR NORMAL REDUNDANCY 2 FAILGROUP OCR_000 DISK '/dev/asm_ocr_11204_2G_disk1' NAME disk1 3 FAILGROUP OCR_001 DISK '/dev/asm_ocr_11204_2G_disk2' NAME disk2 4 FAILGROUP OCR_002 DISK '/dev/asm_ocr_11204_2G_disk3' NAME disk3 5 ATTRIBUTE 'au_size'='4M', 6 'compatible.asm' = '11.2', 7 'compatible.rdbms' = '11.2'; Diskgroup created Verify OCR backup ( check on all nodes to get the most current backup ! ) [grid@grac41 ~]$ ocrconfig -showbackup auto PROT-26: Oracle Cluster Registry backup locations were retrieved from a local copy grac41 2014/07/01 04:48:16 /u01/app/11204/grid/cdata/grac4/backup00.ocr grac41 2014/07/01 00:48:12 /u01/app/11204/grid/cdata/grac4/backup01.ocr .. Restore the latest OCR backup AS the CRS disk group is created & mounted the OCR can be restored - must be done as the root user: [root@grac41 Desktop]# $GRID_HOME/bin/ocrconfig -restore /u01/app/11204/grid/cdata/grac4/backup00.ocr DG_NAME DG_STATE TYPE DSK_NO MOUNT_S HEADER_STATU MODE_ST STATE PATH FAILGROUP ---------- ---------- ------ ------- ------- ------------ ------- -------- ------------------------------ --------------- OCR MOUNTED NORMAL 0 CACHED MEMBER ONLINE NORMAL /dev/asm_ocr_11204_2G_disk1 OCR_000 OCR MOUNTED NORMAL 1 CACHED MEMBER ONLINE NORMAL /dev/asm_ocr_11204_2G_disk2 OCR_001 OCR MOUNTED NORMAL 2 CACHED MEMBER ONLINE NORMAL /dev/asm_ocr_11204_2G_disk3 OCR_002 --> Now Disk header status has changed to MEMBER Add voting disks [root@grac41 ~]# $GRID_HOME/bin/crsctl replace votedisk +OCR CRS-4602: Failed 27 to add voting file 112b412681a04fa1bfd99c0ed4dc991c. CRS-4602: Failed 27 to add voting file 0fd25dd59d954f80bfbd4e9f0431fb11. CRS-4602: Failed 27 to add voting file 36bf8226cc364faebf9b616999771bbc. Failed to replace voting disk group with +OCR. --> Need to set asm_diskstring SQL> alter system set asm_diskstring="/dev/asm*"; System altered. SQL> show parameter asm; NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ asm_diskgroups string OCR asm_diskstring string /dev/asm* [root@grac41 ~]# $GRID_HOME/bin/crsctl replace votedisk +OCR Successful addition of voting disk 7f78f5e770534f74bf4a00e542579bd1. Successful addition of voting disk 7f75b3f24c064f43bfae9ca2d87329f1. Successful addition of voting disk 8b2d9f3224c64fe1bf395647615ba722. Successfully replaced voting disk group with +OCR. CRS-4266: Voting file(s) successfully replaced Recreate ASM SPFIE SQL> create spfile='+OCR/grac4/asmparameterfile/spfileASM.ora' from pfile='/home/grid/ASM_SPFILE/initasm.ora' ; File created.
Verify SPfile location, voting disks and OCR DG status after restore
--> Restart CRS Verify ASM SPFile location [grid@grac41 grac41]$ asmcmd spget +OCR/grac4/asmparameterfile/spfileASM.ora [grid@grac41 grac41]$ crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 7f78f5e770534f74bf4a00e542579bd1 (/dev/asm_ocr_11204_2G_disk1) [OCR] 2. ONLINE 7f75b3f24c064f43bfae9ca2d87329f1 (/dev/asm_ocr_11204_2G_disk2) [OCR] 3. ONLINE 8b2d9f3224c64fe1bf395647615ba722 (/dev/asm_ocr_11204_2G_disk3) [OCR] Located 3 voting disk(s). Verify OCR DG [grid@grac41 grac41]$ asmcmd lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED NORMAL N 512 4096 4194304 6132 5236 2044 1596 0 Y OCR/
Amazing article. Saved my day 🙂