Backup currently active OCR [root@dsctw21 peer]# ocrconfig -manualbackup dsctw21 2017/05/21 09:07:10 +MGMT:/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631 0 [root@dsctw21 peer]# ocrconfig -showbackup PROT-24: Auto backups for the Oracle Cluster Registry are not available dsctw21 2017/05/21 09:07:10 +MGMT:/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631 0 Locate all OCR backups ASMCMD> find --type OCRBACKUP / * +MGMT/dsctw/OCRBACKUP/14348721.293.944515403 +MGMT/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631 ASMCMD> ls -l +MGMT/dsctw/OCRBACKUP/14348721.293.944515403 Type Redund Striped Time Sys Name OCRBACKUP UNPROT COARSE MAY 20 21:00:00 Y 14348721.293.944515403 ASMCMD> ls -l +MGMT/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631 Type Redund Striped Time Sys Name OCRBACKUP UNPROT COARSE MAY 21 09:00:00 Y backup_20170521_090710.ocr.292.944557631 --> Note the first backup was created by root.sh ! After a GNS corruption we need to restore to the OCR created by root.sh List the nodes and cluster resources in your cluster by running the following command on one node: [grid@dsctw21 ~]$ olsnodes dsctw21 dsctw22 [grid@dsctw21 ~]$ crs ***** Cluster Resources: ***** Resource NAME INST TARGET STATE SERVER STATE_DETAILS --------------------------- ---- ------------ ------------ --------------- ----------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE dsctw22 STABLE ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE dsctw21 STABLE ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE dsctw21 STABLE ora.MGMTLSNR 1 ONLINE ONLINE dsctw21 169.254.156.94 192.1 68.2.151,STABLE ora.asm 1 ONLINE ONLINE dsctw21 Started,STABLE ora.asm 2 ONLINE ONLINE dsctw22 Started,STABLE ora.asm 3 OFFLINE OFFLINE - STABLE ora.cvu 1 ONLINE ONLINE dsctw21 STABLE ora.dsctw21.vip 1 ONLINE ONLINE dsctw21 STABLE ora.dsctw22.vip 1 ONLINE ONLINE dsctw22 STABLE ora.gns 1 ONLINE ONLINE dsctw21 STABLE ora.gns.vip 1 ONLINE ONLINE dsctw21 STABLE ora.ioserver 1 ONLINE ONLINE dsctw21 STABLE ora.ioserver 2 ONLINE ONLINE dsctw22 STABLE ora.ioserver 3 ONLINE OFFLINE - STABLE ora.mgmtdb 1 ONLINE ONLINE dsctw21 Open,STABLE ora.qosmserver 1 ONLINE ONLINE dsctw21 STABLE ora.rhpserver 1 ONLINE ONLINE dsctw21 STABLE ora.scan1.vip 1 ONLINE ONLINE dsctw22 STABLE ora.scan2.vip 1 ONLINE ONLINE dsctw21 STABLE ora.scan3.vip 1 ONLINE ONLINE dsctw21 STABLE If OCR is located in an Oracle ASM disk group, then stop the Oracle Clusterware daemon: [root@dsctw21 ~]# crsctl stop crs [root@dsctw22 ~]# crsctl stop crs Start the Oracle Clusterware stack on one node in exclusive mode by running the following command as root: [root@dsctw21 ~]# crsctl start crs -excl -nocrs The -nocrs option ensures that the CRSD process and OCR do not start with the rest of the Oracle Clusterware stack. Ignore any errors that display. Check whether CRSD is running by running the following command: [root@dsctw21 ~]# crsi ***** Local Resources: ***** Resource NAME INST TARGET STATE SERVER STATE_DETAILS --------------------------- ---- ------------ ------------ --------------- ----------------------------------------- ora.asm 1 ONLINE ONLINE dsctw21 Started,STABLE ora.cluster_interconnect.haip 1 ONLINE ONLINE dsctw21 STABLE ora.crf 1 OFFLINE OFFLINE - STABLE ora.crsd 1 OFFLINE OFFLINE - STABLE ora.cssd 1 ONLINE ONLINE dsctw21 STABLE ora.cssdmonitor 1 ONLINE ONLINE dsctw21 STABLE ora.ctssd 1 ONLINE ONLINE dsctw21 OBSERVER,STABLE ora.diskmon 1 OFFLINE OFFLINE - STABLE ora.driver.afd 1 ONLINE ONLINE dsctw21 STABLE ora.drivers.acfs 1 ONLINE ONLINE dsctw21 STABLE ora.evmd 1 ONLINE INTERMEDIATE dsctw21 STABLE ora.gipcd 1 ONLINE ONLINE dsctw21 STABLE ora.gpnpd 1 ONLINE ONLINE dsctw21 STABLE ora.mdnsd 1 ONLINE ONLINE dsctw21 STABLE ora.storage 1 OFFLINE OFFLINE - STABLE If CRSD is running, then stop it by running the following command as root: # crsctl stop resource ora.crsd -init Locate all OCR backups ASMCMD> find --type OCRBACKUP / * +MGMT/dsctw/OCRBACKUP/14348721.293.944515403 +MGMT/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631 ASMCMD> ls -l +MGMT/dsctw/OCRBACKUP/14348721.293.944515403 Type Redund Striped Time Sys Name OCRBACKUP UNPROT COARSE MAY 20 21:00:00 Y 14348721.293.944515403 ASMCMD> ls -l +MGMT/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631 Type Redund Striped Time Sys Name OCRBACKUP UNPROT COARSE MAY 21 09:00:00 Y backup_20170521_090710.ocr.292.944557631 Restore OCR with an OCR backup that you can identify in "Listing Backup Files" by running the following command as root: [root@dsctw21 ~]# ocrconfig -restore +MGMT/dsctw/OCRBACKUP/14348721.293.944515403 Note: If the original OCR location does not exist, then you must create an empty (0 byte) OCR location before you run the ocrconfig -restore command. Ensure that the OCR devices that you specify in the OCR configuration exist and that these OCR devices are valid. If you configured OCR in an Oracle ASM disk group, then ensure that the Oracle ASM disk group exists and is mounted. If the OCR backup file is located in an Oracle ASM disk group, then ensure that the disk group exists and is mounted. [root@dsctw21 ~]# ocrcheck Status of Oracle Cluster Registry is as follows : Version : 4 Total space (kbytes) : 409568 Used space (kbytes) : 3992 Available space (kbytes) : 405576 ID : 2008703361 Device/File Name : +DATA Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded [root@dsctw21 ~]# crsctl stop crs -f Run the ocrconfig -repair -replace command as root on all the nodes in the cluster where you did not the ocrconfig -restore command. For example, if you ran the ocrconfig -restore command on node 1 of a four-node cluster, then you must run the ocrconfig -repair -replace command on nodes 2, 3, and 4. Begin to start Oracle Clusterware by running the following command as root on all of the nodes: [root@dsctw21 ~]# crsctl start crs [root@dsctw22 ~]# crsctl start crs Verify OCR integrity of all of the cluster nodes that are configured as part of your cluster by running the following CVU command: [grid@dsctw21 ~]$ cluvfy comp ocr -n all -verbose Verifying OCR Integrity ...PASSED Verification of OCR integrity was successful. CVU operation performed: OCR integrity Date: May 21, 2017 8:13:54 PM CVU home: /u01/app/122/grid/ User: grid Verify cluster resources [root@dsctw22 ~]# crs ***** Cluster Resources: ***** Resource NAME INST TARGET STATE SERVER STATE_DETAILS --------------------------- ---- ------------ ------------ --------------- ----------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE dsctw22 STABLE ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE dsctw21 STABLE ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE dsctw21 STABLE ora.MGMTLSNR 1 ONLINE ONLINE dsctw21 169.254.156.94 192.1 68.2.151,STABLE ora.asm 1 ONLINE ONLINE dsctw21 Started,STABLE ora.asm 2 ONLINE ONLINE dsctw22 Started,STABLE ora.asm 3 OFFLINE OFFLINE - STABLE ora.cvu 1 ONLINE ONLINE dsctw21 STABLE ora.dsctw21.vip 1 ONLINE ONLINE dsctw21 STABLE ora.dsctw22.vip 1 ONLINE ONLINE dsctw22 STABLE ora.gns 1 ONLINE ONLINE dsctw21 STABLE ora.gns.vip 1 ONLINE ONLINE dsctw21 STABLE ora.ioserver 1 ONLINE ONLINE dsctw21 STABLE ora.ioserver 2 ONLINE ONLINE dsctw22 STABLE ora.ioserver 3 ONLINE OFFLINE - STABLE ora.mgmtdb 1 ONLINE ONLINE dsctw21 Open,STABLE ora.qosmserver 1 ONLINE ONLINE dsctw21 STABLE ora.rhpserver 1 ONLINE ONLINE dsctw21 STABLE ora.scan1.vip 1 ONLINE ONLINE dsctw22 STABLE ora.scan2.vip 1 ONLINE ONLINE dsctw21 STABLE ora.scan3.vip 1 ONLINE ONLINE dsctw21 STABLE