Table of Contents
Check OS and CRS version
$GRID_HOME/OPatch/opatch lsinventory Oracle Home : /u01/app/121/grid Central Inventory : /u01/app/oraInventory from : /u01/app/121/grid/oraInst.loc OPatch version : 12.1.0.1.0 OUI version : 12.1.0.1.0 Log file location : /u01/app/121/grid/cfgtoollogs/opatch/opatch2014-06-25_14-50-13PM_1.log Lsinventory Output file location : /u01/app/121/grid/cfgtoollogs/opatch/lsinv/lsinventory2014-06-25_14-50-13PM.txt -------------------------------------------------------------------------------- Installed Top-level Products (1): Oracle Grid Infrastructure 12c 12.1.0.1.0 Patch level status of Cluster nodes : Patching Level Nodes -------------- ----- 0 gract3,gract2,gract1 # uname -a Linux gract1.example.com 2.6.39-400.109.6.el6uek.x86_64 #1 SMP Wed Aug 28 09:56:40 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux # cat /etc/oracle-release Oracle Linux Server release 6.4 Check whether related ACFS modules are already loaded # lsmod | grep ora oracleacfs 3053229 0 oracleadvm 320180 2 oracleoks 417171 2 oracleacfs,oracleadvm oracleasm 53865 1 You may check whether your kernel version is supported by reading ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)
Create a new ACFS diskgroup
Use asmca and check driver status --> ADVM Driver status sould be : Loaded Create ASM diskgroup with HIGH redundany Make 3 ASM disks available at OS using UDEV rules - Reboot All custer nodes should show the same results # ls -l /dev/acfs* brw-rw----. 1 grid asmadmin 8, 81 Jun 26 08:12 /dev/acfsdisk1_1G brw-rw----. 1 grid asmadmin 8, 97 Jun 26 08:12 /dev/acfsdisk2_1G brw-rw----. 1 grid asmadmin 8, 113 Jun 26 08:12 /dev/acfsdisk3_1G Invoke acma and create a diskgroup with following paramters Change your disk discorvery path to : /dev/asm*,/dev/acfs* Redundancy : HIGH Disk PATH: /dev/acfsdisk1_1G CANDIDATE /dev/acfsdisk2_1G CANDIDATE /dev/acfsdisk3_1G CANDIDATE ASM Compatibility : 12.1.0.0.0 Show Advanced Options and set ASM Compatibility : 12.1.0.0.0 Database Compatibility : 12.1.0.0.0 ADVM Compatibility : 12.1.0.0.0 Verify diskgroup status ( should be mounted on all nodes ) [grid@gract1 ~]$ asmcmd lsdg -g ACFS_DG1 Inst_ID State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name 1 MOUNTED HIGH N 512 4096 1048576 3057 2643 0 881 0 N ACFS_DG1/ 3 MOUNTED HIGH N 512 4096 1048576 3057 2643 0 881 0 N ACFS_DG1/ 2 MOUNTED HIGH N 512 4096 1048576 3057 2643 0 881 0 N ACFS_DG1/ [grid@gract1 ~]$ asmcmd lsdsk -g Inst_ID Path 1 /dev/acfsdisk1_1G 3 /dev/acfsdisk1_1G 2 /dev/acfsdisk1_1G 1 /dev/acfsdisk2_1G 3 /dev/acfsdisk2_1G 2 /dev/acfsdisk2_1G 1 /dev/acfsdisk3_1G 3 /dev/acfsdisk3_1G 2 /dev/acfsdisk3_1G Create and display the volume status grid@gract1 ~]$ asmcmd volcreate -G ACFS_DG1 -s 800m acfs_vol1 [grid@gract1 ~]$ asmcmd volinfo -G ACFS_DG1 acfs_vol1 Diskgroup Name: ACFS_DG1 Volume Name: ACFS_VOL1 Volume Device: /dev/asm/acfs_vol1-443 State: ENABLED Size (MB): 800 Resize Unit (MB): 32 Redundancy: HIGH Stripe Columns: 4 Stripe Width (K): 128 Usage: Mountpath: Create an ACFS filesystem # mkfs -t acfs /dev/asm/acfs_vol1-443 mkfs.acfs: version = 12.1.0.1.0 mkfs.acfs: on-disk version = 39.0 mkfs.acfs: volume = /dev/asm/acfs_vol1-443 mkfs.acfs: volume size = 838860800 mkfs.acfs: Format complete. Register a mount point [root@gract1 rules.d]# mkdir -p /u01/acfs/acfs-vol1 [root@gract1 rules.d]# /sbin/acfsutil registry -a /dev/asm/acfs_vol1-443 /u01/acfs/acfs-vol1 acfsutil registry: mount point /u01/acfs/acfs-vol1 successfully added to Oracle Registry
Mount and dismount an ACFS filesystem
Locate any ACFS mounted filesystem [root@gract1 Desktop]# mount -t acfs /dev/asm/acfs_vol1-443 on /u01/acfs/acfs-vol1 type acfs (rw) Dismount an ACFS filesystem [root@gract1 Desktop]# umount /u01/acfs/acfs-vol1 [root@gract1 Desktop]# mount -t acfs Find ACFS mountpoints and mount ACFS fs [root@gract1 Desktop]# /sbin/acfsutil registry -l Device : /dev/asm/acfs_vol1-443 : Mount Point : /u01/acfs/acfs-vol1 : Options : none : Nodes : all : Disk Group : ACFS_DG1 : Volume : ACFS_VOL1 [root@gract1 Desktop]# mount -t acfs /dev/asm/acfs_vol1-443 /u01/acfs/acfs-vol1 [root@gract1 Desktop]# mount -t acfs /dev/asm/acfs_vol1-443 on /u01/acfs/acfs-vol1 type acfs (rw)
Use lsof to investigate umount problem : device busy
[root@gract1 Desktop]# umount /u01/acfs/acfs-vol1 umount: /u01/acfs/acfs-vol1: device is busy. (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1)) umount: /u01/acfs/acfs-vol1: device is busy. (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1)) [root@gract1 Desktop]# lsof | grep /u01/acfs/acfs-vol1 vi 11553 root cwd DIR 251,226817 4096 2 /u01/acfs/acfs-vol1 vi 11553 root 3u REG 251,226817 12222 44 /u01/acfs/acfs-vol1/.test.swp bash 29897 root cwd DIR 251,226817 4096 2 /u01/acfs/acfs-vol1 --> root run bash shell and this shell has open directory /u01/acfs/acfs-vol1 root also runs vi from the same direactory and vi has opene a .swp file After stopping or killing these processes and the dismount should go through [root@gract1 Desktop]# lsof | grep /u01/acfs/acfs-vol1 [root@gract1 Desktop]# umount /u01/acfs/acfs-vol1
Verify ACFS setup
# mount /dev/asm/acfs_vol1-443 on /u01/acfs/acfs-vol1 type acfs (rw) Commands to verify your setup [root@gract1 rules.d]# lsmod | grep ora oracleacfs 3053229 2 oracleadvm 320180 8 oracleoks 417171 2 oracleacfs,oracleadvm oracleasm 53865 1 $ asmcmd volinfo -G ACFS_DG1 acfs_vol1 Diskgroup Name: ACFS_DG1 Volume Name: ACFS_VOL1 Volume Device: /dev/asm/acfs_vol1-443 State: ENABLED Size (MB): 800 Resize Unit (MB): 32 Redundancy: HIGH Stripe Columns: 4 Stripe Width (K): 128 Usage: ACFS Mountpath: /u01/acfs/acfs-vol1 [root@gract1 rules.d]# /sbin/acfsutil registry -l Device : /dev/asm/acfs_vol1-443 : Mount Point : /u01/acfs/acfs-vol1 : Options : none : Nodes : all : Disk Group : ACFS_DG1 : Volume : ACFS_VOL1 help: help [root@gract1 rules.d]# asmcmd lsdg -g Inst_ID State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name 1 MOUNTED HIGH N 512 4096 1048576 3057 159 0 53 0 N ACFS_DG1/ 3 MOUNTED HIGH N 512 4096 1048576 3057 159 0 53 0 N ACFS_DG1/ 2 MOUNTED HIGH N 512 4096 1048576 3057 159 0 53 0 N ACFS_DG1/ [root@gract1 rules.d]# df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/asm/acfs_vol1-443 819200 118852 700348 15% /u01/acfs/acfs-vol1
Reference
- http://oracleinaction.com/ora-15477-communicate-volume-driver/
- ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)