Configure Network Devices with Virtualbox after cloning/copying for RAC setup

Table of Contents

Typical RAC network device configuration used for RAC setup

eth0  - Virtualbox NAT  - Device used by connecting to VPN network for running yum and other stuff 
        CHECK    automatically obtain IP address with DHCP
        UNCHECK  Automatically obtain DNS information from provider
eth1  - Public cluster interface for VIPS,....
eht2  - Private cluster Interconnect

Methode I – Fixing the network devices after cloning by using Network manager

Disable and delete all bak network devices
Edit scritpts located in /etc/sysconfig/network-scripts to configure devices ( see sample scripts below )
Restart network with : # service network restart
Sample scripts for a typical RAC setup

Configuration for Network devices 
eth0  - Virtualbox NAT  - Device used by connecting to VPN network for running yum and other stuff 
        CHECK    automatically obtain IP address with DHCP
        UNCHECK  Automatically obtain DNS information from provider

Network script location:
/etc/sysconfig/network-scripts

ifcfg-eth0
DEVICE=eth0
BOOTPROTO=dhcp
HWADDR=08:00:27:F8:BA:74
ONBOOT=yes
DHCP_HOSTNAME=ract1.example.com
TYPE=Ethernet
DNS1=192.168.1.50
DNS2=192.135.82.44
DNS3=192.168.1.1
DOMAIN="example.com de.oracle.com"

eth1 : IP  192.168.1.131 : Virtualbox Internal Network : Public RAC Interface  
ifcfg-eth1  
# Intel Corporation 82540EM Gigabit Ethernet Controller
DEVICE=eth1
BOOTPROTO=none
BROADCAST=192.168.1.255
HWADDR=08:00:27:B5:5A:54
IPADDR=192.168.1.131
NETMASK=255.255.255.0
NETWORK=192.168.1.0
ONBOOT=yes
TYPE=Ethernet

eth2 :  IP  192.168.2.131 : Virtualbox Internal Network:  Private Cluster Interconnect
ifcfg-eth2 
DEVICE=eth2
BOOTPROTO=none
BROADCAST=192.168.2.255
HWADDR=08:00:27:26:67:20
IPADDR=192.168.2.131
NETMASK=255.255.255.0
NETWORK=192.168.2.0
ONBOOT=yes
TYPE=Ethernet

Methode II – delete /etc/udev/rules.d/70-persistent-net.rules

Current Network device naming by using VirtualBox Bridged Network Device

# ifconfig
eth4      Link encap:Ethernet  HWaddr 08:00:27:D5:CC:47  

# dmesg | grep eth
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
udev: renamed network interface eth0 to eth4
ADDRCONF(NETDEV_UP): eth4: link is not ready
e1000: eth4: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
ADDRCONF(NETDEV_CHANGE): eth4: link becomes ready

# cat /etc/udev/rules.d/70-persistent-net.rules

# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:ec:21:88", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
...
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:d5:cc:47", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"

Solution

# rm  /etc/udev/rules.d/70-persistent-net.rules
# reboot 
# ifconfig
eth0      Link encap:Ethernet  HWaddr 08:00:27:D5:CC:47

Check that content of /etc/sysconfig/network-scripts/ifcfg-eth0 is correct:

Change HWADDR=08:00:27:EC:21:88 and  IPADDR=10.165.112.150 if needed
# cat ifcfg-eth0
HWADDR=08:00:27:D5:CC:47
TYPE=Ethernet
BOOTPROTO=none
IPADDR=10.165.112.150
PREFIX=23
GATEWAY=10.165.112.1
DNS1=10.165.246.33
DNS2=192.135.82.60
DOMAIN=de.oracle.com
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no

Verify that scripts located in /etc/sysconfig/network-scripts are working

# pwd
/etc/sysconfig/network-scripts
# ifdown eth0
Device state: 3 (disconnected)
# ifup eth0
Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/2
state: activating
state: activated
Connection activated

Vbox Network adapter fails with: Device not managed by NetworkManager or unavailable

Runnung ifup fails with : 
$ ifup eth0 
Error: Connection activation failed: Device not managed by NetworkManager or unavailable

Device status 
[root@gract3 Desktop]# ifconfig
eth0      Link encap:Ethernet  HWaddr 08:00:27:A9:76:E3  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
--> Device is up but IP address was missing
    Workaround by adding NM_CONTROLLED=yes to ifcfg-eth0 doesn't solve the problem

Checking device status more in detail 
[root@gract3 ~]# dmesg | grep eth | grep becomes
ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
--> eth0 never reached ready state

Checking Ethernet addresses
[root@gract3 network-scripts]#  dmesg | grep probe
Fixed MDIO Bus: probed
e1000: 0000:00:03.0: e1000_probe: (PCI:33MHz:32-bit) 08:00:27:a9:76:e3
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
e1000: 0000:00:08.0: e1000_probe: (PCI:33MHz:32-bit) 08:00:27:2f:73:1f
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
e1000: 0000:00:09.0: e1000_probe: (PCI:33MHz:32-bit) 08:00:27:d9:bc:7e
e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection
e1000: 0000:00:0a.0: e1000_probe: (PCI:33MHz:32-bit) 08:00:27:cf:c0:9b
e1000: eth3: e1000_probe: Intel(R) PRO/1000 Network Connection
--> Looks fine with configured address in  ifcfg-eth0: HWADDR=08:00:27:A9:76:E3

Testing 
[root@gract3 Desktop]#  ifup eth0 up
Error: Connection activation failed: Device not managed by NetworkManager or unavailable

Trying to restart Network and Network Manager 
[root@gract3 Desktop]# service network restart
Shutting down interface eth0:                              [  OK  ]
Bringing up interface eth0:  
Determining IP information for eth0... failed; no link present.  Check cable?
                                                           [FAILED]
Restart  NetworkManager  
$ service NetworkManager stop 
$ service NetworkManager start 

VBOX: Reconfigure device eth0 to using  Internal Network  
[root@gract3 network-scripts]# ifup eth0
Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/5
state: activated
Connection activated
[root@gract3 network-scripts]# ifconfig -a eth0
eth0      Link encap:Ethernet  HWaddr 08:00:27:A9:76:E3  
          inet addr:192.168.1.178  Bcast:192.168.1.255  Mask:255.255.255.0
--> Device is up 

Now changing back to NAT 
[root@gract3 network-scripts]#  ifup eth0
Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/7
state: activated
Connection activated
[root@gract3 network-scripts]# ifconfig -a eth0
eth0      Link encap:Ethernet  HWaddr 08:00:27:A9:76:E3  
          inet addr:10.0.2.15  Bcast:10.0.2.255  Mask:255.255.255.0
--> Now device eth0 is up using NAT with a valid IP addresss 

Reference 
What is the difference between 'ifconfig up eth0' and 'ifup eth0'?

Change hostname in the “/etc/sysconfig/network” file for cloend VMs:

NETWORKING=yes
HOSTNAME=hhrac2.de.oracle.com

Problem /etc/resolv.conf gets overwritten on reboot or network restart

Choose “Automatic (DHCP) addresses only” method, so that your DHCP server cannot override your DNS setting. See following link for details.
Also double check /etc/sysconfig/network-scripts/ifcfg-eth0 and related network config files no having out-dated DOMAIN entries like de.oracle.com in case your current domain should be example.com. Always take care that your scripts ifcfg-eth0, ifcfg-eth1 and ifcfg-eth2 are up to date and in sync.

Reference

One thought on “Configure Network Devices with Virtualbox after cloning/copying for RAC setup”

Simply wish to say your article is as astonishing.
The clearness on your put up is just excellent and i can think you’re
knowledgeable on this subject. Fine together with your permission let me
to grasp your feed to stay updated with imminent
post. Thank you 1,000,000 and please keep up the gratifying
work.