How to change DNS or NTP in Exadata?

December 23, 2013

1) The following command should return ‘Yes’ for the grid disks being listed:

cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome

2) Run cellcli command to Inactivate all grid disks on the cell you wish to power down/reboot:

cellcli -e alter griddisk all inactive

3) Confirm that the griddisks are now offline by performing the following actions:

(a) Execute the command below and the output should show either asmmodestatus=OFFLINE or asmmodestatus=UNUSED and asmdeactivationoutcome=Yes for all griddisks once the disks are offline in ASM. Only then is it safe to proceed with shutting down or restarting the cell:

cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome

( there has also been a reported case of asmmodestatus= OFFLINE: Means Oracle ASM has taken this grid disk offline. This status is also fine and can proceed with remaining instructions)

(b) List the griddisks to confirm all now show inactive:

cellcli -e list griddisk

4) Shut down the cell services and ocrvottargetd service using the following commands:

# cellcli -e alter cell shutdown services all
# service ocrvottargetd stop

5) Use the ipconf utility to change the DNS settings using the following command:

# /usr/local/bin/ipconf

>>> LOG
[root@dm01cel01 ~]# /usr/local/bin/ipconf
Logging started to /var/log/cellos/ipconf.log
Interface ib0 is Linked.  hca: mlx4_0
Interface ib1 is Linked.  hca: mlx4_0
Interface eth0 is Linked.  driver/mac: igb/00:21:28:a5:8a:c8
Interface eth1 is … ^[[Unlinked.  driver/mac: igb/00:21:28:a5:8a:c9
Interface eth2 is … Unlinked.  driver/mac: igb/00:21:28:a5:8a:ca
Interface eth3 is … Unlinked.  driver/mac: igb/00:21:28:a5:8a:cb

Network interfaces
Name     State      IP address      Netmask         Gateway         Net type     Hostname       
ib0      Linked                                                                                 
ib1      Linked                                                                                 
eth0     Linked                                                                                 
eth1     Unlinked                                                                               
eth2     Unlinked                                                                               
eth3     Unlinked                                                                               
Warning. Some network interface(s) are disconnected. Check cables and swicthes and retry
Do you want to retry (y/n) [y]: n

The current nameserver(s): 10.50.128.60 10.235.1.60 10.229.156.60 10.236.1.52(custom) 10.235.1.52(custom)
Do you want to change it (y/n) [n]: y
Nameserver: 10.235.1.52
Add more nameservers (y/n) [n]: y
Nameserver: 10.236.1.52
Add more nameservers (y/n) [n]: n
The current timezone: America/Chicago
Do you want to change it (y/n) [n]:
The current NTP server(s): wmntp01p.prod.wedorac.com
Do you want to change it (y/n) [n]:

Network interfaces
Name     State      IP address      Netmask         Gateway         Net type     Hostname       
eth0     Linked     10.253.49.34    255.255.255.0   10.253.49.1     Management   dm01cel01.prod.wedorac.com
eth1     Unlinked                                                                               
eth2     Unlinked                                                                               
eth3     Unlinked                                                                               
bondib0  ib0,ib1    192.168.10.5    255.255.252.0                   Private      dm01cel01-priv.prod.wedorac.com
Select interface name to configure or press Enter to continue:

Select canonical hostname from the list below
1: dm01cel01.prod.ç
2: dm01cel01-priv.prod.wedorac.com
Canonical fully qualified domain name [1]:

Select default gateway interface from the list below
1: eth0
Default gateway interface [1]:

Canonical hostname: dm01cel01.prod.wedorac.com
Nameservers: 10.235.1.52 10.236.1.52
Timezone: America/Chicago
NTP servers: wmntp01p.prod.wedorac.com
Default gateway device: eth0
Network interfaces
Name     State      IP address      Netmask         Gateway         Net type     Hostname       
eth0     Linked     10.253.49.34    255.255.255.0   10.253.49.1     Management   dm01cel01.prod.wedorac.com
eth1     Unlinked                                                                               
eth2     Unlinked                                                                               
eth3     Unlinked                                                                               
bondib0  ib0,ib1    192.168.10.5    255.255.252.0                   Private      dm01cel01-priv.prod.wedorac.com
Is this correct (y/n) [y]:

Do you want to configure basic ILOM settings (y/n) [y]:
Loading basic configuration settings from ILOM …
ILOM Fully qualified hostname [dm01cel01-ilom.prod.wedorac.com]:
ILOM IP discovery (static/dhcp) [static]:
ILOM IP address [10.253.49.45]:
ILOM Netmask [255.255.255.0]:
ILOM Gateway or none [10.253.49.1]:
ILOM Nameserver or none [10.50.128.60]: 10.236.1.52
ILOM Use NTP Servers (enabled/disabled) [enabled]:
ILOM First NTP server. Fully qualified hostname or ip address or none [10.235.33.31]: 10.116.230.105
ILOM Second NTP server. Fully qualified hostname or ip address or none [none]: 10.116.230.108

Basic ILOM configuration settings:
Hostname             : dm01cel01-ilom.prod.wedorac.com
IP Discovery         : static
IP Address           : 10.253.49.45
Netmask              : 255.255.255.0
Gateway              : 10.253.49.1
DNS servers          : 10.236.1.52
Use NTP servers      : enabled
First NTP server     : 10.116.230.105
Second NTP server    : 10.116.230.108
Timezone (read-only) : America/Chicago

Is this correct (y/n) [y]: y
Connected. Use ^D to exit.
-> set /SP/clients/dns nameserver=10.236.1.52
Set ‘nameserver’ to ‘10.236.1.52’

-> Session closed
Disconnected
Connected. Use ^D to exit.
-> set /SP/clients/ntp/server/1 address=10.116.230.105
Set ‘address’ to ‘10.116.230.105’

-> Session closed
Disconnected
Connected. Use ^D to exit.
-> set /SP/clients/ntp/server/2 address=10.116.230.108
Set ‘address’ to ‘10.116.230.108’

-> Session closed
Disconnected

Info. Run /opt/oracle.cellos/validations/init.d/saveconfig
Info. Custom changes have been detected in /etc/resolv.conf
Info. Original file will be saved in /etc/resolv.conf.backupbyExadata

Warning. You modified DNS name server.
         Ensure you also update the Infiniband Switch DNS server
         if the same DNS server was also used by the Infiniband switch.

<<< EOF

6) Restart the server using the following command:

# shutdown -r now

7) Once the cell comes back online – you will need to reactive the griddisks:

cellcli -e alter griddisk all active

8) Issue the command below and all disks should show ‘active’:

cellcli -e list griddisk

9)  Verify grid disk status:

(a) Verify all grid disks have been successfully put online using the following command:

cellcli -e list griddisk attributes name, asmmodestatus

(b) Wait until asmmodestatus is ONLINE for all grid disks. Each disk will go to a ‘SYNCING’ state first then ‘ONLINE’. The following is an example of the output:
DATA_CD_00_dm01cel01 ONLINE
DATA_CD_01_dm01cel01 SYNCING
DATA_CD_02_dm01cel01 OFFLINE
DATA_CD_03_dm01cel01 OFFLINE
DATA_CD_04_dm01cel01 OFFLINE
DATA_CD_05_dm01cel01 OFFLINE
DATA_CD_06_dm01cel01 OFFLINE
DATA_CD_07_dm01cel01 OFFLINE
DATA_CD_08_dm01cel01 OFFLINE
DATA_CD_09_dm01cel01 OFFLINE
DATA_CD_10_dm01cel01 OFFLINE
DATA_CD_11_dm01cel01 OFFLINE

(c) Oracle ASM synchronization is only complete when all grid disks show asmmodestatus=ONLINE.

( Please note:  this operation uses Fast Mirror Resync operation – which does not trigger an ASM rebalance. The Resync operation restores only the extents that would have been written while the disk was offline.)

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: