How to configure the WebLogic AdminServer for High Availability

Introduction

The high availability of applications in the WebLogic environment is realized by clustering. Managed servers in the cluster work together. The information about transactions is distributed cluster-wide. If a cluster member fails, another server takes over the tasks of the failed server and executes them. In this way, the applications are kept running without interruption.

The AdminServer is a single point of failure: if the server fails, the Domain is no longer administrable:

– Configuration changes cannot be performed

– The administration console is not available

The managed servers are still running and can continue to work, even if the AdminServer is not available: this requires the activation of MSI (Managed Server Independence) Mode.

How can the AdminSevrer be protected from failure? In this blogpost I will describe all the steps that are necessary to keep your server running safely.

Environment Overview

My domain consists of two managed servers and an AdminServer. Managed servers are grouped into a Cluster. The environment is installed on two Linux servers based on VMWare:

– host1.example.com

– host2.example.com

Prerequisites:

1. Shared Storage

The configuration of the AdminServer should be shared with both hosts. For this purpose, necessary directories and files can be placed on a shared storage. If shared storage is not available, you can configure replication between host01.example.com and host02.example.com. All necessary data can be synchronized between both servers. I configured the nfs share /nfs_wls based on the share from the Openfiler (the third VM in my test environment):

2. Additional network interface

A virtual IP address (VIP) has to be configured in the network. The AdminSevrer can be listed via this address. The VIP will only be started on the host01.example.com server.

If the machine host01.example.com fails, the following actions must be performed:

– The virtual IP Address must be started on the host host02.example.com

– AdminServer will be started on the host host02.example.com

Solution

The solution consists of the following steps:

1. Definition and configuration of a virtual IP address, which is responsible for the listening to the AdminServer

2. Enabling the IP address on the server host01

3. Creation of the WebLogic Domain

4. Configuring AdminServer for using the virtual IP Address

5. Creation of the NodeManager for the AdminServer

6. Moving of the configuration to the shared storage

7. Test Failover

Configuration of the virtual ip address

The virtual ip address will be configured on both hosts (host01 / host02)

host01: IP Address: 192.168.75.32

host02: IP Address: 192.168.75.33

The AdminServer will be contacted via virtual ip 192.168.75.35 (Interface eth1:1)

On the host01 we will execute following steps: (as OS-user root)

Creating file ifcfg-eth1:1:

cd /etc/sysconfig/network-scripts
cp ifcfg-eth1 ifcfg-eth1
vi ifcfg-eth1:1

Change following lines in the file ifcfg-eth1:1 (in red):

DEVICE=eth1:1
BOOTPROTO=none
ONBOOT=yes
HWADDR=00:0c:29:d5:0f:2b
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
PEERDNS=yes
NETMASK=255.255.255.0
IPADDR=192.168.75.35

Start network interface:

Ifup eth1:1

Check:

Ifconfig -a

Output:

eth1 Link encap:Ethernet HWaddr 00:0C:29:D5:0F:2B
inet addr:192.168.75.32 Bcast:192.168.75.255 Mask:255.255.255.0
...
eth1:1 Link encap:Ethernet HWaddr 00:0C:29:D5:0F:2B
inet addr:192.168.75.35 Bcast:192.168.75.255 Mask:255.255.255.0
...

We will execute the steps on the server host02 (as OS user root):

Creating file ifcfg-eth1:1:

cd /etc/sysconfig/network-scripts
cp ifcfg-eth1 ifcfg-eth1:1

Change following lines in the file ifcfg-eth1:1 (in red):

DEVICE=eth1:1
BOOTPROTO=none
ONBOOT=yes
HWADDR=00:0c:29:38:34:98
NETMASK=255.255.255.0
IPADDR=192.168.75.35
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
PEERDNS=yes

We will move the file ifcfg-eth1:1 to the /root directory: the network interface will not be started on the server reboot:

mv /etc/sysconfig/network-scripts/ifcfg-eth1:1 /root

Creation of WebLogic Domain

The Domain will be created with following settings:

After the creation the Domain we have the following configuration:

Configuring AdminServer for using the virtual IP Address

In the next step we will configure the AdminServer to be able to use the virtual ip address already configured in the previous steps: 192.168.75.35

We can execute these steps either in the AdminConsole or via editing the File $DOMAIN_HOME/config/config.xml. The second way is not supported by Oracle. We will try this via AdminConsole:

Connect to the AdminConsole http://192.168.75.32:7001/console and change to the Domain Tree, then to Servers and AdminServer.

Click on „Lock and edit“ and then change the ListenAddress:

Save the configuration, press Button „Activate Changes“:

Restart the AdminServer.

The AdminServer is now configured to use the virtual IP Address 192.168.75.35.

Access the AdminConsole now via the virtual IP. http://192.168.75.35:7001/console

Creating the NodeManager for the AdminServer for HA reasons

The new created NodeManager will be configured to use the virtual ip: 192.168.75.35:

In the AdminConsole go to Domain Tree -> Environment -> Machines. Then press button „New“ and create the new Machine:

Edit Name: adminmachine and choose Unix for the Machine OS: Press the button „Next“

On the next screen enter the Listen Address: 192.168.75.35 (our virtual ip) and Port: 5557 and press „Finish“:

The new Machine is created:

Choose the new machine, go to the tab „Servers“ and press „Add“:

Select the AdminServer and press „Finish“

Alternatively, we can also create the machine with the wlst script:

connect('weblogic','welcome1','t3://192.168.75.35:7001')

edit()
startEdit(-1,-1,'false')
cd('/')
cmo.createUnixMachine('adminmachine')
cd('/Machines/adminmachine/NodeManager/adminmachine')
cmo.setNMType('SSL')
cmo.setListenAddress('192.168.75.35')
cmo.setListenPort(5557)
cmo.setDebugEnabled(false)
cd('/Servers/AdminServer')
cmo.setMachine(getMBean('/Machines/adminmachine'))
save()
activate()

Create NodeManager directory as the sub directory of the AdminServer

mkdir /u01/app/oracle/user_projects/domains/mydomain/servers/AdminServer/nodemanager
cd /u01/app/oracle/user_projects/domains/mydomain/servers/AdminServer/nodemanager

Create two configuration files:

1. nodemanager.domains with the content:

mydomain=/u01/app/oracle/user_projects/domains/mydomain

2. nodemanager.properties with the content:

DomainsFile=/u01/app/oracle/user_projects/domains/mydomain/servers/AdminServer/nodemanager/nodemanager.domains
LogLimit=0
PropertiesVersion=12.2.1.1.0
AuthenticationEnabled=true
NodeManagerHome=/u01/app/oracle/user_projects/domains/mydomain/servers/AdminServer/nodemanager
JavaHome=/u01/app/oracle/product/JAVA/jdk
LogLevel=INFO
DomainsFileEnabled=true
ListenAddress=192.168.75.35
NativeVersionEnabled=true
ListenPort=5557
LogToStderr=true
weblogic.StartScriptName=startWebLogic.sh
SecureListener=true
LogCount=1
QuitEnabled=false
LogAppend=true
weblogic.StopScriptEnabled=false
StateCheckInterval=500
CrashRecoveryEnabled=false
weblogic.StartScriptEnabled=true
LogFile=/u01/app/oracle/user_projects/domains/mydomain/servers/AdminServer/nodemanager/nodemanager.log
LogFormatter=weblogic.nodemanager.server.LogFormatter
ListenBacklog=50

Change to directory $DOMAIN_HOME/bin and create the start script for the new NodeManager:

cd /u01/app/oracle/user_projects/domains/mydomain/bin
cp startNodeManager.sh startNodeManagerAdmin.sh

Edit the file startNodeManagerAdmin.sh:

Change the following line:

Old:

NODEMGR_HOME="/u01/app/oracle/user_projects/domains/mydomain/nodemanager"

New:

NODEMGR_HOME="/u01/app/oracle/user_projects/domains/mydomain/servers/AdminServer/nodemanager"

Start the newly created NodeManager with the script startNodeManagerAdmin.sh

cd /u01/app/oracle/user_projects/domains/mydomain/bin
./startNodeManagerAdmin.sh &

Stop the AdminServer via AdminConsole and start them via NodeManager:

source /u01/app/oracle/user_projects/domains/mydomain/bin/setDomainEnv.sh
/u01/app/oracle/product/FMW/Oracle_Home/oracle_common/common/bin/wlst.sh
wls:/offline> nmConnect('weblogic','welcome1', '192.168.75.35','5557','mydomain','/u01/app/oracle/user_projects/domains/mydomain','ssl')
wls:/nm/mydomain> nmStart('AdminServer')

Shared Storage Configuration

In the next step we will place the following domain directories on the already configured shared storage (/nfs_wls):

  • /u01/app/oracle/user_projects/domains/mydomain/bin
  • /u01/app/oracle/user_projects/domains/mydomain/config
  • /u01/app/oracle/user_projects/domains/mydomain/servers/AdminServer

Shutdown the AdminServer, all Managed Servers (MS1/MS2) and all NodeManagers on both hosts.

On the host01: (as OS user oracle):

[oracle@host01 bin]$ cd /nfs_wls/
[oracle@host01 nfs_wls]$ mkdir mydomain
[oracle@host01 mydomain]$ mv /u01/app/oracle/user_projects/domains/mydomain/bin /nfs_wls/mydomain
[oracle@host01 mydomain]$ mv /u01/app/oracle/user_projects/domains/mydomain/config /nfs_wls/mydomain
[oracle@host01 mydomain]$ mv /u01/app/oracle/user_projects/domains/mydomain/servers/AdminServer /nfs_wls/mydomain
[oracle@host01 mydomain]$ cd /u01/app/oracle/user_projects/domains/mydomain
[oracle@host01 mydomain]$ ln -s /nfs_wls/mydomain/bin .
[oracle@host01 mydomain]$ ln -s /nfs_wls/mydomain/config .
[oracle@host01 mydomain]$ cd servers/
[oracle@host01 servers]$ ln -s /nfs_wls/mydomain/AdminServer .

On the host02: (as OS user oracle):

[oracle@host02 mydomain]$ cd /u01/app/oracle/user_projects/domains/mydomain
[oracle@host02 mydomain]$ rm -rf bin config
[oracle@host01 mydomain]$ ln -s /nfs_wls/mydomain/bin .
[oracle@host01 mydomain]$ ln -s /nfs_wls/mydomain/config .
[oracle@host02 mydomain]$ cd servers/
[oracle@host02 servers]$ ln -s /nfs_wls/mydomain/AdminServer

Start the Domain:

Host01:

Start both NodeManager:

[oracle@host01 bin]$ cd /u01/app/oracle/user_projects/domains/mydomain/bin
[oracle@host01 bin]$ ./startNodeManager.sh &
[oracle@host01 bin]$ ./startNodeManagerAdmin.sh &

Start AdminServer via wlst:

[oracle@host01 bin]$ source /u01/app/oracle/user_projects/domains/mydomain/bin/setDomainEnv.sh
[oracle@host01 bin]$ /u01/app/oracle/product/FMW/Oracle_Home/oracle_common/common/bin/wlst.sh
wls:/offline> nmConnect('weblogic','welcome1', '192.168.75.35','5557','mydomain','/u01/app/oracle/user_projects/domains/mydomain','ssl')
wls:/nm/mydomain> nmStart('AdminServer')
wls:/nm/mydomain> exit()

Start Managed server MS1:

[oracle@host01 bin]$ source /u01/app/oracle/user_projects/domains/mydomain/bin/setDomainEnv.sh
[oracle@host01 bin]$ /u01/app/oracle/product/FMW/Oracle_Home/oracle_common/common/bin/wlst.sh
wls:/offline> nmConnect('weblogic','welcome1', '192.168.75.32','5556','mydomain','/u01/app/oracle/user_projects/domains/mydomain','ssl')
wls:/nm/mydomain> nmStart('MS1')
wls:/nm/mydomain> exit()

Host02:

Start the NodeManager:

[oracle@host02 bin]$ cd /u01/app/oracle/user_projects/domains/mydomain/bin
[oracle@host02 bin]$ ./startNodeManager.sh &

Start Managed server MS2:

[oracle@host02 bin]$ source /u01/app/oracle/user_projects/domains/mydomain/bin/setDomainEnv.sh
[oracle@host02 bin]$ /u01/app/oracle/product/FMW/Oracle_Home/oracle_common/common/bin/wlst.sh
wls:/offline> nmConnect('weblogic','welcome1', '192.168.75.33','5556','mydomain','/u01/app/oracle/user_projects/domains/mydomain','ssl')
wls:/nm/mydomain> nmStart('MS2')
wls:/nm/mydomain> exit()

Now we have configured the AdminServer for high availability.

Summary of the configuration:

We’ve created the virtual ip for the AdminServer

We’ve configured the AdminServer to be accessible by the virtual ip

We’ve created the separate Nodemanager for the AdminServer.

We’ve moved some directories to the shared storage: $DOMAIN_HOME/bin, $DOMAIN_HOME/config, $DOMAIN_HOME/servers/AdminServer and made there accessible from both hosts: host01/host02:

Failover Test

Scenario: In the case of failure of the server host01 the managed server ms2 on the host02 will still be running and an application will remain online. However we won’t be able to administer the Domain because the AdminServer will be offline (hosted on the host01).

We will start the AdminServer on the host02. Following steps describe the solution:

1. Enable the virtual ip 192.168.75.35 on the host02 (as OS user root):

[root@host02 network-scripts]# cp /root/"ifcfg-eth1:1" /etc/sysconfig/network-scripts/
[root@host02 network-scripts]# ifup eth1:1
[root@host02 network-scripts]# ip a

Output:

...eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
inet 192.168.75.33/24 brd 192.168.75.255 scope global eth1
inet 192.168.75.35/24 brd 192.168.75.255 scope global secondary eth1:1
...

2. Start the NodeManager adminmachine on the host02 (as OS user oracle). The NodeManager configuration is saved on the shared storage and is accessible from the server host02:

[oracle@host02 nodemanager]$ cd /u01/app/oracle/user_projects/domains/mydomain/bin
[oracle@host02 bin]$ nohup ./startNodeManagerAdmin.sh &

3. Start the AdminServer on the host02 (as oracle):

[oracle@host02 bin]$ source /u01/app/oracle/user_projects/domains/mydomain/bin/setDomainEnv.sh
[oracle@host02 bin]$ /u01/app/oracle/product/FMW/Oracle_Home/oracle_common/common/bin/wlst.sh
wls:/offline> nmConnect('weblogic','welcome1', '192.168.75.35','5557','mydomain','/u01/app/oracle/user_projects/domains/mydomain','ssl')
wls:/nm/mydomain> nmStart('AdminServer')
wls:/nm/mydomain> exit()

We are now able to access the AdminConsole on the host02 (via virtual ip): http://192.168.75.32:7001/console

The AdminServer is running on the host02.

Conclusion: after the configuration of the virtual ip and shared storage we are able to start the AdminServer on the second server host02 within the few minutes. The AdminServer is now configured for HA requirements.

Enjoy!

Advertisements

Autor: Neselovskyi, Borys

Oracle Database / Middleware / Engineered System Solution Architect

3 Kommentare zu „How to configure the WebLogic AdminServer for High Availability“

Kommentar verfassen

Trage deine Daten unten ein oder klicke ein Icon um dich einzuloggen:

WordPress.com-Logo

Du kommentierst mit Deinem WordPress.com-Konto. Abmelden / Ändern )

Twitter-Bild

Du kommentierst mit Deinem Twitter-Konto. Abmelden / Ändern )

Facebook-Foto

Du kommentierst mit Deinem Facebook-Konto. Abmelden / Ändern )

Google+ Foto

Du kommentierst mit Deinem Google+-Konto. Abmelden / Ändern )

Verbinde mit %s