Cisco Prime: Configuring HA

The SMTP server settings should already be configured.

2014_07_03_17_38_35_Cisco_Prime_Infrastructure_Mail_Server_Configuration_172.20.74.187

Login to the primary server with your credentials

2014_07_03_17_42_35_Cisco_Prime_Infrastructure_Mail_Server_Configuration_172.20.74.187

Fill in the details for the secondary server

2014_07_03_17_44_10_Cisco_Prime_Infrastructure_HA_Configuration_172.20.74.187

The failover mode can be manual or automatic. Manual requires you to do the failover, for automatic, the secondary server. Status changes to initializing

2014_07_03_17_57_31_Cisco_Prime_Infrastructure_HA_Configuration_172.20.74.187

From the secondary server, this is the status that I am getting

2014_07_03_18_01_03_wldcbpinf02.mkcorp.com_Remote_Desktop_Connection

Had to leave then came back and found that HA was not configured as expected.

2014_07_04_10_32_52_wldcbpinf02.mkcorp.com_Remote_Desktop_Connection

Now to troubleshoot the issue…

So I check the status on the primary server and everything looks ok. Everything is running.

2014_07_04_10_39_05_172.20.74.187_PuTTY

But when i look at the status from the secondary server, everything seems to be stopped except for the health monitor. 2014_07_04_10_37_59_10.44.6.200_PuTTY

So i tried to restart the services once again.

2014_07_04_10_45_31_10.44.6.200_PuTTY

That did not seem to help.2014_07_04_10_46_22_10.44.6.200_PuTTY

Anyway, after trying to search for the cause of the problem and not finding one, I got a hold of Cisco TAC. The error that I was getting was this one

Failed HA registration Can not create physical standby DB: RMAN-06136: ORACLE error from auxiliary database: ORA-03113: end-of-file on communication channel

I had made sure that i did not have connectivity issues between the two servers, the firewall rule between the two was any any.

TACs response was as follows after they checked the logs under root admin.

As checked the HA configurations is stuck in initializing stage and the secondary ha status shows secondary lost primary. From primary logs I can see the below errors which indicates to communication issue between primary and secondary :

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-00601: fatal error in recovery manager

RMAN-03004: fatal error during execution of command

ORA-12537: TNS:connection closed

RMAN-06900: WARNING: unable to generate V$RMAN_STATUS or V$RMAN_OUTPUT row

RMAN-06901: WARNING: disabling update of the V$RMAN_STATUS and V$RMAN_OUTPUT rows

ORACLE error from target database:

ORA-03113: end-of-file on communication channel

Process ID: 7483

Session ID: 246 Serial number: 1

These errors indicate to communication issue between primary and secondary , as you stated the network bandwidth between two servers is not as the required 1Gbps , check that as mentioned below :

During the high-availability registration, ensure that the bandwidth between the primary Prime Infrastructure and the secondary Prime Infrastructure is 1Gbps.

http://www.cisco.com/c/en/us/td/docs/net_mgmt/prime/infrastructure/2-0/administrator/guide/PIAdminBook/config_HA.html

Now now now. I have a big issue with that bandwidth set. Obviously the servers are geographically separated. I think the bandwidth limit set is rather unrealistic especially if you have MPLS links that are so damn expensive!!! The TAC conclusion was as follows.

so this behavior is expected and the HA wont work unfortunately , since these two servers are on different subnet with MPLS it wont be possible to grant such bandwidth so you can sue the two servers as standalone separately.

You will reimage the secondary to have the primary setup . I apologize for your network structure doesn’t meet the needed requirements for HA.

I hope I could be with more help for you. As agreed I will proceed with closing this case .

Yep. so there goes 🙂 I will need to reimage the server and install it as a standalone then just have similar devices register to it. Thats the only redundancy that I will be able to achive with a WAN link less than 1 Gbps 🙂

See you in my next blog as we set up a standalone server 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s