Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

Cluster Failover on W2008 and higher - disable Port Stealth Mode

Daniel Nashed  21 November 2015 09:34:21
I should have blogged about this earlier. It was in my 2013 IBM Connected presentation but beside the TN and my presentation there is not much information.
If you are using Domino clustering on Win2008 or higher you should really disable the port Stealth mode!


This week I ran into a customer crash situation with repeated crashs which took a while to fix.

The failover on their Win2012 R2 servers was painful slow.


In Win2008 Microsoft introduced a feature called the Port Stealth mode.

This new "security feature" is enabled by default and is independent from the Windows Firewall.


If Domino does not listen any more for NRPC port 1352 Windows will discard all TCP IP packets for new and also existing connections.

That means the Notes client still thinks that the server is there and tries again to send TCP packages until the TCP timeout is reached.

The client is hanging for 30 up to 60 seconds until the failover occurs because Windows does not reject the packages from the client.


Once you disabled the Stealth mode via registry values, the client failover is again almost immediate.

You should also enable silent cluster failover in the desktop policy to avoid any prompts and the failover is almost seamless in most of the cases.

And in current Domino releases the client will also fail back to the home-mail-server later on.


To disable the port Stealth mode you have to set the registry values mentioned in the technote and we had to restart Windows to ensure the settings have effect.


Registry Settings:


[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\WindowsFirewall\DomainProfile]

"DisableStealthMode"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\WindowsFirewall\PrivateProfile]

"DisableStealthMode"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\WindowsFirewall\PublicProfile]

"DisableStealthMode"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\WindowsFirewall\StandardProfile]

"DisableStealthMode"=dword:00000001


The changes only take effect when your restart Windows!
We have multiple customers reporting it even for Windows 2012 R2.


References:


IBM Technote --> https://www.ibm.com/support/docview.wss?uid=swg21498755


The IBM TN is referencing the following Microsoft Technote --> http://msdn.microsoft.com/en-us/library/ff720058%28v=prot.10%29.aspx

Comments

1Oliver Regelmann  30.11.2015 11:01:57  Cluster Failover on W2008 and higher - disable Port Stealth Mode

Any idea what they mean with "IBM recommendes the installation of Firewall software which disables this "feature" on the Notes NRPC port"?

2Daniel Nashed  30.11.2015 14:56:30  Cluster Failover on W2008 and higher - disable Port Stealth Mode

@Oliver

I never checked if there are firewalls that can change the behavior for an individual port.

But when you completely disable the stealth mode this affects also all other ports on the machine.

Personally I see not a big issue from security point of view to disable the stealth mode completely.

But there might be customers having security concerns.

-- Daniel

3Oliver Regelmann  08.12.2015 18:18:12  Cluster Failover on W2008 and higher - disable Port Stealth Mode

Thx.

BTW: my first test disabling stealth mode didn't really change the behaviour at the Notes client. It still took some seconds before failing over to the cluster partner. But not 30 to 60, neither before or after the change.

4Konstantinos Psimoulis  16.12.2015 1:27:43  Cluster Failover on W2008 and higher - disable Port Stealth Mode

Unfortunately the proposed solution does not work on W2012 R2 servers and we are having the exact same issues, the LB or http proxy cannot identify if a port is open or closed and we cannot disable the firewall for other reasons. Those registry paths did not even exist and creating them did not change anything and there are no group policy rules for disabling the stealth mode. Windows makes it really hard to solve simple issues. I bet that the Engineer who created this stealth mode did not think about the possibility of a server behind a LB or an http proxy.

5Daniel Nashed  16.12.2015 9:08:40  Cluster Failover on W2008 and higher - disable Port Stealth Mode

@Konstantinos, it should work and the stealth mode has nothing directly to do with the Wndows firewall.

It will be enabled even if the firewall is disabled. The parameters should work also on Win2012 R2.

We are not 100% sure what we needed to restart on the machine so we booted the server and it worked.

I don't like the port stealth mode at all. It will cause issues with all type of high availability solutions.

-- Daniel

6Konstantinos Psimoulis  16.12.2015 16:11:10  Cluster Failover on W2008 and higher - disable Port Stealth Mode

@Daniel,

You were absolutely right. After rebooting the server everything started working. Perhaps you can add in your instructions that a reboot is required after making those changes. Normally people don't think that they need to reboot after modifying the registry.


  • [IBM Lotus Domino]
  • [Domino on Linux]
  • [Nash!Com]
  • [Daniel Nashed]