Fail2Ban Support for Domino on Linux -- Intrusion Detection
Daniel Nashed – 13 August 2019 15:18:23
Introduction
Domino supports Internet password lockout, which is meanwhile working for all internet protocols (it came in thru a fix I think somewhere in the 8.5.x code stream and isn't really documented).
This does already help to protect individual accounts. But it doesn't currently help for the same IP trying to hack different accounts.
There is a AHA idea to improve it. And I think it is an important functionality for Domino. But blocking IPs with suspicious login attempts isn't always simple for an application.
On the one side someone behind a remote proxy could be blocked if there are too many people having bad password attempts at the same time.
On the other side if your server is behind a secure proxy, you don't have full control to block IPs
As long you have remote IPs hitting your server "directly", you could block them on your server.
This will work for many infrastructures and there is already a quite flexible solution for Linux.
Fail2Ban offers a wide range of "filters" for different applications which parse log files to find out which IP is not behaving correctly and blocks them in the local Linux firewall.
The idea is to have Fail2Ban read thru the Domino console log (better notes.log from my start script because it never wraps around) to find failed password attempts.
Fail2Ban is designed to track and block those IPs in the local Linux firewall.
Here is a sample line for an invalid login attempt. All other protocols use the same format.
[10780:00015-00007F4E8FFA6700] 08.08.2019 22:52:04 http: john.doe@acme.com [1.2.3.4] authentication failure using internet password
Once you have the right filter defined, it's quite easy to install and use Fail2Ban.
I wrote a filter for Domino and also have a default configuration which also includes a configuration for sshd.
The following is a installation description provides all you need to be up and running.
It also includes information about operations like status checking, unblocking users and troubleshooting.
The scripts used will be added to my start script in the "extra" directory.
It's only a solution for Linux and right now only for a local server without a proxy.
For Linux this offers also protection for other services like sshd.
Proxy Support
A friend is using NGINX in front of the Domino HTTP stack on the same machine. And he asked if I could help to get fail2ban working in combination with a proxy in front.
From Domino point of view traffic appears to come from the proxy IP address. But I found a solution which isn't what I expected but it works.
Via notes.ini HTTP_LOG_ACCESS_XFORWARDED_FOR=1 you can configure to write an additional field "ForwaredFor" into domlog.nsf.
The log entry (see above) still lists the proxy IP. There is another AHA idea to enhance the logging.
But for now I wrote a small extension manager, which captures the domlog.nsf update and writes the original requesting IP in the same format into log. So Fail2Ban can capture the right IP address data.
Remote Proxies
This works for a locally installed proxy, but for a remote proxy you will have to pass the information to the proxy. This could be done with event monitoring configurations (run a program, start an agent, etc) based on the log information.
This would a more tricky configuration. The basic configuration is pretty simple.
Below you find all the instructions and additional information.
Enjoy and let me know what you think.
-- Daniel
Current Implementation and Feedback
This installation instruction below uses CentOS 7.6. But once you installed Fail2Ban it will also work with other distributions.
I have it also tested with CentOS 6.10 which works a bit different because init.d is used instead of systemd.
The current implementation checks for all protocols (http, smtp, ldap, imap, pop3).
It is a single filter which would count failed login attempts for all protocols together and than blocks the IP for all protocols.
This seems to be the most reasonable configuration. But depending on your needs you might want to have separate filter definitions and configurations.
The current script can be easily adopted to individual protocols.
But to keep it simple and also because I think this should be the most reasonable way in most cases.
I am looking for feedback if this is what you need. Alternatively I could have a separate filter for all protocols like "domino_http.conf".
But it is far easier to just have one definition and one rule set.
-- Installation --
First of all you have install Fail2Ban. It's included in the epel repository, which can be enabled via yum
yum install epel-release
Next you can install the package
yum install fail2ban
Disable SELinux
Before you can run the log filter, you have to disable SELinux (you could also create a profile for the service, but Domino is also not supported with SELinux enabled).
Check the status via
getenforce
The result should be "disabled". If not you can change it the following way.
vi /etc/selinux/config
Change the line
SELINUX=disabled
The next reboot disables SELINUX
You can temporary disable SELinux if you don't want to reboot now (you should reboot at least later to ensure your server will still boot!).
setenforce 0
The application leverages python and works in combination with firewalld used by default in CentOS 7.
You can enable and start the systemd services via systemd commands. A configuration change needs a restart.
systemctl enable fail2ban
systemctl start fail2ban
systemctl restart fail2ban
-- Domino Configuration --
Copy new configuration file jail.local and Domino filter configuration domino.conf (contains filters for multiple protocols)
If you have an existing configuration copy entries manually. The jail.local is a good starting point and also contains an enabled sshd configuration.
You should review the configuration and change parameters as needed. The default configuration and the service configuration contain the same values but can be customized per service.
Copy the two configuration files
cp jail.local /etc/fail2ban/jail.local
cp domino.conf /etc/fail2ban/filter.d/domino.conf
You should review the configuration. But some details might need to be adjusted.
The domino.conf file contains a "datepattern" which is very important for the pattern matching.
fail2ban parses the date first and removes it from the original string line before the regex expessions are used to match the string and get the HOST IP address.
The Script contains two definitions for the mostly used date format. The format widely used in Europe and the US settings.
You could also change the Domino log format to the one Fail2Ban understands (see notes.ini settings in the domino.conf files).
But I would recommend to change the datepattern in the domino.conf file instead.
Example:
# European Date 31.12.2019 22:11:01
datepattern = %%d.%%m.%%Y %%H:%%M:%%S
The second important parameter is in jail.local.
The logpath defines the log file to check By default the standard location used by the Domino start script is configured.
Please use the Domino Start Script log, because the file doesn't rotate like console.log!
Example:
logpath = /local/notesdata/notes.log
Afterwards restart the service
systemctl restart fail2ban
-- Operations --
Check status for a jail
fail2ban-client status domino
Status for the jail: domino
|- Filter
| |- Currently failed: 0
| |- Total failed: 8
| `- File list: /local/notesdata/notes.log
`- Actions
|- Currently banned: 1
|- Total banned: 1
`- Banned IP list: 192.168.100.107
List IP Tables for banned IPs
iptables -L -p
Chain f2b-domino (1 references)
target prot opt source destination
REJECT all -- 192.168.100.107 0.0.0.0/0 reject-with icmp-port-unreachable
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Unban IP for a specific rule
To unban an IP before it expires use the fail2ban-client
Example:
fail2ban-client set domino unbanip 192.168.100.107
-- Troubleshooting --
Check the log file:
cat /var/log/fail2ban.log
Check Python Errors:
abrt-cli list
-- Testing Rules --
In case you want to test rules to see that for example the date format matches, you can use the following regex test tool included in Fail2Ban
Example:
fail2ban-regex /local/notesdata/notes.log /etc/fail2ban/filter.d/domino.conf
You can use the following filters:
--print-all-matched
--print-all-missed
-- Appendix Configuration Files --
Just copy the following configuration files.
The configuration is a basic configuration, which can be changed for your needs.
You find the code currently in the start script extras directory in the IBM Domino Docker script.
I had to change the download location because pasting it into the Domino blog template made some code disappear.
It is really time to find something better than the old blog template...
https://github.com/IBM/domino-docker/tree/develop/start_script/extra/fail2ban
- Comments [8]
![[HCL Domino]](../lotus-domino.gif){kind=small}
![[Domino on Linux]](../domino-linux.gif){kind=small}
![[Nash!Com]](../nashcom.gif){kind=small}
![[Daniel Nashed]](../daniel-nsh.gif){kind=small}