SLES 12 SP2 Issues with Domino running with Systemd
Daniel Nashed – 24 July 2017 10:01:20
There is a new feature introduced in SLES 12 SP2 which could lead to issues with larger Domino or Traveler servers. The default nproc size is still set to 7400. So in most cases this tunable does still not need to be set in your Domino service file.
But there is a new security feature introduced in SLES 12 SP2 which will cause processes fail to start or not able to span more threads.
The error you might see is the following:
Jul 20 11:02:41 dom-srv kernel: cgroup: fork rejected by pids controller in /system.slice/domino.service
The new feature limits processes by default to use more than 512 threads.
Here is the relevant extract from SLES 12 SP2 readme:
-- snip --
2.3.2 Support for PIDs cgroup Controller #
The version of systemd shipped in SLES 12 SP2 uses the PIDs cgroup controller. This provides some per-service fork() bomb protection, leading to a safer system.
However, under certain circumstances you may notice regressions. The limits have already been raised above the upstream default values to avoid this but the risk remains.
If you notice regressions, you can change a number of TasksMax settings.
To control the default TasksMax= setting for services and scopes running on the system, use the system.conf setting DefaultTasksMax=. This setting defaults to 512, which means services that are not explicitly configured otherwise will only be able to create 512 processes or threads at maximum.
For thread- or process-heavy services, you may need to set a higher TasksMax value. In such cases, set TasksMax directly in the specific unit files. Either choose a numeric value or even infinity.
Similarly, you can limit the total number of processes or tasks each user can own concurrently. To do so, use the logind.conf setting UserTasksMax (the default is 12288).
nspawn containers now also have a TasksMax value set, with a default of 16384.
-- snip --
The best solution for Domino is to increase the limit directly in the domino.service file.
In addition to this new setting I also updated the config file with an updated value for LimitNPROC= 8000 in addition to the new setting TasksMax= 8000.
8000 Threads should be sufficient for all Domino server environments.
So in case you are running a larger scale environment with SLES 12 SP2 you really should check those settings in your service file!
-- Daniel
-- snip --
[Unit]
Description=IBM Domino Server (notes)
After=syslog.target network.target
[Service]
Type=forking
User=notes
LimitNOFILE=60000
LimitNPROC= 8000
TasksMax= 8000
PIDFile=/local/notesdata/domino.pid
ExecStart=/opt/ibm/domino/rc_domino_script start
ExecStop=/opt/ibm/domino/rc_domino_script stop
TimeoutSec=100
TimeoutStopSec=300
KillMode=none
RemainAfterExit=no
#Environment=LANG=en_US.UTF-8
#Environment=LANG=de_DE.UTF-8
[Install]
WantedBy=multi-user.target
-- snip --
- Comments [2]
1Georg Meier 20.02.2018 16:33:44 SLES 12 SP2 Issues with Domino running with Systemd
Hello Daniel.
You are using SLES 12.
IBM announced that SLES will not longer be supported. But RHEL should be supported by newer domino versions.
I am using SLES, and it is OK.
Do you have experience with Domino 9.0.1 FP 10 in combination with SLES12??
What are your plans for replacing SLES?
Thanks a lot for your advice.
Best regards from Germany
Georg Meier
2Daniel Nashed 21.02.2018 9:00:55 SLES 12 SP2 Issues with Domino running with Systemd
@Georg Meier,
Right now SLES 12 is fully supported by D9.0.1 and all future Feature Packs.
For new versions only RHEL will be supported -- that's my current status.
My current server is running CentOS -- which is not supported but CentOS is source-code compatible with RHEL.
We are still waiting for an update about Domino on Docker. There might be a way to run it on SLES in future.
But there is no detailed information available yet.
When you look into Docker you have to distinct between the host OS and the Linux version running in the Docker container.
My understanding is that for the container IBM is looking into CentOS because if using RHEL as a base you run into license issues.
But we have to wait what IBM comes back to us about Docker support for Domino.
-- Daniel