Running Domino with SELinux on current REHL/CentOS Stream 9 & Co
Daniel Nashed – 8 October 2023 04:45:45
Domino 12.0.2 added support for SELinux in enforced mode, which is enabled by default by newer installations.
SELinux is a lower level security feature, which can even limit processes running with root permissions.
But the application needs to have a SELinux profile.
I ran into this week on my own on a RHEL 9.2 machine and I got the same problem from a partner yesterday.
It turns out that systemd can't read from /tmp any more. But the Domino service from my Nash!Com start script writes the domino process id into the /tmp folder.
With SELinux enabled you get the following error message when looking into your service status (domino statusd).
The start and stop operations of your server will also hang, because systemctl will hang.
systemd[1]: domino.service: Can't convert PID files /tmp/domino.pid O_PATH file descriptor to proper file descriptor: Permission denied
Admins probably notice first that the service is handing on startup:
Starting systemd domino.service
Systemd seems to not handle this error gracefully and the specified timeout seems not to work.
How to find out what SELinux does not allow
Meanwhile SELinux comes with great tools to analyze issues.
You can see that SELinux prevented systemd to read /tmp/domino.pid.
audit2allow -w -a|grep systemd
type=AVC msg=audit(1696738388.170:348): avc: denied { open } for pid=1 comm="systemd" path="/tmp/domino.pid" dev="dm-0" ino=134293379 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:tmp_t:s0 tclass=file permissive=0
How to address this issue
The right location for process pid files would be /run or /var/run (which is a symbolic link to /run) where pid files are usually stored.
Systemd is allowed to read the directory and all sub directories.
But Domino as an application without root permissions cannot write into this directory.
This looks like a catch-22, but can be solved by creating a sub directory for Domino.
A small complication is that the directory would be deleted on every boot and the start script running with the "notes" user can't create directories.
Systemd offers a way to create those sub-directories per application by specifying a RuntimeDirectory directory created below /run.
Once tested for a couple of days I will change the default configuration of the start script to use this new location and new logic.
The install_script bash script would not overwrite existing configurations without specifying the "upd" option.
But it would be in the right location by default for future installation.
To change the configuration manually, change the two following files as listed below.
I have not found any good documentation about this recent change. So this might also help other Linux admins for their own services.
/etc/systemd/system/domino.service
PIDFile=/run/notes/domino.pid
RuntimeDirectory=notes
RuntimeDirectoryPreserve=yes
ExecStart=/opt/nashcom/startscript/rc_domino_script start
/etc/sysconfig/rc_domino_config
DOMINO_PID_FILE=/run/notes/domino.pid
After you made those changes you have to update systemd
systemctl daemon-reload
Update 18. October 2023:
It tuned out that systemd has a build-in functionality to create a sub-directory for an application.
Systemd offers a way to create sub-directories in /run for applications. This is a cleaner approach instead of using ExecStartPre commands.
I am leaving those older changes here for reference, because they could be helpful for other use cases.
Systemd offers ExecStartPre operations, which are executed before the ExecStart statement starts the Domino service.
Prefixing the command with a plus sign runs the operation with root permissions.
ExecStartPre=+/bin/mkdir -p /run/notes
ExecStartPre=+/bin/chown notes:notes /run/notes
Update 9. October 2023:
There was one more tweak needed. The command to create the directory needs to be mkdir -p -- else the 2nd run of the server fails because the directory is already there.
I pushed the changed start script to GitHub in the develop branch with a new version number 3.7.3.
No new features and I will work on documenting details later. It's more important that this change needs to go out.
You will need to update your configuration! If you did not change anything a "install_script upd" would update the configuration and the service. You will also need to reload the service!
If you changed the configuration you need to change the service and the config manually as outlined above.
- Comments [1]
![[HCL Domino]](../lotus-domino.gif){kind=small}
![[Domino on Linux]](../domino-linux.gif){kind=small}
![[Nash!Com]](../nashcom.gif){kind=small}
![[Daniel Nashed]](../daniel-nsh.gif){kind=small}