Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

alt

Daniel Nashed

Daniel Nashed's Blog

 DACHNUG 

DACHNUG Vienna DNUG Lab preparation done

Daniel Nashed – 22 June 2025 22:51:41

There are many interesting new features in Domino 14.5.
And I am working on a couple projects.


Beside updating the DNUG Lab to Domino 14.5, I also setup a lot of components on my demo notebook.
  • Notebook with Windows 11 (https://blog.nashcom.de/nashcomblog.nsf/dx/black-friday-getting-some-new-ai-ready-hardware.htm)
  • NVIDIA GeForce RTX GPU with 8 GB RAM.
  • Latest Docker Desktop
     
  • The NVIDIA GPU is running native on Windows, on Docker Desktop and also in WSL and inside Docker containers running on WSL Linux like Ubuntu.
  • WSL with Ubuntu 24.04, Alma Linux 10, Kali Linux
  • Docker container on Linux in different modes (Docker Desktop, native Linux in WSL, ...)
     
  • The new Docker model runner
  • VMware Workstation 17 Pro with Windows 2025
  • NGINX on Linux and Windows as a host for the Domino IQ workshop on Monday


There is a lot to discover and we can show a lot of new functionality.


Image:DACHNUG Vienna DNUG Lab preparation done
 Let’s Encrypt 

Let’s Encrypt stops OCSP support

Daniel Nashed – 22 June 2025 22:12:16

Let's Encrypt is stopping OCSP support and goes back to a classical CRL.
You find details about this change here -> https://letsencrypt.org/2024/12/05/ending-ocsp/

I blogged about OCSP stampling a while ago. And just updated the post with this new information.

https://blog.nashcom.de/nashcomblog.nsf/dx/ocsp-stapling-and-why-it-matters-improve-your-https-performance.htm

The Domino TLS Cache supports OCSP caching and provides OCSP data to the HTTP task,
but the back-end code does not distinct between certificates with or without OCSP information.

The TLS Cache will re-check the status every 10 minutes. So it's just a small overhead in the back-end thread.
But it's still good to disable when your certificates have no OCSP responder available in the certificate.

Here is how to check if a certificate has OCSP responder.
Either check the full openssl -text output or just query the OCSP URI as shown below.

I have disabled OCSP stapling on my servers a while ago.

If you never enabled it on Domino, there is nothing to change.
But this information might be also important for other server type that might use OCSP in combination with Let's Encrypt.

openssl x509 -in buypass.pem -noout -text

Authority Information Access:
                OCSP - URI:http://ocsp.buypass.com
                CA Issuers - URI:http://crt.buypass.no/crt/BPClass2CA5.cer


openssl x509 -in buypass.pem -noout -ocsp_uri
http://ocsp.buypass.com
 Domino IQ 

Domino IQ -- Paraphrasing explained

Daniel Nashed – 21 June 2025 13:18:03

Domino IQ 14.5 GA introduces a paraphrasing request invoked before the actual request (added post 14.5 EA3).
This additional request is intended to protect against ticking the LLM to perform other operations then intended by the system prompt.
The idea is to send a request to the same LLM to sanitize and refine the request.

The functionality is documented here -> https://help.hcl-software.com/domino/14.5.0/admin/conf_security_considerations_for_iq.html

The additional request adds overhead and depending on the LLM model it might introduce undesired results.
My personal recommendation is to disable this additional check, because most LLMs today already provide protection and also contain guard functionality for additional protection.
To disable the functionality add the following notes.ini

DOMIQ_DISABLE_PROMPT_PARAPHRASE=1

In my simple example, the meaning of the query slightly changed ("sense" vs "purpose").
I have seen other requests where a German question got translated to English, which causes the response to be returned in English in some cases.

The red part is the system prompt and prefix in front of the actual query.
The actual request uses the configured system prompt with the query returned by the LLM from the first request.


Paraphrasing request is send to the same LLM

{"model":"Qwen2.5","messages":[{"role":"system","content":"Paraphrase the following text:"},{"role":"user","content":"Paraphrase the following text:What is the sense of life?"}],"max_tokens":1024,"temperature":0.0}


Actual request

{"model":"Qwen2.5","messages":[{"role":"system","content":"You are a knowledgeable and helpful assistant. Provide clear and accurate answers to user questions in a concise and friendly manner."},{"role":"user","content":"What is the purpose of life?"}],"max_tokens":1024,"temperature":0.0}



 Domino 14.5 

Domino 14.5 Design task comes with some pretty useful new functionality

Daniel Nashed – 17 June 2025 08:52:21

First of all the design task is now leveraged for updating all system databases during template setup for a new release.
The only important detail to know is that the database needs to have the admin server set to that server.

The background is that an automated process should not run wild on all replicas in your domain trying to upgrade the design multiple times.

There is also a new list of system databases automatically maintained: dominosystemdbs.ind.
The file is updated when the server starts and is leveraged by the design task.
I blogged about it before, but now here it is live in action.

The design task now writes a separate log file IBM_TECHNICAL_SUPPORT/design_upgrade.log.
The file is automatically rotated like other system files with a date and cleaned up like other system logs when configured.

In this file you find additional information like if the admin server is set right or what got updated.
The file also contains run-time information per database and also information about the template version update.

In my case I had to run it again manually after server update, because my admin server was set to a different server for autoupdate.nsf.

Probably you don't want to look into the file on the server. Here is another option you might like.
You can run the design task on all system databases ( -sd ) and get them updated + get a mail with the log file ( -mt ).

If you want to use the standard design task operation only taking into account the admin server settings of the design task, just run the -sd switch.
In case you want the same behavior template setup uses, you can specify the new -adm option.
This would run the design task only on system database (or in general database) with the admin server set to this server also if the "Refresh on admin server only" option not set in the database.

In any case you would get warnings in design_upgrade.log if no admin server is set or "Refresh on admin server only" is not set.

Here is what I just did to upgrade my autoupdate.nsf design after I realized the admin server was wrong and I corrected it.

lo design -sd -mt "daniel nashed"

Refreshing design [autoupdate.nsf]
Updating 'ac Download' into database 'Domino Auto Update' from template 'Domino Auto Update'
Updating 'ac Edit' into database 'Domino Auto Update' from template 'Domino Auto Update'
Updating 'ac Enable' into database 'Domino Auto Update' from template 'Domino Auto Update'
Updating 'ac Formula Check' into database 'Domino Auto Update' from template 'Domino Auto Update'
...
Design refresh completed (1 sec) [autoupdate.nsf] 14.5 (02.01.2025 21:21:08) -> 14.5 (10.04.2025 16:57:53)


Mail example from production today


Image:Domino 14.5 Design task comes with some pretty useful new functionality

New Design Task Switches in Domino 14.5


-sd        Refresh design of System Databases in dominosystemdbs.ind.
-adm       Refresh design only if current server is Administration Server of database.
-mt name   Mail design update log to the Internet Address name.


 Notes.Domino  Traveler 

Notes/Domino/Traveler 14.5 is available

Daniel Nashed – 17 June 2025 08:16:31

Domino 14.5 is out and there is not much to say ...
... because if you are running Domino 14.0 and have Auto Notify enabled, you should see the ProductInfo updated already.

Inside the Notes, Domino and Traveler Product Info documents you find details + links to release notes, what's new and other resources.
See the screen shot below.

There are exciting new features specially on the Domino side.
Some of them will need some detailed info later and I am covering most of it in my sessions at DACHNUG conference next week.
Domino Container project update

I have also pushed out the Domino Container project and I have updated all my servers already.
The Windows machines use AutoUpdate. Most of the Linux machines use the container image.

Along with the Domino 14.5 update, I added the REST API 1.1.4 to the container build.

Adding software means not only changing the software.txt, but also running an end to end automation test after building an image.
This ensures, software is downloadable, installs and runs.
Most of this process is fully automated and is executed before any Git hub repository update -- not just for software updates, but also changes in the project.

To upgrade a container based server, you just build a new image. The build menu remembers the selected software when you save the selection and picks the latest versions automatically for you at build time.
Below I have an example how that looked for me today when updating one of the DNUG Lab servers including all the add-on software.



Image:Notes/Domino/Traveler 14.5 is available


Domino container inspect example switching from 14.5 EA3 to the release + Updating the Domino REST API


Info: New Image Version available!


------------------------------------------------------------------------------------------
 Runtime        :  docker 27.5.1
 Compose        :  docker-compose 1.29.2, build unknown
 Status         :  running
 Health         :  healthy
 Started        :  04.06.2025 09:43:03

 Name           :  /domino
 Image          :  hclcom/domino:latest
 Version CNT    :  14.5EA3
 Version IMG    :  14.5
 Domino Ver CNT :  14.5EA3
 Domino Ver IMG :  14.5
 Add-ons CNT    :  ontime=11.1.1,verse=3.2.4,nomad=1.0.16,traveler=14.5EA3,domrestapi=1.1.3.1-14
 Add-ons IMG    :  ontime=2.3.0,verse=3.2.4,nomad=1.0.16,traveler=14.5,domrestapi=1.1.4-14
 Base Image     :  registry.access.redhat.com/ubi10/ubi-minimal
 BuildTime CNT  :  03.06.2025 16:28:46
 BuildTime IMG  :  17.06.2025 06:45:16

 Hostname       :  ray
 Volumes        :  /local/docker
 Mounts         :  /local

------------------------------------------------------------------------------------------
 Container ID   :  bbf67511762b
 Image-ID CNT   :  90e39ef92404
 Image-ID IMG   :  59f7ca155342
------------------------------------------------------------------------------------------

 Image Size     :  2180 MB
 Network Name   :  host
 Driver         :  overlay2
 Restart Policy :  always
 Restart Retries:  3


 Linux  Windows 2025 

Windows vs Linux update experience

Daniel Nashed – 16 June 2025 01:35:03

The internet is running on Linux for good reasons.
Updating Linux on all distributions I know is a very straightforward operation with a single reboot if needed at all.
In contrast on Windows the download and installation is really painful.

I am running Windows mainly for test and development.
All my Domino servers are running on Linux.
Mainly as a Docker container, but also Domino native on Linux is easy to setup and maintain.

Windows 2025 seems to use more system resources than the previous versions.

Now that Domino 14.5 ships, maybe a good idea for your upgrade plan could be to think about moving to Linux...



Image:Windows vs Linux update experience

Getting the Domino 14.5 container image ready

Daniel Nashed – 16 June 2025 22:48:57

Domino 14.5 is about to ship this week in time for the Domino 14.5 launch event.
For the launch event register here -> https://www.linkedin.com/events/7323258890261868544/

I have just prepared the container image waiting for the MHS download to be available to publish it.
The container image contains all the logic for dependency checking for versions. Like Traveler and the OnTime version shipped with the Domino web-kit.
The default is switched to 14.5. The 14.5 EA releases are removed and the selection allows to switch between 14.5, the latest 14.0 FP and the latest 12.0.2 FP.
Running a container image is the fastest track to Domino 14.5 on Linux.
Most of my servers are container based, which allow me to update by just restarting with a new image.

Image:Getting the Domino 14.5 container image ready


Image:Getting the Domino 14.5 container image ready
 Lotus Script 

Lotus Script detach files not part of an embedded attachment

Daniel Nashed – 8 June 2025 09:09:06

Usually you navigate thru embedded attachments which are part of a richtext item.
But there is also a way to detach files which are not part of an richtext item.

Here is the documentation:

https://help.hcl-software.com/dom_designer/14.0.0/basic/H_GETATTACHMENT_METHOD.html

You can navigate thru the items to find the $File items and extract all of them.

Here is an example:


Dim embedded As NotesEmbeddedObject
...

Forall x In doc.Items
      Set item = x
      If item.Name = "$FILE" Then
              Set embedded = doc.GetAttachment(item.Values(0))
              If Not embedded Is Nothing Then
                      Call embedded.ExtractFile (GitProjectDir + "\json\" + embedded.Name)
                      count = count + 1
              End If
      End If
End Forall
 LS2CAPI  Nomad 

Nomad Web LS2CAPI

Daniel Nashed – 8 June 2025 23:23:25

There isn't documentation for LS2CAPI (C-API from Lotus Script) yet.
But I did some research and asked around.

On Nomad Web the declares are expected to be for Win32 and Linux32.
For Nomad Mobile the declarations are expected to be Linux64.

To check for the Nomad platform you can use @Platform ([Specific]);

On Nomad Web it returns a text list where the first entry is "WebBrowser";
The second entry is "Windows" or "Linux".

On iOS the first entry shows "iOS";

LS2CAPI on Nomad only supports public C-API calls listed in the SDK.
You can't use calls which are not exposed.
So for example checking the delivery sequence number would not work.


Image:Nomad Web LS2CAPI


 Domino  Linux 

What Is the Best Linux Distribution for HCL Domino?

Daniel Nashed – 7 June 2025 19:26:30
What Is the Best Linux Distribution for HCL Domino?



I get this question once in a while and it was also a question which came up in the Domino on Linux Engage session.
The short answer is: It depends.


Any Linux distribution is better than running on Windows

If you don’t have any specific reasons to stay on Windows, any long-term supported Linux distribution can serve as a stable and efficient platform for Domino. But there are important factors to consider.

Linux has a couple of benefits compared to Windows:

- Better resource usage and less overhead
- Better performance
- Easier maintenance and updates
- More secure
- Remote access via SSH (certificate based with multi factor options) instead of RDP
- Easier integration and automation
- Good performance analysis tools and tuning options
- Open Source software stack and community resources


Choose a Company-Standard Distribution


The most important factor is often simple:

Which Linux distribution is already in use within your organization?


Standardizing on one distribution reduces the need for additional expertise and simplifies system management. In most enterprises, the choice of Linux is not made by the Domino team — it’s usually an organization-wide decision.

So, if your company already runs Linux, it’s usually best to deploy Domino on the same platform to streamline support and maintenance.


Do You Need Enterprise Linux Support?


Domino is supported on all major Linux distributions, but if you require a fully certified and tested platform, these are your safest choices:

  • Red Hat Enterprise Linux (RHEL) 9.6 or higher
  • SUSE Linux Enterprise Server (SLES) 15.6 or higher

Keep in mind: Your internal Linux support team may also have specific platform preferences or support agreements. For example, Ubuntu Pro offers enterprise-grade support and extended security patching, which might make it a good fit even though it’s not as widely adopted in traditional enterprise IT environments.


The three main Linux families


There are three major Linux “families” to consider, which differ primarily in package managers, naming conventions, and system tools (such as networking or firewall tools).


Red Hat/CentOS-Based Distributions

  • Red Hat Enterprise Linux
  • CentOS Stream
  • Rocky Linux
  • AlmaLinux

These distributions are all closely related and follow the RHEL release cycle:

CentOS Stream → RHEL → Rocky/AlmaLinux


Package manager: dnf (or yum) using RPM packages


Why it matters: Some software, such as IBM Spectrum Protect for Mail, is only available as an RPM package and not supported on Ubuntu/Debian.

If you're familiar with RHEL-based systems, you'll find the same tooling and administration across all of these variants.


SUSE-Based Distributions

  • SUSE Linux Enterprise Server (SLES)
  • openSUSE

Package manager: zypper using RPM packages Unique feature: YaST ‐ a powerful, menu-driven administration tool

SUSE offers robust enterprise features and excellent SAP integration. While less common for Domino, it is officially supported and a good choice if SUSE is your organization’s standard.



Ubuntu/Debian-Based Distributions
  • Ubuntu LTS (Long-Term Support)
  • Debian

Package manager: apt using .deb packages


Highlights:

  • Ubuntu LTS releases can be upgraded in place
  • UFW (Uncomplicated Firewall) provides simple firewall management
  • Native support for ZFS with advanced features:
    • Compression
    • Deduplication (useful for DAOS and backup)
    • Snapshots
    • Case-insensitive mode
    • Encryption

These features make Ubuntu a strong candidate — especially for virtualized environments or containers.

Note: Proxmox (a popular virtualization platform) is based on Debian and works exceptionally well with LXC containers and ZFS.
Ubuntu and Debian make great guest operating systems in this context.

Other Specialized RPM-Based Distributions

  • VMware Photon OS well suited for VMware environments
  • Oracle Linux ideal if you're already using Oracle products
  • Amazon Linux optimized for AWS instances
These distributions are more niche but can be a good fit in specific environments. All of them support RPM packaging and are generally compatible with Domino deployments.


Conclusion

You’ll find Domino running well on all major Linux distributions — and HCL’s official Domino Linux GitHub repository reflects this diversity:

https://opensource.hcltechsw.com/domino-linux/



Your platform choice should match your operational needs:

  • Follow internal standards when possible
  • Consider enterprise support requirements
  • Evaluate package compatibility for add-ons
  • Leverage ZFS if your use case benefits from it (e.g., backup, DAOS)
I’m running Domino on all of these distributions. The start script and container project support them all, and the tooling abstracts away the differences.

Recent Entries

Feeds

Links

    Archives

    • [HCL Domino]
    • [Domino on Linux]
    • [Nash!Com]
    • [Daniel Nashed]