Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

alt

Daniel Nashed

Domino container image MySQL and PosgreSQL JDBC driver support

Daniel Nashed – 19 November 2025 08:33:37

Traveler supports MySQL and it looks like there will be support for PosgreSQL at some point.
It makes sense to have those JDBC drivers available for using in own applications, if selected for install.

This means:


a.) If Traveler is installed add the JDBC driver to /opt/hcl/domino/notes/latest/linux/Traveler/lib (to be prepared for future use)

b.) Always add it to opt/hcl/domino/notes/latest/linux/ndext when requested independent from Traveler


I also beefed up the automation testing to check if the JDBC drivers are available.
It detects both location of the JDBC driver.

The build script also has a new variable "JVM_LIB_INSTALL_DIRECTORY" where the Java extension directory is detected to allow to install Jar files into the right location.


The two new container build options are:

-mysql-jdbc
-postgresql-jdbc


-- Daniel

Domino Container image - automatic download of files which have a HTTP link.

Daniel Nashed – 19 November 2025 07:24:38

Initially the container image was only consuming files stored locally (e.g. /local/software).
Meanwhile there is an automatic download option with domdownload which can download files once from MHS and store them locally.

But there are 3rd party files like Tika, domprom, JDBC drivers which come from a remote location.

The software.txt entry contains multiple entries in this those cases as shown here:


domprom|0.9.2|domprom-0.9.2-linux-amd64.taz,
https://github.com/nashcom/domino-grafana/releases/download/v0.9.2/domprom-0.9.2-linux-amd64.taz|x|07b0...

The installation process tries to use those comma separated entries from left to right.
But until today it did not download the files to cache them locally instead of loading them every time -- in this example from GitHub.

If the file is available locally, it would be used from the local directory instead of using the download link.

Last night there was a GitHub outage which impacted container builds when using domprom (the servertask for Prometheus/Grafana integration).
The download didn't work any more and the build pipeline failed.

This morning I added functionality to the build script to download the file if not available locally.
That way the first build will cache it and does not need to access GitHub or other external resources for every build.

The download is automatically  in the SOFTWARE_DIR local configuration.


References:


https://opensource.hcltechsw.com/domino-container/howto_download-software/
https://nashcom.github.io/domino-startscript/domdownload/

Domino on Linux NSD -- fgrep: warning: fgrep is obsolescent; using grep -F

Daniel Nashed – 19 November 2025 07:09:05

When you run Domino on Linux, you might have seen this message when running newer Linux versions.

Background about this message is that GNU starts to deprecating those older functions and tell developers how to change it.


There is an environment variable you can set to disable this warning for now. But it would need to be set in the environment of the user before Domino is started.


export GREP_OPTIONS="--no-warnings"


The better way would be to replace those invocations in those scripts.

There are actually two separate warnings depending on which version of Domino you use:


egrep: warning: egrep is obsolescent; using grep -E

fgrep: warning: fgrep is obsolescent; using grep -F


egrep has been already replaced in Domino 14.5.

Still the fgrep warning is showing up when running NSDs.


Those messages are not a big deal, but there is a simple way to address it at least for the Domino container image.

I have just added a simple sed command during the container build to replace those invocations to grep -F/-E.

This replacement also addresses older Domino versions. Not just 14.5.


-- Daniel

 Hetzner  DNS 

Hetzner is moving their DNS to the Cloud Console / Update your DNS TXT API integrations

Daniel Nashed – 11 November 2025 18:55:22

As Hetzner moved their famous "storage boxes" from their separate administration tool to the Hetzner Cloud interface a while ago, they made the same move with the DNS API.
Like the last time this is a very straightforward process. There is a simple migrate button in the existing DNS admin interface to initiate the move, which should finish almost immediate.


Once migrated you find the DNS settings in a separate project.

Separate projects fro DNS can make sense, because each project can have it's separate API token.

The new API token is longer (more secure) than the existing one and if you move DNS domains to separate projects, you can have separate tokens per DNS domain.


DNS TXT API Changes


Along with the GUI changes the API has also been merged into the standard cloud API.

This means that for migrated domains you will need to use the new API.

It is specially important for ACME DNS TXT API integrations.


Check the tools you use, if they already updated their APIs.


The updated Domino CertMgr API has just been published today.


https://opensource.hcltechsw.com/domino-cert-manager/dns_providers/#hetzner
  • Download the new DXL file
  • Switch migrated domains to the new API
  • Set the new API token

If you run into issues, please open an issue in the GitHub project.



Image:Hetzner is moving their DNS to the Cloud Console / Update your DNS TXT API integrations


 Domino 

Fixed in 14.5 FP1 - ID Vault password authentication to a Domino 14.5.0 ID Vault server fails with error: Illegal Security function code

Daniel Nashed – 2 October 2025 22:17:05

Domino 14.5 introduced a new more secure ID Vault format. This format works only for Notes 14.0 and higher clients.
In mixed client environments with older versions this new format caused that the clients could not access their ID in ID Vault.


The format gets updated when the ID is updated in ID Vault.

Here is the updated technote which describes the details
https://support.hcl-software.com/csm?id=kb_article&sysparm_article=KB0122915

With Domino 14.5 FP1 the format has been reverted to the previous format.
But you might have Notes.IDs with the new format, which some user might not be able to read.


Here is how to recover those files:


1. You cloud reset the password which would write the Notes.ID in the previous format

2. You could remove the current Notes.ID from ID Vault and let the client sync it again to ID Vault.


To make sure you don't loose the ID, you could copy the existing document into another database as a backup.

If the user also has a newer client which understands the new format, any kind of vault operations which writes the ID should correct it.


If you did not update your ID Vault servers yet, make sure you update to Domino 14.5 FP1.

 macOS  Notes 

HCL Notes Mac Client 14.5 FP1 is still experiencing issues on macOS 26 with M2 and higher

Daniel Nashed – 1 October 2025 13:32:13

Adam Osborn blogged about this already and we had an offline discussion about it --> https://ab1osborne.blogspot.com/2025/09/the-hcl-notes-mac-client-145-sp1-is.html

The Technote in it's current state does not reflect the issue Adam and others including me as seeing even after updating to 14.5 FP1.
https://support.hcl-software.com/csm?id=kb_article&sysparm_article=KB0124211

The technote states that the first time the client is started does not come up.
What we are seeing on M2 and higher is that the client starts cleanly but when you try to shutdown the client, the client hangs in a way that you can't even shutdown macOS cleanly.

The technote is not yet updated, but HCL is aware of the situation. Adam and also Friedhelm Klein from TIMETOACT both have tickets open.
I updated my M4 mac mini to macOS 26.0.1 and I can reproduce the same hang.

Friedhelm checked with his whole team which CPUs are affected (they tested various combinations of M1-M4 including Max).
It turns out it starts with M2 and the M1 CPU is still working fine.

If you on a Mac with a M2 and higher, I would recommend to wait for the fix before updating to macOS Tahoe.

Thanks to Adam and Friedhelm for this great community work to narrow this down.

-- Daniel


 Domino  Tika 

AHA Idea DOMINO-I-3091 - Allow Domino to use an external Tika server instead of Tika started from Domino

Daniel Nashed – 28 September 2025 09:51:16
Given the current Apache Tika Vulnerability, I want to bring up my request to have support for an external Tika server as an option.

This isn't a request to change what is shipped with Domino nor a change for the default deployment out of the box deployment.
It is a request to add an option to allow companies to use their own Tika deployment (e.g. in a containerized Tika server using the official Apache Tika container image)

Today that's not possible because the Domino FT index back-end starts/stops Tika automatically as a process in a Domino JVM.

If you like the idea, here is the AHA link to vote --> https://domino-ideas.hcltechsw.com/ideas/DOMINO-I-3091


-- Daniel
 Domino  Windows  Linux 

My personal thoughts about keeping OS and Domino versions up-to-date

Daniel Nashed – 26 September 2025 09:55:39

Specially for Domino admin and mail servers upgrading to a later Domino version should not be complicated (unless you have backup or anti-virus dependencies specially on Windows).

I can understand that updates of application servers might take some time.


But staying on older Domino releases like 12.0.x always comes with a risk. No vendor can back port all add-on component new major versions to older code streams.

A dependencies like Java 11 for Tika 3.x where Domino 12.x still is running on Java 8 can't be simply solved if the vendor of the add-on software does not provide a fix of the add-on component of the code stream needed.

If like in the current case Apache Tika does not provide a patch for an older version (which is already out of support) and the new version depends on a newer Java version, the "upstream tool chain dependency" will hit us.


Running on older releases always comes with risk.HCL has quite a number of code streams in support. Plus offers extended support for older versions -- which gives you the chance to stay on an older version if you really have to.

But I would also see all of us admins being responsible to move to later code streams if possible.


GDPR


GDPR by the way implies that companies do what is feasible to protect customer data. This also means being up-to-date with OS and application releases.

I am not saying you have to be on the latest code stream. But today 14.0 with the latest fixpacks is a best practice form security point of view.



Domino Auto Update


Domino 14.0 introduces the first phase of Auto Update to automatically deploy software to install.

Domino 14.5 introduces true Auto Update functionality for Domino on Windows and Linux.

It has been introduced to allow customers to keep their environments patched to the latest version from security and functionality point of view.



Operating system choice and version


Domino is cross platform since the early days and there is a flexibility of choice also today (Windows/Linux/AIX/OS400).

I would personally like to get Domino on ARM supported -- but that's a different story.
But it is impressive how cross platform the Domino code base is on server side.


IMHO Domino on Linux is the better choice than running on Windows for multiple reasons:


  • Security
  • Performance
  • Resource usage
  • Operational costs
  • Maintenance (like patching)
  • Automation
  • Independent from one large operating system vendor

I have spent a lot of time making Domino on Linux easier to deploy manage than on Windows.

Today Domino on Linux is the best choice unless you have special requirements which requires Windows.


For admin and mail servers I would not see any reason to not move to Linux.


Still HCL Domino gives you the flexibility of platform choice and both Windows and Linux are valid platforms.

But if you have Linux available as a platform in your company, running Domino on Linux is really a recommended combination for the reasons stated above.


Domino on Linux will also be the base for the new Domino Workspace Sovereign Cloud offering for exactly those reasons.


HCL does also allow you to run Domino on the Linux Enterprise version of you choice ->
https://opensource.hcltechsw.com/domino-linux/
You are not limited to the two main enterprise vendors.

Some of the distributions like Ubuntu offers a free Linux distribution and commercial support if needed.
But also Redhat and SUSE have free options which are based on the same or very similar code bases if you prefer a free Linux distribution and get Linux level support from a local partner.


Ubuntu for example is a very interesting platform for two additional reasons


  • In place major release updates (which by the way also work on client/desktop side)
  • ZFS support (which would deserve a separate blog post)

---


I am not saying you have to migrate all your environments to Linux. But it would provide a lot of benefits and is even easier to manage today with the tooling the Domino Linux Start Script project offers -->
https://nashcom.github.io/domino-startscript/
This includes an installation and run-time menu driven environment which makes it easy to install, manage, operate and update Domino on Linux.


In case you are having a container strategy, the container project is providing an even easier path Domino on Linux -->
https://opensource.hcltechsw.com/domino-container/.


Keep your operating system update to the latest version


No matter which platform you choose, HCL always stays up to date with operating system support.

Each new enterprise version will be looked at as soon it is available.


One recent addition was Windows 2025 and Redhat Linux 10.
Support most times happens as soon the next fixpack of a code stream ships.


If you are still on Windows 2016 or earlier, it would be time to move up -- even you are on Windows extended support.


The next enterprise platform updating to a new major version will be SUSE updating to SUSE enterprise 16.0.

I took a quick look into the components and there are no surprises from Kernel nor glibc level.


In contrast to the just releases Debian 13, which comes with a very recent glibc which needs a newer OpenJDK version, which is planned to ship with Domino 14.5.1 next year.



My Conclusion


When it comes to security all components in your stack matter. Different application versions have different OS level version support and add-on requirements like Java.

Staying up to date is important for all components of your stack.



 Tika 

Security Bulletin: HCL Notes is affected by an XML External Entity (XXE) vulnerability in Apache Tika (CVE-2025-54988)

Daniel Nashed – 22 September 2025 08:52:18

That's the risk you take when adding external libs to your software: You can be hit by an upstream vulnerability.

In this case Tika has an issue with indexing PDF attachments.


https://support.hcl-software.com/csm?id=kb_article&sysparm_article=KB0124165

Notes and Domino both run Tika as an external stand-alone Java application where the client or server is talking to it over TCP/IP loopback.

The Tika server is started as the same user then the client/server. On server side this should be usually be a none privileged user.


So the risk for Notes/Domino might be not as high as the original CVE rating.

Still it makes sense to replace Tika if you are indexing databases with attachments in your environment.


There will be a fix provided by HCL. But you can also replace the Tika jar file manually today.


Note: Replacing the Tika server jar will only work with Notes/Domino 14.0+ because the current Tika release will only work with Java 11+.

Notes/Domino introduced Java 17 in version 14.0. Older versions are still running Java 8.


https://tika.apache.org/download.html

Container image


The Domino container project supports replacing Tika at build time.

I have removed previous Tika versions from the software list and added the latest 3.2.3 version this morning.

If you are running the container image, you can just use the -tika option
to rebuild your container image with the fixed version of Tika.


Update 26.09.2025:


I had a couple of discussions offline and there is some discussion in the commends of this blog post.

Christian Henserler raised an interesting fine tune option to only exclude PDF instead of adding a whitelist.
You can exclude certain type of attachments to avoid the risk.


notes.ini FT_INDEX_IGNORE_ATTACHMENT_TYPES=*.pdf


https://help.hcl-software.com/domino/14.5.0/admin/modifying_file_attachment_indexing.html

This is specially interested on clients where you can't update quickly and where usually no attachment indexing might be needed.
You can deploy the notes.ini via desktop policy to ensure if someone uses a local FT index with attachment filters, the component in Tika is not invoked.


On the server side my recommendation remains: I would updated to Domino 14.0 or better 14.5 and switch to the newer Tika binary.
Or wait for the upcoming 14.5 FP1 and 14.0 FP5 which both will contain the fixed Tika version.


-- Daniel

ID Vault password authentication to a Domino 14.5.0 ID Vault server fails with error: Illegal Security function code

Daniel Nashed – 27 August 2025 19:59:53

Domino 14.5 introduced an ID Vault incompatibility with older client versions.

The issue is described in the following technote.


https://support.hcl-software.com/csm?id=kb_article&sysparm_article=KB0122915

If you are looking into upgrading and have older clients, you should leave your ID Vault server on 14.0 for now.
Domino 14.5 FP1 is planned to be shipped soon  and will provide an option to keep the older algorithm.


Below is the current info from the technote. I will write up more information as soon we have 14.5 FP1 available.

But it would be expected to have a way to control if the new algorithm is used.


What is important: Once the ID is in the wrong format, an older client cannot access it any more.

The new format is written when the ID is updated.

If you have a current issue, you should open a support ticket.



Notes/Domino 14.5.0 released with a new, stronger, default algorithm setting used during the password authentication protocol to the ID Vault. This new default algorithm only has latent support in 14.0.x clients and servers.

Notes clients and Domino servers that are pre-14.0 do not have the latent support for the new 14.5.0 default algorithm setting used in the ID Vault password authentication transaction.

Therefore any ID Vault password authentication transaction consisting of one endpoint running pre-14.0 code and the other end 14.5.0 code could encounter the ID Vault password authentication failure "Illegal Security function code".


Links

    Archives


    • [HCL Domino]
    • [Domino on Linux]
    • [Nash!Com]
    • [Daniel Nashed]