Daniel Nashed's Blog

Economically Right-Sizing Domino Server Memory on Linux

More RAM is generally good for a Domino server. But more RAM is not automatically a good investment.
This matters more today because RAM has become significantly more expensive again, particularly when building servers with large memory configurations.

A useful concept from economics applies surprisingly well to server sizing: diminishing marginal utility.
The first additional GB of RAM can provide significant performance benefits. The next GB still helps, but usually a little less. Eventually, adding another 32 GB might provide only a small additional benefit.

The interesting question therefore isn't:

How much RAM can Domino use?

It is:

How much RAM provides a meaningful benefit for a workload?

On Linux, unused RAM normally doesn't remain unused. Linux uses available memory as filesystem cache.
A 64 GB Domino server might therefore look approximately like this:


64 GB RAM

8–10 GB   Domino + other processes
~50 GB     filesystem cache
remainder kernel / other usage

It is easy to look at this and conclude that the machine benefits from all 64 GB.
But there is an important distinction:

Memory being used is not the same as memory being required.

Linux will happily use additional RAM to cache filesystem data whenever memory is available.

Diminishing Marginal Utility

The first GBs of filesystem cache are extremely valuable because they contain the most frequently accessed data.
As the cache grows, increasingly less frequently accessed data is retained. Additional RAM still provides a benefit, but the incremental benefit gets smaller.

Conceptually:

Filesystem cache        Marginal benefit

8 GB                   Very high
16 GB                   High
24 GB                   Significant
32 GB                   Useful
48 GB                   Lower
64 GB                   Lower still

The numbers are only illustrative. The actual curve depends on the workload.The important principle is:

Doubling the filesystem cache does not double its performance benefit.

In my tests on one busy Domino mail server, for example, Linux was using around 55 GB of filesystem cache.
Reducing the available cache substantially did not result in a corresponding dramatic increase in physical disk I/O.
Even around 24 GB of cache, the server continued to behave very well.

That suggests that much of the additional cache was useful because it was available—not because the workload actually required it.

Different Domino Workloads Are Different

A mail server, Traveler server, directory server and SMTP server do not have the same memory requirements.
A busy mail server accessing many NSF databases can benefit substantially from filesystem cache.
Traveler is quite different. Its important memory consumers include the JVM, Domino HTTP and native process memory, while an HA configuration uses an external database for Traveler data.

Providing huge amounts of RAM to a Traveler VM simply to create a large filesystem cache may therefore provide little additional benefit.

RAM Isn't Cheap Anymore

For years, a common argument was: RAM is cheap. Just add more. That argument isn't nearly as convincing anymore.
Large server-memory configurations have become expensive again. And in virtualized environments, overprovisioning accumulates quickly.

Consider 20 Domino VMs with 32 GB more RAM each than their workloads meaningfully benefit from: 20 × 32 GB = 640 GB

That's 640 GB of physical infrastructure memory providing progressively smaller benefits.
The same memory could instead provide capacity for additional workloads, HA headroom, or simply reduce the cost of the infrastructure.

Right-Size Instead of Max-Size

The goal should not be to provide every Domino server with the largest possible filesystem cache.
The goal is to provide enough application memory, enough filesystem cache to capture the valuable part of the working set, and reasonable operational headroom.

Beyond that point, additional RAM still helps—but its marginal utility decreases.

That's the economic part of right-sizing:

Don't size memory simply because Linux knows how to use it. Size it according to the value that additional memory actually provides.

• Comments [0]