Daniel Nashed's Blog

Large-Scale Domino Storage Considerations (Last blog post questions reply)

This post is a response to a detailed question about large-scale Domino deployments from Mike.

It is not possible to cover every aspect of a large deployment in a single post. At larger scales, the details matter, and careful planning is required.
The goal here is to highlight considerations that are useful for both smaller and larger environments.

A Domino server with 8 TB data, storage design requires careful planning.

ZFS and TrueNAS SCALE


ZFS is a powerful and highly scalable file system with a lot of flexibility.
At the same time, it must be handled with care, especially at larger volumes.
TrueNAS SCALE is a solid storage appliance and will scale to the limits of the underlying hardware.


Depending on the final design, ZFS usually requires:

• A mix of standard disks and high-speed SSDs
• SSDs for cache and metadata
• Sufficient RAM

Cache design and sizing are separate topics and not covered here.

Compression


Compression is only effective if the data can actually be compressed.

• Attachments are often already compressed
• Attachments may already be stored using LZ1 compression
• This reduces the effectiveness of filesystem-level compression

Notes application data usually compresses well.
In most cases, the additional CPU cost is a good trade-off for reduced I/O.


Deduplication

Deduplication is mainly useful for:

• Backups
• Shared DAOS stores used by multiple servers

Some customers operate many terabytes of DAOS data on storage appliances where deduplication provides significant savings.


Deduplication and Encryption

Deduplication and encryption do not work together.
This is true even if: Multiple servers share the same DAOS encryption keys Encrypted data will not deduplicate.

DAOS Encryption

DAOS uses a hybrid encryption model:

• A public/private DAOS key pair exists per server
• A new symmetric key is generated for each NLO (attachment)
• The attachment data is encrypted with the symmetric key
• The symmetric key is encrypted with the public DAOS key

This is a standard encryption approach and is also used for Notes document encryption and S/MIME.
Because a new symmetric key is generated for each NLO, the same attachment encrypted with the same DAOS key will always result in different encrypted data.


As a result

• DAOS-encrypted data cannot be deduplicated
• This applies across servers and in backups

When DAOS Encryption Is Useful


DAOS encryption is enabled by default to avoid security concerns around DAOS.
In practice, extracting data from an NSF file is usually easier than extracting data from an NLO.

DAOS encryption is mainly useful when:

• NSF files are also encrypted
• The server ID has a password
• The DAOS store is located outside the Domino server (for example in a cloud environment)

If data is highly sensitive, end-to-end encryption using user public keys is required. In that case, the data is not decryptable by administrators or servers.
DAOS encryption is not useless, but in many environments it is not strictly required.
Deduplication benefits for shared DAOS stores or backups can be significant.

ZFS Record Size


The default ZFS record size is 128 KB. This is suitable for:

• DAOS data
• Backup data

Domino databases perform better with smaller record sizes.


Recommended Record Sizes

• NSF / NIF / FT: 16 KB preferred, 32 KB maximum
• Transaction log: 32 KB or smaller
• DAOS and backup data: 128 KB
  (64 KB may improve deduplication but increases RAM usage)

Important Note


If the Domino parameter Create_R85_Log=1 is enabled and the transaction log record size exceeds 32 KB, Domino will crash.
Like most database systems, Domino generally performs best with smaller block sizes.
16K would be the recommended record size.


Additional optimization


Also moving out FT and NIFNSF to different file-systems are helpful.
This would be specially important for snapshot backups. But it would also greatly reduce the file-system size of your server.
You could put both on the same file-system with different folders.


Platform Notes


For large and modern deployments, Linux is usually the most scalable and best tunable platform.
OS/400 (IBM i / iSeries) is fundamentally different from other platforms and does not receive all modern Domino features. Migration to Linux should be considered where possible.
Windows may still be required in some environments, but where there are no specific dependencies, Linux is often the preferred choice from security and TCO point of view.

• Comments [0]