How do you backup DAOS? Some new ideas ...
Daniel Nashed 29 August 2021 09:54:47
Storage optimization before backup
For my server I am testing multiple ways to backup NSF already.
You have seen BorgBackup and I am looking into different ways of snapshot and standard Domino V12 Backups via file-system ( OpenZFS snapshots etc ).
I moved out FT, NIFNSF, enabled all compression options, DAOS and now have a pretty small NSF files to backup.
OK DAOS is a bit overkill for a one person server. But it offers me to optimize my backup as well.
DAOS can be backuped using any backup solution at any time. the files are written once and never change.
If the attachment changes a new NLO is created and the ref-count for the old NLO is lowered by the referencing documents removed.
DAOS Backup
I am now thinking about multiple ways to backup. A standard backup doesn't sound like the best option for me.
A daily incremental backup still causes a lot of overhead on the backup side.
In my case I probably have not to take care too much about deletions.
The back-end storage would be allocated once. And deleting NLOs after a time would be really difficult to handle.
Also I want to push the DAOS backups from my cluster partner to the same location for deduplication.
The target storage is a Hetzner Storage box which can be either accessed over SAMBA file share or SFTP/SSH.
rclone
A great tool for Linux and also Linux with many options for different back-ends and also with an optional encryption layer.
I am currently looking into pushing all my NLOs into a single directory for deduplication reasons.
The copyto option would help to copy only the new files.
https://rclone.org/commands/rclone_copy/
And I have just tested the encryption layer as a connector on top with my One Drive account
https://rclone.org/crypt/
One drive would be another interesting option for a smaller environment.
The standard private or family accounts come with 1 TB of storage included already per user!
Copyto operation with rclone
So rclone is a very interesting data mover and is very flexible to integrate different back-ends.
The encryption on top for DAOS would be and interesting option.
But also Domino V12 offers shared encryption keys for DAOS T1 storage.
So we could have even multi level security...
A nightly sync job would just sync all DAOS objects to the remote.
And yes the biggest challenge would be deleting NLOs. In my case I might just move attachments to an archive database.
So I would not expect many NLOs to be deleted.
For a single server keeping track of deleted NLOs would be still possible.
But for a multi server environment it would be the safer bet to not delete any NLO.
Yes there are regulations for GDPR. But if the NLOs are encrypted and there is no attachment any more to access it, this might be an acceptable way.
- Comments [0]
![[HCL Domino]](../lotus-domino.gif){kind=small}
![[Domino on Linux]](../domino-linux.gif){kind=small}
![[Nash!Com]](../nashcom.gif){kind=small}
![[Daniel Nashed]](../daniel-nsh.gif){kind=small}