Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

 
alt

Daniel Nashed

 

Daniel Nashed's Blog

    rclone - rsync like tool that works on different platforms with multiple back-ends

    Daniel Nashed  19 December 2019 08:06:18
    While looking into additional integration points for my backup project, I was looking for a rsync like solution for Windows.
    On Linux rsync is the defacto standard. But on Windows, there isn't a native implementation I would use.

    But I ran into a very interesting project  https://rclone.org.

    rclone is available for many platforms including Windows and beside sftp and the file-system source and destinations it also provides many other connectors including S3.

    Depending on the backup method used for the Domino NSF/NTF data and the OS, I might need a way to sync the remaining part of the data directory to a backup location.

    With rclone you could use a command-line like this, to sync all your remaining data in the data directory once per day to a backup location.

    rclone.exe sync d:\notesdata b:\backup\domino --ignore-case --local-no-check-updated --exclude "*.{ns*,ntf,box,lck}" --exclude "*.{ft}/**" -v  -v --log-file sync.log

    It will also sync deletions allows to use it in many different ways to integrate, depending on your backup strategy.
    A delta sync is very efficient and fast!

    So rclone will be one of the first choices a an integration component. And I might even use it for other backup requirements for data on my own notebook.

    -- Daniel


    Comments

    1Artur Sobolev  24.12.2019 7:46:47  rclone

    Thanks for your review

    It is very nice tool.

    2Steve Johnson  27.05.2021 8:19:04  rclone - rsync like tool that works on different platforms with multiple back-ends

    Daniel we need to copy NSF files to Dropbox. I have used rclone mount to achieve this with some success but I am not sure if it is a sensible idea. I mounted Dropbox into the data directory so I could replicate files into it.

    I want to PUSH files only so the bulk of the copying can be done in advance of the company closing at which time there would be a final sync and disconnect. I can't find much advice out there. Have you done this or seen it done?

    Perhaps there are good reasons why a NSF file on Dropbox (mounted into the data directory) cannot or should not be expected to work? It only has to work Read Only.

    3Daniel Nashed  27.05.2021 10:14:44  rclone - rsync like tool that works on different platforms with multiple back-ends

    @Steve, syncing NSF ro rclone files while the server is running will not work.

    And I don't know why your are doing it. Sounds like this isn't anything that makes sense from Domino point of view.

    rclone cannot work with files that are modified while it is uploading.

    We are using it to backup databases. and only after creating a snapshot via VSS.

    And we are doing it in combination with Domino V12 backup. That's the only way to copy databases securely.

    For data privacy unencrypted data is not desirable in general...

    -- Daniel

    4Steve Johnson  27.05.2021 13:33:17  rclone - rsync like tool that works on different platforms with multiple back-ends

    Daniel

    Thanks for coming back to me.

    We are closing the business down. We need to upload some NSF files to Dropbox so a successor company can access them if they have to. Dropbox was their choice.

    We have primary mail files on Server A with DAOS (from where they are accessed) with Replicas on Server B without DAOS. On server B the only thing that accesses them is Domino Replication and I can turn that off during the cloud replication to stop them being updated / modified.

    I was hoping to use Push Replication to Dropbox to save time when the users have all been made redundant as some mail files are >50Gb there are several Tb of data.

    I was hoping to be able to change the ACL of the Dropbox replicas as a final act so the successor company can open them for reading (probably only after downloading).

    I am not using V12 maybe I should!

    I take your point about encryption.

    Regards Steve

    Recent Entries

    Feeds

    Links

      Archives

      • [HCL Domino]
      • [Domino on Linux]
      • [Nash!Com]
      • [Daniel Nashed]