Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

    Restoring deleted folders in a Notes database

    Daniel Nashed  25 January 2020 11:46:46

    Feature Request: Trash for Folders


    In many customer environments, restoring deleted folders is number one reason for restoring databases.

    For documents you already have soft-deletions and the trash (default 48 hours which can be increased).

    And you can restore the document to the original folder.


    There is an AHA idea to have a trash for folder deletes, which is really popular (  
    https://domino-ideas.hcltechsw.com/ideas/NTS-I-31 ).
    Actually I could come up with an extension manager which hooks into the folder delete and rename it to DeletedFolders\MyFolderName.
    But this should be there out of the box and fully integrated into Notes and into the mail-file!



    Restoring deleted folders


    Last week I had to recover folders for a customer and there is actually a straight forward way, if you know how replication works in detail.


    Idea


    Thanks to Friedhelm Klein from Timetoact, who came up with a great idea in one of his Admin Camp presentations!

    When you restore a database you just have to update the folder design notes in the restored database to make the sequence time/date newer than the deletion stub in the original database.

    An update beats a deletion and you don't have to remove any deletion stubs in the original database on your own. The replication will also take care for putting back all still existing documents into the folder!


    Out of the box without any extra tools you just need the following steps:


    1. Restore the database with replication disabled

    2. Run convert -u twice on the database -> this will update the folder design of all custom folders from the inbox and modify the folder design element

    3. Enable replication and replicate (push) the changes from the restore database to the original database. You might want to use the -NOPIRC flag depending on when the folder was deleted

    4. Disable replication or delete the restore database


    Those steps are quite straightforward.

    But if you could also add this to your restore routine. I added it to my "nshrestore" helper application, which is also the restore part of my backup solution. So with the restore, you will be able to request automatically folder restores.
    I am still testing and have to integrate it to the UI. But it works like charm!


    C-API based solution


    load nshrestore.exe mailrestore.nsf -k mailtest.nsf

    ..
     Restoring deleted folder [Test1] Deleted: 25.01.2020 13:03:32

      Restoring deleted folder [Test1\Test A] Deleted: 25.01.2020 13:03:32
      Restoring deleted folder [Test1\Test A\Test AA] Deleted: 25.01.2020 13:03:32
      Restored 3 folder(s) from [mailrestore.nsf] to [mailtest.nsf]

    How does it work?
    • I am searching for deleted NOTE_CLASS_VIEW
    • If the Note returns a deletion stub, I am looking for the original document by UNID on the restored database.
    • In case this is really a folder (DESIGN_FLAG_FOLDER_VIEW, etc), I am updating the folder note twice
    • Afterwards I push replicate the restore database, with the local server

    This restores the original folders :-)


    Conclusion


    So even with no add-on tools you can do this today. And there are options for further automation.
    It's a bit tricky to get information from a deletion stub. But beside that, it was quite straightforward.


    And I would really wish we don't have to do those kind of restores less often, once the trash for folders will make it into Domino!


    -- Daniel


    Comments

    1Friehdelm Klein  25.01.2020 15:49:10  Restoring deleted folders in a Notes database

    Hi Daniel,

    thank you for the reference. Two more little notes about the restore process of deleted folders:

    As you notes, you just need to make the backup newer than the production mail file. This can be achieved by several ways, you can e.g. sign the folder design elements, rename a parent folder of an accidentially deleted folder tree, or use convert. Usually one run is enough, but to ensure a higher revision number, I do it twice to be on the safe side.

    The process also works with little variations for documents, but there you might have issues replicating when the backup was restored to a local folder as it does not have access to DAOS objects when restored outside the server. When you have a clustered server, you can delete it on one side, put the restore there, touch the documents you want to restore or delete deletion stubs in the production database, then push the restore to the cluster server.

    If it fails, disable PIRC and remove replication history on both replicas.

    2Daniel Nashed  25.01.2020 20:21:34  Restoring deleted folders in a Notes database

    @Friedhelm,

    your approach to update the design or the documents is the best way, because it doesn't need any other action, like purging deletion stubs, deleting databases etc.

    My program logic will work exactly the same for documents. I have a routine, which detects deletes in a defined range and I would just update those documents in the restore before replicating ;-)

    IMHO DAOS is a completely different discussion. You have to restore to the same server! There is no good way bringing NLOs back.

    In the beta of DAOS in 8.5 I had a working solution where I could take advantage of the object being there. But than they added extra security.

    In future with DAOS repair for NLOs, there might be other ways in the cluster to implement restores.

    Today have a a Linux script which works for TSM and EMC networker in combination with daosmgr list missing. There is no available API to query missing NLOs ..

    I have opened an AHA idea for that --> https://domino-ideas.hcltechsw.com/ideas/DOMINO-I-859

    DAOS restores are not on the top of my list yet, because for normal restores all NLOs should be still there.

    But I am planning DAOS integration for my backup application. The DAOS NLO missing command will be integrated into the solution until a native API way is available. And than we leverage a command to trigger the restore.

    Most applications can handle a text list with original files to restore.

    -- Daniel

    3John  27.01.2020 11:58:27  Restoring deleted folders in a Notes database

    We're using this:

    https://notesapplicationmigration.com/folder-restore-tool-for-lotus-notes-email/

    4Daniel Nashed  27.01.2020 13:55:17  Restoring deleted folders in a Notes database

    Thanks John, for the link!

    They are probably doing it on Lotus Script level and add the documents by script?

    The way I am implementing the folder restore is really low-level.

    The olders are the same as before and also the documents are updated in a way that they appear to be exactly the same.

    For folders I am leveraging the replication functionality. And for documents I am updating the documents in the same way there replicator would update them.

    This will bring back the documents and folders 1:1 ;-)

    -- Daniel

    Archives


    • [IBM Lotus Domino]
    • [Domino on Linux]
    • [Nash!Com]
    • [Daniel Nashed]