Domino V10 Deletion Logging explained
Daniel Nashed – 7 October 2018 09:46:33
During the beta I have played around with deletion logging, gave feedback and had some questions. Now shortly before the Domino V10 Launch event I think it is time to share some more detailed information.
As far I understood there are no changes planned between Beta 2 and GA.
Deletion Logging is implemented on lower database level and is activated per database. The settings are stored inside the database and do replicate.
Enable Deletion Logging per Database
Right now the only official way to enable deletion logging is to use the compact servertask with the -dl option (see examples below).
You can add up to 4 additional fields per database that you want to log. Those fields could be different depending on the type of database and the logging purpose.
The log does distinct between HARD deletes and SOFT deletes and also allows you to trace the deletion of design elements.
The compact operation looks like this (example):
load compact mail/nsh.nsf -dl on "$TITLE, Form, Subject, DeliveredDate"
After you enabled the deletion logging, each kind of delete is recorded in a central text file on the server.
IBM has chosen a text file for performance reasons. I would have wished they write this information into the deletion stub itself.
Here is how the standard log looks like and how it looks with the custom log fields.
In my example I am interested in the Subject, DeliveredDate. And also the Form of the document.
In addition for design elements I am interested in the design element name stored in $TITLE.
Those would be the type of fields I would add for example for a mail-database. The choice of $TITLE for design document delete can be quite helpful. The note class alone might not be sufficient to identify the design element deleted.
The resulting log files are stored in IBM_TECHNICAL_SUPPORT directory and look like this.
"20181006T130419,29+02","del.nsf","C125831B:004BC903","nserver","CN=Daniel Nashed/O=NashCom/CÞ","HARD","0008","45DF55C1:15FE5439C125831E:003CB1F4","$TITLE","TestFolder"
"20181006T130241,00+02","del.nsf","C125831B:004BC903","nserver","CN=Daniel Nashed/O=NashCom/CÞ","HARD","0001","92A7E32E:6A7A0FD9C125831B:0069F662","Form","Memo","Subject","Test Mail äöü / 电脑死机"
The name of the file has the following syntax:
delete_
Like other files the currently active log file has a default name delete.log and the first line contains the name of the file for renaming it when the server is restarted (similar to console.log as shown above).
Here is a short description of the columns. The last columns depend on your configuration and the list is comma separated and has quotes around the values. Quotes are escaped accordingly.
- Timedate in the timezone of the server plus the timezone of the server at the end.
- Database Name
- Replica ID
- Process which deleted the note
- User who deleted the note
- HARD/SOFT Delete, RESTORE for a SoftDelete
- NoteClass of the document (for example 1 for a document, 8 for a view/folder)
- UNID of the document
After those standard fields you see the up to 4 custom fields that you have optionally specified with compact.
The first column always gives you the name of the field. The following column gives you the value.
The text elements and the total log line is limited. The current limits are 400 bytes per item and 4K for the whole log line.
This should be sufficient in most cases because we only need it to find the document.
The field types that can be used are Text, Text_List, RFC822_Text, or Time.
And those fields have to be present at the database when it is enabled!
The current log file encoding is in LMBCS (Lotus Multi Byte charset) which is the internal representation that Notes/Domino uses since day one to store text. But it would be difficult outside Notes do read this charset encoding.
There are plans for the next update to support other formats. But for now it is LMBCS.
Delete Log Application
All those files are located on the server and are not really admin friendly to read. So I am currently building a small application which imports the log files into a server database.
The same application also can collect the final log-files from each server and stores it a global database to archive those logs before deleting the physical file on disk.
So on the one side you will have a analysis database with all the logs in sortable and searchable way. And in addition you have full access to all the delete-log files in a central place.
The application has a scheduled agent which checks the current log file and also existing log files for importing and archives the older log files.
I have a first version implemented and I am still working on it. But I am planning to publish this database soon.
If someone wants to beta test the application, drop me a note. And I will also have it with me at the Domino V10 launch event next week in Frankfurt and also at the DNUG bleed yellow event the evening before.
Daniel
- Comments [9]
1Ulrich krause 07.10.2018 14:26:45 Domino V10 Deletion Logging explained
I am interested in beta testing your application.
2Christian 07.10.2018 16:55:12 Domino V10 Deletion Logging explained
Hallo Daniel,
Ich habe auch Interesse.
Kleiner Tippfehler:
LOAD compact mail/nsh.nsf -dl on "$TITLE, Form, Subject, DeliveredDate"
3Daniel Nashed 08.10.2018 8:35:29 Domino V10 Deletion Logging explained
@Christian, thanks!
I pasted the parameters and I had to read twice what is different .. I was the text before the parameters .. LOL
Are you going to be in Frankurt today or tomorrow? I will bring the database with me .. Else I can send it after the launch event and the first feedback.
-- Daniel
4Daniel Juarez 10.12.2018 14:24:54 Domino V10 Deletion Logging explained
This is great news. We just had a design element deleted from a database this weekend that (of course) caused application problems. It would be great to determine who deleted it.
I would like to beta test your application if that is still possible.
Thank you.
5Daniel Nashed 11.12.2018 15:08:01 Domino V10 Deletion Logging explained
@Dan, you cannot just beta test it. It's intended as a free solution!
6Xiaoping Liu 27.02.2019 2:41:40 Domino V10 Deletion Logging explained
Hi Daniel, Is the application release now? if yes, where to download it? if no, can I join the beta test? Thanks!
7Daniel Nashed 28.02.2019 13:48:27 Domino V10 Deletion Logging explained
@Xiaoping Liu, you can request it by mail as described but I already sent it to you a minute ago.
It's still kind of "beta" because I did not get much feedback. But you can use it on a production server.
I am very interested in feedback about functionality, bugs and documentation.
So far the only feedback I got was "it works".
-- Daniel
8Marcus Kremers 16.07.2019 18:34:10 Domino V10 Deletion Logging explained
Hallo Daniel,
ich hätte auch Interesse deine DB zu testen.
VG Marcus
9Robert Aschenbrenner 12.09.2019 12:59:29 Domino V10 Deletion Logging explained
Hallo Daniel,
wir hätten ebenfalls Interesse die DB zu testen.
VG
Robert