Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

"Design" Compression Issue in 8.5. / 8.0.2

Daniel Nashed  2 March 2009 11:09:00

Compact -ZU can be used to re-compress attachments in a database. This is extremely useful when moving a database to LZ1.
This option is around for a while (since D6.5) but has been changed a couple of times.
You should only use this in Domino 8.x and above because that will allow to re-compress attachments that have no compression or huffman compression before.

Ulrich Krause posted an issue with Java applets in his blog (problem link, support reply).
When I looked into it and opened a PMR to get more details I figured out that the problem has been closed as not reproducible.

I guess part of the problem was a communication issue. L2 could reproduce it but the documentation said it is a problem with design compression.
So L3 tried to reproduce it in an early 8.5.1 build where it did not occur with their configuration.

When we looked into it again it turned out that it was not an issue with design compression itself but with attachment compression in design elements.
The SPR has been reopened and is a pending fix for the next update of 8.5 and 8.0.2(FP1 is also affected).

What is happening in the back-end is that all note types are searched for attachments to compress.
This also includes attachments for Java applets. The applet itself is not converted to LZ1 because it has already a good compression level. But the assoicated attachments in the design element are re-recompressed to LZ1.

In general this should be fine but the backend-code that reads and executes Java applets cannot handle compressed attachments and fails.
This is what Ulrich was seeing in his environment. There is no way to change the attachment back to "not compressed" beside replacing the design of the database. This should recover the problem.

To prevent the problem when using the -ZU switch there is a work-around that will be listed in a TN shortly (we got TN #1370344 created by support based on my additional problem report).

Normal attachments have the host type HOST_MSDOS = 0. The Java files have a different host type (HOST_STREAM).

If you set the notes.ini variable DEBUG_ENABLE_LZ1_HOST_TYPES=0 only normal attachments are re-compressed and the problem should not occur any more.

The current information for the planned fix (subject to change) is to only re-compress attachments in documents (NOTE_CLASS_DOCUMENT) because there might be other Host Types in documents which could benefit from re-compression).

But the work-around above should already give you re-compression of most of the attachments and makes it save again to run compact -ZU.

So as a best practice for Domino 8.5 and 8.0.2 you should set this notes.ini parameter. Once the SPR has been fixed and you are running an updated version you should remove this parameter.

-- Daniel

Comments

1Carl Tyler  02.03.2009 18:14:33   Design Compression Issue in 8.5. / 8.0.2

Do you know if doing this still stops the files from being byte served by the Domino HTTP component?

Currently if you want to byte serve an attachment to the web from Notes/Domino you must attach it without compression.

2Marzel Laning  25.05.2009 8:56:50   Design Compression Issue in 8.5. / 8.0.2

Thanks for making this available!

LZ1 compression is very nice to reduce the amount of data. I came across this issue though: { Link }

It will not be possible to have all attachments compressed with LZ1 even when you enabled all mail files en mail.box-es on servers and clients for LZ1 compression of attachments.

This is very harmful for DAOS which has the most effect when the attachments are in the same compression for deduplication. A MIME message coming in to mail file A will be Huffman compressed, but when forwarded, stored as LZ1 in sent items in the same mail file A and LZ1 compressed in recipient mail file(s) B,C,etc... Thus 2 .NLO files are stored instead of one when incoming MIME messages are replied or forwarded and DAOS is used.


  • [IBM Lotus Domino]
  • [Domino on Linux]
  • [Nash!Com]
  • [Daniel Nashed]