Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

Details About ODS 52 shipped with Notes/Domino 9.0.1

Daniel Nashed  29 April 2014 07:09:14
I got a couple of questions from multiple customer about ODS 52 which has been introduced in 9.0.1.
There is a bit of confusion about the new ODS and there is not much public available information.


First of all the new ODS 52 is optional and you only need it in some special cases.

It is not enabled by default and in the same way that you needed to set the new ODS it will also be implemented in 9.0.1


How to migrate to the new ODS?


You will need to set notes.ini CREATE_R9_DATABASES=1.


And the new ODS is available and important for clients and servers.

There are different ways to move databases to the new ODS on servers and clients.


For clients you will need to set NSF_UpdateODS=1 in combination with CREATE_R9_DATABASES=1 which lets the client convert to the new ODS.

On the server side you will need to set CREATE_R9_DATABASES=1 and use a copy-style compact.


You can either leverage the compact or the preferred method would be to leverage DBMT which would also generate an unfragmented new NSF file by default.


e.g. DBMT –compactThreads 6 –updallThreads 0



Why to migrate to the new ODS?


There are multiple reasons to migrate to the new ODS.



a.) Issue with encrypted databases


The best public available information about it is from John Paganetti's IBM Connect 2014 presentation. Thanks John for sharing those details!
Everthing else I found is either not detailed or not public..



Issue 1: Medium and Strong Encrypted Databases


- Problem – Rare note corruption when updating a note, only occurs with Medium or Strong encrypted databases

- Has existed since Notes/Domino began using Medium and Strong encryption

- Not noticed because vast majority of databases have replicas and fixup would discard the corrupted note and next replication the note would come back in just fine


- Resolution – Best way to maintain backward compatibility and interoperability was to address with a change to the on-disk-structure (ODS)



Issue 2: Medium Encrypted Databases


- Problem – Rare note corruption when updating a note, only occurs with Medium encrypted databases

– Has existed since Notes/Domino began using Medium encryption

– Not noticed because vast majority of databases have replicas and fixup would discard the corrupted note and next replication the note would come back in just fine


Resolution – The fix for this issue would affect the vast majority of the data and hence there were security concerns it could potentially weaken the current Medium encryption strength.
As a work around, Security team recommends customers go to ODS52 and upgrade existing Medium Encrypted databases to Strong



If you are using encrypted databases either on Notes client or on Domino server you should update to the new ODS!
But this requires to be on 9.0.1 code -- also on the client.

You will have more likely encrypted databases on a client than on a server.


IMHO On the server -- unless you have a password on your server.id (and a tool to manage that server.id on server startup) -- you should disable encryption.

Without a password on the server.id there is not much sense encrypting databases (and NLOs).


But in case you need encryption you should update to ODS52 and switch to strong encryption.


There is also another detail that John shows in his presentation.

I have not seen any public information for the overhead that encryption has on CPU utilization. And this information is quite useful.



NRPC run of Win2008 R2 Server 64-Bit @ 4000 Users, mail9 template
 Not Encrypted  35% CPU
 Medium Encrypted  39% CPU
 Strong Encrypted  48% CPU








On a client this is not really much overhead -- unless you are on a Citrix server.

But for a server this can be quite some overhead.


If you don't want that additional overhead there is a fix that helps also with medium encrypted databases.

But you will need to compact the database to the "new" medium encryption with ODS52 as well.

This is clearly more a work-around and the security team recommends to upgrade to strong encryption if you can.


Here is the way to enable the fix:


notes.ini ENABLE_MEDIUM_ENCRYPTION_FIX
= FFFFFFFB

- Next copy style compact of existing Medium Encrypted databases will be ODS52 with new Medium Encryption which has fix applied



You can update all your medium encrypted databases to strong encryption leveraging copy style compact.

The notes.ini setting you need for that is COMPACT_UPGRADE_MEDIUM_ENCRYPTION_TO_STRONG=1.


This parameter can be quite helpful because it would be a manual step to migrate to strong encryption without it.

And you should disable the parameter when you are done with upgrading all databases to strong encryption.


On Notes clients databases are usually encrypted by default. The notes.ini setting LOCAL_DB_ENCRYPT_DEFAULT determines which encryption strength to use
(0 = No Encryption, 1 = Simple Encryption, 2 = Medium Encryption, 3 = Strong Encryption)


So you should have enabled the following for new databases that should be encrypted with strong encryption.


LOCAL_DB_ENCRYPT_ENABLE=1

LOCAL_DB_ENCRYPT_DEFAULT=3


Note: In case your workstation uses local disk encryption and/or you are using shared login there is also not much sense in encrypting databases.



a.) Issue with large attachments


There is an issue with attachments larger than 2 GB which is fixed in ODS52 in 9.0.1


Fix for ZXZG85KJRK: Large attachments above 2 GB fail


You need Notes 9.0.1 clients and Domino 9.0.1 servers in combination with ODS 52 to get this completely addressed.


Details are available in the following technote:


http://www.ibm.com/support/docview.wss?rs=899&uid=swg21648607


This issue is another reason to upgrade to the new ODS even this is an issue that might only hit you in very rare conditions.



Additonal Note:

There are also settings to log the database encryption used. They will report the current encrpytion level based on the settings the first time a database is opened.


Administrators may now easily identify which databases are currently encrypted and the encryption level, by setting the following notes.ini variable

SHOW_ENCRYPTED_DATABASES=


Utilizes a Bit Mask

1 is “Show Simple”

2 is “Show Medium”

4 is “Show Strong”


To see all Encrypted Databases

Simple, Medium and Strong (1+2+4 = 7)

Set SHOW_ENCRYPTED_DATABASES = 7 in notes.ini


When encrypted databases are opened for the first time - 0 to 1 transition, one of the following messages will be logged

“Current encryption strength: SIMPLE - < absolute file path >”
“Current encryption strength: STRONG - < absolute file path >”


Legacy Medium encrypted database

“Current encryption strength: MEDIUM - < absolute file path >”


New Medium encrypted database with fix (+)

“Current encryption strength: MEDIUM+ - < absolute file path >”


As long as running Release 9.0.1, SHOW_ENCRYPTED_DATABASES works for all database ODS levels




Conclusion:


It makes sense to switch to the new ODS in some cases but you don't need to necessarily put it directly into your upgrade path -- at least on server side.

This can be done afterwards with a copy-style compact that you should run once in a while on any database.

DBMT in 9.0.1 helps you to keep databases defragmented -- check one of my recent blog entries for details.

And in the same step you can upgrade the ODS if needed.


On the server side there is most of the times really no reason to use encrypted databases in the first place.

So as not mentioned in other postings about the new ODS52 the most important step is to migrate to the new ODS on client side.


Unless you have users storing 2 GB attachments in their mailfiles...




Comments

1Tinus Riyanto  30.04.2014 2:09:43  Details About ODS 52 shipped with Notes/Domino 9.0.1

I thought that ODS 52 also allow you to move a database full text index (or is it view index ?) outside of your domino data directory. Ideally this means that you can move your full text index to a separate disk. A side benefit of this is that your backup will be faster since total backup size is smaller.

Please correct me if I am wrong on this.

2Daniel Nashed  30.04.2014 16:45:16  Details About ODS 52 shipped with Notes/Domino 9.0.1

Currently you can only move FT indexes outside the data directory.

This functionality is available since 8.5.3 and you use FTBasePath=c:\FTBasePath

IBM is thinking about also optionally moving NIF to a separate file.

But there is no officially information if that is already something that we can expect soon.

Technically it can be done and it will also improve performance.

There will be less contention in contrast to a single NSF file and separate files can be better optimized from file-system level as well.

-- Daniel

3I have a question re: your comment re: passwords on server ID  02.05.2014 1:01:52  Details About ODS 52 shipped with Notes/Domino 9.0.1

Daniel, thanks for the helpful post.

I have a client with a Domino server on a dedicated box at GoDaddy. Presently the server ID has no password. I would like to put a password and then encrypt the databases so that anyone with physical access to the server or databases can do nothing without the server password.

Do you have any thoughts or recommendations around this kind of setup?

Eric

4Daniel Nashed  02.05.2014 13:10:48  Details About ODS 52 shipped with Notes/Domino 9.0.1

not sure what kind of recommendations you are looking for.

you should use 9.0.1 with ODS52 and strong encryption.

and you will need a solution to apply the password.

either do it manually but in case you need to restart the server you have to type it in again.

or you need a tool to do that.

I wrote an extension manager for a customer to bind the ID ot the machine.

the password is automatically set and binds the ID to the machine.

also it does only allow the server process to use that password.

you cannot revert it and need to save the original ID for that case.

not sure what else you are looking for as a recommendation.

-- Daniel

5Amit H Bora  15.05.2014 12:08:14  Details About ODS 52 shipped with Notes/Domino 9.0.1

Hi Daniel,

Can you please tell me if I created the DB using CREATE_R9_DATABASES=1 on 9.0.1 server and trying to open the same DB in IBM Notes 9.0 client, gives me error as "Invalid NSF version",

Is there any beackward compatibility issue? Or it is not supported to open the ODS 52 DB on lower version of client than 9.0.1.

Your help much appreciated.

6Daniel Nashed  15.05.2014 23:54:30  Details About ODS 52 shipped with Notes/Domino 9.0.1

ODS 52 is only supported with 9.0.1 code or higher. 9.0 does not support this ODS.

If you have a 9.0.1 and run ODS52 on the server that will work with a 9.0 and also 8.5.x clients.

also replication with older servers will work fine.

-- Daniel

7Pierre  21.08.2015 16:11:09  Details About ODS 52 shipped with Notes/Domino 9.0.1

I have seen those CPU numbers but in my case, I have enabled DB encryption on a mail server with a server ID with AES256 and to my despair the machine became totally unresponsive with the router choking with the decryption/encryption operations every time an email is to be delivered. There were still about 60% of CPU available but the system did not take it.

I then did the same thing on a failover mail server where there is little to no router activity and there is no such problem.


  • [IBM Lotus Domino]
  • [Domino on Linux]
  • [Nash!Com]
  • [Daniel Nashed]