Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

    Important Update on Traveler iOS 8 Support -- You have to install an IF!

    Daniel Nashed  15 September 2014 22:23:53

    There are some last minute changes in iOS which are only in the final version.

    Apple changed the EAS Sync ID which used to match the Device ID. There has been planning for that change for a while but Apple should have introduce that change already in the Beta releases.
    However this change causes issues in device mapping for the companion/todo app.

    IBM released a IF for 9.0.1/ UP2 today to address this issue and added some background logic to map the device ID.

    There is a ARPA describing the issue

    The problem mainly occurs when you register new devices and causes issues with todo and companion app.
    Existing devices with existing profiles should keep their ActiveSync Device ID. But you will run into issues with new registered users and companion/todo app.

    The IF does also address a couple of other issues. Some are also iOS 8 releated.

    You should update your Traveler servers ASAP.

    Detlev put together a nice details description of what happens in the backend.
    See his blog for additional details -->

    References for fixes for all supported versions.
    You should update even if you are not using the companion/todo app.

    IBM should have sticked with their own rule not announcing support for something that has not yet shipped.
    But we as partners and customers wanted to know in advance what version will support iOS 8.

    Thanks for this very fast response from the Traveler team!
    There are always changes in new software releases and I was surprised that we get a support statement before the release.

    -- Daniel

    9.0.1 IF6 IF7

    8.5.3 UP2 IF7

    Traveler iOS 8 Support

    Daniel Nashed  10 September 2014 17:23:05
    Update: IBM released and IF to address some last minute fixes required for iOS 8!!

    See this blog post for details


    iOS is released soon (hopefully 17.9 for existing devices) and I already got some customer questions about it.

    There is a technote describing the Traveler support for iOS 8.

    The good news everything should work fine and new app versions for iOS are on their way.

    Traveler supports iOS 8 with 8.5.3 Upgrade Pack 2 and higher but I would highly recommend that you update to the latest and greates release 9.0.1 IF5 anyway.

    Only the latest IFs will recognize iOS 8 correctly because they have the build-in codes for the new OS release.

    See all details in the official support technote

    -- Daniel

    Important Platform Support Additions in Notes/Domino 9.0.1 FP2

    Daniel Nashed  21 August 2014 17:59:38
    The new fixpack adds the following platform support:

    9.0.1 FP2 adds support for the following:

        Citrix XenApp 7.5 for Client
        Internet Explorer 11 for xPages
        RHEL7 for Server

    I got the question for RHEL7 already a couple of weeks ago and I think it is great news to have RHEL7 support introduced with a fixpack! That does not always happen!

    The release notes have been updated today and tests are completed.

    A big thanks to IBM also for the other two important platform version updates!!

    -- Daniel

    Traveler 9.0.1 IF5 shipped

    Daniel Nashed  30 July 2014 08:06:19
    Traveler 9.0.1 IF5 shipped just in time for updating a customer yesterday -- after we planned the downtime for more than a month -- funny.
    First updated my Linux box before updating the customer server on Windows.

    The Linux silent install on Linux was a lot quicker than the one on Windows.

    There are a couple of important fixes for all devices types and a new version of the Android client.

    IBM Notes Traveler 9.0.1 Interim Fix 5

    Release Date Component Build Levels Release Documentation
    July 28, 2014 Server
    Android Client
    9.0.1 IF5 Release Documentation

    APAR # Component Abstract
    LO78514external link Server Accepting meeting reschedule or update on iOS 7 device may not update the server copy.
    LO79236external link Server Exception thrown processing repeating event with empty date time stamp.
    LO79453external link Server Reschedule from BlackBerry device may not show correctly for attendees.
    LO79507external link Server Some calendar entries may be missing on device after issues Traveler reset.
    LO79517external link Server Extra reply notice may be generated for non-repeating event.
    LO79665external link Server Import of Notes Calendar may generate duplicate events.
    LO79714external link Android LED notification not working on some Android devices.
    LO79747external link Android Unable to reply to or forward e-mail to user name that contains an ampersand.
    LO79754external link Android Uncommon file extension may not launch when selected from Notes Traveler client on Android.
    LO79796external link Android Field used to edit Out of Office message doesn't scroll in Notes Traveler client for Android devices.
    LO79811external link Server ActiveSync provision loop may cause resync of all data.
    LO79824external link Server Traveler tracking field may grow to large in mail document.
    LO79933external link Server Device security view may not display all devices in Traveler HA Pool.
    LO79952external link Server BlackBerry device may send incorrect date for event instance.
    LO79960external link Server Third e-mail address for contact created on mobile device may get replaced by first e-mail address when edited by Notes Web.
    LO79975external link Server Plain test mail from Android should use UTF-8 encoding.
    LO79999external link Server Return receipt message not consistent with Notes Client.
    LO80087external link Server Create contact on Apple device and IM Address field may appear unexpectedly.
    LO80092external link Server Event summary data may be too large for document processing.
    LO80163external link Android German translation for ToDo not correct on Android client.
    LO80183external link Server Send mail from Apple device with no text and an image may loose the image.
    LO80296external link Android Tablet view instead of phone view displayed on Sony Xperia T2 Ultra Phone.
    LO80340external link Server Apple devices running iOS 7.1.x may periodically resync folders and other content.
    LO80343external link Server Out of Office formatting error being logged by Traveler Server.
    LO80373external link Server BB 10 device may get stuck in Calendar event sync loop.
    LO80415external link Server Double incompatible with String error message on Traveler server.
    LO80422external link Server Too many unsupported start date warning messages on Traveler server.
    LO80423external link Server Cleanup does not always cleanup all users.
    LO80425external link Server May see field too large to save document error due to presence of BlackBerry fields.
    LO80552external link Android Samsung Galaxy S5 fingerprint scanner is not recognized as valid option when password type is unrestricted.
    LO80595external link Server Deadlock on mail server table.
    LO80777external link Server Change read statice for Calendar notices when processing from mobile device.
    LO80925external link Server Support confirmation notice on Apple devices when changes are included.
    LO81006external link Android Calendar alarm may not dismiss on some Android devices.
    LO81091external link Server Improve event fixup Traveler command.
    LO81158external link Server Handle NTS_BODY_THRESHOLD like normal truncation scenario

    Force Traveler to use IPv4 instead of IPv6

    Daniel Nashed  28 July 2014 15:41:26
    We ran into this in a customer situation.
    The code used in Traveler is Java based. For the Servlet and also for the Travler servertask.
    Even if you specify notes.ini NTS_HOST_IP_ADDR with an IPv4 address Traveler might use IPv6.
    If you are in stand-alone mode this should not cause any issues.
    But if you are in HA mode connecting to a remote machine might cause trouble in some situations.

    My recommendation would be to completely disable IPv6 on the machine unless you really need it.
    At some hosted environments like a hosted virtual server (not a root server) you cannot disable IPv6 on Linux completely -- because it needs to be changed on kernel level and you don't control that on those machines.

    On my machine hosted at a provider I went thru the hotline to find out that I cannot disable it.

    But I still wanted to use a IPv4 address for my Traveler server.

    There are two steps that you have to follow.

    First of all convince the Traveler servertask to use IPv4 addresses.
    That can be done with the following options that you pass to Java.

    Second you have to convince the Traveler servlet to use IPv4 only.

    You can pass Java parameters via the HTTP configuration.

    Create a read-only file that is owned by root like this -- Can be located in the data directory if you set the right permissions.

    -rw-r--r-- 1 root root 31 Jul 21 16:27

    Add the following parameter:

    And specify the file in the notes.ini of your Traveler server:

    notes.ini JavaUserOptionsFile=/local/notesdata/

    After restarting the Traveler servertask and HTTP you should see via netstat, that 50125 (Traveler Servertask) and 50126 (Traveler OSGI Servelet) are binding to IPv4.

    -- Daniel

    DAOS NLO Encryption and Decryption

    Daniel Nashed  28 May 2014 10:18:57
    We have been asking for this functionality since DAOS was releases and now there is finally a solution.

    In some cases customers have to either switch of DAOS NLO encryption for a server or enable it later on. Or even want to move from one to another

    There are two SPRs (#PMAO9C6R9G / #GFAL9AKKJZ) described in the following technote -->
    The TN also describes how to use this new functionality.

    There are a couple of details that you should be aware of. First of all the two SPRs are not included in shipping code and are also not yet listed in the fixlist database.
    But they have been submitted to the 9.0.1 code-stream as far I understood.

    The output of the commands are printed to the console (using xprintf which is the equivalent of the internal console write call).
    I have asked if the output can be written to a file via -o opton in future. But for now you have to use the redirect invoking the daosmgr command.

    The TN also mentioned this fix-numbers. So if you need this functionality urgently you can try to request a hotfix from IBM.

    And as described in the TN you should use the migration to either encrypted or unencrypted offline. The move is a major migration. All NLOs will be rewritten most cases. This should be planned for a weekend and should be a one time action only.

    What are the szenarios and reasons to change the encryption of the NLOs?

    In many cases NLOs are encrypted because when DAOS was introduced to an environment someone forgot to set the notes.ini parameter to disable DAOS_ENCRYPT_NLO=0.

    But most customers don't require encryption of NLOs.

    If the NSF files on your Domino server are not encrypted and the is not protected by a password, it does not make much sense to have the NLOs encrypted.
    It is even harder to find the right information in a NLO than in a NSF file. And if you copy the NLOs to a different machine including the if it has no password, you can read the NLO anyway.
    So in most cases not having NLO encryption enabled is a best practice for a couple of reasons and the encryption only adds security when the is protected as well.

    Encryption adds not that much overhead at runtime but there are a couple of other reasons.

    First of all if you want to use another cluster member to copy missing NLOs as more simple restore scenario when a NLO is missing this is only possible if NLOs are not encrypted.

    Second if you have storage like a NetApp where you have enabled block-level deduplication and point multiple DAOS stores to the same NetApp volume you can save a lot of disk storage because the same NLOs will have the same blocks. This does only work if the NLO is not encrypted because the same NLO on different servers will be encrypted with a different key (actually even on the same server when encrypted later the file could be different because of a different "session key").

    On top of that some backup solutions support block-level deduplication. And that could save space on the backup side as well if encryption is disabled.
    With encryption enabled there is amost no block-level deduplication.

    In addition moving DAOS stored among servers when you switch the is much more simple without encryption.
    But if you have the new options in the daosmgr you could now re-encrypt NLO files with a new
    I would only do this if you really really need it. In normal cases in such a migration scenario I would use the new functionality to disable NLO encryption for the above reasons.

    IMHO it is still good to have NLO encryption enabled by default to avoid discussions about DAOS security.
    But in reality in at least 80% customer environments NLO encryption is not required overhead and complexity.

    I know others think differently about it and that's just my humble opinion...

    On the other side we also have customers who started without encryption and now need to encrypt all databases, NLOs and also protect the with a password (including the need for a solution to apply the password on server start in a secure way).

    Thanks to IBM to make this change and have it implemented in a flexible way to do it both ways including a verification options for encryption status of all NLOs.

    -- Daniel

    Details About ODS 52 shipped with Notes/Domino 9.0.1

    Daniel Nashed  29 April 2014 07:09:14
    I got a couple of questions from multiple customer about ODS 52 which has been introduced in 9.0.1.
    There is a bit of confusion about the new ODS and there is not much public available information.

    First of all the new ODS 52 is optional and you only need it in some special cases.

    It is not enabled by default and in the same way that you needed to set the new ODS it will also be implemented in 9.0.1

    How to migrate to the new ODS?

    You will need to set notes.ini CREATE_R9_DATABASES=1.

    And the new ODS is available and important for clients and servers.

    There are different ways to move databases to the new ODS on servers and clients.

    For clients you will need to set NSF_UpdateODS=1 in combination with CREATE_R9_DATABASES=1 which lets the client convert to the new ODS.

    On the server side you will need to set CREATE_R9_DATABASES=1 and use a copy-style compact.

    You can either leverage the compact or the preferred method would be to leverage DBMT which would also generate an unfragmented new NSF file by default.

    e.g. DBMT –compactThreads 6 –updallThreads 0

    Why to migrate to the new ODS?

    There are multiple reasons to migrate to the new ODS.

    a.) Issue with encrypted databases

    The best public available information about it is from John Paganetti's IBM Connect 2014 presentation. Thanks John for sharing those details!
    Everthing else I found is either not detailed or not public..

    Issue 1: Medium and Strong Encrypted Databases

    - Problem – Rare note corruption when updating a note, only occurs with Medium or Strong encrypted databases

    - Has existed since Notes/Domino began using Medium and Strong encryption

    - Not noticed because vast majority of databases have replicas and fixup would discard the corrupted note and next replication the note would come back in just fine

    - Resolution – Best way to maintain backward compatibility and interoperability was to address with a change to the on-disk-structure (ODS)

    Issue 2: Medium Encrypted Databases

    - Problem – Rare note corruption when updating a note, only occurs with Medium encrypted databases

    – Has existed since Notes/Domino began using Medium encryption

    – Not noticed because vast majority of databases have replicas and fixup would discard the corrupted note and next replication the note would come back in just fine

    Resolution – The fix for this issue would affect the vast majority of the data and hence there were security concerns it could potentially weaken the current Medium encryption strength.
    As a work around, Security team recommends customers go to ODS52 and upgrade existing Medium Encrypted databases to Strong

    If you are using encrypted databases either on Notes client or on Domino server you should update to the new ODS!
    But this requires to be on 9.0.1 code -- also on the client.

    You will have more likely encrypted databases on a client than on a server.

    IMHO On the server -- unless you have a password on your (and a tool to manage that on server startup) -- you should disable encryption.

    Without a password on the there is not much sense encrypting databases (and NLOs).

    But in case you need encryption you should update to ODS52 and switch to strong encryption.

    There is also another detail that John shows in his presentation.

    I have not seen any public information for the overhead that encryption has on CPU utilization. And this information is quite useful.

    NRPC run of Win2008 R2 Server 64-Bit @ 4000 Users, mail9 template
     Not Encrypted  35% CPU
     Medium Encrypted  39% CPU
     Strong Encrypted  48% CPU

    On a client this is not really much overhead -- unless you are on a Citrix server.

    But for a server this can be quite some overhead.

    If you don't want that additional overhead there is a fix that helps also with medium encrypted databases.

    But you will need to compact the database to the "new" medium encryption with ODS52 as well.

    This is clearly more a work-around and the security team recommends to upgrade to strong encryption if you can.

    Here is the way to enable the fix:


    - Next copy style compact of existing Medium Encrypted databases will be ODS52 with new Medium Encryption which has fix applied

    You can update all your medium encrypted databases to strong encryption leveraging copy style compact.

    The notes.ini setting you need for that is COMPACT_UPGRADE_MEDIUM_ENCRYPTION_TO_STRONG=1.

    This parameter can be quite helpful because it would be a manual step to migrate to strong encryption without it.

    And you should disable the parameter when you are done with upgrading all databases to strong encryption.

    On Notes clients databases are usually encrypted by default. The notes.ini setting LOCAL_DB_ENCRYPT_DEFAULT determines which encryption strength to use
    (0 = No Encryption, 1 = Simple Encryption, 2 = Medium Encryption, 3 = Strong Encryption)

    So you should have enabled the following for new databases that should be encrypted with strong encryption.



    Note: In case your workstation uses local disk encryption and/or you are using shared login there is also not much sense in encrypting databases.

    a.) Issue with large attachments

    There is an issue with attachments larger than 2 GB which is fixed in ODS52 in 9.0.1

    Fix for ZXZG85KJRK: Large attachments above 2 GB fail

    You need Notes 9.0.1 clients and Domino 9.0.1 servers in combination with ODS 52 to get this completely addressed.

    Details are available in the following technote:

    This issue is another reason to upgrade to the new ODS even this is an issue that might only hit you in very rare conditions.

    Additonal Note:

    There are also settings to log the database encryption used. They will report the current encrpytion level based on the settings the first time a database is opened.

    Administrators may now easily identify which databases are currently encrypted and the encryption level, by setting the following notes.ini variable


    Utilizes a Bit Mask

    1 is “Show Simple”

    2 is “Show Medium”

    4 is “Show Strong”

    To see all Encrypted Databases

    Simple, Medium and Strong (1+2+4 = 7)

    Set SHOW_ENCRYPTED_DATABASES = 7 in notes.ini

    When encrypted databases are opened for the first time - 0 to 1 transition, one of the following messages will be logged

    “Current encryption strength: SIMPLE - < absolute file path >”
    “Current encryption strength: STRONG - < absolute file path >”

    Legacy Medium encrypted database

    “Current encryption strength: MEDIUM - < absolute file path >”

    New Medium encrypted database with fix (+)

    “Current encryption strength: MEDIUM+ - < absolute file path >”

    As long as running Release 9.0.1, SHOW_ENCRYPTED_DATABASES works for all database ODS levels


    It makes sense to switch to the new ODS in some cases but you don't need to necessarily put it directly into your upgrade path -- at least on server side.

    This can be done afterwards with a copy-style compact that you should run once in a while on any database.

    DBMT in 9.0.1 helps you to keep databases defragmented -- check one of my recent blog entries for details.

    And in the same step you can upgrade the ODS if needed.

    On the server side there is most of the times really no reason to use encrypted databases in the first place.

    So as not mentioned in other postings about the new ODS52 the most important step is to migrate to the new ODS on client side.

    Unless you have users storing 2 GB attachments in their mailfiles...

    IBM Notes & Domino are not vulnerable to OpenSSL "Heartbleed" bug (CVE-2014-0160)

    Daniel Nashed  9 April 2014 21:41:51
    In case you are wondering. IBM Domino is not affected by the OpenSSL "Heartbleed" issues.
    Also Traveler (leveraging the Domino HTTP stack) nor the IBM HTTP Stack in Domino 9 on Windows does not use OpenSSL and is not affected.

    You still have to update your machines to a current OpenSSL package if you are running a 1.0.1 OpenSSL package.

    Here is the technote from IBM -->

    And here is some additonal information I got from my ISP -->

    You have to install a current version. on RHEL/CentOS for example 1.0.1e-16 is not affected any more.

    After updating the package you have to restart applications using it.

    -- Daniel

    Passing a document to an agent without saving it first

    Daniel Nashed  6 April 2014 13:43:43
    How cool is that new functionality introduced in 8.5.2.  Simple but important addition.
    Looks like this has been implemented for XPages but you can also use it in normal Java and LotusScript.
    Before you had to save a document before passing the document context to an agent.
    Now you can just pass a new in-memory document and you don't need to save it at all.

    This is really useful when passing parameters to and from agents that you invoke.
    For example if you want output for a Java agent that you need to call -- like in my case right now.

    Thanks to Michael Gollmick who pointed me to this documentionation! This really made my day. I wasn't aware of this new functionality!

    -- Daniel


    Release 8.5.2 introduces a new API for Agents to allow them run with a Document context that can be set by the caller, either an outer Agent or an XPage.

    The Agent.runWithDocumentContext() API runs an agent and passes a saved or unsaved in-memory document to the DocumentContext property of the called agent:

    New APIs

    The new APIs are :

    JavaScript (XPages) Agent.runWithDocumentContext(doc:NotesDocument) : void
    Agent.runWithDocumentContext(doc:NotesDocument, noteID:string) : void
    Java public void Agent.runWithDocumentContext(Document doc)
    public void Agent.runWithDocumentContext(Document doc, String noteID)
    LotusScript NotesAgent.RunWithDocumentContext(doc As NotesDocument, noteID As String) As Integer

    Getting the In-Memory Document

    The called agent can access the in-memory document via the existing API for accessing an in-memory document context. For example

    public Document AgentContext.getDocumentContext()
    Dim doc As NotesDocument
    Set doc = NotesSession.DocumentContext

    The document can be updated within the agent and when control returns to the XPage the updated values can be read from the document.

    Run as Web user

    Domino Server-based Agent code must run in an Agent with "Run as Web user" selected on the Security tab under Properties.

    Traveler 9.0. IF4 has shipped

    Daniel Nashed  31 March 2014 08:20:22
    Traveler 9.0.1 IF 4 has shipped end of last week. There are some important fixes on the server side and also some fixes in the Android client.

    After doing the update over the weekend I thought about building a small script to automate Traveler updates on Linux.
    First I thought it would make sense to have it in my start script but I am not sure about it.

    Silent install works like a charm. What do you think? Should I add a customizable script to shutdown, install, startup?
    It could be even interesting to directly copy the install files from a central location -- specially with larger Traveler environments.
    Or the changes could be pushed centrally and Traveler would just check if the files are there when the restartinstall command is executed.
    Just an idea not sure if this would be really something customers would like to run.

    Maybe I should start this up separately and not include it into the start script. But it would work in combination with the start script.

    What are you guys doing? Is someone already automating server updates on Linux in a similar way?

    -- Daniel

    IBM Notes Traveler 9.0.1 Interim Fix 4

    Release Date Component Build Levels Release Documentation
    March 24, 2014 Server
    Android Client
    9.0.1 IF4 Release Documentation

    APAR # Component Abstract
    LO78645external link Server Save and Security buttons are not enabled for Notes Traveler Web Administrator.
    LO78732external link Android Calendar entries may be missing in Agenda view after upgrading the client.
    LO78762external link Android Traveler client on Android may have connection issues if connected to a server.
    LO78786external link Server Line returns may be lost in Out Of Office message body.
    LO78825external link Server Corporate lookup may not work from Android device if message headers altered by Network.
    LO78876external link Server Plain text mail with pre tag may format too small on mobile device.
    LO78924external link Server Send mail to all invitees from iNotes shows on Mobile device as a Prevent Copy mail.
    LO78929external link Android Contact search does not work on Samsung Galaxy S3 device.
    LO78948external link Server Duplicate mail sends may occur due to device resending with different identifier.
    LO78965external link Server Some attachments may not download to mobile device correctly.
    LO78973external link Server Slow sync performance due to DB threads growing and/or long running PS or DS threads.
    LO78997external link Server Personal contact group may interfere with personal contact sync.
    LO79011external link Server Workaround to prevent BB devices from re-syncing all data when syncing To Dos and Mail.
    LO79012external link Server Delivery failure on send mail from device if domain is found to be empty string.
    LO79015external link Server Unable to send encrypted mail from device if the recipient does not have internet address defined.
    LO79041external link Android Notes Traveler To Do widget only displays one item on Android 4.4 OS.
    LO79070external link Server Unable to forward a calendar entry with no description from Windows device.
    LO79104external link Server Mime format mail sent from device will be converted to Rich Text format.
    LO79234external link Server Passcode History setting is applied differently on Apple devices than other mobile devices.
    LO79412external link Server Long running PS thread on server due to invalid filter window stored in database.
    LO79435external link Server Encoded attachments will not download to BB or WP devices.
    LO79465external link Android Unable to view some folders on Android OS 4.4 devices.
    LO79492external link Server Re-accept meeting on iOS may remove the event from the device, server not effected.
    LO79498external link Server Mail send from device may be sent twice if experience time out or connection drop during send.
    LO79499external link Server Traveler server slow to start if IPv6 addresses specified on host.
    LO79503external link Server Traveler shut down hang due to orphaned thread, may result in Domino server crash.
    LO79504external link Server Support sync of embedded icons with WP and BB devices.
    LO79516external link Server Domino API crash if attachment name greater than 253 characters on 32 bit system.