Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

    Traveler 9.0.1.18 with new Security Mode for Mail-File Access

    Daniel Nashed  22 June 2017 10:07:40
    Traveler 9.0.1.18 comes with a couple of minor fixes and a big change in the way Traveler Server access mail-databases.
    In 9.0.1.15 IBM introduced a new check if the Traveler server is listed in Trusted Servers (Server Security Tab) to show a warning if not.

    Now we know what IBM was preparing for. The server now acts as the user instead of the server. That's only possible if listed in Trusted Servers.

    You still need the Traveler server to be listed in the ACL of the mail databases. Trusted Servers means that to server itself can make the session on a database look like it would be the user session.
    But the remote server still needs access to the database.

    I have done a quick test. Without the proper ACL an error is logged and also the user status reports an error.

    The IBM Traveler server encountered an internal error validating your User ID CN=John Doe/O=Acme/CÞ.  Please contact your server administrator.
    [CN=notes.acme.de/OU=Srv/O=Acme-Net, mail/johndoe.nsf] is not reachable, status(0x4ac) "Unexpected internal error".

    The new method for accessing mailfiles solves a couple of limitations. See details from the documentation below.


    -- Daniel

    What's new?


    Traveler Server Run as User


    Starting with IBM Traveler 9.0.1.18, the run as user feature will now be enabled by default. When running as the user, the Traveler server will access the user's mail file as the user ID instead of the server ID. This feature resolves several long standing issues with accessing the user's mail file as the server ID, including:

    • Honor ACL controls on mail file and corporate lookup for the user.
    • Prevent event notices and automated responses from being sent from the server ID.
    • Prevent the server ID from being assigned as the owner of the mail profile when there is no owner defined.

    Note:
    For run as user feature to function properly, the Traveler server must be listed as a trusted server in the user's Mail Server document. To disable run as user, set this notes.ini parameter: NTS_USER_SESSION=false



    APAR # Abstract
    LO90096 Info update continues to be ghosted on mobile device after the event is processed.
    LO91797 Empty comments displayed on iOS native Calendar application when event processed in iNotes.
    LO91836 Invalid this and future reschedule generated by iOS native Calendar application.
    LO91875 Ghosted event not displayed on mobile device.
    LO91956 Maill attachment does not sync to mobile device when contains angle brackets < and >.
    LO91997 IBM Traveler web administrator may show iOS Verse 9.4 device as not supporting security capabilities.
    LO92010 Better handling of special character in mail header fields.
    LO92080 Ignore a reply message with out a valid action defined.
    LO92085 Hard delete processed notices vs soft delete to prevent from filling up trash folder.
    LO92209 Second meeting room may be lost if event updated from mobile device.
    LO92210 Unable to turn off iOS Verse application password via Domino policy document setting.
    LO92257 Two instances of a previously processed event may show on mobile device if the daylight savings rules change for the time zone.
    LO92303 SQL Syntax error adding index TSGUDTSTAMPCREATEIDXSQL9 on DB2.




    Notes Client/Windows Crash with Windows 10 Creators update

    Daniel Nashed  3 June 2017 16:36:03
    Just got that question today at DNUG. There is an issue with the Notes Client with the current Windows 10 Update - aka Creators Update (Build 1703).

    According to the responsible person who is at DNUG today, this happens because of changed Windows graphics APIs.
    IBM is working on a fix which will be available in FP9.

    FP9 will also have full High Resolution support! We saw a demo with FP9 which really looked great!

    Here are the two relevant SPRs:


    SPR LHEYALMCEP : Domino Designer crashes the OS after Windows 10 Creators update [For Designer BSOD issue]
    SPR AYAVALMCJK : Windows 10 Creators update and OS crashes while using Notes/Designer

    IBM said that you should remove the following registry setting to avoid the blue screen after the Notes Client start.
    (Updated: By mistake I wrote notes.ini parameter but correct is registry setting which might not exist).

    -- snip --

    Delete this registry entry and the crash should go away

    PageHeapFlags, VerifierFlags from
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\notes.exe  

    -- snip --

    Ram also said that the problem is not impacting all configurations and the other bug is not happening very often.

    So I hope that if you updated your Windows 10 environment already, your are not running into this issue or the temporary work-around helps you.

    Another Update I got in comments  of my posts is that this happened often with customized welcome pages.

    Note: I updated the blog post and got a replication/save conflict. So I deleted and added the post again because the blog template does not like replication/save conflicts.
    Some comments might be lost but I added them to the post anyway. Thanks for your feedback!

    --. Daniel

    Security Bulletin: IBM Domino TLS server Diffie-Hellman key validation vulnerability (CVE-2016-6087)

    Daniel Nashed  1 June 2017 07:27:46
    There is a vulnerability in the TLS stack which could lead an exploit which could lead a less secure connection.
    The good news is that the fix is already included in FP8. So you should upgrade to 9.0.1 FP8 if you have a public facing Domino Server with HTTPS.

    See the details and reference below.

    -- Daniel

    A vulnerability in the IBM Domino TLS server's Diffie-Hellman parameter validation could potentially be exploited in a small subgroup attack which could result in a less secure connection.
    An attacker may be able to exploit this vulnerability to obtain user authentication credentials.

    Vulnerability Details

    CVEID: CVE-2016-6087 / DESCRIPTION: IBM Domino could allow an attacker to steal credentials using multiple sessions and large amounts of data using Domino TLS Key Exchange validation.

    CVE-2016-6087 is tracked as SPR# DKEN9WGMYE.


    http://www.ibm.com/support/docview.wss?uid=swg22002808

    Important Security Fix for IMAP

    Daniel Nashed  22 April 2017 11:13:16
    In case you are running IMAP on a server that is reachable over the internet you should look into this fix ASAP.

    It might not be that critical for internal services.

    See details about this vulnerability here --> http://www.ibm.com/support/docview.wss?uid=swg22002280

    All versions of Domino are affected!


    NIFNSF Supported Maximum Size above 64 GB! -> 1 TB is officially supported!

    Daniel Nashed  21 April 2017 19:02:31
    After getting that question offline and having a discussion on my blog, I checked with IBM if they plan support NIFNSF sizes above 64 GB.
    Since it is kind of a database container and needs a database handle someone could think that the maximum limit is also 64 GB.

    That would give us at least 64 GB room for the NIF index -- which would be already a big improvement.


    But from what I recall from some comments at Connect some years ago the maximum limit was not around 64 GB when they designed it.


    On the other side it is difficult to test and such large view / folder index sizes. And you will not run into many situations where you need such a large size.


    From what I heard, IBM is about to publish a supported official size for the NIFNSF indexs that is far beyond 64 GB.
    Stay tuned for the official statement. For now I can tell you that it will work above the 64 GB limit!


    On the other side a databases with that index size will reach other limits like application responsiveness issues because of the nature of complex views with many documents.


    But it is good to know that it was designed to support larger sizes and also the counters in the database will continue to work as I have tested earlier for DAOS.


    Once we get an official statement I will update my post and share the link.

    25.04.2017 Update:

    The technical documentation has been updated -->
    https://www.ibm.com/support/knowledgecenter/SSKTMJ_9.0.1/admin/admn_moving_views_out_of_databases.html

    Here is the official statement for NIFNSF .NDX File size :-)

    ".NDX files have a limit of 1 TB. The real determination of how large the views can grow is based on application responsiveness or if any other limits are reached before the 1 TB .NDX file limit is reached."


    -- Daniel

    Disclaimer Attachment Issue not yet fixed in IF1

    Daniel Nashed  14 April 2017 20:28:40
    As Rob Kirkland commented in one of my last blog posted, the fix in IF1 does not solve the iusse.

    We both checked with IBM and got the reply that the SPR just changes back the default and disables the change introduced in FP8 for Google calender integration.

    IBM is working on a fix hopefully makes it into FP9.

    So for now you should keep the notes.ini Parameter MIMEDisclaimersNoEncode=0 disabled.

    Thanks to Rob to bring this up!

    -- Daniel

    TPONAKFJLP

    After upgrade to FP8, with disclaimers enabled, .pdf attachments have content-transfer-encoding of binary  



    Get Notes FP Version in @Formulas

    Daniel Nashed  12 April 2017 08:19:07
    In C-API and Lotus Script, Java developers the version information already shown for each FP.
    For example Lotus Script returns the full version string with session.NotesVersion.

    But if you want to check the version information in @Formulas @Version will still return 405.

    There is a new optional parameter which returns the Feature Pack version.
    So you use @Version to check the version and if it is 405 you check @Version(1) which will return 8 for Feature Pack 8.
    It is used in the new mail-template and will be updated for every new FP.

    @Version(1) returns 0 for older versions (I checked with a Notes 7 client to verify).

    -- Daniel




    Notes/Domino 9.0.1 FP8 IF1 released

    Daniel Nashed  11 April 2017 07:06:32
    Notes and Domino 9.0.1 FP8 IF1 has been shipped and there is also a separate fix for iNotes.

    All those IFs have the same version number but contain different SPRs!

    The most important IF is for the Domino Server. It fixes the disclaimer issue I reported before.
    And also the performance fix for the new feature NIFNSF which was introduced in in FP8.

    Be aware that NIFNSF is a server feature and not a client feature. It requires translog enabled on your server!
    Translog is not officially available on Notes clients.




    Domino

    YJIAAJ4MXV
    Add new JNI wrappers to jNotes        

    NNUZAG78H3
    the tag includes some illegal characters, will be output not commentted as before.        

    TPONAKFJLP
    After upgrade to FP8, with disclaimers enabled, .pdf attachments have content-transfer-encoding of binary        

    SWASAKELQ8
    Perf issue w/ nifnsf enabled.        

    LHEYAKALAH
    Form Validation not working on an XPage        


    Notes

    LHEYAKBJSQ
    SSJS editor stops working in an NSF with a managed bean        

    LHEYAKALAH
    Form Validation not working on an XPage        

    LHEYAKBJSQ
            SSJS editor stops working in an NSF with a managed bean        

    MDOYAKFPE2
    XPages iNotes calendar control icons do not display the correct hover help text


    iNotes

    JJCMAKV3DT
    iNotes "Starts With" window does not show when pressing any letter or number key


      Current Information about NIFNSF

      Daniel Nashed  1 April 2017 00:42:31
      Domino 9.0.1 Feature Pack 8 introduced "NIFNSF" which allows to separate the view/folder index into a separate file.

      Let me try to summarized my current experience from my tests and from the field.


      There are multiple benefits moving the index to a separate file.

      1. Backup Storage Reduction

      First of all having the index in a separate file reduces the amount of data that you need to backup.
      For mail databases the index is around 10%. If you have DAOS enabled from the remaining data it's about 30%.
      So the backup time and backup storage in total is reduced.

      2. Size Limit of the data above 64 GB

      The total size of a NSF is 64 GB. With DAOS enabled you can increase the logical size of a server based database by moving attachments to the DAOS store.
      For DAOS you can have external attachments up to 1 TB. Beyond that size the internal counters might overflow.

      But in some cases you still need more that 64 GB for NSF data and the view/folder indexes. With NIFNSF the limit is the 64 GB data in the NSF without the view/folder index.

      3. Performance

      NIFNSF is intended to deliver better performance than having all data in a single NSF file.

      There is a current performance issue. For mail databases there should not be big difference.
      But for more complex views in applications the performance with NIFNSF might be not as good as without it.
      Tests have shown that it can take double the time.

      There is a pending fix that might be delivered with an IF for FP8 which should bring back the performance to almost the same as without NIFNSF.

      And for FP9 there is optimization planned to have better performance for concurrent operations. Those changes did not make it into FP8.


      So for now you might want to wait at least for an IF before enabling NIFNSF for complex applications.



      -- Storage Location for NIFNSF --


      There are multiple options to configure where to store the .NDX files which store the NIF data.
      What you choose depends on your environment,platform and your requirements.

      a.) Have NDX files stored next to your NSF files

      b.) Have NDX files stored in a separate folder in the data directory

      c.) Have NDX outside the data directory on the same disk

      d.) Have NDX stored on a separate disk

      There are no one size fits all recommendations. It really depends what storage situation and platform you are running on.

      If you can for example on Windows I would store NDX files at least outside the data directory.
      On Linux often without a new mount point you might not be able to move the NDX files outside the data directory, because often the data directory is a mount.

      If you need to increase your storage anyway because the NSF disk is full, having a separate disk (most of the times virtual disk) makes sense.,
      This is a good way for a clean new allocation and it will separate the I/O operations.



      -- Enabling NIFSNSF on your Server --


      Translog

      First requirement is that you are using transactions logging. Circular translog is perfectly OK for that.
      And translog is general recommendation for Domino anyway! For stability, fault recovery and also for performance!

      ODS 51 or higher

      You will need at least ODS 51 for NIFSNSF. But I would recommend using ODS 52 for all databases on your server.

      notes.ini Create_R9_Databases=1 will ensure the ODS is updated the next time you run a copy-style compact.

      Notes.ini Settings

      There are a couple notes.ini settings. The most important setting  NIFNSFEnable=1 enables NIFNSF on your server.

      To store the NDX files in different locations (see options above) you can leverage NIFBasePath=path depending on your preferences.

      In addition if you want all new databases to be NIFNSF enabled there is another notes.ini setting CREATE_NIFNSF_DATABASES=1 which will ensure that all new databases are automatically NIFNSF enabled.



      -- Enabling NIFSNSF on a Database --


      Once your server is NIFNSF enabled you can start enabling NIFNSF on your databases via compact.

      Please take care not to run the compact operation on all databases. We have seen customers who enabled NIFSNSF also on the DAOS catalog -- even the special database has no views.

      I would currently start with mail databases only! And you just specify the right mail directory.

      The normal recommendation is to use

      compact -c -NIFNSF ON mail/

      This will enable the feature and also move existing indexes out of the NSF.
      But if the database is in use, the copy-style compact will not be possible.

      Instead you could enable NIFNSF on databases without copy-style compact and have a copy-style compact later on with either compact -c or leveraging the DBMT tool which you might have configured anyway.

      Once the database is on ODS 51 or higher and NIFNSF is enabled new indexes are created in the NDX file.
      But only the copy-style compact will move the views to the NDX file.



      -- Checking NIFNSF --


      You can check which databases are already NIFNSF enabled and there is also a way to see the size of the NDX. But this command shows all databases.

      The most useful commands shows all NIFNSF enabled databases.

      show dir -nifnsfonly

      show only NIFNSF enabled databases

      show dir -nifnsf

      show all databases with NDX files also



      -- Maintaining Databases with NIFNSF enabled --


      I have done some tests. Only with copy style compact the NDX will be compacted.
      Many customers are still using compact -B for an inplace, space reduction compact.

      There are also other reasons to leverage DBMT which is using copy style compacts and does use space pre-allocation to ensure the NSF is not allocated fragmented.

      The copy style compact will also shrink the NDX if needed. A compact -B did not free any space from the NDX file in my tests.

      However the free space in a NDX file should be still be reused if released from a purged view/folder index during normal runtime.



      -- Tuning for NIFNSF --

      A NDX file is a NSF file. The index data needs a container. Therefore if you are running a large server you have to make sure you have sufficient dbcache entries, because the NDX file will also need a cache entry.

      By default the dbcache handles depend on the size of the NSF Buffer Pool (which is 1024 MB for 64bit). The number of cache entries is around 3 times the buffer pool size in MB.

      3000 DbCache entries should be OK for most servers. But if your server is already on the limit you have to increase the limit.

      Here are the relevant server statistics from a current customer example:

              Database.DbCache.CurrentEntries = 4498
              Database.DbCache.HighWaterMark = 4500
              Database.DbCache.MaxEntries = 3000
              Database.DbCache.OvercrowdingRejections = 15220

      Your CurrentEntries and HighWaterMark should be alwass below the MaxEntries.
      And the OvercrowdingRejections should be always zero!

      So in this case it would make sense to increase the number of cache entries to 6000 via:

      notes.ini NSF_DbCache_Maxentries=6000

       

      CREATE_R9_LOG is not a valid notes.ini parameter and does not exist!

      Daniel Nashed  24 March 2017 11:29:50
      After hearing this question twice a week I think it is time for a blog entry.

      There is a notes.ini parameter for the different ODS formats in different releases.
      The latest one you should use is Create_R9_Databases=1 to create databases with ODS 52.

      ODS 52 is needed for local databases which are encrypted (there was an underlying ODS issue that has been addressed in ODS 52).
      Also for hew new LargeSummary (16MB instead of 32KB per document) you need to be on ODS 52.

      But there is no new notes.ini parameter for the optimized aligned translog extends.

      The only parameter valid is still CREATE_R85_LOG=1.

      I have dumped the string resources from the current binaries in 9.0.1 FP8 to double-check.

      There is no change for translog notes.ini parameters in Domino 9.0.1!

      Sounds like someone posted on his blog or added it to his presentation and others copied from there.

      -- Daniel


      libnotes.so:Create_R9_Databases
      libnotes.so:Create_R85_Databases
      libnotes.so:Create_R8_Databases
      libnotes.so:CREATE_R85_LOG

      • [IBM Lotus Domino]
      • [Domino on Linux]
      • [Nash!Com]
      • [Daniel Nashed]