Notes Domino Platform - What is Notes/Domino and what is special about it?
Daniel Nashed – 10 January 2019 21:26:42
In the last couple of month I did workshops with customers to check where they are with their Notes and Domino environment and what they are going to do in the future.
My part was to explain what Notes/Domino really is and why this is so different than other application platforms.
Based on my workshop and the presentation I put together a summary. If you want a copy of my German workshop slides, drop me a note.
The following gives you a high level overview of Notes/Domino which describes the basic architecture and some of the most important core features.
It is not a complete list of functionality nor describes all the details but it gives you an idea about the concept behind it.
There is an interesting new Forrester study "The Total Economic Impact™ of IBM Domino" which provides some additional background.
https://www.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=01022901USEN
My summary is more a technical summary that gives you an idea about what is great about the platform.
And this is not including the new exiting new development options introduced in Notes/Domino 10 which we should look into separately.
-- Daniel
Introduction
When Notes was first released most of the core functionality was already in place and the core architecture never had to change because it was from ground up designed in a way that makes it very flexible, robust and secure.
Looking into the first memorandum written by Mitch Kapor you can already see the most important aspects and the core idea behind Notes --> https://app.box.com/s/lv7lm64d1gbvkwyzxdct9sqzavot2mh6).
Later there was a very interesting book published by Lotus Oktober 2000 which is still available. It describes the architecture and core components in a way, which has never been published before or after.
The foreword already describes the core idea behind the product in a few words.
„In 1989 Lotus Notes® made the world a little smaller.
Since its first release, Notes™ has enabled teams of people to work together, even when separated by great distances or when individual contributions are made at different times.
.. Notes continues to evolve innovative, new capabilities that support its initial core concept: Collaboration made possible through shared, secure, online databases."
(Source : http://www.lotus.com/ldd/doc/uafiles.nsf/docs/inside-notes).
Today we are used to work in shared teams in different countries and timezones but when Notes was first designed there wasn't a wide spread network that everyone could use.
I recall the times where our servers had modems and later ISDN cards to communicate. Times changed but the concept behind it is still modern and very powerful.
And when it comes to an integrated, rapid application development platform there is stil no comparable product on the market.
What is IBM Notes
When we look into the basic concepts behind Notes/Domino we see a very flexible, document oriented. none-relational databases system (today you would name it "NoSQL").
It is a Rapid Application Development & Deployment platform with integrated development and run-time environment.
Lotus integrated email/calendar functionality into the base environment on same infrastructure and APIs
Thus extendibility, customization of functionality and design is part of the core product.
Notes and Domino has support for @Formula, Lotus Script/Com, Java, Javascript and C.
This includes integration options like own servertasks or even Extension Managers, client extension points
All of this has been designed into the product since day one.
The application is document centric with flexible fields and field types per document (including Richtext and MIME).
So you can have any number of fields of any type in any document. There is no fixed structure that limits you to a certain structure.
It is not the best choice for large number of often changing records/documents that have more relational structure.
But you can integrate via ODBC, JDBC and today with flexible Web-Services/REST services.
Notes/Domino separates UI/Design/Code and Data which is also still a modern approach.
Leveraging simple but very flexible and powerful architecture: e.g. "forms, views, folders, agents" you can write and extent applications flexible and in rapid application development style.
The NSF storage (Notes database) holds the design and data at the same time. Both use separated but very similar "note-types" in the database.
This makes easy to replicate among servers and clients. And this also helps to maintain the design leveraging "templates".
Notes/Domino supports XML since Version 5 and allows to export/import documents and also design elements!
The own client/server communication protocol "NRPC" allows transparent client/server access which allows you to work with the same database either on-line on a server or on a local instance "replica" of the database.
For that reason Notes/Domino has build-in replication for Notes databases between clients and servers with full local support for databases.
At the time Notes was first releases there wasn't any central directory.
So Notes/Domino invented an integrated directory containing information for users, groups, servers and other information.
Domino was one of the first products to support external access and sync via LDAP. And there are also other interfaces to access directory data.
Today integration with central directories like Active Directory is important. There are multiple options to integrate and sync and also for Single Sign On (e.g. SAML or Kerberos).
Out of the box you can open, edit and save back attachments with external applications like Office and other applications in any document.
In addition the Notes client offers optional MAPI support for Office applications for mail integration for local applications.
One feature that has been taken for granted by many users and developers is the integrated, performance optimized full text index for server and also local databases!
You can use it in any database and you can use it on Lotus Script. Java and also C-API level!
The tight integration with email and the private public key infrastructure allows to build flexible work-flow applications where you can digitally sign documents.
Tight integration means that all functionality is available in one application in contrast to other applications that need separate products to implement comparable functionality -- if other products support the same functionality at all.
In many cases you end up having multiple other application platforms depending on the solution you choose.
Development Platform
Since Version 8/8.5 Notes integrates with the Eclipse development and run-time environment.
This offered new integration points and application options while keeping all existing functionality from the C/C++ application platform.
At the time it was introduced this was a good idea. But using Eclipse was quite challenging specially from performance point of view.
Today we see a move away from this integration and we will probably see other client options soon.
The development platform (Admin/Design Client) has low footprint on normal desktop with local or server based applications.
One simple but very powerful part of the application platform is the formula engine which can be used in forms, views and folders to calculate information that is displayed or stored in the application.
This is is comparable to the @formula uses in spread sheet applications (no wonder looking into the roots of the inventors working on Lotus 1-2-3, Lotus Symphony and earlier products).
This is on one side a very simple but also a powerful and very effective way to process and present data.
Full Upward Compatibility
From the very beginning Notes was design to be upward compatible!
So all functionality remains supported in a newer version without much modification needed.
A new version might come with a new On Disk Structure (ODS) which brings new database back-end functionality.
Clients and servers encapsulate the ODS details for the local NSF so even earlier clients can access servers with later versions.
Once you updated the client and server code you can simply update the ODS with standard tools/maintenance compact servertasks.
Updates -- even major releases -- are simple without data migration or export/import!
Custom applications usually work without or with little modification on new major release.
Standard unmodified applications are "just" updated via updated templates by the standard design servertask.
The client UI is consistent cross versions, even when new functionality is added.
But on the other side this means UI/design enhancements for your existing applications and application maintenance is often neglected because everything just works.
In fact many applications worked for years without any change. That's why nobody touched them even for UI modernization.
JVM Support
It took quite a while until IBM decided to implement JVM 1.8 in Domino and they implemented just in time before Oracle stopped supporting JVM 1.6.
But we now have JVM 1.8 support since 9.0.1 FP8 (runtime) 9.0.1 FP10 (compile time)
An IBM JVM Team provides JVM based on Oracle JVM which is updated in every FP to the latest version available.
Notes/Domino Security
Flexible databases access with multiple levels is integrated part of the database design since day one.
- Manager/Designer, Editor/Author, Reader, Depositor with additional roles
- Access on document-level with simple but effective mechanisms like Reader/Author fields with names, groups and roles
- Encryption on field/document level and local database level
- Execution Control List (ECL) for detailed controlled for signed application code!
- Integrated Public Key Infrastructure (PKI) document/email encryption based on RSA technology
- Native S/MIME integration (public certs need deployment process, most customers use gateway encryptions)
- Certificated based authentication via Notes.ID
- Access via Certificate and optional password checking
- Since Domino 8.5 simplified Notes.ID management via "ID-Vault"
- HTTP via user/password with native SSO (LTPA token, also used in other IBM products)
- Optional Authentication via X.509 Certificate
- Optional Authentication with SAML 2.0
Current Encryption Standards
- HTTP/ SMTP, LDAP, IMAP, POP3 use current encryption like
- TLS 1.2 with ECDHE Ciphers (since 9.0.1 FP4) and SHA256 certificates
- SSL-Labs „A“ Rating
- Notes NRPC Port Encytion (9.0.1 FP8)
- Encryption using AES-128 CBC / AES-128 GCM / AES-256 GCM / RC2-128
- Ticket Algorithm HMAC-SHA 1 - HMAC-SHA 512 (Default: HMAC-SHA 256)
- Notes data encryption AES 128 or AES 256
HTTP/HTTPS Support
Any database can be presented in web. Web representation with current technologies for example via XPages "Responsive-Design" with responsive design, infinite scolling etc
- Hybrid applications with Web/Notes Client/mobile access
- New modern web interface for mail.
- Verse on Prem (VOP) does currently use a simple add-on installer and leverages the existing infrastructure
- REST Interfaces for read/write with specific rest services for mail, calendar etc
- Strategic use of REST with Open Standard documentation with new functionality in the latest Feature Packs
Domino 64bit Support
Domino supports native 64bit on all platforms.
64bit has benefits specially in virtualized environments because it allows better optimization if all components leverage 64bit architecture.
Larger memory support is also helpful for special applications servers like Traveler, specific HTTP work-loads or the IBM Enterprise Integrator.
Domino as an application usually does not need more than 4 GB RAM for a normal server configuration.
IBM put a lof of energy into I/O optimization. There fore not more RAM is needed by the application.
You would still can benefit on larger memory configurations where the OS will leverage the file-system cache (specially in virtualized environments).
But the existing code is performance and resource optimized and does not need more RAM on it's own like other applications do.
64 GB Database Limit
64 GB remains the physical limit of a Notes/Domino database
But there are many options to optimize databases to either
a.) reduce the storage needed
b.) move some data outside the physical NSF file
Document Design/Compression → Reduces Design and Document data (everything that is not Index or attachment) up to 50%
DAOS (Domino Attachment and Object Services) = External Storage for "attachment" data (database objects are move to the DAOS storage)
Leveraging DAOS officially supports 1 TB per NSF database. Usually for mail-databases attachments would cover 70% of the mail-data.
NIFNSF = Put view/folder index to a separate file (.ndx) → Maximum size of the .ndx file: 1 TB!
So when leveraging those technologies the physical limit of the NSF does not matter any more on server level.
In addition a single NSF file would only hold data for a single mail user or a single application. So the 64 GB is per Notes NSF.
On client side the 64 GB limit still applies because the optimization is not available in Notes 9.0.1.
But on the other side more than 64 GB of data in a single application on client side might not be the best idea.
Domino offers integrated archive functionality which would help to reduce data on the archive mail database or other application databases.
For Notes/Domino 10 the maximum physical database size is increased to 256 GB.
Database Optimized Storage
Leveraging Database Compression, DAOS and NIFNSF also helps your to optimized performance and helps you to reduce storage and backup space dramatically!
With Document/Design Compression you save additional storage space.
It is transparent to the application with up to 50 % of data and design reduction just by enabling the feature via standard compact
DAOS
Domino Attachment and Object Services (DAOS) moves attachment data to a central store with one file per attachment (configurable threshold usually 256 KB generating NLO files with the attachment) where it is stored deduplicated.
In addition to deduplication and NSF data reduction you can store data on a NetApp or other technologies which allow deduplication on storage container level to further reduce storage needs.
DAOS also allows to reduce backup costs leveraging online, incremental backup. The NLO is only stored and backuped once per server!
NIFNSF
A quite new feature introduced in Domino 9.0.1 Feature Pack 8 allows to store the index in a separate file for performance reasons and also to further reduce backup costs.
The separate file on disk allows optimized locking and the NIF index does not need to be backuped.
Domino Clustering
Application level Active/Active Clustering with separate, independent server instances with separate storage
No SAN mirroring or special storage environment required.
The Notes Client is "cluster aware" with automatic fail-over and fail-back
A cluster replicator servertask is used for almost real-time replication.
Downtime is minimized downtime because maintenance is performed on each server separately and Domino clusters are operating completely independent!
A single databases can also be taken off-line. The key here is that Domino as a "service" is hight-available and the individual servers can be down for planned maintenance.
Domino Transaction Logging/Backup
Transaction Logging is fully integrated into Domino NSF infrastructure since Domino 5.0
Ensures database consistence and performance for each NSF file.
Fast restart without fixup after failure (similar to journal file-system on Linux).
Translog functionality has been derived from DB2 functionality.
Dedicated backup API interface used by 3rd parties for on-line backup without downtime
Full backup/Incremental backups with Point in Time recovery per NSF file
Notes Domino Diagnostic & Fault Recovery
Outstanding, fully integrated diagnostic functionality for crash, hand, memory leaks etc.
NSD (Notes System Diagnostic) provides detailed diagnostic information for IBM to pinpoint and fix issues.
Fault-Recovery provides fast restart after server crash
Automated diagnostic collection (ADC) after restart collects diagnostic data from client or server.
Mobile Mail/PIM Access
„Mobile First“ is one of the important IBM strategic goals.
IBM Traveler as a separate, easy to install solution leverages the standard Domino server stack.
Mobile access via iOS, Android, BB10 for Mail, Calendar, Contacts (and Todo)
Native Verse Client (App) for iOS and Android via SyncML
iOS Mail App Support via Active Sync
BB10 Support via Active Sync
My Conclusion
Today mail is commodity. Notes and other products support similar features compared to other platforms.
Replacing Notes as a mail-platform alone does not provide much benefit and has migration risks and costs.
Specially coexistence during the migration or with existing applications can be tricky.
The key benefit for Notes/Domino as a platform is unmatched rapid application development.
There is no easy way to migrate applications and if you end up replacing just part of the applications you have to support and pay for both platforms for a longer time period.
Alone the full integration of application with mail, calendar and contacts and workflow functionality is difficult to replace by one single product.
Customers migrating most of the times have to move applications to multiple platforms and write new applications.
Notes/Domino is stable, integrated, performing, high available platform.
It has good TCO as an application platform compared to other platforms. There is no single platform providing all functionality used by customers today.
In addition today offers Domino REST APIs to be more flexible and used in a micro-services landscape and can be even better integrated with other enterprise applications.
- Comments [13]
![[HCL Domino]](../lotus-domino.gif){kind=small}
![[Domino on Linux]](../domino-linux.gif){kind=small}
![[Nash!Com]](../nashcom.gif){kind=small}
![[Daniel Nashed]](../daniel-nsh.gif){kind=small}