Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

 
alt

Daniel Nashed

 

Notes/Domino 10 FT Index Auto Update

Daniel Nashed  30 January 2019 07:39:20
There is a new feature in Notes/Domino 10 which is enabled out of the box.

When you do a FT search, the backend checks if the index is up to date and will index up to 200 documents before the search is executed.
If there are more than 200 documents to index, it will queue an update request for immediate action.

This happens on client and server based databases and the index update is done in the backend where the database resists.

I really like the feature but on my client with a larger local mailfile (8 GB) this causes some delay in some cases where the dialog was displayed for a couple of seconds once in a while.

If you have the same experience I would be interested in your feedback (here or by mail).
Time it takes, how often you see detail, database size, number of documents, version, client/server...

If this happens too often that you have longer delays you can disable the auto update via notes.ini

FT_SUPPRESS_AUTO_UPDATING=1 (dynamic on client and server)

-- Daniel
Comments

1Sean Cull  30.01.2019 10:26:23  Notes/Domino 10 FT Index Auto Update

It would be useful if there was a notes.ini setting for the "200" threshold value although this is from the perspective of Domino web applications rather than mail.

It is not uncommon to modify >200 documents with back end processes and some may contain attachments.

Would attachment scanning slow this down ?

This is in the context of a user simply wanting a filtered list of documents. In many web apps this is achieved via a FTI search although the user does not think in terms of "search".

2Marcelo Castro  30.01.2019 11:00:42  Notes/Domino 10 FT Index Auto Update

In my case, I needed the result as the return page after the update.

In XPages it is taking 50 seconds for the page to display.

3Stuart Bogom  30.01.2019 14:14:13  Notes/Domino 10 FT Index Auto Update

I really like this feature. I have a local mail file, and I use FT search on it frequently. Before version 10, if I was looking for a recent message, I had to remember to update the local index manually in order to include newer messages in the search. Often I forget to do it and I struggle with the search until I remember the index is out of date.

My mail file has about 40K docs. the auto update time is likely proportional to the number/size of documents being indexed, so it is hard to generalize the time it takes. I'd say it is commonly between 2 and 15 seconds. I don't mind the delay, but I could see an argument for a prompt box that asked the user if they wanted to update the index at this time or not - if I was not looking for a recent document, I would probably defer the update to later. Even better if the client would queue a deferred update if I didn't want it done immediately in the foreground.

In a non-mail application, I might want the developer to be able to turn this on/off at the database level.

4Reid Canavan  30.01.2019 17:32:44  Notes/Domino 10 FT Index Auto Update

I think this feature should be a property setting on if this should be enabled or not. Certain applications would benefit from this feature whereas others would not.

5Torben Busch  31.01.2019 9:02:48  Notes/Domino 10 FT Index Auto Update

Same issues here from time to time. I just had to wait about 45 seconds for the index update to update 4 documents in a local 3GB/4k docs ODS53 database having high cpu usage meanwhile. There are some encrypted fields in those docuements. Maybe that makes a difference?

6Heinz Mathys  01.02.2019 7:10:16  Notes/Domino 10 FT Index Auto Update

Same issue here. Time to wait vary from some seconds to more than a minute. Time to wait doesn't change after updated ODS.

On user perspective, this interrupts your work and it bothers me.

My opinion: It is a good idea to add new indexed documents in search results, but it should be 'never' interrupt your work.

Idea: Run the update process in the background always - but display in the full text search bar a progress information (as today in 10.0.x the annoying dialog box). As soon as all documents are indexed, display additional found documents according to the search criterias (or add continuously each indexed document in search results if appropriate).

7Sjef Bosman  27.02.2019 17:53:17  Notes/Domino 10 FT Index Auto Update

Thank you for the info on how to deactivate the updater! Our XPages application uses FTSearch all over the place, and there are regular updates everywhere. We used to have, in R9, our own extension to FTSearch, in that we do an FTSearch first followed by a standard search in the non-indexed documents. We had hoped V10 would solve this issue, so we could leave the additional search out, but instead it complicates matters for us, and quite a lot!

We get lots of messages in our logs:

Error full text indexing document NT00000000 (rc=3859) Database is currently being indexed by another process

With your setting we're able to stop the real-time indexing and use our own solution again. Thanks!!

8RenĂ© van der Weide  01.03.2019 9:06:47  Notes/Domino 10 FT Index Auto Update

We have a many customers with huge databases.

The delay after a search really is an issue, cause the documents are updated all the time.

If I set the config FT_SUPPRESS_AUTO_UPDATING=1

does it mean the indexes off the server aren't update at all anymore, do I need to update the indexes manually?

Any comments would be great. My thanks in advantage.

9RenĂ© van der Weide  01.03.2019 10:56:09  Notes/Domino 10 FT Index Auto Update

In the meantime I've tested this setting:

Setting FT_SUPPRESS_AUTO_UPDATING=1 works like a charm!

The searching of the databases is quick again and the

database setting:

"update frequence (servers only)" immediate

still updates the index when the server has the time for it.

Thank you very much for the solution.

10Ty Rex  22.06.2019 20:11:08  Notes/Domino 10 FT Index Auto Update

Lifesaver - we host an app for lots of clients and many of their databases are massive. One particular client complained their app was running slowly, but it took us a long time to pin it down to the fact it was only the search screens that were the problem. Over 60 seconds for a page refresh in many cases!

Searched for solutions and found this blog entry - switched on this INI setting and now everything is running sweetly again. Thank you for posting :-)

Links

    Archives


    • [HCL Domino]
    • [Domino on Linux]
    • [Nash!Com]
    • [Daniel Nashed]