Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

 
alt

Daniel Nashed

 

Updated: C-API - NIFReadEntries with SIGNAL_MORE_TO_DO in combination with NIFFindByName

Daniel Nashed  6 August 2020 05:35:08


Yesterday I have been debugging an application issue in my sync tool. It was a very strange behavior which finally lead me to the following.

This is only interesting for my C-API development friends in the community and if you use NIFReadEntries for more than 16351 entries you should check your application.


The examples in the C-API toolkit shows the normal navigation without searching.
The Tumbler is set to 0 and NAVIATE_NEXT with a skip value of 1 is used.
But that doesn't work the same way with NIFFindByName().
The comment from Markus for this post pointed me to the difference. And this makes it even more confusing, but explains what is going on.


So I am updating this blog post. The example in the documentation is right. But it works differently when you use NIFFindByName().


If you start without a search operation and set the following, you are actually before the first element and need skip to the first entry:


 CollPos.Level = 0;

 CollPos.Tumbler[0] = 0;


For a find operation you are at a distinct current position where you want to get a current entry found.

So in that case you need a different way to navigate thru the collection.

By the way if you start at this position without a skip no documents are returned!


Here is what will not work if you used "find" and want the documents at the current position:


a.) Skip 1 for all including the first loop --> this will let you miss the first entry

b.) Skip 0 for all entries -> this will result in duplicate processing of the last element from the previous round --> happened in my case


The right way looks like the example appended. It will handle the first entry correctly and have no duplicate processing for the last entry in a round.

In my case double processing of the last element was problematic.


The following section in the doc explains what is happening:


" After a call to NIFReadEntries(), the collection position (parameter 2) points to the last entry read. You can use repeated calls to NIFReadEntries to move incrementally through a collection.  However, upon repeating the call to NIFReadEntries(), use one of the NAVIGATE_NEXT_xxx skip navigate flags (the third parameter) and use a skip count of 1L (the fourth parameter) to advance the collection position past the last one read so that it begins reading with the next entry in the collection.  "



If you use this function in a similar way in your code you should double check.

The point I did not take into account was reading without searching. My issue was after NIFFindByName()
.

There are examples in the SDK showing the correct way as well.
That shows that the examples are an important part of the SDK documentation ..

-- Daniel


Example code with the important parts highlighted


 error = NIFFindByName(

   hCollection,            /* collection to look in */

   "key",                  /* string to match on */

   FIND_CASE_INSENSITIVE,  /* match rules */

   &CollPos,               /* where match begins (return) */

   &dwMatchSize);          /* how many match (return) */


   AddInLogMessageText("%s: Entries matched :%u", 0, gTaskName, dwMatchSize);


bFirstLoop = TRUE;


do {

  error = NIFReadEntries(

    hCollection,               /* handle to this collection */

    &CollPos,                  /* where to start in collection */

    (WORD)(NAVIGATE_NEXT),     /* order to use when skipping */

    bFirstLoop ? 0L : 1L,      /* number to skip */

    NAVIGATE_NEXT,             /* order to use when reading */

    0xFFFFFFFF,                /* max number to read */

    READ_MASK_NOTEID,          /* info we want */

    &hBuffer,                  /* handle to info (return)   */

    NULL,                      /* length of buffer (return) */

    NULL,                      /* entries skipped (return) */

    &dwNotesFound,             /* entries read (return) */

    &wSignalFlag);             /* signal and share warnings (return) */


 
  bFirstLoop = FALSE;
...


Comments

1Karsten Lehmann  06.08.2020 12:54:57  Updated: C-API - NIFReadEntries with SIGNAL_MORE_TO_DO in combination with NIFFindByName

Unfortunately the C API does not contain any example how to properly handle/recover from SIGNAL_ANY_CONFLICT, e.g. when the view index changes while reading.

HCL has worked on this topic over the years and added NIFFindByKeyExtended2 that supports reading the first 64K of lookup data after the key lookup in an atomic operation. Last time I checked the C API documentation, NIFFindByKeyExtended2 contained the wrong description text.

And the undocumented NIFFindByKeyExtended3 even provides a callback to return the data while the view index is locked to ensure stable data.

That method should be documented and exposed in my opinion.

2Markus Sablatnig  06.08.2020 14:25:46  Updated: C-API - NIFReadEntries with SIGNAL_MORE_TO_DO in combination with NIFFindByName

Hey Daniel,

this is very interesting, you made us double check our code :)

Markus Seitz and I went through it, wondering why it works, because we are actually using the call like the example shows.

What might be different is that we are setting the collection position Level and Tumbler[0] to 0 first, and then do a read with 1 as SkipCount. This delivers the first entry in the view.

Here is a snippet of the (actual) code from our GetFirstEntry function in our view wrapper class. This function is called after NIFOpenCollection to start reading entries.

mcp_current.Level = 0;

mcp_current.Tumbler[0] = 0 + ul_offset;

WORD w_signalFlags = 0;

STATUS st_retVal = NOERROR;

st_retVal = NIFReadEntries( mhc_entryCollection, &mcp_current, NAVIGATE_NEXT, 1, NAVIGATE_NEXT, mul_entryLimit, READ_MASK_NOTEID + READ_MASK_NOTEUNID + READ_MASK_NOTECLASS + READ_MASK_SUMMARY, &mh_viewBuffer, NULL, NULL, &mdw_entriesFound, &w_signalFlags );

if( st_retVal != NOERROR )

throw ND::Exception( __FUNCTION__, L"Could not read first entry", st_retVal );

For example, we use this to read the first three entries in a view (mul_entryLimit is 3, ul_offset is 0), and then (if needed) we call it a second time with ul_offset set to 3 to start on the fourth entry. This works.

The only case where we use 0 for SkipCount is when we use NIFReadEntries after NIFFindByName. In this case NIFFindByName sets the collection position to the first found entry, so you need to start with that.

Interesting case, I'd be curious to what is different between your and our usage here to get so different results.

-- Markus

Links

    Archives


    • [HCL Domino]
    • [Domino on Linux]
    • [Nash!Com]
    • [Daniel Nashed]