Updated! Limit the system cache on Win64 for Domino32

Daniel Nashed  1 July 2009 23:04:50

Note: The referenced technote confused me and my tests have been misleading. After checking with a customer, re-reading the TN and speaking with IBM I updated the blog post. Sorry for the confusion!


Domino 32bit on a Windows 64bit platform can address up to 4 GB of memory because the full 32 bit address space can be used.
At the same time the limit for the file-system cache on Win64 has been increased from 1 GB to 1 TB.

Because of the way Windows uses the cache memory it competes with application memory like Domino shared and local process memory. In some situations when a lot of databases are opened with file-caching Domino can run low on memory.
Currently the only way to limit the cache memory is to use a 64bit Windows API call SetSystemFileCacheSize() which can not be called directly from Domino as a 32 Bit application. Therefore Domino ships a small 64 Bit helper binary "cacheset".

But Domino 32bit (in contrast to Domino 64bit) by default does not set the limit in 8.0.x. There is a notes.ini parameter OS_CACHESET_ON=1 Which will not work without additional fixes in 8.0.x and 8.5.
In D8.5 the setting is enabled automatically without the notes.ini setting (like in native 64bit).But there are pending fix to get this working (SPR #DADS7NCLU2).
TN #1391477 describes the required tuning and fixes detail but the current version of the TN at least confused me.

You would have to request a hotfix to get this working or use the cacheset utility in a program document as a workaround.
If you are on 8.0.1 you should update at least to 8.0.2 to have the cacheset tool fixed (there is a calculation error for the memory percentage used for machines with larger amount of physical memory).

If you are on 8.0.2 or 8.5 there is no need to request a hotfix just for this issue. I misinterpreted the current version of the TN and had the impression that those fixes are already in those releases (and my tests did hit the wrong binary directory on my 64bit test environment). I would use the work-around to use cacheset manually or use the following work-around in D8.5.

In Domino 8.5 there is a work-around to fix the last pending SPR #DADS7NCLU2 (planned for D8.5.1). You can work-around this issue by renaming "cacheset.exe" to "ncacheset.exe" (add the platform specifier for Windows).

Once you have either a hotfix for the mentioned SPRs (see TN) for your release or the work-around in place you can use the same configuration as described for the native 64bit platform and the cacheset tool is automatically started.
The default setting is 30% of the memory and you can change this using notes.ini MEM_FSCachePercentMem=n (in percent).

Without using this notes.ini parameter you have to manually start the cacheset tool shipped with Domino 8.x.

There is a new ARPA describing also a debug parameter (DEBUG_OS_CACHESET=1).
This debug parameter is new in 8.5 and helps you to check in detail how the server calculates the cache size and how it invokes the cacheset tool.
Here is a sample output when I set the cache to 10%.
 
 [0A3C:0002-0798] OSAdjustFileSystemCache> PercentPhysRam: 10, FileSystemCacheMax KB: 104755 
[0A3C:0002-0798] OSAdjustFileSystemCache> Has 64-bit processor architecture 
[0A3C:0002-0798] OSAdjustFileSystemCache> 
[0A3C:0002-0798]        osvi.dwOSVersionInfoSize: 156 
[0A3C:0002-0798]        osvi.dwMajorVersion: 5 
[0A3C:0002-0798]        osvi.dwMinorVersion: 2 
[0A3C:0002-0798]        osvi.wServicePackMajor: 2 
[0A3C:0002-0798]        osvi.dwBuildNumber: 3790 
[0A3C:0002-0798]        osvi.dwPlatformId: 2 
[0A3C:0002-0798]        osvi.szCSDVersion: Service Pack 2 
[0A3C:0002-0798] OSAdjustFileSystemCache> Valid OS platform - 2 
[0A3C:0002-0798] Enabling cacheset launch based on ini. 
[0A3C:0002-0798] Launching cacheset with cmdline '-s 0 104755' 


It is a an important requirement that you limit the file-system cache on Windows 64 and you should test if it works by running cacheset -g after you started the Domino server independent on how you set the cache (either invoking cachset via program document or using the work-around).

Sorry for the confusion in my first version of this post and I hope the posting is now more clear and helps to understand the back-ground.

-- Daniel

• Comments [2]