Re: (ITS#8823) LMDB on ramdisk

[hyc@symas.com]

luca.foppiano@inria.fr wrote:
> Full_Name: Luca Foppiano
> Version: 2.4
> OS: linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (37.169.254.52)
> 
> 
> Dear OpenLDAP team,
>      I'm using one of your component (LMDB) via a java JNDI bindings
> implementation (https://github.com/deephacks/lmdbjni) and I'm having an issue
> when I deploy my LMDB file on a tempfs filesystem in RAM.

The ITS is for bug reports, not for help requests. Use the -technical mailing 
list. Closing this ITS.
> 
> The issue do not occur when the LMDB files are stored on a "normal" filesystem.
> When the data is in the tempfs ramdisk all the allocated memory ends up being in
> the Dirty area (it has not been written back to the Filesytem).
> 
> Here an example using the ramdisk:
> 
> 7ce320000000-7cfc20000000 r--s 00000000 00:26 2459
> /ramfs/nerd/data/db/db-en/entityEmbeddings/data.mdb
> Size:           104857600 kB
> Rss:             1255680 kB
> Pss:             1255680 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:   1255680 kB    <---
> Referenced:      1255680 kB
> Anonymous:             0 kB
> AnonHugePages:         0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Locked:                0 kB
> VmFlags: rd sh mr mw me ms sd
> 
> and here an example without:
> 
> 7ca4fc000000-7cbdfc000000 r--s 00000000 fd:00 11154951
> /data/workspace/shared/nerd-data/db/db-en/entityEmbeddings/data.mdb
> Size:           104857600 kB
> Rss:              838124 kB
> Pss:              838124 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:    838124 kB       <----
> Private_Dirty:         0 kB
> Referenced:       764872 kB
> Anonymous:             0 kB
> AnonHugePages:         0 kB
> ShmemPmdMapped:        0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Locked:                0 kB
> VmFlags: rd sh mr mw me ms sd
> 
> 
> According to my understanding the memory is dirty when 1)there are open
> transactions, 2) the data has not been written back to the filesystem
> 
> What I don't understand is why there is a difference between filesystem and
> ramdisk?
> Is there any reason? The application (listed above) is not writing on the lmdb,
> but just reading (using reading transaction).
> 
> Thank you
> Luca
> 
> 


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/