[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Re: LMDB usage on windows - to much memory needed


On Thu, 29 Jan 2015, Frank Offermanns wrote:


thanks for the information, but unfortunately this information is not
helpfull for me.

Is there a way to configure LMDB on Windows 32 bit, so that a database can
be larger?
Or is it mandantory to use 64 bit with lmdb (at least on windows file
systems) when handling larger data?

You need 64 bits on any kind of OS to support multi gigabyte databases.

No 32 bit OS will allow a single process to memory map much more than 1 or perhaps 2 GB of address space.



Von:    Kristoffer Sjögren <stoffe@gmail.com>
An:     Frank Offermanns <Frank.Offermanns@caseris.de>,
Kopie:  openldap-technical@openldap.org
Datum:  28.01.2015 16:32
Betreff:        Re: LMDB usage on windows - to much memory needed

Regarding size.

"If an application specifies a size for the file mapping object that is
larger than the size of the actual named file on disk and if the page
protection allows write access (that is, the flProtect parameter specifies
PAGE_READWRITE or PAGE_EXECUTE_READWRITE), then the file on disk is
increased to match the specified size of the file mapping object. If the
file is extended, the contents of the file between the old end of the file
and the new end of the file are not guaranteed to be zero; the behavior is
defined by the file system. If the file on disk cannot be increased,
CreateFileMapping fails and GetLastError returns ERROR_DISK_FULL."


On Wed, Jan 28, 2015 at 3:27 PM, Frank Offermanns <
Frank.Offermanns@caseris.de> wrote:

we would like to change our backend from hdb to mdb.

I did some initial tests and found out the following:
When setting maxsize to 1.4 GB, the size of the database on the file
system is about 1.4 GB. I thought I read that the maxsize has nothing to
do with the actual physical size needed. (If I have only 10 MB of data my
database should be about 10 MB).
Even worse is the fact, that slapd.exe then needs about 1.5 GB virtual
size memory.
And if I use 32 Bit slapd.exe the process will crash when reaching 2 GB
virtual size. (as every 32 bit process will do)

Now my conclusion is, that the things described in the mdb paper are only
valid for unix/linux, because windows uses another memory system. Is this
correct? Or is there a way I haven't found to configure OpenLDAP on
windows so that it does not need so much RAM.
The size on the disk does not matter for us, but since we use only 32 bit
slapd.exe the RAM does matter.

Best regards,

Christian Kratzer                   CK Software GmbH
Email:   ck@cksoft.de               Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0       D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9       HRB 245288, Amtsgericht Stuttgart
Mobile:  +49 171 1947 843           Geschaeftsfuehrer: Christian Kratzer
Web:     http://www.cksoft.de/