[Date Prev][Date Next]
Re: clarifications on cachesize, preferred db, et. al. from admin guide
In the email that started this thread, I asked several questions about
clarifications or updates to the Admin Guide or man pages.
Quanah's first response is below with my status updates inline.
I received further clarifications from Marco Pizzoli and Howard.
I have status updates on three of the items and a draft I would like
feedback on for one of the other items.
In regard to: Re: clarifications on cachesize, preferred db, et. al. from...:
--On Thursday, April 07, 2011 6:46 PM -0500 Tim Mooney <Tim.Mooney@ndsu.edu>
- Admin Guide, section 18.104.22.168. The tuning chapter is a godsend and
Every single one of my index .bdb files is of type Btree, though, not
Hash. Is that section of the docs outdated, and all indexed attributes
are now in Btree databases (for back-bdb and presumably back-hdb), or am
I fundamentally misunderstanding what the index-related cache calculations
They are btrees. You can find this is corrected in the FAQ:
I submitted a patch last week to fix up that section of the Admin Guide,
it's ITS #6906.
If you agree that it would be useful to explicitly list which backends
would block the use of slapd-config and someone can provide me with the
list of blockers, I would be happy to file an ITS and provide a patch to
the current docs to spell things out. I personally think it will help
adoption of slapd-config.
Check against OpenLDAP 2.4.25. I believe nearly all backends and overlays
support slapd-config now.
With Marco's tip and a bit of subsequent reading, it looks like only meta
and sql backends block the use of slapd-config. I've updated the intro
section of both chapter 5 and chapter 6 to make it more explicit that
* chapter 6 is about slapd.conf, which is on its way out
* chapter 5 and chapter 6 show two different but roughly equivalent
ways to configure slapd.
* only slapd-meta and slapd-sql block the use of slapd-config
* you should never edit the slapd-config LDIF files directly, you
should use ldap commands instead.
The patch for the updates to the intro for chapters 5 & 6 is ITS #6905.
- man page for slapd.backends(5). The man page entry states that
bdb is the preferred backend. I've seen enough hints and comments on
the mailing list to suggest that it will eventually be supplanted by hdb.
How soon is that going to happen (2.5?), and is it worth mentioning that
hdb is as good as bdb now and will be the new preferred backend soon?
Again, I'll submit the ITS with the doc patch if it's worth making that
assertion in the docs now.
It was done with OpenLDAP 2.4. The man page needs updating.
There are two separate patches as part of ITS #6904 that correct this.
- Admin Guide, chapter 21. The tuning chapter doesn't mention the
potential benefits of using an alternate memory allocator on Linux, as
Quanah clued me in to on the mailing list last month. Should it? If
people feel it would be worthwhile to mention, I would be happy to write
the first draft and supply the patch in an ITS.
Probably. It doesn't just apply to Linux. It applies to all *nix systems.
I haven't touched this yet, probably won't get to it for a week or so.
- Admin Guide, chapter 21. The tuning chapter doesn't mention the
potential benefits of using sysv shared memory vs. mmap'ed files on
some platforms. Should it? Same offer for documentation patch applies,
though I expect this one will need more feedback from the experts.
Do you mean the shared memory keys, or something else?
For tuning information on shared memory keys, I added that to the guide
as section 21.4.4, basically right after the section on the IDL cache
and before the section on threads.
The draft I have for shared memory keys is below. I'll submit it as a
patch once I've integrated any feedback. I'm of course most interested
in making certain the information provided is actually correct, but any
and all feedback is welcome.
21.4.4. Berkeley DB shared memory regions: mmap(2) vs. System V shared
Berkeley DB uses shared memory regions to hold the per-process and
per-thread shared information for each environment. On most modern
platforms, the default for shared memory regions is to use
filesystem-backed memory, via the mmap(2) system call. It's this use of
mmap(2) that results in the creation of the __db.00N files in the database
directory for back-bdb or back-hdb.
Another option for Berkeley DB shared memory regions is to instead use
System V style interprocess communication (IPC) shared memory. On some
platforms, shared memory may be much faster than using mmap(2) for
Berkeley DB shared memory regions. This is true on Solaris 10 and earlier,
as well as some releases of the 2.6 Linux kernel.
To use a shared memory key with a particular bdb or hdb backend, specify
the shm_key configuration option with an integer key id.
If you have multiple bdb or hdb databases and you wish to use shared
memory for each of them, you should specify a different key id for each.
Because the key id you specify is actually the base segment ID that is
passed to the Berkeley DB set_shm_key function, there will be additional
shared memory segments created beyond the key id you specify.
# ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x7402ce23 3473408 root 600 4 0
0x7402cdf9 4161537 root 600 4 0
0x00000000 4521986 root 644 80 2
0x7402cdf8 4128771 root 600 4 0
0x00000000 4554756 root 644 16384 2
0x00000000 4587525 root 644 280 2
0x00000015 6094854 ldap 600 24576 1
0x00000016 6127623 ldap 600 30105600 1
0x00000017 6160392 ldap 600 536870912 1
0x00000018 6193161 ldap 600 2359296 1
0x00000019 6225930 ldap 600 811008 1
0x0000001a 6258699 ldap 600 32768 1
In the example output shown above, shared memory key 0x00000015 owned by
the ldap user corresponds to the shm_key 21 specified in the slapd
configuration. The other subsequent shared memory segments owned by the
ldap user correspond to the additional shared memory regions created
automatically by Berkeley DB.
If you elect to use a shared memory key with one or more Berkeley DB
databases, be certain the System V shared memory on your server has
suitable settings for the number of shared memory segments, the maximum
amount of memory usable system-wide for shared memory, and especially the
maximum size of any one shared memory segment. These tunables are
frequently called shmmni, shmall, and shmmax, respectively, but vary by
platform. Consult the documentation for your operating system for
information on how to configure shared memory.
Tim Mooney Tim.Mooney@ndsu.edu
Enterprise Computing & Infrastructure 701-231-1076 (Voice)
Room 242-J6, IACC Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164