[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: big directories (more than 1M entries)

Howard Chu wrote:
>> [...]
>> noticed that i have to restart (or at least stop) slapd (same for
>> slapadd, as i notice) before db_archive shows me unused logs.
> You need to set a checkpoint value in slapd.conf. See slapd-bdb(5).

Ah, ok. I Should read more Manual Pages, they seem to be much more 
uptodate than the Adminguide. Great stuff to read.

I encountered a Problem with that setting.
In slapd.conf i had "checkpoint 100000 7200" and first it seemed to work 
good and as i expected. After writing about 10 log files, db_archive 
showed me 9 old files to remove.
But at that point ldapadd (running with a big LDIF file) suddenly slowed 
down horribly. CPU statistics showed me 30% "system" load, "user" load 
was just 3% to 5% anymore. (Whereas usually during population there's 
about 15% "system" and 30% "user".)
I think this might be due to a bug in the checkpoint feature, 'cause 
from this point db_archive showed me every logfile quite after it 
wasn't needed any more. (Instead of waiting for another 100MB to be 
written, as i expected.) I think slapd was checking whether to release 
transaction logs after each write or something like that..

>> Next i noticed that slapdindex grows quite large when
>> indexing the whole backend, i.e. it grows larger than 256MB [...]

> This shouldn't happen. There have been other reports of memory leaks
> on AIX but we have never been able to track any of them down.

Ah, ok. Thought this is usual behaviour.
I experienced lots of memory problems with older Releases of Openldap 
(1.2.X (multithreaded), 2.0.X) with ldbm backend, but 2.1.8 (meaning 
slapd of OpenLDAP 2.1.8) seems to be rock solid so far, at least with 
BDB backend.

Ill try to track that down as soon as I have time, I've to get the 
installation working first..

>> Running slapindex should have created all needed indexes, shouldn't
>> it?
> As long as you uncommented them again before running slapindex, yes.

Thought that. (And actually I really forgot to uncomment them the very 
first time.. :o)

Ah, got it. I forgot to index objectclass attribute. Stupid me.

Hmm, I'm nevertheless wondering why slapd didn't use the uid index at 
all then. Is that some kind of optimization? 'Cause we have to pull out 
every Entry anyway to see whether it's a referral object, we're 
checking for uid=foo at the same time?

>> I vaguely remember someone on the list mentioned a possibility to do
>> this without referrals? With a glue backend or something like
>> that?
> See the "subordinate" keyword in slapd.conf(5) and in the archives of
> this mailing list.

Yes, that's it, thanks.
And I have a question about that too..
Manpage says, someone should use the same rootdn for each of the 
subordinate backends. I remember having troubles with a setting like 
that, 'cause AFAIK a bind() operation is always done against one 
backend, isn't it? And so if I'm binding with the rootdn of the 
backends, I'll always bind to the first one, and so I'll always be 
anonymous for the other ones. Did I mess something up here?

thanks a lot,
I had a letter in the post today. It said 'Gas Bill'.
It sounds a tempting offer.
                -- Alan Cox