[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: slapd memory and performance problems


Thanks for the information.  I'm not familiar with GSSAPI accept to
understand that it's a generic security services API.  I'm not inclined
to hack the Exim code to incorporate another API.  I have heard that
performance improvements can be gained by using unix sockets, but I'm
not familiar with the use of those either and am looking for more info
on that topic.  But your performance seems enviable to me.  I've not
heard any complaints about the thread implementation in this version of
RedHat, but I'll check into that.

You say you created an eq index for the maildrop attribute that I assume
belongs to your main entry which is accessed by the uid filter.  I was
wondering about creating an eq index on my mailquota attribute, but it
seemed unnecessary since the attribute is part of an entry already
accessed by uid.  Can you comment on your reasoning for adding an index
for that attribute if you don't explicitly search for an entry by
filtering on that attribute?  I may be incorrect in my thinking here as
I am not really a db person by vocation or by nature.



-----Original Message-----
From: Quanah Gibson-Mount [mailto:quanah@stanford.edu] 
Sent: Tuesday, April 01, 2003 6:04 PM
To: Mike Denka; OpenLDAP-software@OpenLDAP.org
Subject: Re: slapd memory and performance problems

--On Tuesday, April 01, 2003 5:25 PM -0800 Mike Denka

> After experimenting with threads (max number and idletimeout),
> to v.2.1.16 and applying Howard's new cvs version of tpool.c, I am
> experiencing some performance issues with slapd.  The most immediate
> problem I see is that the response time for a query under peak loads
> very slow.  The most obvious oddity is that the slapd threads are
> growing in size over time.
> I am currently running four load balanced mail servers which query the
> ldap server once for each incoming message to extract the user's mail
> quota.  The user's entry is indexed by uid and a user defined
> in the user's entry stores the mail quota.  At peak times I am getting
> combined total of between 1 and 4 queries per second for these quotas
> from the mail servers.  A search on this server during peak load times
> takes several seconds (10 to 20) to complete.  If I try to add
> additional loads on this same server (e.g., radius authentication
> maybe one query every two to 4 seconds), queries start timing out and
> applications (like radius) making those queries start to fail.
> Upon starting slapd, each thread has a size of about 4800KB.  This
> thread size seems to double after 8 hours or so and grows to about
> Also, the total memory available on the box seems to degrade over
> not just from the increased thread size, but from what appears to be a
> memory leak somewhere since just restarting slapd does not increase
> available amount of system memory.  Only a reboot recovers system RAM.
> I'm running RedHat 8.0 on a platform with an Intel 1G processor and
> 512MB of RAM.  Processor idle time never dips below 90%, no swap space
> is ever used but memory, as I said, diminishes over time.  However,
> performance stays constant and doesn't change as memory degrades
> memory drops to below 5M or so, at which time the server must be
> rebooted).
> Each query for a mail quota does a simple bind via the ldap:// url
> the entire dn as the base and '(objectclass=*)' as the filter.  This
> base and filter is composed internally by Exim (our MTA).
> I am considering using Unix sockets as was suggested in an earlier
> thread but I don't understand that approach enough to engage it yet.
> Could anyone comment on the performance of this box?  Is 1 to 4
> queries/second the best I  can expect on a system of this kind?  Any
> ideas what might be causing the memory in the slapd threads to grow
> time?  Slapd is the only user application running on this machine.  I
> can't be sure that slapd is causing the memory leak, but I don't see
> this problem on any of our other similar machines.  Are there any
> issues involving memory leaks in this latest version of slapd?  Also,
> could someone comment on the efficacy of using unix sockets and point
> to some docs somewhere explaining them?


I can't comment on the redhat portion of this, as we are running under 
Solaris 8, but I can offer the following:

We began a production rollout of OpenLDAP today, using 2.1.16 with the 
tpool.c patch.  So far, only 1 mail router has been cut over to one 
machine.  It is doing about 10,000 queries/hour.  So far, I have seen no

degradation of performance, and the memory footprint is remaining

A big difference between our setups, is that we have sendmail using
to bind, as we hacked our sendmail to use GSSAPI authentication.  In
testing of these systems, I consistently saw 66 queries/second as the
for the particular hardware we are using, and that was leaving the test 
running overnight, with over 1 million queries made in about 6 hours 
against the machine (18 querying clients).  The hardware is:

SunFire v120 with 650MHz CPU & 4GB of memory.

I will keep you posted if we encounter any problems with threads.  Maybe

Redhat's thread implementation is buggy.  I have had performance
using Solaris 9, due to a change in Sun's thread implementation.

For indexing, we index the filter (essentially uid) and the attribute we

are searching for (maildrop) with eq as well.  Our searches continue to 
take less than a second to complete.

Software wise, we are running:

OpenLDAP 2.1.16 + tpool.c patch
Cyrus-SASL 2.1.12
BDB 4.1.25
OpenSSL 0.9.6i


Quanah Gibson-Mount
Senior Systems Administrator
ITSS/TSS/Computing Systems
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html