Issue 3954 - slapd stability problems with bdb backend
Summary: slapd stability problems with bdb backend
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-08-19 13:21 UTC by adrian.gschwend@bfh.ch
Modified: 2014-08-01 21:05 UTC (History)
0 users

See Also:


Attachments
openldap-its-3954.tar.gz (3.67 KB, application/x-gzip)
2005-08-19 14:24 UTC, adrian.gschwend@bfh.ch
Details

Note You need to log in before you can comment on or make changes to this issue.
Description adrian.gschwend@bfh.ch 2005-08-19 13:21:38 UTC
Full_Name: Adrian Gschwend
Version: 2.2.27
OS: FreeBSD 5.3
URL: ftp://ftp.openldap.org/incoming/adrian-gschwend-050819.gdb-backtrace.log
Submission from: (NULL) (147.87.65.205)


slapd consumes 100% cpu and never quits after add or delete operations. I
described the issues detailed under the following URLs:

http://article.gmane.org/gmane.network.openldap.general/30543
http://article.gmane.org/gmane.network.openldap.general/30574
http://article.gmane.org/gmane.network.openldap.general/30604
http://article.gmane.org/gmane.network.openldap.general/30609
http://article.gmane.org/gmane.network.openldap.general/30624
http://article.gmane.org/gmane.network.openldap.general/30633
http://article.gmane.org/gmane.network.openldap.general/30644

Summary: Our DB contains about 4000 entries (bdb backend), it gets updated via a
meta database which connects to OpenLDAP via the perl-interface. During updates
it often happens that the slapd process hangs, which means it consumes 100% CPU
and does not quit anymore. No more queries can be done. On FreeBSD we have to
kill the process hard (-9) and run db_recover -c afterwards to get a clean DB
before we can restart it.

The problem also occurs on Debian 3.1 with slapd 2.2.23 (all precompiled debian
packages). We redid the database several times (slapcat/slapadd) with various
DB_CONFIG options, see the URLs above for details.

Note that the same add operation that fails works perfectly well once we
restarted slapd so the add itself must be ok. Also it was much harder to
reproduce the problem with a debug version of slapd (CFLAGS+= -g). But it still
does lock, see the gdb thread backtrace attached to this bug

We couldn't do a dbstat -CA yet during a lock, we will provide that as soon as
it locks up again.

If necessary we could provide an ssh (root) login to a test machine with gdb and
a test-environment. If you need that please contact me by email.

Comment 1 adrian.gschwend@bfh.ch 2005-08-19 14:24:31 UTC
The server just hung again, with a simple add operation. I took the 
chance to do the thread backtrace again and to get db_stat logs as well.

Attached is a tar.gz file that contains the following files:

openldap-its-3954/cn.bdb.stat
openldap-its-3954/dn2id.bdb.stat
openldap-its-3954/entryUUID.bdb.stat
openldap-its-3954/gidNumber.bdb.stat
openldap-its-3954/id2entry.bdb.stat
openldap-its-3954/memberUid.bdb.stat
openldap-its-3954/objectClass.bdb.stat
openldap-its-3954/uid.bdb.stat
openldap-its-3954/uidNumber.bdb.stat
openldap-its-3954/slapd.log
openldap-its-3954/db-backtrace2.log

slapd.log contains the add-entry that locked it, db-backtrace2.log is 
the gdb output.

cu

Adrian
-- 
Adrian Gschwend
System Administrator
Berne University of Applied Sciences
Biel, Switzerland
Comment 2 Howard Chu 2005-08-28 05:26:24 UTC
adrian.gschwend@bfh.ch wrote:
> Full_Name: Adrian Gschwend
> Version: 2.2.27
> OS: FreeBSD 5.3
> URL: ftp://ftp.openldap.org/incoming/adrian-gschwend-050819.gdb-backtrace.log
> Submission from: (NULL) (147.87.65.205)
>
>
> slapd consumes 100% cpu and never quits after add or delete operations. I
> described the issues detailed under the following URLs:
>   
The backtrace indicates a deadlock in the syncrepl persistent search 
code. This code is known to suffer from a number of deadlock issues in 
OpenLDAP 2.2, the only fix is to use OpenLDAP 2.3.

-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc
  OpenLDAP Core Team            http://www.openldap.org/project/

Comment 3 Howard Chu 2005-08-28 05:27:26 UTC
changed notes
changed state Open to Feedback
Comment 4 Howard Chu 2005-10-08 19:39:21 UTC
changed state Feedback to Closed
Comment 5 Howard Chu 2009-02-17 05:29:57 UTC
moved from Incoming to Archive.Incoming
Comment 6 OpenLDAP project 2014-08-01 21:05:55 UTC
no fix for RE22