[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: multiple writes in a short period cause db instability



Jonathan Higgins wrote:

working on a new build with the following software:
OpenLDAP-2.2.15
db-4.2.52 w/ patches
(other components are included but do not relate to the current issue:
cyrus-sasl-2.1.19, heimdal-0.6, openssl-0.9.7d)

build on the following hardware:
OS= Redhat 7.3
processor=2.8ghz
memory=1gig


some of the config info that I am using: DB_CONFIG: set_cachesize 0 300000000 10 set_lg_regionmax 262144 set_lg_bsize 2097152

partial slapd.conf(db portion):
threads 20
database        bdb
lastmod on
checkpoint 1024 5
cachesize 30000
idlcachesize 30000

some of these were guessed, based on hardware and software
configurations of others that I found on the net using similar
hardware.

Reads are super fast.. I mean stupid fast! .. never seen it perform
this well.. ever.
a db_stat -m in the database directory shows:
357MB 644KB 540B        Total cache size.
10      Number of caches.
35MB 784KB      Pool individual cache size.
0       Requested pages mapped into the process' address space.
35M     Requested pages found in the cache (100%).

Writes work, as long as its only a few ... but if I try to add 100 user
accounts using ldapadd -f file.ldif .. it bombs out fairly quickly, with
no obvious errors(loglevel -1).  The technical term "bombs" is defined
as, slapd stops responding, but is still running.

I need to run a db_recover on the db before slapd will come up
properly.

what other information would I need to provide so that I or someone
else can help me diagnose this problem.


Make sure you've built slapd with debug symbols enabled (compile with "-g") and the binary is not stripped. Run the command sequence that causes the hang, attach to slapd using gdb, and get a trace of all the running threads during the hang:
gdb> thread apply all bt


It sounds like you've hit a deadlock, but that's rather surprising as we've subjected this code to some pretty intensive write loads already without trouble. The trace should reveal what's happened.

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support