[Date Prev][Date Next]
Re: 8 hours tests ends with inconsistent DB.
You are of course right Quanah. ldapadd is surley not the right tool for
adding so much entries.
But this way has two advantages. First every application we use from day
to day use
the same mechanism and second I could "simulate" what happens to
it run's for days or month without restarting. I think adding an entry
is surley a "heavy"
operation compared with the other one's.
I'm now relatively sure that it is a memory leak. I've done the same
test today again and
after about 1 1/2 h. the SIZE of slapd was nearly 3 GB and RSS nearly 1
GB (which was
expected because of the kernel resources configured) according to "top".
This time I only
changed the kernel settings:
Theese settings are good settings for a Oracle 10g DB which could handle
quite a lot
queries on a FSC RX300 so I'm sure it must be enough for a simple
add's a lot of enties granted, but nothing impossible since it's the
only command running.
So I will use this kernel settings for my further tests.
What's interessing to note is that after sending slapd a kill -HUP the
SIZE (according to
top) went down slowly from 3 GB to 2,5 GB but the I/O was very high.
After not going
down anymore after 30 min. I killed slapd with kill -6.
I must verify tomorrow if the same happens when using slapadd. With the
help of one
of our programmers we tried to profile/debug slapd with memprof and
OpenLDAP didn't like that. It started but didn't opend a port on 389.
When I try to
attach with strace -p <PID> to slapd I get only one line of output
(while there is still
a lot I/O). Btw. before profiling we build a debug enabled version of
and OpenLDAP 2.2.13.
I've never used gdb. So I don't know how to set breakpoints. Our
that this wouldn't help very much since it's only a malloc() that fails
and we don't know
if the malloc that fails is responsable for the leak. Maybe we will try
to profile slapd
tomorrow again. Today wasn't that much time.
Maybe another thing which could be interesting. With "ulimit -v" I
forced slapd to
use only 400 Mbyte of virtual memory. I reduced also the cachesize in
set_cachesize 0 104857600 0
and cachesize in slapd.conf from 10000 to 100. The result was the same
the memory errors described appeared earlier.
That's for now.
Quanah Gibson-Mount wrote:
--On Sunday, June 13, 2004 3:43 PM +0200 RW <email@example.com> wrote:
For the problems I mentioned above it now really seem's to be my own
fault. For the case of the DB corruptions I could now reproduce it. In
this case I'm loading 500.000 entries with ldapadd into the
entry consists of about 23 attributes. After about 440.000 entries I
get the following messeages:
If you load the 500,000 entries via slapadd, do you see the same
thing? ldapadd as a process to add large amounts of entries isn't
generally advisable. Perhaps there is a memory leak somewhere. Have
you used gdb to set a breakpoint when this error occurs, and do a
backtrace? Some profiling of the running process to see its memory
usage? It is hard to tell here if the issue is in OpenLDAP or BDB.
Although I could see a memory leak causing a loss of resources...
Principal Software Developer
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html