Re: Overlays: freeing resources on a hijacked Operation

[David Hawes <dhawes@vt.edu>]

On Thursday 02 September 2004 05:30 am, Howard Chu wrote:
> David Hawes wrote:
> >  I have recently finished an overlay that I call 'addpartial' that
> >  intercepts an add operation, determines if the entry to be added
> >  exists, and if so, determines if anything has changed and modifies
> >  the appropriate attributes. If the entry does not exist, the add
> >  proceeds as expected. If the entry does exist yet is identical to
> >  the existing entry, the add will silently return LDAP_SUCCESS. The
> >  intent of this overlay is to do partial record replication from a
> >  master to its slaves, while upstream replication data (from our
> >  Registry, in this case) can be full records. Before this overlay, we
> >  were doing deletes and adds for each entry changed in our Registry
> >  (ugh!), so we were effectively slowing down our directories. The
> >  overlay can do a steady 400-500 records compared per second on
> >  entries with approximately 20-40 attributes (very rough numbers). I
> >  know other people faced with this problem effectively diff entries at
> >  another level--this overlay simply does that at the LDAP level. I
> >  provide this information in the hope that others will find this
> >  useful. I'd love to polish it up and contribute it, if so.
>
> Sounds good. This is exactly the behavior that I want to add to the
> syncrepl consumer, which currently does a full Delete/Add on modifications.
>
> >  My apologies for that long-winded description. Now on to my
> >  question.
> >
> >  After the overlay has finished doing what it needs to do, it sends
> >  LDAP_SUCCESS to the client and then returns. If I return
> >  LDAP_SUCCESS, my server runs out of memory at around 1 million
> >  entries (starts eating up swap and then segfaults). However, if I
> >  return SLAP_CB_CONTINUE, the server does not run out of memory
> >  (tested up to 20 million entries). I assume that something in the
> >  original add Operation needs to be freed, but I am not sure what.
> >  Could anyone point me in the right direction for this case? The
> >  overlay works fine when I return SLAP_CB_CONTINUE, but there are some
> >  misleading errors left in the logs (68: entry already exists) as it
> >  tries to complete the original operation.
> >
> >  I appreciate any help or guidance for this. Also, I have to say I
> >  really love the concept of overlays.
>
> Looking at the current HEAD code, I don't see anything obvious. Since
> you didn't mention what version you're working with, we have no point of
> reference from which to start guiding you. Without knowing what your
> code looks like, there's no way for us to tell what is being leaked, and
> guessing won't be very helpful.
>
> I suggest you run your code under a malloc debugger and let it tell you
> exactly what is being leaked. valgrind on linux is really good for this
> sort of thing. If you're using Solaris you can use my FunctionCheck
> 1.5.3 package. http://highlandsun.com/hyc/fncchk153.tgz
>
> FunctionCheck works on Linux also, of course. Which to use depends on
> the problem you're chasing; valgrind can work with your existing
> binaries but since it runs your code using a CPU emulator, it is
> hundreds of times slower than normal execution. FunctionCheck requires
> you to recompile your code specially with gcc, but it runs on the real
> CPU so it's only marginally slower than normal. When the problem you're
> chasing can be reproduced quickly and easily, valgrind is probably the
> easier approach. When the problem occurs rarely, and requires a lot of
> operations to reproduce, it may be faster overall to use FunctionCheck.

I had been testing my overlay on 2.2.13 and have been testing with 2.2.15 this 
week.  The problem I reported is almost completely nonexistent with the new 
version.  Memory use reaches a peak and then seems to level off.  The same 
operations on 2.2.13 use an inordinate amount of ram and then the server 
eventually segfaults.  I still may take your suggestion and try out a malloc 
debugger just to appease my curiosity.  For now I'll just upgrade to 2.2.15.  

Thanks,

dave