Re: ldap_pvt_mp_ multiprecision stuff

[Howard Chu <hyc@symas.com>]

Pierangelo Masarati wrote:
> On Wed, 2006-02-08 at 22:25 -0800, Howard Chu wrote:
>   
> > I'm seeing a significant amount of CPU use after a search completes, 
> > apparently just from updating the op counters. The CPU really should not 
> > be 50-90% busy after processing a single query, after the client has 
> > already disconnected. I think it might be better to prefer long long's 
> > to the bignum or gmp stuff. Are we really concerned that any 64 bit 
> > counter is going to overflow? Even on 10Gbit ethernet it will take ~544 
> > years to overflow a 64 bit byte counter, and that's assuming a 
> > continuous output stream at 100% bandwidth utilization.
> >     
>
> The reaon that stuff was introduced is essentially the byte counter
> would wrap in hours with a 32 bit long.  I guess this can be disabled,
> or made a configure option that's "no" by default.  How can you explain
> that excessive load?  GMP is supposed to be pretty efficient (I can't
> say anything about BIGNUM).
>   

It turns out I was looking at a pathological case; the process had 
already grown to 2.4GB of memory (only 2GB physical present) due to the 
BDB entry cache growing out of control. The BIGNUMs need to malloc 
additional space to store data, and most of the CPU seemed to be 
consumed in libc malloc. I suppose we might be able to avoid some of the 
problem if we can force the BIGNUMs to preallocate some space, but that 
will only help when multiple operations occur on a single connection. In 
the common case of a single Bind followed by a single Search, it's a 
noticable hit.

--
 -- Howard Chu
 Chief Architect, Symas Corp.  http://www.symas.com
 Director, Highland Sun        http://highlandsun.com/hyc
 OpenLDAP Core Team            http://www.openldap.org/project/