Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases

[hyc@symas.com]

Henrik Bohnenkamp wrote:
> On Mon, Jul 15, 2019 at 02:26:59PM +0100, Howard Chu wrote:
>>
>> Fyi, on our problematic test database with 11M entries and 3.7M aliases, a search with -a always , starting from the
>> DB suffix, took 4 minutes without this patch, and 1235 minutes with this patch.
>>
>> Needless to say, that's not looking good. Still checking other test cases.
> 
> Interesting, so the behavior is reversed now :-). I assume you have
> found an alternative approach to solve the problem. That's fine with
> me, I want the problem solved, not my patch integrated.  I'm of course
> interested in how you do it. Surely you did not get the 4 minutes with
> a stock 2.4.48 slapd?

For this size of DB we needed the ITS#8977 patches to accommodate larger IDLs.
(I used 24 bits for IDLs, 16.7M slots)
Also at this size, the IDL processing itself is the main bottleneck now. We would
need to switch to bitmaps or trees to avoid this bottleneck, but that's also a
much larger change than we can consider for this release.

-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/