(ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases

[hbohnenkamp@united-internet.de]

Full_Name: Henrik Bohnenkamp
Version: >= 2.4.44, HEAD
OS: Ubuntu 18.04, Coreos 7.5
URL: https://github.com/hbo/openldap-mdb-deref-problem
Submission from: (NULL) (77.176.95.241)


This is a followup to
http://www.openldap.org/lists/openldap-technical/201805/msg00065.html

When MDB is used as backend database and with large DITs (O(10^6)) with many
alias entries (O(10^5)), search requests with deref=always and scope=sub will
take prohibitively long. Servers with a high request rate might become utterly
unresponsive. This problem is not present in the HDB backend.

In this issue I want to contribute two things:
- a means to demonstrate the problem; this in the form of two scripts
(bash/python) which set up a large test  DIT and start two slapds (one HDB, one
MDB) to allow easy comparisons of the performance
- a patch to fix the problem

The patch is certainly not production ready (or, if it is, it needs still
exhaustive testing to inspire  confidence), however, I think it is far enough to
at least discuss the approach. 

Both the scripts and the patch, together with instructions how to use the former
can be found here: 

https://github.com/hbo/openldap-mdb-deref-problem

I am looking forward to discuss the patch.