Issue 9001 - try_read1msg() takes O(n) steps to lookup a matching request
Summary: try_read1msg() takes O(n) steps to lookup a matching request
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: 2.5.3
Assignee: Ondřej Kuzník
URL:
Keywords:
: 6301 (view as issue list)
Depends on:
Blocks:
 
Reported: 2019-03-29 13:38 UTC by Ondřej Kuzník
Modified: 2021-04-01 04:05 UTC (History)
1 user (show)

See Also:


Attachments
searcher.c (1.35 KB, text/x-csrc)
2020-04-19 11:25 UTC, Ondřej Kuzník
Details

Note You need to log in before you can comment on or make changes to this issue.
Description Ondřej Kuzník 2019-03-29 13:38:30 UTC
Full_Name: Ondrej Kuznik
Version: re24/master
OS: 
URL: https://github.com/mistotebe/openldap/tree/its9001
Submission from: (NULL) (82.10.24.68)


If a client has thousands or more requests in flight at the same time, it has to
look up the right request every time it receives a response. As the requests are
tracked in a doubly-linked list, lookup tends to take O(n) steps whichever
direction we take and things generally get out of hand from there on.

The linked branch puts them into a TAvl for quicker lookup, with another commit
to let people test as (t)avl code is part of liblutil at the moment.
Comment 1 Ondřej Kuzník 2019-03-29 13:50:12 UTC
On Fri, Mar 29, 2019 at 01:38:30PM +0000, ondra@openldap.org wrote:
> Full_Name: Ondrej Kuznik
> Version: re24/master
> OS: 
> URL: https://github.com/mistotebe/openldap/tree/its9001
> Submission from: (NULL) (82.10.24.68)
> 
> 
> If a client has thousands or more requests in flight at the same time, it has to
> look up the right request every time it receives a response. As the requests are
> tracked in a doubly-linked list, lookup tends to take O(n) steps whichever
> direction we take and things generally get out of hand from there on.
> 
> The linked branch puts them into a TAvl for quicker lookup, with another commit
> to let people test as (t)avl code is part of liblutil at the moment.

A trivial program to expose this behaviour is available at
ftp://ftp.openldap.org/incoming/searcher.c

The same program takes under a second to handle 100000 searches for
cn=monitor and children when linked to the patched libldap.

-- 
Ondřej Kuzník
Senior Software Engineer
Symas Corporation                       http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP

Comment 2 Ondřej Kuzník 2020-03-19 16:58:03 UTC
*** Issue 6301 has been marked as a duplicate of this issue. ***
Comment 4 Ondřej Kuzník 2020-04-19 11:25:23 UTC
Created attachment 711 [details]
searcher.c
Comment 5 Quanah Gibson-Mount 2021-01-28 17:28:18 UTC
Ondrej to test moving needed liblutil code into libldap
Comment 6 Quanah Gibson-Mount 2021-03-30 15:53:24 UTC
Commits: 
  • e36d1e31 
by Ondřej Kuzník at 2021-03-30T15:46:40+01:00 
ITS#9001 manual changes


  • 3bd1b090 
by Ondřej Kuzník at 2021-03-30T15:46:40+01:00 
ITS#9001 Use a TAvl for request tracking in libldap