Full_Name: Quanah Gibson-Mount Version: 2.4.44 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.52.177) After setting the sessionlog for syncprov to a large size so as to avoid ITS#8125. However, due to the sessionlog being a linear linked list, it can be vastly inefficient to iterate, and slows down the master to the point that searches can time out. Howard suggests converting it into a tree instead to help obviate the load this currently induces.
--On Monday, August 29, 2016 9:16 PM +0000 quanah@openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.44 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.52.177) In doing testing for ITS#8491, I encountered this problem in my lab. Any time a single replica is parsing the session log, all other operations on the server come to a complete halt. I.e., slapd does *nothing* other than handle the sessionlog query. This seems like a fatal problem. --Quanah -- Quanah Gibson-Mount
moved from Incoming to Software Bugs
changed notes
quanah@zimbra.com wrote: > --On Monday, August 29, 2016 9:16 PM +0000 quanah@openldap.org wrote: > >> Full_Name: Quanah Gibson-Mount >> Version: 2.4.44 >> OS: Linux 2.6 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (75.111.52.177) > > In doing testing for ITS#8491, I encountered this problem in my lab. Any > time a single replica is parsing the session log, all other operations on > the server come to a complete halt. I.e., slapd does *nothing* other than > handle the sessionlog query. This seems like a fatal problem. A patch for this is available for testing in https://github.com/quanah/openldap-scratch/tree/its8486. It no longer keeps the sessionlog mutex locked for the entire time that it's playing the log. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Using the patch in the openldap-scratch repository, this is no longer a problem. Test case: 2 masters with identical DBs Approximately 4 million user DB Modify every user on master 1 (replicated to master 2) Stop master 2, reload it with the original DB Let time pass so the accesslog DB empties out a record of all changes Start master 2 master 2 will then start querying the sesison log to find any deleted entries while this is ongoing, perform 4 concurrent queries against master 1 (in this case, a search against the entire DB with filter "(uid=*)" requesting 1.1 All four searches execute w/o issue once this patch is in place. Master 2 logs: Feb 10 19:09:54 anvil2 systemd[1]: Started LSB: Symas OpenLDAP. Feb 10 19:09:54 anvil2 slapd[5580]: do_syncrep2: rid=100 LDAP_RES_SEARCH_RESULT Feb 10 19:09:54 anvil2 slapd[5580]: do_syncrep2: rid=100 delta-sync lost sync, switching to REFRESH Feb 10 19:09:54 anvil2 slapd[5580]: do_syncrep2: rid=100 (4096) Content Sync Refresh Required --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
changed notes changed state Open to Release
changed notes changed state Release to Closed
See also ITS#8125, ITS#6467, ITS#8768 Fixed in master Fixed in RE24 (2.4.46)
Note: OpenLDAP 2.5 has further work converting the sessionlog to a btree: Commits: • d2036cec by Ondřej Kuzník at 2020-09-25T00:07:50+00:00 ITS#8486 Switch sessionlog to use TAVL • 8f8774c0 by Howard Chu at 2020-09-25T00:07:50+00:00 ITS#8486 Minor play_sessionlog cleanup Fix logmsg uuidstr. Shortcut finding start of playback. Allow dup CSNs in log, but with different UUIDs. All non-present deletes in a refresh use the same CSN. • 98d5c5c6 by Ondřej Kuzník at 2020-09-25T00:07:50+00:00 ITS#8486 Protect tavl_* calls in play_sessionlog • 1915cb96 by Howard Chu at 2020-09-25T00:07:50+00:00 ITS#8486 use kbtree for sessionlog Saves about 20% CPU time and RAM
RE24: commit 234230f286f8d67c4d4bf590accbd483172dbdf8 Author: Ondřej Kuzník <ondra@openldap.org> Date: Thu Oct 26 12:00:20 2017 +0100 ITS#8486 Switch sessionlog to use TAVL