[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: openldap 2.4.23 hangs
Thanks for the response. It'll take me a couple of days to work this in my schedule. Just so that you know the new auth servers (ldap & samba) will be 64-bit and this is what I will compile to.
Unfortunately we may never get to the 2.5 release.
Bob
--bs
Sent from my iPad
On Aug 14, 2012, at 8:15 PM, Howard Chu <hyc@symas.com> wrote:
> rwsmith@bislink.net wrote:
>>
>>
>> Hi,
>>
>>
>>
>> Not sure if I should post this here or with the CentOS mailing list (I am
>> hoping they are monitoring this). I am using a stock CentOS 6.3 32-bit
>> installation with
>>
>>
>>
>> # rpm -qa | grep openldap
>> openldap-devel-2.4.23-26.el6_3.2.i686
>> openldap-2.4.23-26.el6_3.2.i686
>> openldap-clients-2.4.23-26.el6_3.2.i686
>> openldap-servers-2.4.23-26.el6_3.2.i686
>>
>>
>>
>> I have a 4-way multi-master sync replication set up on four virtual servers
>> using Citrix XenServer 6.2. I am also running Samba 3.5.10 as a PDC on one
>> machine and BDC on the other three. All servers are also running sssd-1.8.0
>> for the Linux authentication.
>>
>>
>>
>> The problem is that one or more of the LDAP servers will hang, usually the one
>> that acts as the PDC, since this is hit the hardest and is the more critical
>> of the four. Usually but not always the "hang" will trickle to the other
>> servers--usually when I am not watching during the middle of the night.
>> Fortunately we are not yet in full production.
>>
>>
>>
>> Compiling from source is not yet an option. I must use the stock RPMs from
>> CentOS per our design guidelines.
>>
>>
>>
>> LDAP will appear to hang but what appears to be happening is that only the
>> listener becomes busy and will not get out this state. Here is a short pull of
>> the logs that I am collecting:
>
> Sounds like this was fixed in 2.4.25, git commit ID
> 0ae659ad87c64bef938f729e87573ff3ce04bd28 (master),
>
> commit a3f40e5601c0c522f2bda418374fb415bdcbd75c (release).
>
> There was no ITS submitted for this change, so it is not in the CHANGES file.
>
> If you can reproduce the problem with 2.4.32 please submit an ITS for it. I
> will note that all of this listener code is targeted for a major rewrite in
> OpenLDAP 2.5.
>
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: read active on 69
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: epoll: listen=7 active_threads=0
>> tvp=zero
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:34:44 auth-us slapd[10357]: conn=1742 op=0 EXT
>> oid=1.3.6.1.4.1.1466.20037
>> Aug 14 20:34:44 auth-us slapd[10357]: conn=1742 op=0 STARTTLS
>> Aug 14 20:34:44 auth-us slapd[10357]: conn=1742 op=0 RESULT oid= err=0 text=
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: activity on 1 descriptor
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: activity on:
>> Aug 14 20:34:44 auth-us slapd[10357]:
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: epoll: listen=7 active_threads=0
>> tvp=zero
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: activity on 1 descriptor
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: activity on:
>> Aug 14 20:34:44 auth-us slapd[10357]: 69r
>> Aug 14 20:34:44 auth-us slapd[10357]:
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: read active on 69
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: epoll: listen=7 active_threads=0
>> tvp=zero
>> Aug 14 20:34:44 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:34:46 auth-us slapd[10357]: daemon: epoll: listen=7 active_threads=0
>> tvp=zero
>> Aug 14 20:34:46 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:34:51 auth-us slapd[10357]: daemon: epoll: listen=7 active_threads=0
>> tvp=zero
>> Aug 14 20:34:51 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: activity on 1 descriptor
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: activity on:
>> Aug 14 20:34:54 auth-us slapd[10357]: 39r
>> Aug 14 20:34:54 auth-us slapd[10357]:
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: read active on 39
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: epoll: listen=7 active_threads=0
>> tvp=zero
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: activity on 1 descriptor
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: activity on:
>> Aug 14 20:34:54 auth-us slapd[10357]:
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:34:54 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:34:56 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:34:56 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:01 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:01 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:06 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:06 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:11 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:11 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:12 auth-us slapd[10357]: daemon: activity on 1 descriptor
>> Aug 14 20:35:12 auth-us slapd[10357]: daemon: activity on:
>>
>> Aug 14 20:35:12 auth-us slapd[10357]: 42r
>> Aug 14 20:35:12 auth-us slapd[10357]:
>> Aug 14 20:35:12 auth-us slapd[10357]: daemon: read active on 42
>> Aug 14 20:35:12 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:12 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:14 auth-us slapd[10357]: daemon: activity on 1 descriptor
>> Aug 14 20:35:14 auth-us slapd[10357]: daemon: activity on:
>> Aug 14 20:35:14 auth-us slapd[10357]: 40r
>> Aug 14 20:35:14 auth-us slapd[10357]:
>> Aug 14 20:35:14 auth-us slapd[10357]: daemon: read active on 40
>> Aug 14 20:35:14 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:14 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:16 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:16 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:21 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:21 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:26 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:26 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>> Aug 14 20:35:31 auth-us slapd[10357]: daemon: epoll: listen=7 busy
>> Aug 14 20:35:31 auth-us slapd[10357]: daemon: epoll: listen=8 active_threads=0
>> tvp=zero
>>
>>
>>
>> Every log entry prior to this looks normal in that epoll: listen=7 goes
>> between active_threads=0 to busy when a connection comes in, sets up the
>> connection, and then goes back to active_threads=0. I have yet to understand
>> what is going on to cause it to go into the busy state and never to return
>> until I manually stop and restart the slapd process. It does appear however
>> that slapd can still process any queries on active connections as noted on
>> descriptors 40r and 42r--it just can't process any new connection requests as
>> epoll: listen=7 has hung.
>>
>>
>>
>> Looking through the archives this problem still appears to be present in a few
>> additional later releases but I have not found any thread yet which points to
>> a specific solution or patch that fixes this problem. Unless I can point to a
>> specific solution and/or patch I won't get approval to do a pull from the
>> latest source and compile--I'll have to stick with an hourly cron job that
>> stops and restart slapd.
>>
>>
>>
>> Thanks,
>>
>> Bob Smith
>>
>> --bs
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> -- Howard Chu
> CTO, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/