Issue 7503 - slapcat: retry a few times when retrieving an entry
Summary: slapcat: retry a few times when retrieving an entry
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-25 22:31 UTC by balint@balintreczey.hu
Modified: 2014-08-01 21:03 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description balint@balintreczey.hu 2013-01-25 22:31:09 UTC
Full_Name: Balint Reczey
Version: master
OS: Linux
URL: http://rbalint.cs.bme.hu/0001-slapcat-retry-a-few-times-when-retrieving-an-entry-f.patch
Submission from: (NULL) (84.0.168.17)


From: Axel Beckert <beckert@phys.ethz.ch>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: slapd: slapcat output truncated every now and then
Date: Tue, 15 May 2012 18:33:57 +0200

Package: slapd
Severity: grave
Version: 2.4.23-7.2
Justification: Causes data loss

Every now and then slapcat's output does not contain the whole content
of the LDAP but is truncated at some LDIF entry border (i.e. all printed
LDIF records seem complete).

To reproduce run "while sleep 1; do slapcat | wc; done" on a moderately
busy LDAP server (writes every few seconds to minutes; about 10000
entries) while slapd is running and notice the occasionally occurring
huge change in wc's printed values, e.g.:

# while sleep 1; do slapcat | wc; done
 471698 1015498 15336677
 471698 1015498 15336677
 471698 1015498 15336677
...
 471698 1015497 15336635
 471698 1015497 15336635
 281829  606820 8632165  <--
 471698 1015497 15336635
 471698 1015497 15336635
 471698 1015497 15336635
 308627  664573 9476751  <--
 471698 1015497 15336635
 471698 1015497 15336635
 471698 1015497 15336637

Please see email thread starting at
http://www.openldap.org/lists/openldap-technical/201301/msg00195.html

The attached modifications to OpenLDAP Software are subject to the following
notice:

Copyright 2013 Balint Reczey <balint@balintreczey.hu>
Redistribution and use in source and binary forms, with or without modification,
are permitted only as authorized by the OpenLDAP Public License. 
Comment 1 Quanah Gibson-Mount 2013-01-27 18:09:11 UTC
--On Friday, January 25, 2013 10:26 PM +0000 balint@balintreczey.hu wrote:

> Full_Name: Balint Reczey
> Version: master
> OS: Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (84.0.168.17)

Hi Balint,

The patch as submitted is rejected, due to the fact that it changes 
slapadd, rather than fixing back-bdb/hdb, which is where the actual problem 
occurs.   I.e., your change is too broad.  Feel free to submit an updated 
patch correctly updating the back-bdb/hdb code rather than slapadd.

Thanks,
Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Comment 2 Quanah Gibson-Mount 2013-01-27 18:27:32 UTC
--On Sunday, January 27, 2013 6:09 PM +0000 quanah@zimbra.com wrote:

> --On Friday, January 25, 2013 10:26 PM +0000 balint@balintreczey.hu wrote:
>
>> Full_Name: Balint Reczey
>> Version: master
>> OS: Linux
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (84.0.168.17)
>
> Hi Balint,
>
> The patch as submitted is rejected, due to the fact that it changes
> slapadd, rather than fixing back-bdb/hdb, which is where the actual
> problem  occurs.   I.e., your change is too broad.  Feel free to submit
> an updated  patch correctly updating the back-bdb/hdb code rather than
> slapadd.

s/slapadd/slapcat/g ;)

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Comment 3 balint@balintreczey.hu 2013-01-27 21:03:19 UTC
2013/1/27 Quanah Gibson-Mount <quanah@zimbra.com>:
> --On Friday, January 25, 2013 10:26 PM +0000 balint@balintreczey.hu wrote:
>
>> Full_Name: Balint Reczey
>> Version: master
>> OS: Linux
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (84.0.168.17)
>
>
> Hi Balint,
>
> The patch as submitted is rejected, due to the fact that it changes slapadd,
> rather than fixing back-bdb/hdb, which is where the actual problem occurs.
> I.e., your change is too broad.  Feel free to submit an updated patch
> correctly updating the back-bdb/hdb code rather than slapadd.
>
> Thanks,
> Quanah
>
Hi Quanah,

I'll let OpenLDAP developers fix the problem where they find it
appropriate instead.
My intention was adding a retry logic where there was a skip and
continue logic already, in slapcat.

Cheers,
Balint

Comment 4 Quanah Gibson-Mount 2013-01-27 22:59:36 UTC
--On Sunday, January 27, 2013 10:03 PM +0100 Bálint Réczey 
<balint@balintreczey.hu> wrote:

> I'll let OpenLDAP developers fix the problem where they find it
> appropriate instead.
> My intention was adding a retry logic where there was a skip and
> continue logic already, in slapcat.

Hi Balint,

I understand your intent, I'm simply noting it was the wrong approach. 
Supplying a patch that fixes the actual issue, if you choose to do so, 
simply means it will have a greater chance of being fixed sooner than 
later.  I.e., it is not a high priority issue at this time for the openldap 
devs.

--Quanah



--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Comment 5 Howard Chu 2013-02-07 18:27:16 UTC
I've committed an alternate fix for this to git master, filed against 
ITS#6365. Please test and let us know your results, thanks.
The patch is in commit 853b9d1335d27e280751e9cfb8ca6b5356ffec73

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 6 balint@balintreczey.hu 2013-02-07 18:47:14 UTC
Hi Howard,

Thank you!
I have updated the Debian bug report [1].

Cheers,
Balint

[1]: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=673038

2013/2/7 Howard Chu <hyc@symas.com>:
> I've committed an alternate fix for this to git master, filed against
> ITS#6365. Please test and let us know your results, thanks.
> The patch is in commit 853b9d1335d27e280751e9cfb8ca6b5356ffec73
>
> --
>   -- Howard Chu
>   CTO, Symas Corp.           http://www.symas.com
>   Director, Highland Sun     http://highlandsun.com/hyc/
>   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 7 Howard Chu 2013-04-24 08:25:05 UTC
changed notes
changed state Open to Closed
Comment 8 OpenLDAP project 2014-08-01 21:03:57 UTC
fixed as ITS#6365