Issue 8360 - apparent MDB corruption when hitting maxsize
Summary: apparent MDB corruption when hitting maxsize
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: 2.4.43
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-01-25 14:28 UTC by peter.van.dijk@powerdns.com
Modified: 2016-02-11 00:51 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description peter.van.dijk@powerdns.com 2016-01-25 14:28:32 UTC
Full_Name: Peter van Dijk
Version: 2.4.43
OS: Mac OS X 10.10.5
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (2001:610:666:0:c92b:7aa7:bc73:c430)


The procedure follow is fully repeatable with consistent problematic results for
me.

(1) ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/openldap-2.4.43.tgz,
installed via osx homebrew with configure flags
--prefix=/usr/local/Cellar/openlp%p/2.4.43 --sysconfdir=/usr/local/etc
--localstatedir=/usr/local/var

(2) config and related files as uploaded on the given URL (cosine.schema is
unedited but included for completeness)

(3) /usr/local/var/openldap-data/ is empty

(4) sudo /usr/local/opt/openldap/libexec/slapd -d 255 -u peter 2> slapd.log

(5) ldapmodify -D 'uid=peter,o=habbie' -w secret -c < named.conf.ldif  2>
ldapmodify.log ; ldapdelete -D 'uid=peter,o=habbie' -w secret -r ou=dns,o=habbie
2> ldapdelete.log
modify is unhappy, initially because of some unknown attributes (thish is why I
used -c), then later because it is hitting the MDB maxsize of 10 megabytes.
ldapdelete then fails to delete the ou=dns tree due to MDB errors

(6) increasing maxsize after this 'corruption' has happened does NOT enable
ldapdelete to suddenly work

(7) (with slapd stopped, not sure it matters): $
/usr/local/opt/openldap/sbin/slapcat outputs 191771 lines to stdout before
aborting with:
56a63063 mdb_entry_decode: attribute index 24 not recognized
Segmentation fault: 11

(8) as requested by hyc on IRC, copying here for completeness 

$ mdb_dump -p -s ad2i /usr/local/var/openldap-data/
VERSION=3
format=print
database=ad2i
type=btree
mapsize=10485760
maxreaders=126
tetegerkey=1
db_pagesize=4096
HEADER=END
 \01\00\00\00
 objectClass
 \02\00\00\00
 o
 \03\00\00\00
 structuralObjectClass
 \04\00\00\00
 entryUUID
 \05\00\00\00
 creatorsName
 \06\00\00\00
 createTimestamp
 \07\00\00\00
 entryCSN
 \08\00\00\00
 modifiersName
 \09\00\00\00
 modifyTimestamp
 \0a\00\00\00
 ou
 \0b\00\00\00
 dc
 \0c\00\00\00
 associatedDomain
 \0d\00\00\00
 dNSTTL
 \0e\00\00\00
 sOARecord
 \0f\00\00\00
 nSRecord
 \10\00\00\00
 mXRecord
 \11\00\00\00
 aRecord
 \12\00\00\00
 cMEReRecord
 \13\00\00\00
 LocRecord
 \14\00\00\00
 hInfoRecord
 \15\00\00\00
 tXTRecord
 \16\00\00\00
 aAAARecord
DATA=END
Comment 1 peter.van.dijk@powerdns.com 2016-01-25 14:31:08 UTC
The configure line got messed up somewhere (probably my terminal), 
should be
==> ./configure --prefix=/usr/local/Cellar/openldap/2.4.43 
--sysconfdir=/usr/local/etc --localstatedir=/usr/local/var

Comment 2 peter.van.dijk@powerdns.com 2016-01-25 14:32:26 UTC
Apparently I did not actually enter the right URL. All data is at 
https://downloads.powerdns.com/tmp/openldap-its-20160125/

Comment 3 Howard Chu 2016-01-26 12:51:00 UTC
peter.van.dijk@powerdns.com wrote:
> Full_Name: Peter van Dijk
> Version: 2.4.43
> OS: Mac OS X 10.10.5
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (2001:610:666:0:c92b:7aa7:bc73:c430)
>
>
> The procedure follow is fully repeatable with consistent problematic results for
> me.

Unable to reproduce this on Linux using RE24.

> (1) ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/openldap-2.4.43.tgz,
> installed via osx homebrew with configure flags
> --prefix=/usr/local/Cellar/openlp%p/2.4.43 --sysconfdir=/usr/local/etc
> --localstatedir=/usr/local/var
>
> (2) config and related files as uploaded on the given URL (cosine.schema is
> unedited but included for completeness)
>
> (3) /usr/local/var/openldap-data/ is empty
>
> (4) sudo /usr/local/opt/openldap/libexec/slapd -d 255 -u peter 2> slapd.log
>
> (5) ldapmodify -D 'uid=peter,o=habbie' -w secret -c < named.conf.ldif  2>
> ldapmodify.log ; ldapdelete -D 'uid=peter,o=habbie' -w secret -r ou=dns,o=habbie
> 2> ldapdelete.log
> modify is unhappy, initially because of some unknown attributes (thish is why I
> used -c), then later because it is hitting the MDB maxsize of 10 megabytes.
> ldapdelete then fails to delete the ou=dns tree due to MDB errors
>
> (6) increasing maxsize after this 'corruption' has happened does NOT enable
> ldapdelete to suddenly work

ldapdelete works fine for me after raising maxsize.
>
> (7) (with slapd stopped, not sure it matters): $
> /usr/local/opt/openldap/sbin/slapcat outputs 191771 lines to stdout before
> aborting with:
> 56a63063 mdb_entry_decode: attribute index 24 not recognized
> Segmentation fault: 11

Never saw this, slapcat runs to completion with no complaints.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 4 Howard Chu 2016-01-26 13:41:11 UTC
Howard Chu wrote:
> peter.van.dijk@powerdns.com wrote:
>> Full_Name: Peter van Dijk
>> Version: 2.4.43
>> OS: Mac OS X 10.10.5
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (2001:610:666:0:c92b:7aa7:bc73:c430)
>>
>>
>> The procedure follow is fully repeatable with consistent problematic results
>> for
>> me.
>
> Unable to reproduce this on Linux using RE24.

I had the sequence of steps a bit wrong - running slapcat immediately after 
ldapmodify masked the problem.

Fixed now in git.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 5 Quanah Gibson-Mount 2016-01-29 01:07:43 UTC
changed notes
changed state Open to Release
moved from Incoming to Software Bugs
Comment 6 OpenLDAP project 2016-02-11 00:51:48 UTC
Fixed in master
Fixed in RE25
Fixed in RE24 (2.4.44)
Comment 7 Quanah Gibson-Mount 2016-02-11 00:51:48 UTC
changed notes
changed state Release to Closed