Re: OpenLDAP as an enterprise level LDAP provider

[Howard Chu <hyc@symas.com>]

Your mailer is doing something weird with linebreaks, so this is a bit 
tedious to answer...

1) The patches are known to work, but it's possible that they did not 
apply correctly on your source. The problem is known to be fixed in BDB 
4.3. The patches required to support BDB 4.3 are trivial. I've already 
rolled them into the 2.2 release branch, so BDB 4.3 support should 
appear in OpenLDAP 2.2.19. I personally am now using BDB 4.3 
exclusively, and no longer run BDB 4.2.

2) Yes, this is unfortunate. It seemed to be worth the performance gain 
at the time. We may revisit this in OpenLDAP 2.3 but we really can't 
introduce a database format change at this stage of the 2.2 release. re: 
slapadd/slapcat - it sounds like you haven't configured your BDB cache 
correctly. On a 1GHz PIII with an old ATA66 hard drive and 768MB RAM I 
can slapadd 300,000 entries (~360MB ldif) in 8 minutes (8:21) with a few 
indexes enabled, using a 512MB BDB cache. This is with OpenLDAP 2.2 and 
BDB 4.2, by the way. slapcat'ing the entire database (to /dev/null) 
takes 1:40s. That's on 6 year old desktop hardware, a real server-class 
machine would do even better. You probably need to visit the OpenLDAP FAQ:

http://www.openldap.org/faq/data/cache/1075.html

Fortin, John {PBG} wrote:

> First of this, this message is = intended to open a discussion about 
> using OpenLDAP in the = enterprise. I do not want to start a flame war 
> concerning the = pros and cons of various LDAP implementations. 
> Currently we are using OpenLDAP as our = initial implementation for 
> authentication and authorization with = Weblogic and other J2EE 
> providers for our enterprise application. = Our initial rollout was 
> successful, although we did not have a large = population of users in 
> the directory (<1000) Performance was = fine, and we had no issue with 
> loading data etc as the ldif files were = small. However, as we are 
> now looking to roll = this out to a much larger population (600K+) we 
> are starting to run = into some issues, one of which I sent a note 
> about recently. The = issues we are currently seeing, and could 
> potentially be a show stopper = for us are as follows: 1) Log 
> archiving and transactions - With the = current bdb and version of 
> OpenLDAP (2.2.18), I cannot = archive/delete files without stopping 
> slapd. This manifested = itself as we were testing bulk loading of 
> data and consistently ran out = of log space. I have tested with the 
> various patched suggested to = no avail. I have not tested with the 
> newest version of bdb (4.3) = as I have no indication that this fixes 
> the issue. 2) The ability to backup data - Using the bdb utilities = 
> (db_load and db_dump) do not work. It seems that this is based on = 
> OpenLDAP using custom hashes in the creation of the configured 
> indexes. = (This is based on some discussion I found in the maillist = 
> archives). The two workarounds that I am aware of, creating ldif = 
> files with slapcat, and backing up the bdb files themselves so not 
> seem = to be adequate for the following reasons: = slapadd - with 600K 
> users and no = indexes it takes about 2 hrs to load. The creation of 
> indexes = afterwards with slapindex takes an additional 6-12 hours. To 
> me, = this seems like too long of a time for recovery. = *.bdb file 
> backup - we've had limited = success with this. There also seems to be 
> an issue, even after = doing a db_checkpoint and a db_recover of a 
> dependency on logs = files. As we are looking to do a 'cold' backup of 
> our master ldap = directory, we do not want to be dependent on logs 
> files. I have searched the archives quite a = bit looking for similar 
> issues with limited success. I know the = basics of how OpenLDAP works 
> and tuning of the system, but I am by no = means a guro in the 
> internals. At this point, I am looking for = some direction as to how 
> to proceed. System: OS: RH ES 3.0 OpenLDAP 2.2.18 BDB 4.2.52 (with 
> current = patches) Thanks!! --John John Fortin PBG Middleware and Web 
> = Services (914) 767-7844 



--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support