[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: testing openLDAP vs Netscape (any hints)

On Monday, July 2, 2001, at 11:09 AM, Dan Shriver wrote:
I'm about to start doing tests of openLDAP vs Netscape.
I'm interested in doing various speed tests.
   -loading a slave db

Well, I'm not sure what the point of benchmarking an initial load is... an LDAP database is something that should be built once, and then modified as needed, not something where you constantly rebuild it. (It's a bit like benchmarking how fast Oracle can be installed and set up as a measure of its overall usage speed.). It's good information to know if you want to rebuild in the future, but it's not that valuable if you're running a live ldap server farm.

   -propigating changes to a slave (also time for those changes
on the master)

Hm. Never seen (or tried) benchmarks for this. Since LDAP serving really isn't tuned as a write/modify database, I'd be very cautious about this one, as well.

-search times to find n records in a db of size m

This is always a sticky one, as it depends on usage. Almost all (99.9999%) of my lookups are for single entries. Group lists are a single entry, user mail lookups are a single entry, etc. Also, for speed reasons (regardless of the engine), wildcard searches are heavily frowned upon by my programming teams... of course, it depends on the directory deployment. I've seen some applications that wildcard _everything_. Very slow.

Ideally I'd like to use the same (command-line) tools, but it
looks like I'll be using different ones (ldif2db vs slapadd for
the loading...).

It does make sense to use tools like ldapadd, over a network, to add individual entires to a running db and time that.... especially because it _is_ portable, and you avoid netcraft-like silliness with a "maximum speed" far beyond what a decent network can handle (where you're benchmarking on the same machine as you're running the server on). If you have a 100Mb lan, but the ldap lookups are all coming in via a T-1, speed beyond a T-1 is moot. :-)

 Do any of you have tips, for the "fairest" way
to do tests, and any results you people might have.  I did see
some threads on this subject but most of them were old (~1 yr).

Most of those still hold true (was this the Network World thread? Where Joel used an "out of the box" config?)

Some general benchmark issues to consider: If you're running with specific features on each, those features should match. Which seems obvious, but they're hidden in some funny ways. For example, making the ACL's match would be a serious challenge, and schema management is an equal headache. Which is why, I suppose, the out-of-box thing started.

Based on my past experience with Netscape/iPlanet, you'd want to use optimizations similar to the following, in your slapd.conf... these are from 1.x servers, so they may be a bit out of date, but it'll point out the differences needed to make openldap similar to iPlanet(Netscape).
# Under your database enttry....
# Run without locking during serial (sync) additions, such as one host running all additions, or during initial load
# dbnolocking
# Don't sync to disk constantly.
# Don't sync to disk constantly, documentation bug from old (20 January 1999) slapd.conf man page?
# dbnowsync

# Manually set the number of ldap entries to cache, iPlanet does this automatically.
# To determine best openldap performance, set as high as your number of ldap entries, if RAM will allow.
cachesize 15000

# Manually set the size of the db cache in bytes, iPlanet does this automatically.
# To determine best openldap performance, set as high as your total db sizes, if RAM will allow.
dbcachesize 6000000

# Set a low number of return entries, because many users don't actually look up 500 entries.
sizelimit 50

# Reduce logging down
loglevel 0

# Index like mad. Everything you're keying off of should always be indexed.
index cn,sn,uid,email pres,eq,approx,sub
index objectclass pres,eq,approx,sub
# internal arc below, not applicable to other servers, but here to show custom indexing
index agstore pres,eq,sub


That at least gets it in the ballpark. The most important components if you're using ldbm are the cache and disk (sync) based operatons. As far as schemachecking and ACL's, if you turn them off on both machines you can test raw engine speed, but you'll also want to turn them on to test the schemachecking and ACL checking portions of the respective programs.

One of the sticky issues (as mentioned before, in various ways) is that iPlanet/Netscape requires less manual tuning. So is tuning for the benchmark data fair, or unfair? You might want to try it both ways, for academic purposes, but assume that a live deployment would be a tuned deployment, for both servers.

Even better, if you're testing for live application deployment, you _should_ be testing with live operations and data.... rather than fabricated ACL's, schemas, and load that may not reflect your load pattern. Live data will allow you to see what your expected performance really is, and let you know where/how you need to tune (or even dump ldap alltogether, in some cases).


ron@opus1.com, 520-326-6109, http://www.opus1.com/ron/
The opinions expressed in this email are not necessarily those of myself,
my employers, or any of the other little voices in my head.