[Date Prev][Date Next]
Re: thread pools, performance
Howard Chu wrote:
Howard Chu wrote:
Finally we got to running the actual authrate tests, which yielded a
of 4526 auths/second with 40 client threads. The rate declined from
more clients were added; AD clearly isn't capable of handling very many
Correction, the peak rate was with only 24 client threads. The rate
dropped to 4400/sec and stayed there as more client threads were added
after that point.
It's enlightening to look at the actual CPU time used during the
For ldifde on W2K3 we got:
time ldifde.exe -i -f examp3.ldif -h -q 8
261.10u 140.73s 4:23:46.85 2.5%
For slapadd on FC6 we got:
time .slapadd -f slapd.conf.slam -q -l example.ldif.1mil
260.75u 80.86s 7:05.17 80%
One interesting part here is that the amount of user CPU time is nearly
identical in both cases. That implies that both slapadd and ldifde are
about the same amount of work to parse the input LDIF.
Eh, I take that back. The slapadd time includes both parsing and all of
the BDB manipulation, while the ldifde time just includes parsing and
encoding to BER. The database manipulation time in the server isn't
reflected at all in these numbers. So really there's not much comparable
in these figures at all...
Comparing the rest of the time isn't really fair since it seems that
just feeds data into a running server using LDAP, while slapadd simply
to the DB directly. I guess for the sake of fairness we'll have to
OpenLDAP import using ldapadd next.
For completeness we would have to time both ldapadd against AD and
ldifde against OpenLDAP. Since ldifde only exists for Windows we'd have
to do those runs all on Windows, against a Windows build of OpenLDAP. I
guess that can come later; we already know that ldapadd in OpenLDAP is
now pretty well optimized.
One interesting feature of ldifde is that it also supports multiple
threads, unlike our ldapadd/ldapmodify. Of course, the MS implementation
is pretty braindead: if you have your entire tree in a single LDIF file,
and try to import it with multiple threads in parallel, it will fail
because there's no check to make sure that the thread importing a parent
entry completes before the threads that import child entries start. So,
to take advantage of the multithreading, you need to break out all of
the parent entries into a separate file, import them single-threaded,
and then do all the children in a separate invocation.
It seems that whoever added that feature to ldifde didn't really think
about how Directories work, or what's actually useful for a directory
Thanks for all this Howard. It certainly makes it clear where OpenLDAP
lies in the LDAP world (looking down from the top and around at everyone
else ;-) ).
OpenLDAP Engineering Team.
Community developed LDAP software.