[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: indexing & hardware questions -- again



On Monday, August 6, 2001, at 07:54 AM, oberwetter, josh wrote:
All, thanks for responding. Here are some follow-up
clarifications/remarks:
Finally, is slurpd actually more efficient at handling modify
operations? While a directory is handling a modify operation from a
client, its performance is pretty bad. So, by "efficient" I mean, do
the slaves take significantly LESS of a performance hit if the slurpd
process sends modify operations than if a slave handles client
requests directly?
slurpd doesn't perform modify operations.  It mirrors such
operations on a
master slapd to the slave slapd.  At least as I understand
it.  slapd still is
the one performing the modify op.
My specific question is this:
I am concerned about the costs and benefits of processing modify
operations via slurpd, with respect to how modify operations hurt the
speed of searches.

Well, This is one of the reasons to not write applications that modify the directory an awful lot... :-) I generally think of LDAP as a read-only database, with write functions only for general maintenance... I frequently have clients who, once they "discover" LDAP, want to put real-time data changes into LDAP, which is generally difficult to do. It's not really optimized for that.... if you are expecting many changes simply because of your entry count, you might want to split out your servers into a tree, so each server is only handling a subsection's worth of changes, or build a "change loading only" server, or use a combination of row level (or even row-column) locking RDBM's and LDAP. (MySQL suffers similar problems under load, because of the locking mechanism)


ldapadd is much more faster at importing a lot of data than some client
that I write would be. Is slurp also more efficient?

slurpd is simply a method of "replaying" an add/modify operation. In relation to a "client that I write", I suppose you're wanting to know if slupd is more efficient at processing 300 changes than, say, a java/C/Perl application looping through and acting as a slurpd, replaying changes on 3 different ldap servers. The answer is a qualified "maybe". Slurpd is no slouch, and was written to have fairly low overhead.... but you may be a coding fiend who could write faster C code that did the same thing, or was optimized for the way your data was changing (say, batching up an 24 hours of changes), or used a different methodology, for example, playing thousands of changes into a "loading only" ldap database on the slaves, and then swapping out the live database with the "load only" db, or immediately moved all changes to all servers (if you needed such a thing).


There's lots of ways of doing speed tuning, so I repeat my usual mantra:
"Test with a pilot, tune as needed, try a few different things and choose the best one."


-Ronabop

--2D426F70|759328624|00101101010000100110111101110000
ron@opus1.com, 520-326-6109, http://www.opus1.com/ron/
The opinions expressed in this email are not necessarily those of myself,
my employers, or any of the other little voices in my head.