[Date Prev][Date Next]
Re: why is adding records slow?
Thanks for your response, Gabor. Yes, I have misjudged the appropriate
tool for my job. I will have to find something other than LDAP. But on
my way out the door, I was just wondering if anyone could tell me where
that computing time is being spent during ldapadds. I'm curious you see.
So in my previous note I explained how I attempted to set up a run of
adding 10K records all in an ldif file with ldap add. I quit it after it
was about 25% complete and project that it was on its way to consuming
7692 seconds. By contrast, I wrote a perl script to write 10K records
(the same ones) to a dmb... that took 2 seconds.
For comparison sake, thats:
1) a 4 order of magnatude difference
2) an over 380,000% difference
3) porportionally the same as the variance in difference between the
driving distance between down Town San Jose and Fishermans Wharf San
the earth and the moon. Really*
Now I hope I'm not comming across as insulting or any thing. It's just
an engineering question. This is not LDAPs design purpose, I've made the
mistake by picking a wrong tool for my job. But that kind of speed
discrepency just shocks me and my brain is screaming "WHY?!? What is
consuming the time?".
SJ to SF = about 60 miles
Earth to moon = 240,000 miles
60/240,000 is about equal to 2/7692
GOMBAS Gabor wrote:
> On Mon, Jun 11, 2001 at 12:04:58PM -0700, Ricky Charlet wrote:
> > OK, I know that LDAPs design goal was to optimize reading/browsing
> > records, not adding/modifying records. But can someone tell me the
> > fundamental reasons that adding records do take such a long time?
> You just said. LDAP was designed for many reads and few writes.
> > Even though I knew that record adding was not optimized, I am really
> > very supprised at how slow it is.
> It is not only a matter of optimalization but rather a design principle.
> > I ran a simple test of adding 10,000
> > person records to a database. After 30 minutes it was about 1/4
> > finished. According to my measurements it was adding records at about
> > 1.3/second. That is plainly unworkable at my intended scale.
> > The command line I used was `ldapadd -x -D "cn=Manager,dc=ricky,dc=com"
> > -f tenK.ldif -w *******`
> If you want to add such many records then shut down slurpd and use slapadd.
> The LDAP protocol was _not_ designed for that amount of writes. If you
> need so much modifications all the time you should leave LDAP alone and
> use some relational database.
> Gabor Gombas Eotvos Lorand University
> E-mail: email@example.com Hungary
Ricky Charlet : Redcreek Communications : usa (510) 795-6903