[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: why is adding records slow?
Howdy,
Thanks for your response, Gabor. Yes, I have misjudged the appropriate
tool for my job. I will have to find something other than LDAP. But on
my way out the door, I was just wondering if anyone could tell me where
that computing time is being spent during ldapadds. I'm curious you see.
So in my previous note I explained how I attempted to set up a run of
adding 10K records all in an ldif file with ldap add. I quit it after it
was about 25% complete and project that it was on its way to consuming
7692 seconds. By contrast, I wrote a perl script to write 10K records
(the same ones) to a dmb... that took 2 seconds.
For comparison sake, thats:
1) a 4 order of magnatude difference
2) an over 380,000% difference
3) porportionally the same as the variance in difference between the
driving distance between down Town San Jose and Fishermans Wharf San
Francisco, and
the earth and the moon. Really*
Now I hope I'm not comming across as insulting or any thing. It's just
an engineering question. This is not LDAPs design purpose, I've made the
mistake by picking a wrong tool for my job. But that kind of speed
discrepency just shocks me and my brain is screaming "WHY?!? What is
consuming the time?".
*
SJ to SF = about 60 miles
Earth to moon = 240,000 miles
60/240,000 is about equal to 2/7692
GOMBAS Gabor wrote:
>
> On Mon, Jun 11, 2001 at 12:04:58PM -0700, Ricky Charlet wrote:
>
> > OK, I know that LDAPs design goal was to optimize reading/browsing
> > records, not adding/modifying records. But can someone tell me the
> > fundamental reasons that adding records do take such a long time?
>
> You just said. LDAP was designed for many reads and few writes.
>
> > Even though I knew that record adding was not optimized, I am really
> > very supprised at how slow it is.
>
> It is not only a matter of optimalization but rather a design principle.
>
> > I ran a simple test of adding 10,000
> > person records to a database. After 30 minutes it was about 1/4
> > finished. According to my measurements it was adding records at about
> > 1.3/second. That is plainly unworkable at my intended scale.
>
> [...]
>
> > The command line I used was `ldapadd -x -D "cn=Manager,dc=ricky,dc=com"
> > -f tenK.ldif -w *******`
>
> If you want to add such many records then shut down slurpd and use slapadd.
> The LDAP protocol was _not_ designed for that amount of writes. If you
> need so much modifications all the time you should leave LDAP alone and
> use some relational database.
>
> Gabor
>
> --
> Gabor Gombas Eotvos Lorand University
> E-mail: gombasg@inf.elte.hu Hungary
--
Ricky Charlet : Redcreek Communications : usa (510) 795-6903