[Date Prev][Date Next]
Re: Another observation about the beta release ...
> Anthony Brock (firstname.lastname@example.org) writes:
> > While attempting to import some of our current data, we have noticed some
> > significant resource requirements by the beta software. Basically, as the
> > import progresses, the server steadily starts to slow from one student per
> > second until student records are being add once every couple minutes. This
> > is usually around 700 to 800 records into the import.
> > As it progresses, we also noticed available memory on the machine dropping
> > from over 200 meg until we only have 5 meg available with over 100 meg of
> > swap space in use. The server then dumped it's core and exited.
> > I haven't had time to look into it, but this seems a bit extreme for a
> > simple import, especially when the source LDAP database (we used a perl
> > script on a different machine to read from a remote LDAP server and import
> > to the beta) backend only occupies about 10 meg of disk space.
> Sounds like a database issue to me. This often occurs because as each new
> record is added, the (steadily growing) index must be searched to avoid adding
> duplicate records. Hence, the slowdown.
> If this were an exposed database, I'd recommend removing the indices, adding
> the records, then re-adding the indices. That would speed things up quite a
> Ed Carp, N7EKG email@example.com 940/367-2744 cell phone
One approach is to use slapadd, which will rebuild your database from
scratch (an ldif file). I know Kurt prefers ldapadd.... so what you
can do to speed this up is to set dbcachenosync (sp?) in slapd.conf
to disable flushing the database on each write. This is a good idea
when you are pumping large amounts into the database, but should be
avoided during normal times.