Re: Another observation about the beta release ...

[kunkee@neosoft.com (Randy Kunkee)]

> Anthony Brock (abrock@georgefox.edu) writes:
> 
> > While attempting to import some of our current data, we have noticed some 
> > significant resource requirements by the beta software.  Basically, as the 
> > import progresses, the server steadily starts to slow from one student per 
> > second until student records are being add once every couple minutes.  This 
> > is usually around 700 to 800 records into the import.
> > 
> > As it progresses, we also noticed available memory on the machine dropping 
> > from over 200 meg until we only have 5 meg available with over 100 meg of 
> > swap space in use.  The server then dumped it's core and exited.
> > 
> > I haven't had time to look into it, but this seems a bit extreme for a 
> > simple import, especially when the source LDAP database (we used a perl 
> > script on a different machine to read from a remote LDAP server and import 
> > to the beta) backend only occupies about 10 meg of disk space.
> 
> Sounds like a database issue to me.  This often occurs because as each new
> record is added, the (steadily growing) index must be searched to avoid adding
> duplicate records.  Hence, the slowdown.
> 
> If this were an exposed database, I'd recommend removing the indices, adding
> the records, then re-adding the indices.  That would speed things up quite a
> bit.
> --
> Ed Carp, N7EKG  	erc@pobox.com		940/367-2744 cell phone
> 			http://www.pobox.com/~erc
> 

One approach is to use slapadd, which will rebuild your database from
scratch (an ldif file).  I know Kurt prefers ldapadd.... so what you
can do to speed this up is to set dbcachenosync (sp?) in slapd.conf
to disable flushing the database on each write.  This is a good idea
when you are pumping large amounts into the database, but should be
avoided during normal times.

Regards,
Randy