[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: ldapsearch slow with +110000 entries

On Wed, 2002-06-19 at 02:19, David Wright wrote:
> > Are the configurations bellow:
> > 
> > index	 uid,uidNumber	eq
> > 
> > and
> > 
> > index	uid		eq
> > index	uidNumber	eq
> > 
> > the same thing?
> Yes. By the way, once you do eq, you might as well do pres too, as it 
> adds only a tiny additional memory requirement.

Ok, eq and pres now!

> >>Why are you only using a 10 MB dbcache when you have 1 GB of memory?
>  >
> > Ok, I'm using now 50MB dbcachesize
> I assume you actually proceeded by benchmarking with varying cachesize 
> and dbcachesize, then choosing values at which you saw no more return on 
> your resource comitments. If not, this would be a useful exercise.

Yes, I did it. At least during the tests more dbcache didn't bring more

> > For these tests I'm using ide disks. The production configuration are
> > scsi u2w/lvd disks. IMHO, I SHOULD have acceptable timings in the test
> > plataform for at least ONE search, since in the production I'm going to
> > have many concurrent searchs!
> You should definitely tune your ide disks using hdparm. Disk throughput 
> is likely killing you on that first search. See, for example
>    http://linux.oreillynet.com/pub/a/linux/2000/06/29/hdparm.html
> on how to do this. This is good for your system in general, independent 
> of OpenLDAP.

Even following the guidelines pointed by this doc, my ide disk had an
increase of 5MB/s only. Although a good hardware, it's from 98/99... :(

> > index	default						eq,pres
> I think you don't want to do this. Indexing on what you search on is a 
> very good idea. Indexing on what you don't search on will actually slow 
> you down some.

Ok, removed.

> > With these settings cachesize is twice the number of entries and
> > dbcachesize become larger than the largest file in /var/lib/ldap.
> The (approximate) criterion for being able to hold everything in memory 
> is that dbcachesize be as large as the sum of all files in /var/lib/ldap.
> > The first search is slow (about 1min10s) but the others take 3,5s to 4s.
> I suspect that by doing the stuff I mentioned above, you will be able to 
> improve on these numbers further. Do watch the size of your slapd 
> process to make sure you don't run out of memory. And do watch your CPU 
> consumption during searches, to make sure you are not CPU limited.
> Good luck! And feel free to recommend me to your friends. :-)

Thanks! But none of my friends are too crazy to put 100k entries on a
single directory... :)

"Let the source be with you" - Tux