Re: ldapsearch slow with +110000 entries

["Roberto P.Martins Jr" <roberto@intermega.com.br>]

On Wed, 2002-06-19 at 02:19, David Wright wrote:
> 
> > Are the configurations bellow:
> > 
> > index	 uid,uidNumber	eq
> > 
> > and
> > 
> > index	uid		eq
> > index	uidNumber	eq
> > 
> > the same thing?
> 
> Yes. By the way, once you do eq, you might as well do pres too, as it 
> adds only a tiny additional memory requirement.
> 

Ok, eq and pres now!

> >>Why are you only using a 10 MB dbcache when you have 1 GB of memory?
>  >
> > Ok, I'm using now 50MB dbcachesize
> 
> I assume you actually proceeded by benchmarking with varying cachesize 
> and dbcachesize, then choosing values at which you saw no more return on 
> your resource comitments. If not, this would be a useful exercise.
> 

Yes, I did it. At least during the tests more dbcache didn't bring more
performance...

> > For these tests I'm using ide disks. The production configuration are
> > scsi u2w/lvd disks. IMHO, I SHOULD have acceptable timings in the test
> > plataform for at least ONE search, since in the production I'm going to
> > have many concurrent searchs!
> 
> You should definitely tune your ide disks using hdparm. Disk throughput 
> is likely killing you on that first search. See, for example
>    http://linux.oreillynet.com/pub/a/linux/2000/06/29/hdparm.html
> on how to do this. This is good for your system in general, independent 
> of OpenLDAP.
> 

Even following the guidelines pointed by this doc, my ide disk had an
increase of 5MB/s only. Although a good hardware, it's from 98/99... :(

> > index	default						eq,pres
> 
> I think you don't want to do this. Indexing on what you search on is a 
> very good idea. Indexing on what you don't search on will actually slow 
> you down some.
> 

Ok, removed.

> > With these settings cachesize is twice the number of entries and
> > dbcachesize become larger than the largest file in /var/lib/ldap.
> 
> The (approximate) criterion for being able to hold everything in memory 
> is that dbcachesize be as large as the sum of all files in /var/lib/ldap.
> 
> > The first search is slow (about 1min10s) but the others take 3,5s to 4s.
> 
> I suspect that by doing the stuff I mentioned above, you will be able to 
> improve on these numbers further. Do watch the size of your slapd 
> process to make sure you don't run out of memory. And do watch your CPU 
> consumption during searches, to make sure you are not CPU limited.
> 
> Good luck! And feel free to recommend me to your friends. :-)
> 
> 

Thanks! But none of my friends are too crazy to put 100k entries on a
single directory... :)

-- 
"Let the source be with you" - Tux