[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: bdb bad performance again (really need help)

--On Tuesday, November 04, 2003 10:27 PM +0100 Vadim Tarassov <vadim.tarassov@swissonline.ch> wrote:

Hi Matt,

Thanx a lot for repyling. We increased cache size up to the value enough
to keep all data in memory. This resulted in much better performance.
However I still have odd feeling about it as I don't really understand
why performance descreased so much (several orders of magnitude) when
cache size was small. This is due to my miserable knowledge of berkeley
db. And why preformance decreased in irreversable manner? Why even
restart of LDAP server does not bring performance to proper value? Do you
know what exactly happens to bdb backend when cache size is too small?
You see, I feel unconfortable if someone tells me "put contents of your
backend in memory and everything will be fine", because

1) I don't understand why should I do it ... Look there must be an error
somewhere, if performance degradates so much suddenly and does not
recover . Why db_recover "recovers" performance?
2) I can not increase cache size as fast as ammount of data in our LDAP
server grows ...

We do not put our entire DB into memory, however, we do run a very large cache (2.5GB).

The reason is essentially this -- If you do not have a sufficient cache size, the system must read almost continuously from disk to answer queries. This trashes the disk, and eventually has the end effect of corrupting the database. What the db_recover command does is restore the database from a corrupted state. So, if db_recover was restoring your performance, it was because your database was getting thrashed from having too little cache space in memory.

At least, that is how I understand it. ;)


-- Quanah Gibson-Mount Principal Software Developer ITSS/TSS/Computing Systems ITSS/TSS/Infrastructure Operations Stanford University GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html