[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: back-bdb performance/configuration: Suggestions?
Today at 8:24am, Quanah Gibson-Mount wrote:
> Interesting to see the difference in load times, but what I don't see here
> is how this is a measure of the performance of back-bdb in answering
> queries. I.e., I'm not convinced that time to load a database is an
> indication how the DB will perform in answering queries (in fact, my
> testing for our setup did not indicate that there was any relation between
> the two). Also, how many indexes are you indexing? There is a bug in
> 2.1.17 if you have over 50 or so indexes with back-bdb, that will be fixed
> in 2.1.18 (or I have a patch I can send you if you are interested). I can
> tell you, that to load our account LDIF (which has 70,221 entries and is
> 72MB in size) takes approximately 11 minutes via slapadd with BDB.
I never meant to indicate that it didn't respond to queries -- I haven't
gotten to testing that yet.
My environment is such that I perform my nightly batch updates by
dumping the database, applying changes and loading a new database, so
the slapadd performance is of high concern to me (I have about 45
minutes and two servers). Today on my production machines (2.0.27) I
can load the database in just under 5 minutes (my production machines
are much better than this test machine -- but still have just one
drive), so my whole batch update process takes about 20 minutes -- 10
minutes to update, 10 minutes to load on two servers. If it suddenly
takes 90 minutes to load the two servers and 10 minutes to build the new
database -- there's a real problem there.
Other things are happening that are driving me to develop a batch update
to the live server. So, I'll probably have to figure out a process that
can handle updating 10,000 entries in under an hour. Unfortunately, I
don't get the feeds from the authoritative data sources until 2am and
other processes depend on the updates being finished before 3am.
No, I'm only generating 15 indexes -- so the over 50 bug isn't an issue.
What is your hardware? Have you specified that before and I've
forgotten?
> The one major thing I see with your configuration, is that your logfiles &
> temporary files appear to be going to the same partition as your BDB
> database. The point of seperating those files out is not that they be in a
> different directory than the BDB database, but (preferably) on a completely
> different disk than the BDB database. Since you only have 1 drive, I don't
> see that that is a possibility. Have you tried putting them on different
> partitions?
No, I have not tried putting them on a different partition. I can try
that, but on the face of it, I didn't see that a different partition (on
the same physical drive) would really make a lot of difference -- am I
wrong?
--
Frank Swasey | http://www.uvm.edu/~fcs
Systems Programmer | Always remember: You are UNIQUE,
University of Vermont | just like everyone else.
=== God Bless Us All ===