RE: (ITS#3564) Programatic Insert Scaleability Problem

[Armbrust.Daniel@mayo.edu]

>Bulk loading of a running server is not specifically addressed in any 
>current releases.

So how exactly is one supposed to put things into an ldap database if you can't load a running server?
I certainly shouldn't have to manually convert my data to ldif to be able to load it.


>We need to see a backtrace 
>from the crash to understand what has failed. You should probably also 
>run with debug level 0; there may be diagnostic messages from the 
>BerkeleyDB library as well that you're not seeing at the moment.

I'll try to get more logs, but trying to load this much stuff with a debug level of 0 completely tanks the performance.  I don't know if I even have enough disk space to hold the file it would generate given how verbose it is....

>bdb backend
>DB_CONFIG file:
>set_flags       DB_TXN_NOSYNC
>set_flags       DB_TXN_NOT_DURABLE
>set_cachesize   1       0       1

>If that's all that you have in your DB_CONFIG, then you're definitely 
>limiting yourself. Remember that you MUST store the transaction logs on 
>a separate physical disk from your database files to get reasonable 
>write performance.

If my understanding of the DB_CONFIG file is correct, the configuration I have above should be disabling logging entirely.  Performance with logging turned on is to pathetic to mention for this type of operation....

I'll post additional information relating to the crash to the other bug (http://www.openldap.org/its/index.cgi?findid=3565 ) when I am able to generate it.

But your first comment that I responded to above has significantly cooled our interest in using an ldap backend in our project (http://informatics.mayo.edu/LexGrid/index.php), so its not as high of priority for me anymore.