[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: back-bdb performance
At 02:14 AM 2001-12-15, Howard Chu wrote:
>> -----Original Message-----
>> From: owner-openldap-devel@OpenLDAP.org
>> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Marijn Meijles
>
>> You wrote:
>> >
>> > If you'd care to give me some hints about what you saw in the update
>> > order, I'm open.
>>
>> Ok, I've looked at the latest cvs version of back-bdb/add.c and it's
>> clear why you get so much deadlocks. The problem is that you look
>> up the parent
>> WITH the transaction, thereby rlocking the parents page. You see,
>> the whole
>> bottleneck are the parents db entries, because in most ldap cases
>> concurrent
>> adds share the parent. So (in the ideal case) you should:
>> - lookup the parent with a null txn
>> - defer parent updates (think old school id2children if you will) as long
>> as you can. That way you minimize the time you hold a lock on the parent
>> (the time between update and commit)
>
>OK, this makes sense, thanks.
>
>> However, I don't think this will work in your case, because you don't have
>> an entry cache which can protect the parent from being changed
>> during an add,
>> therefore you need the parent read to be within the transaction.
>
>As I understand it, reading outside the transaction is still safe (locked),
>it just has no repeatability guarantees. That is acceptable in the case of
>an add where we're only looking for the existence of the parent and checking
>some ACLs. If the parent is changed after we fetch it, that is irrelevant
>for the span of the current add transaction. Of course, if the current
>transaction aborts and we retry from the beginning, re-fetching the parent,
>we may get a different one with different properties. That's still OK
>because we will never leave things in an uncertain state.
Note that locking the parent may also be used to ensure multiple
children of the parent cannot be created with the same name.
>Things might be more problematic for something like modrdn, I'll have to
>look at that later. This is another point in back-hdb's favor, the parent is
>completely independent of changes made to child nodes. (Although I have to
>wonder that anyone needs convincing on this front at all. That's like trying
>to convince someone that MSDOS 2.11 with hierarchical directories is better
>than DOS 1.0, or Mac HFS is better than MFS. It should already have been
>explained ad nauseum by now, and now be self-evident. Flat data structures
>are evil...)
>
>> As for the child txn, I use one to encapsulate the parent updates. That
>> doesn't hurt throughput and reduces the retry time. Although I
>> might drop it again
>> as I went from 3000 to 5 deadlocks during a concurrent add of
>> 2x5000 entries.
>
>This is why I mentioned better deadlock avoidance as an ultimate goal;
>whether or not the increased granularity helps your retry time, it's a lot
>of effort for very little gain. And the gain from deadlock avoidance far
>overshadows it.
> -- Howard