Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index

[h.b.furuseth@usit.uio.no]

Can cn=config have 2 lock levels? That fixes this ITS (my
ldapwhoami hang) without introducing cn=config failures.

The outer lock serializes cn=config ops including the indexer.
Other ops do not lock it.  The inner lock blocks slapd like
cn=config does now, and should only be held briefly.  Always
hold the outer lock when taking the inner lock.

Also, the inner lock could be taken and released several
times during a config op when feasible.  That might improve
non-config op latency, for the price of slower config ops.

On 1/20/19 4:42 PM, Howard Chu wrote:
> Hallvard Breien Furuseth wrote:
>> On 1/19/19 8:04 PM, Howard Chu wrote:
>>> modifications to cn=config require no other threads to be active. Once the background indexer
>>> starts running, its activity will of course block further cn=config modifications until it
>>> completes.
>>
>> Nothing "of course" about it.  I don't know why running
>> the background indexer is considered a change to cn=config.
>
> It is not. But the indexing rules can be changed by further config mods, and we
> don't want the config to change out from under it.

OK.  I was just parsing your worlds too literally.
Also, I'm trying to think like an admin, not an OpenLDAP insider:

>> The ldapmodify did the change to cn=config and completed.
>>
>> I can guess that it might block cn=config simply because that's
>> the safe way to have both an indexer and dynamic config.  Or
>> I can read the code.  But "read the code" != "of course".
>
> This was all discussed on the -devel mailing list when the feature was first implemented.
> http://www.openldap.org/lists/openldap-devel/200504/msg00090.html

Thanks.

>> So... back to start, I guess I was suggesting that the
>> indexing task should stop if another config mod comes in,
>> if it can resume afterwards.  And the config change need
>> only fail if it might prevent the indexer from resuming.
>> I suppose that gets complicated in a hurry though.
> 
> Stopping and resuming is problematic, especially if other config changes to the DB occur
> that would invalidate the in-progress indexing.

OK.

> We could arrange to return LDAP_BUSY to config mods if any background indexers are running.
> Or as I already mentioned, we could try to only do this for relevant config mods, e.g. to
> the active backend, or to global schema.

I'm mostly interested in the worst expected case during
normal cn=config use - whether hang or error.

Limiting the problems beyond that is a bonus.  Or maybe not.
A cn=config which fails in some obscure cases is flaky.  A
cn=config which fails more frequently but in predictable ways
can be annoying, but is easy to figure out how to operate.

I suppose a simple test is how simple the documentation can be:
If I want non-config operations to not hang, what do I do and
what do I avoid?  And same thing if want to write a config
script which won't need to cope with expected failures.