[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: OpenLDAP 2.5

> We should also walk thru the Software Enhancement requests and
> decide which to accept and which to reject. Currently there are 37
> outstanding.

Here's a few things we talked about on IRC:

* Re-design and re-implemenation of the C LDAP (and LBER) API.

per http://scratchpad.wikia.com/wiki/LDAP_C_API

** No global state; everything in app or connection handles.  No
   exceptions, no mercy!
** Use function pointers to allow override of
*** Memory allocation
*** Non-reentrant functions
*** Have sane internal defaults as well as defaults for nspr, apr,
    glib, etc
** Better defaults!  (v3 etc)
** Simple function alternatives for simple apps
** Use structures instead of many arguments.

* Overhaul Debug():

** Clean up multi-arg issues
** Revise (create) log formatting guidelines
** Better macros to normalize function trace enter/leave
** Support adding an optional high-res timestamp

* Revised memory allocation:

** Common memory allocator interface with explicit scoping/extent tags
** Consider built-in substitutes for malloc with better performance
** Possible interactions/conflicts/advantages of back-mdb integration

* Back-MDB concerns:

** Schema info in non-mmap ram pointed to from an mmap()d database?
   (no, and you mentioned this, but we'll need to re-arrange things,
   as discussed on IRC)  Per DB schemas?

* Ensure fork() can based safely in more places - Essential for a
  fully operational slapo-(shell, perl, lua, python, etc)

* With API cleanups here and there, start supporting more languages
  like those.

* liblforth for dynamic syntax extensions and other features.
  (back-forth just sounds like an apr1 proposal...)

* Entry cache overhaul

** The right fix for this is MDB instead, but...
** Would be very nice to specify limits in terms of memory size rather
   than object count.  Yes, this is hard. - Allocation of entry blocks
   into usage pools with generational mark/sweep collections?

> > 2. If support for slapd.conf is completely dropped would it also
> > be possible to implement a more client-friendly back-config schema
> > which does not have to be backward compatible to slapd.conf?

> Describe what would be more client-friendly.

* Make database suffixes be the RDN for database configuration -
  Use unique RDNs with inherently ordered characteristics where-ever
  possible Obviously not for ACLs, but many other things are fixable

* Similarly, Drop the concept of backend ordering entirely and move
  slapo-glue into more generic VFS-like support of the DIT

* Databases should only store one suffix.  No, I'm not sure how that
  should integrate with people putting "" in a database yet.  Need to
  figure that out.

* slapmodify - too useful to not implement now that we have a
  configdb.  Bonus points if it (and slapadd) can be made safe to use
  with a running directory in all cases.  (Perhaps a cache-
  invalidation-hint unix domain socket?)

* Full two-phase commit support at all layers, including
  read-transaction support.  Back-MDB should make this feasible, and
  it offers lots of nice data reliability guaranties.

** Use multiple dbroots in MDB to track read/write transactions as
   they develop.

** Commited data in MDB is effectively read-only.  Read transactions
   just hold their relevant generational root open.

** Write transactions hold open a parent-generation and maintain a log
   of updates as well as a local difference-tree holding just modified
   branch/leaf nodes.  On prepare, this is reconciled with the new
   master-commited root and entered into the mmap zone.  (or rejected)
   On commit, that new root is promoted to master-commited root.

** Roots disappear when no one is watching them anymore.  If all roots
   and temp-roots are known, we can easily remove internal nodes and
   leaves at the same time, so this is much simpler than full garbage

** Root cleanup can be done asynchronously with respect to the
   triggering reads or writes, potentially in another thread.

** This method is equivalent to, and replaces a journal file.
   Commited MDB tree pages could be mprotect()ed if desirable.

Matthew Backes
Symas Corporation