On 2/16/07, Howard Chu <hyc@symas.com> wrote:
> The Admin Guide still has not been updated with all of the relevant changes,
> so here are some notes on new features in the 2.4 release... I believe all of
> the manpages are up to date, so you can get specifics from them.
>
> More complete cn=config functionality:
> There is a new slapd-config(5) manpage for the cn=config backend.
>
> the original design called for auto-renaming of config entries when you
> insert or delete entries with ordered names, but that was not implemented in
> 2.3. It is now in 2.4. This means, e.g., if you have
> olcDatabase={1}bdb,cn=config
> olcSuffix: dc=example,dc=com
>
> and you want to add a new subordinate, now you can:
> ldapadd olcDatabase={1}bdb,cn=config
> olcSuffix: dc=foo,dc=example,dc=com
>
> this will insert a new BDB database in slot 1 and bump all following
> databases down one, so the original BDB database will now be named
> olcDatabase={2}bdb,cn=config
> olcSuffix: dc=example,dc=com
>
> In 2.3 you were only able to add new schema elements, not delete or
> modify existing elements. In 2.4 you can modify schema at will. (Except for
> the hardcoded system schema, of course.)
>
>
> More sophisticated syncrepl configurations:
> the original implementation of syncrepl in OpenLDAP 2.2 was intended to
> support multiple consumers within the same database, but that feature never
> worked and was removed from OpenLDAP 2.3. I.e., you could only configure a
> single consumer in any database.
>
> In 2.4 you can configure multiple consumers in a single database. The
> configuration possibilities here are quite complex and numerous. You can
> configure consumers over arbitrary subtrees of a database (disjoint or
> overlapping). Any portion of the database may in turn be provided to other
> consumers using the syncprov overlay. The syncprov overlay works with any
> number of consumers over a single database or over arbitrarily many glued
> databases.
>
> As a consequence of the work to support multiple consumer contexts, the
> syncrepl system now supports full N-way multimaster replication with
> entry-level conflict resolution. There are some important constraints, of
> course: In order to maintain consistent results across all servers, you must
> maintain tightly synchronized clocks across all participating servers (e.g.,
> you must use NTP on all servers). The entryCSNs used for replication now
> record timestamps with microsecond resolution, instead of just seconds. The
> delta-syncrepl code has not been updated to support multimaster usage yet,
> that will come later in the 2.4 cycle.
>
> On a related note, syncrepl was explicitly disabled on cn=config in 2.3.
> It is now fully supported in 2.4; you can use syncrepl to replicate an entire
> server configuration from one server to arbitrarily many other servers. It's
> possible to clone an entire running slapd using just a small (less than 10
> lines) seed configuration, or you can just replicate the schema subtrees,
> etc. Tests 049 and 050 in the test suite provide working examples of these
> capabilities.
>
> In 2.3 you could configure syncrepl as a full push-mode replicator by
> using it in conjunction with a back-ldap pointed at the target server. But
> because the back-ldap database needs to have a suffix corresponding to the
> target's suffix, you could only configure one instance per slapd.
>
> In 2.4 you can define a database to be "hidden" which means that its
> suffix is ignored when checking for name collisions, and the database will
> never be used to answer requests received by the frontend. Using this hidden
> database feature allows you to configure multiple databases with the same
> suffix, allowing you to set up multiple back-ldap instances for pushing
> replication of a single database to multiple targets. There may be other uses
> for hidden databases as well (e.g., using a syncrepl consumer to maintain a
> *local* mirror of a database on a separate filesystem).
>
>
> More extensive TLS configuration control:
> In 2.3, the TLS configuration in slapd was only used by the slapd
> listeners. For outbound connections used by e.g. back-ldap or syncrepl their
> TLS parameters came from the system's ldap.conf file.
> In 2.4 all of these sessions inherit their settings from the main slapd
> configuration but settings can be individually overridden on a
> per-config-item basis. This is particularly helpful if you use
> certificate-based authentication and need to use a different client
> certificate for different destinations.
>
>
> Various performance enhancements:
> Too many to list. Some notable changes - ldapadd used to be a couple of
> orders of magnitude slower than "slapadd -q". It's now at worst only about
> half the speed of slapadd -q. A few weeks ago I did some comparisons of all
> the 2.x OpenLDAP releases; the results are in the slides from my SCALE
> presentation and you can find a copy here:
> http://www.highlandsun.com/hyc/scale2007.pdf
>
> That compared 2.0.27, 2.1.30, 2.2.30, 2.3.33, and HEAD (as of a couple
> weeks ago). Toward the latter end of the "Cached Search Performance" chart it
> gets hard to see the difference because the runtimes are so small, but the
> new code is about 25% faster than 2.3, which was about 20% faster than 2.2,
> which was about 100% faster than 2.1, which was about 100% faster than 2.0,
> in that particular search scenario. That test basically searched a 1.3GB DB
> of 380836 entries (all in the slapd entry cache) in under 1 second. i.e., on
> a 2.4GHz CPU with DDR400 ECC/Registered RAM we can search over 500 thousand
> entries per second. The search was on an unindexed attribute using a filter
> that would not match any entry, forcing slapd to examine every entry in the
> DB, testing the filter for a match.
> Essentially the slapd entry cache in back-bdb/back-hdb is so efficient
> the search processing time is almost invisible; the runtime is limited only
> by the memory bandwidth of the machine. (The search data rate corresponds to
> about 3.5GB/sec; the memory bandwidth on the machine is only about 4GB/sec
> due to ECC and register latency.)
>
> I think it goes without saying that no other Directory Server in the world is
> this fast or this efficient. Couple that with the scalability, manageability,
> flexibility, and just the sheer know-how behind this software, and nothing
> else is even remotely comparable.
Great, great, great, and great!
Have you done any testing with 2.4/HEAD on bdb 4.5 or any of the newer
oracle/bdb stuff? I heard they were making a lot of performance
improvements for writes. (at least, that's what our local oracle fan
club says)