Here at Carnegie Mellon we're running the back-sql backend on our
production master OpenLDAP server and we've reached a very stable
environment. I have made about 10 enhancements to the backend and I
would like to begin sending them back up to the repository. I have
them in my source tree added to 2.2.18.
Two questions:
Is 2.2.18 a good tree to have them in, or is there a better tree, e.g.
HEAD or the theoretical 2.3?
How would you like to see the changes? A series of unified diff's,
each building on the previous? A tar of the whole back-sql?
Here's a summary of the enhancements, sorted somewhat in order of
their code impact:
* Added an option to set the SQL query for the "id_query" operation.
* Added the backend API functions for the entry_get and entry_release
entry points. The functions are needed for performing ACLs which
employ "dnattr", "group", and other clauses.
* Added an option to the SQL backend called "use_subtree_shortcut".
When a search is performed, the SQL query includes a WHERE clause
which says the DN must be "LIKE %<searchbase>". The LIKE operation
can be slow in an RDBM. This shortcut option says that if the
searchbase of the LDAP search is the root DN of the SQL backend,
and thus all objects will match the LIKE operator, do not include
the "LIKE %<searchbase>" clause in the SQL query (it is replaced
instead by the always true "1=1" clause to keep the "AND"'s working
correctly). This option is off by default, and should be
turned on only if all objects to be found in the RDBM are under the
same root DN. Multiple backends working within the same RDBM table
space would encounter problems. LDAP searches whose searchbase are
not at the root DN will bypass this shortcut and employ the LIKE clause.
* Oracle has reserved the keyword "uid", and so tables cannot be
created with a column with that name. This is unfortunately a common
word for an LDAP server that stores information about computer accounts.
In order to have an LDAP attribute named "uid", a special exception
was added for fetching a column named "uid". The SQL column can be
called anything, e.g. "username", but when the SQL query to fetch an
attribute named "uid" is made, the "AS" clause changes the column name to
"uidtmp" instead of "uid". When ODBC is reading back results for the
"uid" attribute, it will seek the result column "uidtmp". This option is
shielded by a #define macro.
* Improved sr_text messages to tell what went wrong in backend,
instead of just "SQL-backend error".
* Added a "create_hint" column to ldap_oc_mappings table. Allows
taking the value of an attr named in "create_hint" and passing it to
the create_proc procedure. This is necessary for when an objectClass's
table is partition indexed by some indexing column and thus the value
in that indexing column cannot change after the row is created. The
value for the indexed column is passed into the create_proc, which
uses it to fill in the indexed column as the new row is created.
* When loading the values of an attribute, the count(*) of the number
of values is fetched first and memory is allocated for the array of
values and normalized values. The old system of loading the values one
by one and running realloc() on the array of values and normalized
values each time was badly fragmenting memory. The array of values and
normalized values would be side by side in memory, and realloc()'ing
them over and over would force them to leapfrog each other through all
of available memory. Attrs with a large number of values could not be
loaded without crashing the slapd daemon.
* Added code to interpret the value returned by stored procedures
which have expect_return set. Returned value is interpreted as an LDAP
return code. This allows the distinction between the SQL failing to
execute and the SQL running to completion and returning an error code
which can indicate a policy violation.
* Added RDBM connection pooling. Once an operation is finished the
connection to the RDBM is returned to a pool rather than closing.
Allows the next operation to skip the initialization and authentication
phases of contacting the RDBM. Also, if licensing with ODBC places
a limit on the number of connections, an LDAP thread can block waiting
for another thread to finish, so that no LDAP errors are returned
for having more LDAP connections than allowed RDBM connections. An
RDBM connection which receives an SQL error is marked as "tainted"
so that it will be closed rather than returned to the pool.
Also, RDBM connections must be bound to a given LDAP connection AND
operation number, and NOT just the connection number. Asynchronous
LDAP clients can have multiple simultaneous LDAP operations which
should not share the same RDBM connection. A given LDAP operation can
even make multiple SQL operations (e.g. a BIND operation which
requires SASL to perform an LDAP search to convert the SASL ID to an
LDAP DN), so each RDBM connection now has a refcount that must reach
zero before the connection is returned to the free pool.
* Added ability to change the objectClass of an object. Required
considerable work to copy all attributes out of old object and into
new object. Does a schema check before proceeding. Creates a new
object, fills it in, deletes the old object, then changes the
oc_map_id and keyval of the entry in the "ldap_entries" table.
-Mark Adamson
Carnegie Mellon