Re: slapd new backend

[Hallvard B Furuseth <h.b.furuseth@usit.uio.no>]

Malcolm Cowen writes:
> I've got the API documentation, and the setup type documentation from
> the website.  None of that covers the backends much however, so I'm
> currently going through the various call hooks provided by the system
> using debug to see what is going on.

slap.h contains quite a bit of comments about the API.

Here are some more notes in C-like format I've written while trying to
figure the API out.  Note that I haven't checked yet that I've gotten it
straight.  See also ITS#5328.

struct BackendInfo {
    ...;

    /*
     * be->be_<operation> aka bi->bi_op_<operation>:
     *
     * Handle one LDAP operation, except what auxiliary function below handle.
     * Set the SlapReply, in some cases send it too.
     * Return an LDAP result code.
     * ? [What is the relation between sent and returned result code?] ?
     *
     * bind
     *  If returning LDAP_SUCCESS, do not send it first - the frontend will.
     *  Do send non-success responses before returning.
     * unbind, abandon:
     *  These LDAP operations have no response, so don't send one.
     * search, modify, modrdn, add, delete:
     *  Send response.
     * compare:
     *  Send response, but return LDAP_SUCCESS after sending
     *  LDAP_COMPARE_FALSE or LDAP_COMPARE_TRUE.
     * bi_extended (be->be_extended), bi_op_cancel (be->be_cancel):
     *  Do not send response, just set up the SlapReply.
     *
     * Operations that can be abandoned may return SLAPD_ABANDON if
     * op->o_abandon is set.  It is OK to "send" SLAPD_ABANDON with
     * send_ldap_response(); that function will catch it.
     */
    typedef int BI_op_func( Operation *op, SlapReply *rs );
    BI_op_func  *bi_op_bind;
    BI_op_func  *bi_op_unbind;
    BI_op_func  *bi_op_search;
    BI_op_func  *bi_op_compare;
    BI_op_func  *bi_op_modify;
    BI_op_func  *bi_op_modrdn;
    BI_op_func  *bi_op_add;
    BI_op_func  *bi_op_delete;
    BI_op_func  *bi_op_abandon;
    /* Extended Operations Helper */
    BI_op_func  *bi_extended;
    BI_op_func  *bi_op_cancel;

    /* Auxilary Functions */

    /*
     * Insert backend-specific operational attributes in
     * rs->sr_operational_attrs.  The frontend handles entryDN and
     * subschemaSubentry, unless REP_NO_ENTRYDN and REP_NO_SUBSCHEMA are
     * set.  Called from backend_operational().
     */
    BI_op_func          *bi_operational;    /* be->be_operational() */

    /*
     * Called before LDAP operation handlers.
     * May send and return a referral if the operation should do so.
     * The existence of this function makes LDAP operations non-atomic.
     *
     * It should send and return LDAP_REFERRAL, or return but NOT send
     * another result code.  With LDAP_SUCCESS, slapd will proceed to
     * the operation handler.  With other result codes, slapd will put
     * them in rs->sr_err and send the error.
     *
     * Called from backend_check_referrals().
     */
    BI_op_func          *bi_chk_referrals;  /* be->be_chk_referrals() */

    /*
     * bi->bi_chk_controls() aka be->be_chk_controls():
     * May react to controls in op->o_ctrlflag from the LDAP operation,
     * before slapd checks bi->bi_controls and globally handled controls.
     * Called before LDAP operation handlers.
     *
     * Do not send results.
     * Return SLAP_CB_CONTINUE for slapd to check other supported controls.
     * Return LDAP_SUCCESS to proceed to the operation without checking
     * other controls.  Otherwise return an error code which slapd will
     * send to fail the operation.
     * 
     * Called from backend_check_restrictions().
     *
     * See overlays/pcache.c for an example.
     * Presumably it returns LDAP_SUCCESS, it must first do the work
     * normally done by backend_check_controls().
     */
    BI_op_func          *bi_chk_controls;   /* be->be_chk_controls() */

    /*
     * bi->bi_entry_get_rw() aka be->be_fetch():
     *
     * Fetch an entry for operation 'op' with normalized DN 'ndn', for
     * read operations if 'rw' == 0, for write operations if 'rw' == 1.
     * If 'oc' and/or 'at' are non-NULL but the entry does not contain
     * them, the function may fail with LDAP_NO_SUCH_ATTRIBUTE.
     *
     * Return an LDAP result code.  *e is NULL on entry, and must be
     * non-NULL on exit if and only if LDAP_SUCCESS is returned.  I think.
     *
     * Slapd returns the entry to the backend with bi_entry_release_rw()
     * if that is set, otherwise hopefully frees it with entry_free().
     * ? [See below.]
     *
     * Called from be_entry_get_rw().
     *
     * ? [What can the caller and the backend do with the entry before
     *    it is released?  Can it be modified?  No current callers
     *    use rw!=0, can it be modified in that case? ]
     *
     * ? [Callers of entry_release seem to choose rw based on which
     *    LDAP operation is being used, which maybe means that
     *    be_fetch() callers should do the same - i.e. set rw for
     *    the Add operation even for entries which the caller will
     *    only read.  Need to check.  Except, do any callers use
     *    * rw != 0 anyway? ]
     */
    int (*bi_entry_get_rw)(
        Operation *op, struct berval *ndn,
        ObjectClass *oc, AttributeDescription *at, int rw, Entry **e );

    /*
     * bi->bi_entry_release_rw() aka be->be_release():
     *
     * Returns an entry to the backend, possibly fetched with be_fetch.
     *
     * Slapd will also call it with entries that did not originate
     * with the backend, I do not know if that is intentional.
     * ? [After some overlay did entry_dup(), maybe.  Must check.]
     *
     * Called from be_entry_release_rw(), which frees the entry with
     * entry_free() if there is no be_entry_release_rw function.
     * ? [However frontendDB->be_release() also calls this function,
     *    but does not free the entry if no function is found.  Then
     *    it just returns LDAP_NO_SUCH_OBJECT.]
     *
     * Return an LDAP result code, which is hardly ever used.
     * Only SLAP_CB_CONTINUE seems to matter: If there are overlays,
     * this means other bi_entry_release_rw() functions are called,
     * or entry_free() if all else fails.
     *
     * ? [Maybe back-relay has a memory leak and should call entry_free
     *    if there is no backend function.  frontendDB does not, overlay
     *    translucent does, backglue does but has a FIXME comment about
     *    it.  Fix what?]
     */
    int (*bi_entry_release_rw)( Operation *op, Entry *e, int rw );

    /*
     * bi->bi_has_subordinates() aka be->be_has_subordinates():
     *
     * Generate the value for compare(entry, "hasSubordinates=TRUE").
     * Returns an LDAP result code.  When it returns LDAP_SUCCESS, it
     * must set *hasSubs to LDAP_COMPARE_TRUE or LDAP_COMPARE_FALSE.
     *
     * This function really should succeed except on internal errors:
     * test_filter() with the presence filter (hasSubordinates=*) does
     * not call this function, it assumes attribute hasSubordinates will
     * exist if this function does.  And for equality match it turns
     * other result codes than LDAP_SUCCESS into LDAP_OTHER.  (Though it
     * can also be used for the Compare operation, in that case a result
     * code of LDAP_NO_SUCH_ATTRIBUTE also makes sense.)
     *
     * back-relay sets this function, which is wrong when relaying to a
     * backend which does not.  OTOH slapi_entry_has_children() exists
     * even though slapi does not set bi_has_subordinates.
     *
     * bi->bi_entry_get_rw must exist if this function does; Compare
     * will call it if !defined(SLAP_COMPARE_IN_FRONTEND).
     */
    int (*bi_has_subordinates)( Operation *op, Entry *e, int *hasSubs );

    ...;

    /*
     * Connection management, called when a connection is opened or closed.
     * Return an LDAP result code.
     *
     * Overlays must normally return SLAP_CB_CONTINUE, so the backend
     * also gets a chance to prepare for and clean up after connections.
     *
     * Slapd discards other result codes, which is just as well since the
     * overlay code turns SLAP_CB_CONTINUE into LDAP_UNWILLING_TO_PERFORM
     * when the backend has no connection handler.
     *
     * Slapi can set up a fake connection and call backend_connection_init,
     * but I see no matching call to backend_connection_destroy.  Maybe
     * the connection is given to slapd which destroys it?
     */
    typedef int BI_conn_func( BackendDB *bd, Connection *c );
    BI_conn_func    *bi_connection_init;    /* be->be_connection_init */
    BI_conn_func    *bi_connection_destroy; /* be->be_connection_destroy */

    /* Hooks for slap tools: bi->bi_tool_<act> aka be->be_<act> */
    int (*bi_tool_entry_open)( BackendDB *be, int mode );
    int (*bi_tool_entry_close)( BackendDB *be );
    ...;
};