CVS log for servers/slapd/back-bdb/search.c

Up to [OpenLDAP] / servers / slapd / back-bdb

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Happy New Year

ITS#6758 back-bdb: Delay setting sr->sr_entry until we know it will be used

ITS#6758 Reset some SlapReply flags & data.
Reset data more consistenlty.  When resetting sr_entry after sending
data, reset sr_flag as well.  Reset rs_attrs and Extended Operation
oid/data after setting and sending them.

happy belated New Year

more on ITS#6507

silence warning

Revert 1.274

ITS#6103 send response on abandon

ITS#6095 experimental fix: avoid purging other cases of in-use EntryInfos,
avoid using purged EntryInfos that have not yet been fully unlinked.

ITS#6056 pagedresults stacked control is not overlay-friendly, add
slap_add_ctrls() function

Update copyright notices

fix indentation

ITS#5730, patch for #5728 broke referrals

ITS#5728 release entries before send_ldap_ber()

More deadlock propagation fixes

Propagate deadlock results for internal searches on write ops

Use read-only txn's instead of read lockers. Support BDB 4.4-4.7

Fix prev commit

Use o_extra instead of o_private

ITS#5409 must reset pagedresults state if no cookie is provided

Warning cleanup: function ptr <=> void* at ldap_pvt_thread_pool_<set/get>key()

Happy New Year (belated)

rename ldap_pvt_thread_pool_setkey_x() to ldap_pvt_thread_pool_setkey() (as part of ITS#5309)

ITS#5232 report and fail on internal errors

Fix pagedresults resume

Cache-thrashing protection: if the DB has more entries than the cache,
don't allow large search operations that touch more entries churn the
existing cached entries.

ITS#5191 do all pagedresult control parsing in the frontend, don't
re-parse in the backend. Keep track of pagedresult controls and state
in backglue.

add helper for internal searchs for objectClass=*

check for shutdown (need to add to other backends?)

More for BDB 4.6 compatibility

ITS#4838 fix fake root handling broken by prev commit

use dn2id index for extended filters using entryDN

Unify use of BDB lockers

happy new year

Cleanup db_lock/unlock parameters

unifdef -DSLAP_ACL_HONOR_DISCLOSE
unifdef -DSLAP_ACL_HONOR_MANAGE
unifdef -DSLAP_OVERLAY_ACCESS
unifdef -DSLAP_ORDERED_PRETTYNORM
unifdef -DSLAP_AUTHZ_SYNTAX
Move LDAP_COLLECTIVE_ATTRIBUTES, LDAP_COMP_MATCH, and LDAP_SYNC_TIMESTAMP
back behind LDAP_DEVEL (as these either too experimental or not yet
implemented).

ITS#3950 delete gratuitous yield()s

Happy new year!

anticipate sr_err/sr_ref cross-check, so that the right error gets logged

unifdef -DBDB_SUBENTRIES -DLDBM_SUBENTRIES

use LDAP_SIZELIMIT_EXCEEDED; rework the return values of slap_send_search_entry(); silence a few warnings

check sizelimit in frontend (ITS#4213)

ITS#4030 fix alias deref for Finding, fix alias error code propagation.

harmonize back-ldbm and back-bdb handling of searchBase disclose access; fixes ITS#3964, ITS#3963 still open

Make sure we release entry lock before sending paged response

minor cleanup

Silence warning

assert expects int. (int)<nonnull ptr/long> can be 0. Use assert(arg!=0/NULL).

goto loop_begin; skipped initialization of scopeok.

cleanup

Fixing memory leakage in attribute aliasing

Cleanup unused variables

Always return SLAPD_ABANDON in response to op->o_abandon being set.

initialize AttributeAssertion before using

condition compilation of DISCLOSE checking

More cleanup of unused stuff

Delete obsolete syncprovider stuff

fix use of uninitialized var

honor 'disclose' ACL on searchBase/compare/referral/matched (ITS#3472 and comments on -devel)

Happy New Year!

adaptive caching implementation

More BDB_PSEARCH cleanup

Disable back-bdb native syncrepl support, enable syncprov overlay,
remove syncrepl stuff from Operation struct

Dynamic control management

log the right function names

rework static backend initialization

fix suspicious capitalization to make subentries control work properly

unifdef -DBDB_ALIASES
delete dead code

further cleanup of search data upon exit, abandon, cancel

syncrepl consistency patch: utilize BDB txn rollback

syncrepl consistency patch

Revert ITS#3353 patch, it needs to be reworked.

minor cleanup

ITS#3353 consolidate slapd globals into a single struct

Modified @oc supprot code to the share anlist routines; Revised anlist routines

unifdef -UNEW_LOGGING

Check for opinfo->boi_txn before using it - sometimes it is NULL even if
opinfo isn't

add paged results support to back-ldap

rework op/rs structures to deal with opeartional attributes

minor cleanup

Make sure DB result codes are returned to caller; add missing retries to
callers of bdb_cache_find_id().

propagate search result (maybe this is not what bdb_psearch() is intended to do...)

use ltid if it exists; use NULL otherwise.

use txn locker if it exists; get one otherwise.

patch for ITS#3173
- passing transaction ptr to psearch

More paged results cleanup

Fix nentries logging

Don't return useless estimate.

improve pagedResults ignoring (hopefully)

return pagedResultsControl with empty cookie at end of search (ITS#3161)

ITS#3182 ? clear ctrls after sending paged response

clarify no limits in (internal) searches

Better fix for ITS#3063, #3186

Another crash for ITS#3063

cleanup

ITS#3092: Rename sl_free() and friends to slap_sl_free()

remove unused structure member ps_id

Fix paged search initialization

completion of limits w/ paged results control

improve previous commit; now special limit on the number of total entries of a paged results can be set via the limits.  Need to decide whether the default should be unlimited or what

honor size limits when using paged results

Fix ITS#3063, special case handling of "" suffix

Zero out rs->sr_err before send_search_entry

use BER_BVNULL

cleanup for release engineering

reset flags (need to check consistence across slapd)

move limits evaluation to frontend

allow search limits based on groups (ITS#2967)

Fix the missing delete problem (reported by Doug Council)

Happy new year

a patch for persistent search

Quick hack for PSEARCH crash.

Initial subordinate implementation (for chaining)

Update no-op control to return LDAP_NO_OPERATION instead of LDAP_SUCCESS.
(back-sql needs to be updated)

Misc code cleanup

add to 'val[.<style>=<value>' ACLs special match styles for DN-valued attributes; add negated objectClass to attribute name lists for ACLs and partial replication

fix for concurrent persistent searches

unifdef -DLDAP_CONTROL_PAGEDRESULTS (as it is always defined)

detect problems in saslauthz

Notices

Support multiple sync replication at the consumer :
1) simultaneous operation of multiple active sync replication threads
2) cookie management for individual sync replication thread
   (include rid=%3d to the slapd cookie command line option (-c))

Collective entryUUID transmission of PRESENT messages in a single PDU
(refer to draft-zeilenga-ldup-sync-04.txt)

update ldapsync/syncrepl code according to the new version of
	the protocol (draft-zeilenga-ldup-sync-04)

1. Session history support
	- memory based session history to minimize sync traffic
	- when client is covered by a session history, then
      [add+delete] mode is used
	- when client cookie is not covered by the history because
      the cookie is too outdated and/or the history is truncated,
	  [add+present] mode is used
2. Sync cookie syntax : comma separated name=value pairs
	- csn=yyyymmddhh:mm:ssZ#0xSSSS#r#ssssr,sid=nnn

Don't return garbage when plist is empty.

Shortcut syncrepl candidate selection if CSN is up to date

ITS#2684 keep psearch operations on the connection's active list so they
can be located by connection_abandon() during an unexpected close. Remove
the operations in bdb_abandon() etc.

Finish off berbuf fix

do not cache internal search results for present entry priming

Initial support for pre/post read controls.
TODO:
	Fix transactional consistency
	Add client response control handling

add bdb_get_commit_csn() for use in bdb_search

memory mgmt cleanup

more patches for ITS#2692 (Thanks Luke!)

unifdef LDAP_SYNC and LDAP_SYNCREPL

cleanup

LDAP Sync protocol extension
- do not send updates in state mode when
  syncCookie >= contextCSN (or omitCSN if changelog is used)

Context CSN Patch (5)
- persistent search patch
- cascading search patch

Context CSN Patch (2)
- BDB context csn codes moved to a separate function in back-bdb/ctxcsn.c

Context CSN patch (1) - cleanup
- serialize search_candidates() with the read of search_context_csn

Context CSN patch (1)
- currenty works for refreshOnly mode of LDAP Sync
- Context CSN for add / modify is implemented
- code for delete / modrdn / refreshAndPersist will be soon committed

LDAP Sync protocol change : state mode (add+present) and log mode (add+delete)

Various udpates for SyncRepl

LCUP primitive routines removed

not returning glue entries unless manageDSAit is set

Add server side assert control support.

Merge partial and intermediate responses

base entry checking for refreshAndPersist as well

Restructured to avoid direct use of sl_malloc functions. Minimizes impact
of #ifdef USE_SL_MALLOC...

If the sync mode is refreshAndPersist,
search does not return error when base entry is not found

persistent search updates with recent changes

ITS#2506 - fix prev patch, don't ignore undefined filters. treat as False,
test_filter will never match them anyway.

Fix HDB search_aliases crash, cleanup search scope processing: call dn2idl
directly instead of going thru filter_candidates. Use hdb_ function prefix
in dn2id.c to avoid ambiguities...

Use computed filters to pass in search scope.

- manageDSAit is implicitly assumed for LCUP / SYNC
- LCUP / SYNC search both entries and subentries at the same time

Function renaming, make internal funcs static, etc.

filter_candidate tweaks, search_stack tweaks

Mem context tweaks for bdb_dn2idl

Minor fixes

ITS#2453, Fix filter candidate generation

Use EntryInfo navigation for search scope checks

Finish alias scope support

Hierarchical cache management.

More memory context tweaks

Better detection of new search scopes

cleanup & more

Range check cleanup, added alias processing comments

Fix cursor initialization, scope IDs

Cleanup

Fix locking etc. in prev commit

Added code for BDB_ALIASES. Please test.

Consolidated slap_callbacks into one function. Removed send_search_result.

Fix typo in prev commit

Major API change - (SLAP_OP_BLOCKS) All request parameters are
consolidated into the Operation structure. All reply parameters
are consolidated into the new SlapReply structure. Most operations
now have identical call signatures... Changes are not #ifdef'd,
revert to -r NO_SLAP_OP_BLOCKS if necessary to back out.

reorder to statements for debugging purposes

init var

SLAP_NVALUES:
	schema engine updated (but not schema routines so things don't run yet)
	nvalues mostly populated, enough for tests 0-2 to pass
	schema routines needs lots of work
	modify/mods codes needs lots of work

Don't need TXN/opinfo support in read-only operations

log message fix

Misc LCUP cleanup

Cancel exop updates

rename "permit modify" control to "permissive modify"
rename "no referrals" controls to "domain scope"
misc cleanup

Rework #ifdef'ing of expermental controls to ease release engineering
using LDAP_DEVEL (to indicate experimental) to enable experimental features
(true for HEAD unless LDAP_REL_ENG is defined)

cleanup

use of pre-assigned locker in bdb_group() / bdb_attribute()

LDAP Sync Operation (draft-zeilenga-ldup-sync) as a groundwork for an LDAP replication design

LDAP cancel operation

Fix LBER_ERROR vs. -1 confusion.

Happy new year

Use ber_flatten2

Added config keyword "searchstack" for setting search stack cache depth.
Default is still 16.

Added IDL caching, modified from ITS#2182. Uses "idlcachesize" config
keyword. Code is #ifdef'd, currently defined, with default cache of 0.

From ITS#2183, use a long-lived IDL stack per thread.

Fixed minor compile errors in LDAP_LOG debug statements.

#unifdef paged results, remove lint

add limits to entries count for paged results

seems to be leaking a ber

improve limits handling and consistency; return "Admin limit exceeded" instead of "Unwilling to perform"

- allow interactive pagedResults size change
- implement pagedResults abandon by setting size=0
- misc code cleanup

Support for RFC 2696:
LDAP Control Extension for Simple Paged Results Manipulation

contributed by Lynn Moss <lynnmoss@us.ibm.com> (ITS#2189)

applied with changes.

Fix referrals and alias. For ldbm and I think for bdb, this is necessary
now that we normalize to lower case. Should be merged into 2.1.

fix hasSubordinate filtering; now it can be safely turned on

LCUP persistent search code drop

Wrap csn filter frees with sync mode check.

Fix typos in last commit

Cleanup up LDAP_CLIENT_UPDATE code... including some bug fixing.

LCUP Response Control Code

only optimize objectClass filters when (objectClass=*) is
presented... avoids (&(objectClass=*)(objectClass=foo)) problems.

Fix: if slapadd was used with an un-ordered LDIF file, we can't rely on
entries having ascending entry IDs. Always start from the beginning of
the candidate list. (This should be fine in the general case as well,
since the subtree index will also have refined the candidates.)

Some LCUP clean up

LDAP Client Update Protocol - non-persistent update (TODO: response control ...)

this test is useless, since noop is masked SLAP_CTRL_UPDATE

- added the capability to filter based on hasSubordinate attribute
  to back-bdb, back-ldbm and back-sql (the latter with limitations);
- added handling of ":dn" attributes to extended rfc2254 filters
  and to matched value filter
- altered the behavior of get_mra() when a matching rule is given:
  now it checks whether it is compatible with the attribute syntax
  and, in case it is, the given mr is used.  In case of no type,
  the check is delayed when filtering

Fix oc_filter, it could exit early before counting the full depth of the
filter tree. Also, only give special treatment to objectclass=*.

Quick check of LOCK_ID() return.  Likely should retry... but not forever.

More places where LOCK_ID() calls need to be checked.

Plug config leaks

Coverted LDAP_LOG macro to use subsystem ID int values instead of string values

Minor cleanup

reference scope checks

redesign of back-bdb locking :
cache entry is locked by Berkeley DB lock primitives.

s/LDAP_UNWILLING_TO_PERFORM/LDAP_ADMINLIMIT_EXCEEDED/ when search candidates exceed unckecked limit

honor limits if below soft limit

Patch for ITS# 1643

fix: changing bdb_cache_return_entry_w() to bdb_cache_return_entry_r()

ITS#1733 eliminate o_abandonmutex

Mostly based on patches provided by Hallvard B. Furuseth

ITS#1677 - cast away const warnings
ITS#1678 - unsigned char args to ctype funcs
ITS#1682 - don't redefine ldap_debug
ITS#1683 - uninitialized vars
ITS#1703 - ldo_debug initialization
ITS#1705 - unsigned testing
ITS#1706 - socklen_t args
ITS#1719 - back-tcl update (other cleanups/fixes/improvements; yet untested)
ITS#1724 - integerNormalize/integerFilter/integerIndexer bugs
ITS#1725 - libdes not required

Implement back-null (/dev/null style backend)
Cleanup some misc warnings ("%lu" format, unused/uninitialized vars,
        ambiguous operator precedence)

Kurt, please regenerate configure

Use a single malloc'd block for all the temporary IDL storage in the
filter processing, to avoid runtime stack blowout. Also removes the
need for gigantic thread stacks.

Added LDAP_LOG messages to the above files to match the existing Debug messages

Update handling of AND/OR indexing where element is undefined.

BDB_ALIASES and BDB_SUBENTRIES cleanup

Add search no-op support.

Changed dnParent to void instead of int. (It always returned success...)

Changed be_issuffix and dnParent to struct bervals

Entry caching. Based on ITS#1545 from Jong Hyuk Choi, jongchoi@us.ibm.com

final dn_parent elimination in back-bdb

Renamed BVarray to BerVarray. Moved slapd:bvarray_{add,free} to
liblber:ber_bvarray_{add,free}.

Round 2 of subentry changes

Add very basic subentry search support.

Fix compile errors

leftover ber_bvdup eliminated; rewritten ber_bvdup in terms of ber_dupbv; eliminated wrapper macro; other ber_* improvements

Update copyright statements

Change struct berval ** to BVarray

Changed search attrs from struct berval ** to AttributeName *

Use dnIsSuffix

Change struct berval * to struct berval in various structures

Misc cleanup

Quick struct berval DN support for defer alias API.
Underlying code needs work.

Changed Filter.f_dn to struct berval*

Some more (incomplete) struct berval conversion

Update referral handling to use struct berval DNs.

Move search, modify, and modrdn APIs to struct berval DNs.
Much clean up is needed.

Changed get_limits to struct berval*

Changed search attrs to struct berval **.
Use typedefs for all backend functions, to minimize work in future API
updates. (back-*/external.h will never need updating in the future.)

More struct berval DNs

Changed sai_dn, sai_ndn to struct berval. (Affects op->o_dn,o_ndn,
conn->c_dn,c_ndn, Access->a_dn_pat)

misc cleanup

Converted all use of dn_parent to the in-place version. (Fixed a pdn leak
in back-sql along the way.)

Switched to dn_parent1, cleaned up dn2id_add/delete

Fix typos in previous commit

Port reference scope fix from back-ldbm

Misc code cleanup

Some tweaks to cut down on IDL stack usage. idl_intersection and idl_union
now take only two arguments instead of 3, overwriting the result onto the
first argument. (glibc2.0.7 defaults to a 2MB stack per thread; 3 IDLs at
1.5MB plus various other runtime overhead is enough to trash the stack.)
Also pass in a tmp IDL from search_candidates instead of allocating it in
each candiate function.

Removed unused "range" argument from indexing functions. Fixed more
bugs in indexing. Uncommented #define to turn on indexing in back-bdb.h.
It looks like it's working.

First pass at indexing cleanup. Both filterindex.c and search.c now
compile cleanly when BDB_FILTER_INDICES is defined, but I have not
yet seen whether any of it actually works. In particular, I don't
understand the "range" argument to the candidate functions...

moved the empty dn static entry into a global const object

hack to make '' base work; please check side-effects

Fixes ITS#1433

Ported referral patches from back-ldbm. I believe search.c still needs
the patch from ITS#1289 to be ported here.

BDB updates...  haven't had a chance to work on this for a while,
committin what I have so that someone else might be able to work it.

candidate check using new macro from Kurt

forward porting of time/size limits from back-ldbm (pending the definition of the most appropriate way to determine the number of candidates)

Work in progress codes.  !UNTESTED!

fix up bdb_entry_return() calls

Don't complain about non-existance IDL "range" candidates

Rough in extended ops, modify, and modrdn.
General code cleanup.

Fix cursor logic

replace ALLIDS with RANGE IDLs

Add kludge to get working tests add/populate.

Got base entry...

include external.h

Add bdb_bind, clean up init, s/Backend/BackendDB/

Finish search rough in (w/o alias nor indexing support)

Rough in search routine

Fix last commit

Use BerElementBuffer

ITS#2453 import fix from HEAD

Lots of fixes from HEAD

init var

    Fix slapd directoryString exact index normalization bug
    Fix back-bdb bdb_cache_find_entry* retry bug
    Fix back-bdb log message bug
    Fix back-bdb group/atttribute txn code
    Updated slapadd to complain about holes in the DIT

More sync'ing

First round of imports from HEAD

Add IDL caching codes (only slab enabled)

limits

Alias/Referral bug fix

LCUP and hasSubordinates, both behind #ifdefs
modrdn newSuperior fix

client update and disable bogus filter optimizations

Fix cursor

Sync with HEAD

Sync with HEAD

Sync with HEAD, including
	- subclass indexing
	- bdb locker id fixes

Check LOCK_ID() return

Memory alloc issues

mostly new logging

Sync with HEAD

Suck in latest changes from HEAD

Latest changes from HEAD.

Suck in HEAD changes since 2.1alpha

Happy new year!

ITS#3448 fixes

Sync with HEAD:
   updated Debug labels, unifdef NEW_LOGGING, add BDB 4.3 support
   fix ITS#3356, 3358, 3385 (back-bdb part), ctxcsn memleaks

psearch consistency patch for releng

CSN, LDAPsync and related fixes

Changes from HEAD, namely back-bdb updates
but also SASL, DN, matchedValues, and misc other fixes

Sync with HEAD

Changes from HEAD, including
+       Fixed back-bdb ignore deadlock bug (ITS#3188)
+       Fixed back-bdb pagedResults no end cookie bug (ITS#3161)
+       Fixed back-bdb pagedResults clear controls bug (ITS#3182)
+       Fixed back-bdb pagedResults ignore control bug
+       Fixed slapd internal search limit bugs (ITS#3183)
+       Added slapd -l USER/DAEMON options (ITS#3187)
+       Build environments
+               Fixed back-sql trace build

ITS#3076: fix back-bdb paged results bug

From HEAD:
New module build stuff
updateref updates, new SHADOW DBFLAGS, etc.
paging control updates
limits updates
fixed duplicate value checks
fixed permissive modify
sasl auxprop propname changes
back-bdb "" suffix fixes
fixed back-bdb/ldbm approx non-indexing fixings
updated back-sql

Sync with HEAD

Sync with HEAD as of 14-March-2004

Sync with HEAD

Import syncrepl psearch add bug fix

Happy new year

Sync with HEAD

Sync with HEAD

Sync with HEAD

Sync with HEAD

Sync with HEAD

First rounded of changes in prep for 2.2.beta3

Sync with HEAD

Merge in latest changes from HEAD

Changes from HEAD for beta

Suck in changes from HEAD

Fix send_search_reference in prev commit

ITS#5728 release entries before send_ldap_ber()

ITS#5409 fix from HEAD

Happy new year! (belated)

Set e_ocflags on dummy context entry

happy new year

remove yields

Happy new year!

Sync back-bdb with HEAD

Sync with HEAD

Sync with HEAD
Changes needs updating for release...

Sync with HEAD (need to update CHANGES)

Sync with HEAD

Sync with HEAD
rebuild configure

Sync with HEAD

Sync with HEAD

Initial round of changes for 2.3 beta

Sync with HEAD

Sync with HEAD

Sync with HEAD

ITS#6758

Happy New Year!

ITS#6507

happy belated new year

silence warnings

Revert 1.246.2.23

ITS#6103

ITS#6095

ITS#6056

Update copyright for next release

fix indentation

ITS#5730

ITS#5728

Missed deadlock checks

Propagate deadlock results for internal searches on write ops

Use read-only txn's instead of read lockers. Support BDB 4.4-4.7

Use o_extra instead of o_private

ITS#5409

Warning cleanup: function ptr <=> void* at ldap_pvt_thread_pool_<set/get>key()

Happy new year! (belated)

ITS#5309 work

ITS#5232

Fix pagedresults resume

Cache-thrashing protection: if the DB has more entries than the cache,
don't allow large search operations that touch more entries churn the
existing cached entries.

ITS#5191
Fix paged results control handling

Forced commit, prev was:
Import changes from HEAD
+ Helper for internal searches for objectClass=*
+ proxy cache fixes (ITS #5112, #5113, #5114)

/tmp/msg

Sync with HEAD for OL 2.4.5

Quick merge: everything from HEAD

Sync with HEAD

New access_allowed()