CVS log for servers/slapd/syncrepl.c

Up to [OpenLDAP] / servers / slapd

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

More for ITS#6807, make sure to check_syncprov() after initial startup

ITS#6807 always use complete cookie on refresh requests

Happy New Year

FORCED COMMIT: Last commit was for ITS#6758 (sr_type=REP_SEARCH before be_operational)

Set sr_type=REP_SEARCH before be_operational

noop commit: cleanup 0 -> REP_RESULT in SlapReply

ITS#6758 Avoid SlapReply reuse.
Caller provides a SlapReply and reads the result after the operation,
then reuses it.  Reinitialize before reuse, or move declaration inwards.

ITS#6758 Avoid SlapReply reuse, simple cases.
Caller provides and reuses a SlapReply it does not read after
initialization.  So reinitialize or move the declaration inwards.
In case of refint, move the rs parameter into refint_repair().

Remove lint (unused vars/labels/funcs/default:s)

Plug memory leaks

For ITS#6739: Fix last commit (private debuging)

For ITS#6739: Move variables into do_syncrep2 loop

initialize rc in syncrepl_add_glue_ancestors (ITS#6719)

ITS#6707: Free the presentlist (if any) before connecting.

Fix rewriterule, free rewrite info in syncinfo_free

Fix prev commit, return NULL if rewrite returns original DN

Add suffixmassage processing

Add config support for suffixmassage. Note - no actual massage code
present in this commit.

ITS#6533 put rid first, in front of url

log rid when syncrepl stanza successfully added (ITS#6533)

remove redundant logs (underlying backends already log enough)

alreadyExists is a legitimate result code (further improve ITS#6472 commit; also fix ITS#6528)

don't use dangling stack pointers (more on ITS#6472 fix)

fix syncrepl when attr has no matching rule (ITS#6458)

fix out of order add/rename, when rename to newly added new superior occurs while consumer is offline (ITS#6472)

happy belated New Year

ITS#6473 fail delta-sync when unknown attrs are received

ITS#6425 fix uninit'd vars

ITS#6419 also init for ldaps:// URIs

ITS#6413 Ensure task don't get queued while being freed after config change.

Revert pending CSN if the update fails

check result of str2filter; syncrepl: can be broken; retcode: can't; pcache: shouldn't (ITS#6391)

ITS#6394 partially revert ITS#5108. Use RID_MAX, as originally designed.

use macros for fake connid

in updateCookie, no-op if nothing changed

Use unique IDs for fake syncrepl connections

Braindead serialization for #6368/test050

Always log rid with incoming cookie

Fix typo in 1.487

Fix prev commit

Reject old ops based on pending ops, not just committed ops

ITS#6368 add another call to check_syncprov to make sure cookie is fresh

don't test a NULL BackendDB * (ITS#6385)

warn if retry is not forever

ITS#6367 use correct SID in our request cookie

Fix uninit'd var

ITS#6377, fix rev 1.478

ITS#6373 more for subentry creation

ITS#6373 create subentry if missing

ITS#6373 add sync_use_subentry keyword

add pedantic checks

Revert prev commit, use ldap_pvt_ funcs

For ITS#6152 add slapd_str2scope(), slapd_scope2bv()

ITS#6152 add syncrepl_diff_entry()

ITS#6256 from Jonathan Clarke

ITS#5942 consolidate server URL match to config_check_my_url().
In syncrepl, assume consumer pointing to same server but different
searchbase is OK otherwise ignore config.

Add ldap_pvt_thread_pool_retract() to cancel pending threads

Avoid multiple attempts to free the same syncinfo

check dn normalization result

ITS#6064: "//" comment from rev 1.445 -> "/**/" comment

ITS#6054 quit using global malloc

Fix 1.461, forgot to release mutex when persistent conn exists

Fix prev commit

More for prev commit

More for #6011, rearrange teardown steps

ITS#6011 more for refcount'd cookieState

ITS#6011 better fix for connection queue

ITS#6011 use refcount for shared cookieState

cleanup prev commit

More for ITS#6011

More debugging

More for prev commit

ITS#5988 avoid redundant slap_graduate_commit_csn calls

ITS#6020 better tracking of where changes came from

Revert 1.447 ITS#5793, slap_queue_csn is mandatory

ITS#6011 don't hang around during a shutdown...

Forced commit: prev commit for ITS#5454, deadlock when replicating cn=config

Once more, mutex_lock -> trylock...

Fix modrdn / delOldRDN leaving empty mods

Delete bogus code from 1.435

ITS#5973 use the modify op to pass csn updates from syncrepl to syncprov

ITS#5972 Added newCookie sync info messages.

ITS#5954 from jclarke@linagora.com: fix be_syncinfo order

Add some asserts on CSN validity

ITS#5935 fix prev commit

ITS#5935 better check for active syncrepl sessions, avoid double-free

Cleanup, treat modifyTimestamp specially too

Blind fix to runqueue_remove assert. unable to reproduce the problem...

ITS#5881 consumer must use ManageDSAiT control to retrieve referrals
as entries. provider must preserve incoming ManageDSAiT status.

ITS#5843, fallout from #5470. Just test against our newest CSN.

Update copyright notices

Fix prev commit

ITS#5866 skip CSN check for glue entries

add shadow mask

make sure the right control is being used (a little bit paranoid...)

More for prev commit - if doing both modrdn and modify at once, only
set the CSN on the modify op.

Refine prev commit

ITS#5809 - fixed again, based on 1.423 (reverted 1.424-1.425)

retry every hour, but don't save it; on the contrary, complain every time syncrepl is configured without retry (ITS#5825)

don't leak generated filters (ITS#5826)

minimal fix for ITS#5825, pending discussion

s/not|got/(non)present/ (more readable, isn't it?)

cleanup prev commit, delete unused vars

ITS#5809 fix prev commit - always turn modrdn into modrdn+modify, and
only propagate the Modify op.

ITS#5809 fix 1.421, avoid modrdn falling thru to modify unnecessarily

ITS#5809 fewer false detections of renames

ITS#5809 renames are not exclusive from modifies

ITS#5798, don't use per-DB schema-check flag

ITS#5710 fix stupid updateCookie hack, use explicit o_dont_replicate flag

ITS#5781 fix modification of MV attrs with no EQ rule

ITS#4467: Fix snprintf/WHATSLEFT buffer overflow tests.
Also make WHATSLEFT unsigned.

check for overflows

Warning cleanup: signed meets unsigned.  ber_flatten2() returns -1 on
error, not LBER_ERROR.

don't free and reuse filter structure; don't rewrite undefined filter values (ITS#5731,ITS#5732)

ITS#5726: Revert 1.392, the runqueue must always be inspected.

Revert 1.410, i.e back to lock again.  ITS#5725 fixes the lock situation.

ITS#5719 Reset interval after rescheduling a paused task.

Revert 1.394, see ITS#5454. It still locks up without a trylock.

use the most recent contextCSN (ITS#5675)

ITS#5618 log Search result errors, force Deref to Never

ITS#5600 Replace objectClass, delete/add can fail if superior is added.

ITS#5597 fix CSN compare when provider has CSNs with other sids.

ITS#5596 contextCSN updates are internal operations.

More for ITS#5536

ITS#5536 send SID for glued DBs as well as multimaster

ITS#5403 fix sasl_ssf retrieval on 64 bit Solaris

Fix prev commit

Fix prev commit (again) - set in wrong order

Fix prev commit

More for ITS#5470, add csns to presence search filter in MMR

Fix typo in 1.389

ITS#5470 also ignore presentlist if number of CSNs don't match

ITS#5470 ignore presentlist on partial multimaster refreshes

ITS#5454 regular mutex lock is safe

ITS#5376, #5378 delta-sync should fallback in other cases too

ITS#5454 proposed fix. May still need to change initial trylock to a
regular mutex_lock.

ITS#5442 slapd_rq mutexes must always be used

ITS#5430 syncprov/glue can be used two different ways

ITS#5432 fix compare_csns loop conditions

do_syncrep1 always reads contextCSN from the main DB

ITS#5426 replace the entire contextCSN in updateCookie

Fix prev commit, check for NULL syncCookie

More fixes for syncCookie updating

Partially revert prev, redundant.

In mirrormode, also ask syncprov for initial cookie. It may have been
cached from a recent update.

ITS#5413 fix typo

ITS#5407 more checks for pool pausing

ITS#5376: Drop permissive modify, it isn't necessary

ITS#5376 must abandon current session before starting new refresh

ITS#5376 modifies need to be permissive due to refresh/fallback sync

ITS#5376 - deltasync must fallback for missing entries

In do_syncrepl, don't wait if a previous syncrepl thread is still running.

Cleanup debug msg, check for redundant updates

ITS#5362 syncrepl_updateCookie doesn't need to graduate_commit_csn, the
backends already do it.

Happy New Year (belated)

fix declarations of buffers for numeric strings; other related cleanup

Streamlined Winsock connection management

Fix test050

ITS#5231 don't maintain presentlist for entries added in Persist phase

Fix typo

de-normalize simple filter values consistently

Fix syncrepl_entry to propagate cookieCSN when present

s/strtoul/strtol/ (ITS#5165)

don't malloc the berval separately for presentlist

also log cookie parsing

back out hex RID

remove potential buffer overflow, now that rids can be up to 4096

Use attr_valfind instead of value_find_ex to optimize lookups

Preliminary sorted value support

More for a_numvals

Attribute/Modification value counters

add helper for internal searchs for objectClass=*

log unknown message type

re: prev commit, init j

make sure n,o are non-zero, so j is initialized (coverity)

allow shutdown while syncrepl_del_nonpresent()

plug another leak (hope it's general enough)

back out previous two commits

fix uninit'ed var crasher

first round of CSN normalization consequences

improve previous commit; fix ITS#5108 by letting both SID and RID be 0..4095, input/output in decimal, but consistently handled in hexadecimals within CSN

add support for CSNSIDMatch; use it to set contextCSN as appropriate when slapadd'ing -w

Fix prev commit

ITS#5077 again

getting an abnormal number of empty syncUUID...

rework controls API

More for ITS#5077

make lutil_uuidstr_from_normalized() return the length of the string

move uuid normalized to string to liblutil

Remove lint

recent check for syncrepl searchbase broke test048; fixing...

ITS#5077 fix attr_cmp loop termination

ITS#5065 don't crash if old cookieCSN is sent

syncrepl searchbase must be within the database naming context (ITS#5073)

Reworked recent backend API changes, now using a separate struct,
(ConfigReply) for the error messages.

Fix mirrormode/tool mode

Fallout from ITS#4986 - remove unused param of select_backend()

Fix uninit'd op req DN

ITS#4933 use ldap_set_option for search timelimit

test goes __after__ done (ITS#5014 :)

don't forget to log errors (ITS#5014)

more ITS#4973 initialization

Give OperationBuffer normal struct members instead of using
LBER_ALIGNED_BUFFER(OPERATION_BUFFER_SIZE), as discussed in ITS#4078.
Add a corresponding SyncOperationBuffer in overlays/syncprov.c.

Fix typo in prev commit

ITS#4952 use memcmp to compare CSN values

Struct/array initializers must be constant expressions in C89

coverity scan, ch_realloc won't return NULL

ITS#4915 log failure retry status

ITS#4904 fix delta-sync modlist free

cleanup struct declarations

re-fix previus commit the other way 'round (rids are < 1000)

fix type/format mismatch

Plug leaks

Fix rev 1.312

syncinfo_free must be able to free multiple syncinfos

Coverity error

Fix fetching of initial contextCSN, updating from syncprov

ITS#4832 fix unint'd var

Partially revert prev commit, delete unused parameter, fix psearch response

Fixes for glue and persist chaining

Add a debug message when skipping an old entry.

Optimization: send sid in cookie to avoid getting our changes sent
back to us.

Fix compare_csns to accept new cookie when there are more CSNs than
we knew about before

More for contextCSN checks

Add entryCSN comparison for consistency check

Fix prev commit, doesn't need permissive modify

Use permissive modify on contextCSN

Preliminary multi-context support for syncrepl. Passes all single-master
tests, needs multi-master testing.

Fix prev commit

Fix ITS#4827, was losing the entryCSN mod.

plug one-time leaks related to configuration

fix modify opattrs sync replication (ITS#4820; test019 can't be fixed right now)

Check for duplicate UUIDs due to ITS#4813 patch

ITS#4805 plug remaining leak

Listener commit broke test048, skip listener check on Hidden DBs

Don't touch any other opattrs when updating contextCSN

ITS#4809 add reqMod attribute to modrdn records so that operational
attributes may be propagated. use them in delta-sync.

Finish prev fix, entryUUID was getting dropped

Partial fix for prev commit

New modify handler, compares old and new entry and to generate deltas.
Avoids unnecessary rewrites of unchanged attributes.

Preliminary steps to enabling sync replication of config. Needs work.
Allow configuring a consumer on the provider that points to the provider
but is a no-op. This ensures that the consumer will continue to be a
consumer after the replication occurs.

fix leak (ITS#4805)

Log errors in delta sync

Make syncrepl inherit default TLS settings from main slapd config (except
for reqcert, default demand)

happy new year

fix RID format

Pass op to slap_mods_check()

cleanup

ITS#4768 more be_add fallout

improve error message

ITS#4755 add rid to syncrepl debug msgs

ITS#4746 additional leaks

ITS#4746 plug leaks

ITS#4695 same fix for delta-sync

ITS#4695 fix modrdn newSup

add sb_version to slap_bindconf;
rework slap_client_connect() and slap_discover_feature() accordingly

make sure we got the right control...

Entry/Attribute struct caching, to minimize malloc fragmentation
Note: this breaks LDAP_COMP_MATCH and SLAP_ZONE_MALLOC. But they
were probably broken already anyway.

Forced commit.  Previously logged 1.269 commit was deleted instead of reversed.

factor connection code out of syncrepl

ITS#4622
  syncrepl: don't save contextCSN on incoming context entry
  syncprov: delete contextCSN on checkpoint if ctxcsn is empty

ITS#4622 additional fix

ITS#4534, don't update the context cookie with the delete syncID set
cookie, that's only an intermediate value.
ITS#4622, don't update the context cookie until all the nonpresent
deletes are finished.

ITS#4582 complain if no rootDN was set when configuring syncrepl

Fix modrdn with back-ldap

ITS#4567 use unique timestamps for ops for cascading delta-sync

ITS#4534 propagate CSN for delete syncID sets

Detect TLS config changes

searchbase is a required config parameter, log missing params.

Add logging for ITS#4423

more ITS#4503 cleanup

cleanup

Use TLS context stuff in syncrepl

Warnings: Unused vars. funcptr=0, not =NULL. if(b=...) -> if((b=...) != NULL).

Add "subord" alias for subordinate scope

Fix ITS#4401 teardown persistent clients when unconfiguring

Add debug message when deleting nonpresent entries

filter2bv can't de-normalize UUIDs, must do it explicitly for back-ldap
in test045

ITS#4369 add log messages for delta syncrepl ops

(incorrect) variable used before set (ITS#4331)

fix typos in previus commit

Don't update in-memory cookie until db update succeeds

Propagate local errors up, fix test045

o_csn memory handling cleanup and more resource release

Happy new year!

__should__ fix ITS#4296

fix another crasher

Add orr_modlist member to request structure, containing a list of
modifications to be applied by the backend atomically with a modify
RDN request.

Handle updating of RDN values in frontend.

Code that calls a backend's be_modrdn should call slap_modrdn2mods(op, rs)
before invoking the backend and cleanup with slap_mods_free( op->orr_modlist, 1 ).
See do_modrdn() for an example.

fix potential overflow in syncrepl_unbind() and related helpers

note to developers

add more checks on input values

ATTRSONLY doesn't take a value (ITS#4146, #4266)

use macros to handle retry

fix filter generation (back-ldap uses string form)

plug leaks

ITS#4227 delta-sync should use subtree search

Remove some unnecessary #ifdefs

detect if trying to make back-monitor shadow

first round of number/time parsing fixes

Fix retry on stale cookie result

more URI into bindconf

don't trust strchr/strrchr with bervals

error if syncUUID is empty

ITS#4135 fix prev commit

Update result code to IANA-assigned values
Provide some backwards compatibility.
Misc cleanup

ITS#4135 remove client socket from frontend control before unbinding

Fix fallback retries

Trim down delta attr list

Fix delta deletes

Better delta syncrepl support, fallback to full refresh.

Send reloadHint TRUE by default

Move CSN invocations to backends

Align Operation buffers using LBER_ALIGNED_BUFFER

Use ldap_unbind_ext, don't depend on ldap-int.h

Cleanup

ITS#4050 compare normalized DNs when detecting modrdn

ITS#4031 don't update syncCookie if it has invalid ctxcsn

Silence return val warnings in prev commit

Add syncdata keyword, to select different sync data formats. Defaults
to "default", can be set to "accesslog" or "changelog". changelog is
not fully implemented yet.

need stricter sanity checks...

merge sml_managing into sml_flags (SLAP_MOD_MANAGING)

zero out sml_managing any time a Modifications is built (use calloc?)

silence warning

ITS#3917 don't check csn syntax if schema isn't inited yet

ITS#3922 fix syncrepl ctxcsn leak

slap_mods2entry() should not zero pointers into the modifications
list because checking is performed on these, rather than the entry.

slap_mods_free() now takes an extra argument indicating whether to
free the values or not.

Cleanup

assert expects int. (int)<nonnull ptr/long> can be 0. Use assert(arg!=0/NULL).

Add missing "static" for functions declared static but defined non-static

Silence "unused <something>" warnings

fix ITS#3834

address ITS#3831 (mostly code readability)

Format fixes (%d<->%lu etc)

Add SLAP_MOD_INTERNAL flag to Modifications, allow internal ops to bypass
ACL checks when modifying user-modifiable attributes as non-root user.

ITS#3710 add missing space

Add task names to runqueue entries. Display running tasks under
cn=Runqueue,cn=Threads,cn=Monitor

Cleanup task tracking

Isolate syncrepl stuff in syncrepl.c

set alias dereferencing (please check)

more bindconf cleanup

rework few members of slap_bindconf; silence few warnings

Fixing memory leakage in attribute aliasing

SASL/TLS fixes to prev commit

Added replica retrieval

ITS#3546 fix consumer crash on interrupt (provider crash still unresolved)

initialize AttributeAssertion before using

cleanup

More cleanup

small improvements; declare org_dn/org_ndn

Fix cascading deletes

More consolidation for single consumer-context syncrepl

Single consumer context per database

Use a single sessionlog per context, delete sid

Deleted syncrepl updatedn

cosmetic changes (the problem was not there)

Happy New Year!

ITS#3443 fix shutdown test

Fix ITS#3425, use less restrictive search filter when getting info about
an old entry that must be deleted.

improve usability of global ovrlays for write operations; may need to anticipate SLAPI accordingly.  More work needed for add (overlays need to understand they're global and call slap_mods2entry() explicitly)

Fix typo in prev commit

Fix delete/refresh switch

ITS#3423 implement delete for ID sets

Rename syncrepl_rq to slapd_rq, move to daemon.c

Add comments, fix search args in syncrepl_entry

retry if addition of an entry previously added as glue fails; entryUUID must be added to mods in pretty and normalized form

glue also in case of no such object, indicating that the suffix entry is missing yet (unordered results of initial content poll)

Remove syncinfo si_UUID_dn, make consumer use modify/modrdn instead of
just deleting and adding modified entries.

More migration of syncrepl from frontend to syncprov overlay
define OPERATION_BUFFER_SIZE for dummy Operations

Fix prev commit

More syncprov fixes. test019 now passes.

Split Operation into Opheader and op

Disable back-bdb native syncrepl support, enable syncprov overlay,
remove syncrepl stuff from Operation struct

Dynamic control management

Fix modtail initialization in loop

patch for ITS#3379

syncrepl consistency patch: utilize BDB txn rollback

syncrepl consistency patch

Revert ITS#3353 patch, it needs to be reworked.

cleanup & silence warnings

simplify attrs parsing and process objectClass differently

ITS#3353 consolidate slapd globals into a single struct

preserve attrs="*"

misc syncrepl cleanup/fixes

multiple precision with BIGNUM/gmp/ulong

A follow-on commit to the previous @oc support change to use anlist

Modified @oc supprot code to the share anlist routines; Revised anlist routines

*oc support in attr list

release syncinfo mem upon syncrepl failure as well as shutdown

memory leak fixes

unifdef -UNEW_LOGGING

added avl_free() at the end of do_syncrep2()

Avoid gratuitous reindexing of cn and objectclass in syncrepl_updateCookie

ITS#3289 - exattrs support

Use global alloc for now

Further fixes for ITS#3297

TLS SSF does not imply a transport_ssf.

ITS#3297 (double free)

attrs=*,+ becmoes the default behavior when attrs are not specified in syncrepl definition. When attrs are specified, operational attributes other than objectClass, structuralObjectClass, and entryCSN need to be explicitly specified to be included in the replication content.

ITS#3289 - add SLAP_AT_DYNAMIC flag for dynamically generated attributes.
Set flag for hasSubordinates and subschemaSubentry. Strip dynamic attributes
before storing entries in syncrepl.

Reworked fix for ITS#3140 - add access parameter to backend_attribute

Patch for ITS#3259

rework op/rs structures to deal with opeartional attributes

internal operation return value processing cleanup

more timelimit cleanup

limit checking in syncrepl

syncrepl retry code fix for correct timing

cleanup limits for internal searches...

syncrepl retry-on-error code

Fix ITS#3131 properly - set operation SSF to MAX(TLS, SASL) SSFs

ITS #3131 - propagate SSF to syncrepl operation

ITS#3092: Rename sl_free() and friends to slap_sl_free()

s/saslAuthz/authz/

silence warnings

bvalue cleanup

updatedn fix for syncrepl

use BER_BVNULL

Fix for ITS#3055

fix for ITS#2995

patch for ITS#2948

ITS#3032: retry when GSSAPI creds are not available

Add connection_fake_init()

for ITS#3105

checking limits at the frontend

supress log message for glue add case

misc addition to the previous glueing commit

Glue entry pruning: delete parent entries when they are glues and become leaves

syncrepl update: 1) improve error handling 2) glueing support for non-leaf deletion (TODO : deletion of leaf glue entries in the delete / modrdn code)

fix for ITS#2947

Fix syncrepl slapd_shutdown detection, would not stop on a gentlehup.

Happy new year

Misc cleanup

Added sc_next and sc_cleanup to slap_callback

Move SLAB_SIZE def to slap.h

Fix format errors: %d -> %ld, and add (long) casts as appropriate.

Updated notices

misc updates
 - syncrepl : id -> rid
 - man page update

Another patch for the multi syncrepl thread support

Support multiple sync replication at the consumer :
1) simultaneous operation of multiple active sync replication threads
2) cookie management for individual sync replication thread
   (include rid=%3d to the slapd cookie command line option (-c))

Collective entryUUID transmission of PRESENT messages in a single PDU
(refer to draft-zeilenga-ldup-sync-04.txt)

exchange normalized entryUUID

add slap_uuidstr_from_normalized() - any better place to put this ? liblutils ?