[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: syncrepl and mass deletes

Here is my provider, and we use syncprov-sessionlog

database bdb
suffix "dc=univ-yy,dc=fr"
directory /var/ldap/base/
sizelimit -1
cachesize 50000
dbconfig set_cachesize 0 140000000 1
dbconfig set_flags DB_LOG_AUTOREMOVE
dbconfig set_tas_spins 1
dbconfig set_lg_regionmax 262144
idlcachesize 150000
checkpoint 1024 5
# flush tous les 1Mo ou 5 minutes
syncprov-checkpoint 100 10
syncprov-sessionlog 100
index objectClass,entryCSN,entryUUID eq


Should we use a session log? I'll try without. That's tricky to understand each parameter.
Would be great to have something anaylizing more than syntax, relations between parameters and database cache optimization.


2011/3/23 Howard Chu <hyc@symas.com>
LALOT Dominique wrote:

I was obliged to remove slapd package on consumer. Then compile in 2.4.24 and
restart. doing the same tests, there was nothing diffferent. provider is still
My test: delete 30000 entries, stop consumer when deleting; start again
consumer when it's finished. Consumer is then out of sync.

You haven't posted your provider config. It appears you're using syncprov-sessionlog. Obviously both ends of this system are vital to solving the puzzle.

Mar 23 17:27:50 ldaprelay slapd[4358]: slapd stopped.
Mar 23 17:34:45 ldaprelay slapd[4577]: @(#) $OpenLDAP: slapd 2.4.24 (Mar 23
2011 16:52:04)
Mar 23 17:34:45 ldaprelay slapd[4579]: slapd starting
Mar 23 17:34:45 ldaprelay slapd[4579]: do_syncrep2: rid=040
Mar 23 17:34:45 ldaprelay slapd[4579]: do_syncrep2: rid=020
Mar 23 17:34:48 ldaprelay slapd[4579]: last message repeated 175 times
Mar 23 17:34:48 ldaprelay slapd[4579]: do_syncrep2: rid=020
Mar 23 17:34:48 ldaprelay slapd[4579]: do_syncrep2: rid=020
Mar 23 17:34:48 ldaprelay slapd[4579]: slap_queue_csn: queing 0x1a35560
Mar 23 17:34:48 ldaprelay slapd[4579]: slap_graduate_commit_csn: removing
0x1a35620 20110323163416.105518Z#000000#020#000000

I changed the provider to 2.4.24 that makes deletes. Hopefully this one was
built on tar.gz
That makes no difference, after restarting the consumer, it does not delete
extra entries


2011/3/23 LALOT Dominique <dom.lalot@gmail.com <mailto:dom.lalot@gmail.com>>

   Hi Howard,

   We were told to migrate to 2.4.23 sometimes ago, and we did some work to
   update our production servers. Can I try 2.4.24 only on the consumer side?
   It would be a pain to migrate all servers to 2.4.24 without package.

   is this related to the last fixes?

   Fixed slapd syncrepl reuse of presence list (ITS#6707)
   Fixed slapd syncrepl uninitialized return code (ITS#6719)
   Fixed slapd syncrepl variable initialization (ITS#6739)

   Fixed slapd syncrepl refresh to use complete cookie (ITS#6807)



   2011/3/23 Howard Chu <hyc@symas.com <mailto:hyc@symas.com>>

       LALOT Dominique wrote:


           I am testing the replication feature in a multimaster environment
           into a single database. As stated before, I added serverid to my
           providers. I
           just have two providers for test purpose.
           I tested mass updates on a provider, stopped my replica during
           updates, then
           start again and it's OK, it updates the entries
           If I do the same for mass deletes. I deleted 40000 entries while
           stopping the
           consumer. My consumer is still with 30000 undeleted entries. I
           left the
           consumer for hours, restarting it twice.
           It seems there is no regular compare between consumer or provider
           in such
           situation. I'll simplify to test in a single provider setup, to
           see if it works.

           All servers are 2.4.23

       Please try your test with 2.4.24 instead.

 -- Howard Chu
 CTO, Symas Corp.           http://www.symas.com
 Director, Highland Sun     http://highlandsun.com/hyc/
 Chief Architect, OpenLDAP  http://www.openldap.org/project/

Dominique LALOT
Ingénieur Systèmes et Réseaux