Issue 3262 - SyncREPL Behaves Inconsistently
Summary: SyncREPL Behaves Inconsistently
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: documentation (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-08-02 20:41 UTC by adam@morrison-ind.com
Modified: 2009-02-17 06:48 UTC (History)
0 users

See Also:


Attachments
ldap.consumer.gz (1.17 MB, application/x-gzip)
2004-08-06 12:19 UTC, adam@morrison-ind.com
Details
ldap.master.gz (1017.32 KB, application/x-gzip)
2004-08-06 12:19 UTC, adam@morrison-ind.com
Details

Note You need to log in before you can comment on or make changes to this issue.
Description adam@morrison-ind.com 2004-08-02 20:41:22 UTC
Full_Name: Adam Tauno Williams
Version: 2.2.15
OS: LINUX
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (206.114.45.106)


SyncRepl behaves inconsistently.

Three servers -
1.Master 2.2.15 slapd on Redhat Linux 9, littleboy
1.1.kernel-2.4.20-19.9
1.2.glibc-2.3.2-27.9.7
2.Consumer 2.2.15 slapd on RedHat Linux 9, turnstone
2.1.kernel-2.4.20-19.9
2.2.glibc-2.3.2-27.9.7
3.Consumer 2.2.14 slapd on SuSe Linux 9.1, throw
2.1.kernel-smp-2.6.4-54.5
2.2.glibc-2.3.3-97

Relevant sections from server 1 (master) slapd.conf -

sessionlog 100 256
sessionlog 101 256
sessionlog 102 256
sessionlog 103 256
sessionlog 104 256
sessionlog 105 256

We also tried with a single session log,  thought perhaps we needed one per
consumer;  but this didn't make any difference.

Relevant sections from server 2 (consumer) slapd.conf -
syncrepl rid=50
  provider=ldap://littleboy.morrison.iserv.net:389
  type=refreshAndPersist
  searchbase="o=Morrison Industries,c=US"
  filter="(objectclass=*)"
  scope=sub
  schemachecking=off
  updatedn="uid=syncrepl,ou=Replication,ou=SubSystems,o=Morrison
Industries,c=US"
  bindmethod=simple
  binddn="uid=turnstone,ou=Replication,ou=SubSystems,o=Morrison
Industries,c=US"
  credentials=**************

Credentials are stored in the mentioned object in cleartext,  we used MD5 at
first to encode the password but switched to cleartext encoding to see if that
would help.  It didn't have any effect.  This slapd DOES seem to create and
maintain a connection to the master slapd.

Relevant sections from server 3 (consumer) slapd.conf -
syncrepl rid=50
  provider=ldap://littleboy.morrison.iserv.net:389
  type=refreshAndPersist
  searchbase="o=Morrison Industries,c=US"
  filter="(objectclass=*)"
  scope=sub
  schemachecking=off
  updatedn="uid=syncrepl,ou=Replication,ou=SubSystems,o=Morrison
Industries,c=US"
  bindmethod=simple
  binddn="uid=throw,ou=Replication,ou=SubSystems,o=Morrison Industries,c=US"
  credentials=**************

Both consumers will come into sync if RESTARTED, but then syncronization stops.

Both consumers were loaded via -
Master
------
/opt/dsa/sbin/slapcat -n1 -m > /tmp/consumer.ldif
scp /tmp/consumer.ldif root@{consumer}:/tmp
Consumer
--------
cd /var/lib/ldap
rm -f *
ln -s /etc/openldap/DB_CONFIG
/opt/dsa/sbin/slapadd -r -i 50 -n1 < /tmp/consumer.ldif
chown ldap.ldap *

And both consumers immediately create a connection to the master -
# netstat -ap | grep slap
tcp 0 0 *:ldap *:* LISTEN 5296/slapd
tcp 0 0 {consumer}:33161 {master}:ldap ESTABLISHED 5296/slapd
unix 2 [ ] DGRAM 16836 5296/slapd 

IF I have JUST loaded and start ONE consumer it stays in sync.  If I start the
second consumer it sync's itself and then BOTH consumers stop syncing.  They
subsequently only sync on restart even if all slapds are stopped (including the
master) and restarted, or if all slapds are stopped (including the master) and
only ONE consumer is started.

Example......

Two consumers (localhost [turnstone] & throw) and master (littleboy), one
consumer
out of sync....

[root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
morrisoninternalrate
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 50
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 75
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 50

Two consumers (localhost & throw) and master (littleboy), after throw is
restarted it comes into sync.....
 
[root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
morrisoninternalrate
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 50
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 50
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 50

Two consumers (localhost & throw) and master (littleboy), change made on
master is not seen by consumers....

[root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
morrisoninternalrate
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 50
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 50
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 60


Two consumers (localhost & throw) and master (littleboy), both consumer
slapds restarted, they come into sync......

[root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
morrisoninternalrate
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 60
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 60
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 60

Two consumers (localhost & throw) and master (littleboy), change made on
master, both consumer out of sync......
[root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
morrisoninternalrate
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 60
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 60
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 42

All three DSA installs compiled as -
 $ mkdir /opt/dsa
$ cd /tmp
$ tar xzvf openssl-0.9.7d.tar.gz
$ cd openssl-0.9.7d
$ env CC=gcc LDFLAGS="-L/opt/dsa/lib -R/opt/dsa/lib" \
PERL=/usr/bin/perl LD_RUN_PATH=/opt/dsa/lib \
./Configure --prefix=/opt/dsa --openssldir=/opt/dsa/openssl \
shared linux-ppro
$ nice -n19 make
$ make install
$ cd /tmp
$ tar xzvf heimdal-0.6.2.tar.gz
$ cd heimdal-0.6.2
$ export CFLAGS='-O2'
$ export CXXFLAGS='-O2'
$ export CCFLAGS="-O2 -D_REENTRANT"
$ ./configure --prefix=/opt/dsa CCFLAGS="-O2 -D_REENTRANT" --enable-shared
--with-krb4=/opt/dsa --with-openssl=/opt/dsa --without-readline --without-hesiod
--without-ipv6
$ nice -n 19 make
$ make install
$ cd /tmp
$ tar xzvf bdb-4.2.52.tar.gz
$ cd db-4.2.52
$ patch -p0 < ../patch.4.2.52.1
$ patch -p0 < ../patch.4.2.52.2
$ cd build_unix
$ ../dist/configure --prefix=/opt/dsa --enable-compat185
$ nice -n 19 make
$ make install
$ cd /tmp
$ tar xzvf cyrus-sasl-2.1.18.tar.gz
$ cd cyrus-sasl-2.1.18
$ export CFLAGS='-O2'
$ export CXXFLAGS='-O2'
$ ./configure --disable-sample --without-dblib --without-pam --without-des
--disable-des --with-openssl --with-saslauthd=/var/run --disable-checkapop
--disable-cram --disable-digest --disable-otp --disable-anon --enable-plain
--enable-login --enable-gssapi=/opt/dsa -with-plugindir=/opt/dsa/lib/sasl2
--prefix=/opt/dsa
$ nice -n 19 make
$ make install
$ cd /tmp
$ tar xzvf openldap-2.2.15.tgz
$ cd openldap-2.2.15
$ export CPPFLAGS="-I/opt/dsa/include"
$ export CXXFLAGS="-I/opt/dsa/include"
$ export CFLAGS="-I/opt/dsa/include"
$ export LDFLAGS="-L/opt/dsa/lib"
$ export LD_LIBRARY_PATH=/opt/dsa/lib
$ ./configure --prefix=/opt/dsa --sysconfdir=/etc --localstatedir=/var/run/slapd
--libexecdir=/opt/dsa/libexec --libdir=/opt/dsa/lib --mandir=/opt/dsa/man
--sbindir=/opt/dsa/sbin --datadir=/opt/dsa/share --localstatedir=/opt/dsa/var
--includedir=/opt/dsa/include --enable-aclgroups --enable-spasswd
--enable-modules --enable-shared --enable-dynamic --with-tls --with-cyrus-sasl
--enable-crypt --enable-ipv6=yes --enable-aci --enable-bdb --enable-rewrite
--enable-ldap --enable-meta --enable-monitor --enable-ldbm --enable-sql
--enable-lmpasswd --with-dyngroup --with-proxycache
$ nice -n19 make depend
$ nice -n19 make
$ make install  

Comment 1 Jonghyuk Choi 2004-08-02 23:51:10 UTC
I've tested with a similar setup but could not observe the same behavior.
After loading the master and replica databases in the same manner,
additional adds and modifies were performed at the master. Both of the
operations were replicated to two replicas as expected. I'm wondering
whether you can send me the debug output at the master and replicas. It will
significantly reduce the debug time. If it is not possible to send all the
log, can you send me that part of the log files which were generated when
you performed modifications to the morrisoninternalrate attribute ?
- Jong-Hyuk

----- Original Message ----- 
From: <adam@morrison-ind.com>
To: <openldap-its@OpenLDAP.org>
Sent: Monday, August 02, 2004 4:41 PM
Subject: SyncREPL Behaves Inconsistently (ITS#3262)


> Full_Name: Adam Tauno Williams
> Version: 2.2.15
> OS: LINUX
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (206.114.45.106)
>
>
> SyncRepl behaves inconsistently.
>
> Three servers -
> 1.Master 2.2.15 slapd on Redhat Linux 9, littleboy
> 1.1.kernel-2.4.20-19.9
> 1.2.glibc-2.3.2-27.9.7
> 2.Consumer 2.2.15 slapd on RedHat Linux 9, turnstone
> 2.1.kernel-2.4.20-19.9
> 2.2.glibc-2.3.2-27.9.7
> 3.Consumer 2.2.14 slapd on SuSe Linux 9.1, throw
> 2.1.kernel-smp-2.6.4-54.5
> 2.2.glibc-2.3.3-97
>
> Relevant sections from server 1 (master) slapd.conf -
>
> sessionlog 100 256
> sessionlog 101 256
> sessionlog 102 256
> sessionlog 103 256
> sessionlog 104 256
> sessionlog 105 256
>
> We also tried with a single session log,  thought perhaps we needed one
per
> consumer;  but this didn't make any difference.
>
> Relevant sections from server 2 (consumer) slapd.conf -
> syncrepl rid=50
>   provider=ldap://littleboy.morrison.iserv.net:389
>   type=refreshAndPersist
>   searchbase="o=Morrison Industries,c=US"
>   filter="(objectclass=*)"
>   scope=sub
>   schemachecking=off
>   updatedn="uid=syncrepl,ou=Replication,ou=SubSystems,o=Morrison
> Industries,c=US"
>   bindmethod=simple
>   binddn="uid=turnstone,ou=Replication,ou=SubSystems,o=Morrison
> Industries,c=US"
>   credentials=**************
>
> Credentials are stored in the mentioned object in cleartext,  we used MD5
at
> first to encode the password but switched to cleartext encoding to see if
that
> would help.  It didn't have any effect.  This slapd DOES seem to create
and
> maintain a connection to the master slapd.
>
> Relevant sections from server 3 (consumer) slapd.conf -
> syncrepl rid=50
>   provider=ldap://littleboy.morrison.iserv.net:389
>   type=refreshAndPersist
>   searchbase="o=Morrison Industries,c=US"
>   filter="(objectclass=*)"
>   scope=sub
>   schemachecking=off
>   updatedn="uid=syncrepl,ou=Replication,ou=SubSystems,o=Morrison
> Industries,c=US"
>   bindmethod=simple
>   binddn="uid=throw,ou=Replication,ou=SubSystems,o=Morrison
Industries,c=US"
>   credentials=**************
>
> Both consumers will come into sync if RESTARTED, but then syncronization
stops.
>
> Both consumers were loaded via -
> Master
> ------
> /opt/dsa/sbin/slapcat -n1 -m > /tmp/consumer.ldif
> scp /tmp/consumer.ldif root@{consumer}:/tmp
> Consumer
> --------
> cd /var/lib/ldap
> rm -f *
> ln -s /etc/openldap/DB_CONFIG
> /opt/dsa/sbin/slapadd -r -i 50 -n1 < /tmp/consumer.ldif
> chown ldap.ldap *
>
> And both consumers immediately create a connection to the master -
> # netstat -ap | grep slap
> tcp 0 0 *:ldap *:* LISTEN 5296/slapd
> tcp 0 0 {consumer}:33161 {master}:ldap ESTABLISHED 5296/slapd
> unix 2 [ ] DGRAM 16836 5296/slapd
>
> IF I have JUST loaded and start ONE consumer it stays in sync.  If I start
the
> second consumer it sync's itself and then BOTH consumers stop syncing.
They
> subsequently only sync on restart even if all slapds are stopped
(including the
> master) and restarted, or if all slapds are stopped (including the master)
and
> only ONE consumer is started.
>
> Example......
>
> Two consumers (localhost [turnstone] & throw) and master (littleboy), one
> consumer
> out of sync....
>
> [root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
> morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
> morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
> morrisoninternalrate
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 50
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 75
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 50
>
> Two consumers (localhost & throw) and master (littleboy), after throw is
> restarted it comes into sync.....
>
> [root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
> morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
> morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
> morrisoninternalrate
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 50
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 50
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 50
>
> Two consumers (localhost & throw) and master (littleboy), change made on
> master is not seen by consumers....
>
> [root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
> morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
> morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
> morrisoninternalrate
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 50
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 50
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 60
>
>
> Two consumers (localhost & throw) and master (littleboy), both consumer
> slapds restarted, they come into sync......
>
> [root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
> morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
> morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
> morrisoninternalrate
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 60
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 60
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 60
>
> Two consumers (localhost & throw) and master (littleboy), change made on
> master, both consumer out of sync......
> [root@turnstone root]# ldapsearch -x -LLL -h localhost uid=adam
> morrisoninternalrate; ldapsearch -LLL -x -h throw uid=adam
> morrisoninternalrate;ldapsearch -LLL -x -h littleboy  uid=adam
> morrisoninternalrate
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 60
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 60
> dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
> morrisoninternalrate: 42
>
> All three DSA installs compiled as -
>  $ mkdir /opt/dsa
> $ cd /tmp
> $ tar xzvf openssl-0.9.7d.tar.gz
> $ cd openssl-0.9.7d
> $ env CC=gcc LDFLAGS="-L/opt/dsa/lib -R/opt/dsa/lib" \
> PERL=/usr/bin/perl LD_RUN_PATH=/opt/dsa/lib \
> ./Configure --prefix=/opt/dsa --openssldir=/opt/dsa/openssl \
> shared linux-ppro
> $ nice -n19 make
> $ make install
> $ cd /tmp
> $ tar xzvf heimdal-0.6.2.tar.gz
> $ cd heimdal-0.6.2
> $ export CFLAGS='-O2'
> $ export CXXFLAGS='-O2'
> $ export CCFLAGS="-O2 -D_REENTRANT"
> $ ./configure --prefix=/opt/dsa CCFLAGS="-O2 -D_REENTRANT" --enable-shared
> --with-krb4=/opt/dsa --with-openssl=/opt/dsa --without-readline --without-
hesiod
> --without-ipv6
> $ nice -n 19 make
> $ make install
> $ cd /tmp
> $ tar xzvf bdb-4.2.52.tar.gz
> $ cd db-4.2.52
> $ patch -p0 < ../patch.4.2.52.1
> $ patch -p0 < ../patch.4.2.52.2
> $ cd build_unix
> $ ../dist/configure --prefix=/opt/dsa --enable-compat185
> $ nice -n 19 make
> $ make install
> $ cd /tmp
> $ tar xzvf cyrus-sasl-2.1.18.tar.gz
> $ cd cyrus-sasl-2.1.18
> $ export CFLAGS='-O2'
> $ export CXXFLAGS='-O2'
> $ ./configure --disable-sample --without-dblib --without-pam --without-des
> --disable-des --with-openssl --with-saslauthd=/var/run --disable-checkapop
> --disable-cram --disable-digest --disable-otp --disable-anon --enable-plai
n
> --enable-login --enable-gssapi=/opt/dsa -with-plugindir=/opt/dsa/lib/sasl2
> --prefix=/opt/dsa
> $ nice -n 19 make
> $ make install
> $ cd /tmp
> $ tar xzvf openldap-2.2.15.tgz
> $ cd openldap-2.2.15
> $ export CPPFLAGS="-I/opt/dsa/include"
> $ export CXXFLAGS="-I/opt/dsa/include"
> $ export CFLAGS="-I/opt/dsa/include"
> $ export LDFLAGS="-L/opt/dsa/lib"
> $ export LD_LIBRARY_PATH=/opt/dsa/lib
> $
./configure --prefix=/opt/dsa --sysconfdir=/etc --localstatedir=/var/run/sla
pd
> --libexecdir=/opt/dsa/libexec --libdir=/opt/dsa/lib --mandir=/opt/dsa/man
> --sbindir=/opt/dsa/sbin --datadir=/opt/dsa/share --localstatedir=/opt/dsa/
var
> --includedir=/opt/dsa/include --enable-aclgroups --enable-spasswd
> --enable-modules --enable-shared --enable-dynamic --with-tls --with-cyrus-
sasl
> --enable-crypt --enable-ipv6=yes --enable-aci --enable-bdb --enable-rewrit
e
> --enable-ldap --enable-meta --enable-monitor --enable-ldbm --enable-sql
> --enable-lmpasswd --with-dyngroup --with-proxycache
> $ nice -n19 make depend
> $ nice -n19 make
> $ make install
>
>

Comment 2 Jonghyuk Choi 2004-08-02 23:52:32 UTC
changed state Open to Feedback
Comment 3 adam@morrison-ind.com 2004-08-04 14:11:17 UTC
> I've tested with a similar setup but could not observe the same behavior.
> After loading the master and replica databases in the same manner,
> additional adds and modifies were performed at the master. Both of the
> operations were replicated to two replicas as expected. I'm wondering
> whether you can send me the debug output at the master and replicas. 

What would be the optimal debug level?  I checked the man page and there
isn't a log level specific to replication operations,  I tried dumping
"everything" (4095) and it seems like an awful lot.

> It will
> significantly reduce the debug time. If it is not possible to send all the
> log, can you send me that part of the log files which were generated when
> you performed modifications to the morrisoninternalrate attribute ?

No problem.

BTW, one consumer has 'suddenly' become reliable, at least for the last
+24 hours.  We made no changes to its config but we did restart it
several times.  The other consumers still exhibit the same behaviour of
only updating at startup.

------------------------
Sardine just brought online, throw & hoopoe aleardy out of sync.

throw:~ # echo "Master (littleboy, rh9):";ldapsearch -LLL -x -h
littleboy uid=adam morrisoninternalrate;echo "Consumer (turnstone,
rh9):";ldapsearch -LLL -x -h turnstone uid=adam
morrisoninternalrate;echo "Consumer (throw, suse91)";ldapsearch -LLL -x
-h throw uid=adam morrisoninternalrate;echo "Consumer (sardine,
rh9)";ldapsearch -LLL -x -h sardine uid=adam morrisoninternalrate;echo
"Consumer (hoopoe, rh9)";ldapsearch -LLL -x -h hoopoe uid=adam
morrisoninternalrate
Master (littleboy, rh9):
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 15
Consumer (turnstone, rh9):
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 15
Consumer (throw, suse91)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 67
Consumer (sardine, rh9)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 15
Consumer (hoopoe, rh9)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 12

Restarted throw & hoopoe, now all in sync....

throw:~ # echo "Master (littleboy, rh9):";ldapsearch -LLL -x -h
littleboy uid=adam morrisoninternalrate;echo "Consumer (turnstone,
rh9):";ldapsearch -LLL -x -h turnstone uid=adam
morrisoninternalrate;echo "Consumer (throw, suse91)";ldapsearch -LLL -x
-h throw uid=adam morrisoninternalrate;echo "Consumer (sardine,
rh9)";ldapsearch -LLL -x -h sardine uid=adam morrisoninternalrate;echo
"Consumer (hoopoe, rh9)";ldapsearch -LLL -x -h hoopoe uid=adam
morrisoninternalrate
Master (littleboy, rh9):
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate:
15                                                                             
Consumer (turnstone, rh9):
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate:
15                                                        
Consumer (throw, suse91)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate:
15                                                        
Consumer (sardine, rh9)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate:
15                                                        
Consumer (hoopoe, rh9)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 15

Changed value on master (littleboy),  only one consumer tracked the
change.

throw:~ # echo "Master (littleboy, rh9):";ldapsearch -LLL -x -h
littleboy uid=adam morrisoninternalrate;echo "Consumer (turnstone,
rh9):";ldapsearch -LLL -x -h turnstone uid=adam
morrisoninternalrate;echo "Consumer (throw, suse91)";ldapsearch -LLL -x
-h throw uid=adam morrisoninternalrate;echo "Consumer (sardine,
rh9)";ldapsearch -LLL -x -h sardine uid=adam morrisoninternalrate;echo
"Consumer (hoopoe, rh9)";ldapsearch -LLL -x -h hoopoe uid=adam
morrisoninternalrate
Master (littleboy, rh9):
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 16
Consumer (turnstone, rh9):
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 16
Consumer (throw, suse91)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 15
Consumer (sardine, rh9)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 15
Consumer (hoopoe, rh9)
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 15

That consumer has proved reliable for +24 hours now,  not certain what
changed to keep it in sync.  None of the other sync beyond the initial
start-up phase.

Comment 4 Jonghyuk Choi 2004-08-04 14:34:32 UTC
255 will be good enough to trace the problem.

One question: do you have group acl configured ?

- Jong-Hyuk

Comment 5 adam@morrison-ind.com 2004-08-04 19:31:02 UTC
> 255 will be good enough to trace the problem.

Will do this evening.

> One question: do you have group acl configured ?

Yes, all DSAs are configured with...

./configure --prefix=/opt/dsa --sysconfdir=/etc
--localstatedir=/var/run/slapd --libexecdir=/opt/dsa/libexec
--libdir=/opt/dsa/lib --mandir=/opt/dsa/man --sbindir=/opt/dsa/sbin
--datadir=/opt/dsa/share --localstatedir=/opt/dsa/var
--includedir=/opt/dsa/include --enable-aclgroups --enable-spasswd
--enable-modules --enable-shared --enable-dynamic --with-tls
--with-cyrus-sasl --enable-crypt --enable-ipv6=yes --enable-aci
--enable-bdb --enable-rewrite --enable-ldap --enable-meta
--enable-monitor --enable-ldbm --enable-sql --enable-lmpasswd
--with-dyngroup --with-proxycache

We are currently testing with "access to * by * write", to see if that
helped the problem (it didn't).  Prior to that we were testing with the
following ACL as our first rule....

access to dn.subtree="o=Morrison Industries,c=US"
  by dn.base="cn=Administrator,o=Morrison Industries,c=US" write
  by group/groupOfUniqueNames/uniqueMember="cn=DSA
Administrators,ou=ACLGroups,o=Morrison Industries,c=US" write
  by group/groupOfUniqueNames/uniqueMember="cn=DSA
Replicators,ou=ACLGroups,o=Morrison Industries,c=US" write
  by group/groupOfUniqueNames/uniqueMember="cn=Full SyncRepl
Consumers,ou=ACLGroups,o=Morrison Industries,c=US" read
  by * break

...with all the SyncREPL consumer's bind dns in "cn=Full SyncRepl
Consumers,ou=ACLGroups,o=Morrison Industries,c=US"


Comment 6 adam@morrison-ind.com 2004-08-06 12:19:30 UTC
> > 255 will be good enough to trace the problem.
> Will do this evening.

Before:
throw:/tmp # ldapsearch -LLL -x uid=adam -h localhost
morrisoninternalrate;ldapsearch -LLL -x uid=adam -h littleboy
morrisoninternalrate
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 15
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 16

Stop throw (consumer) and littleboy (master),  change loglevel to 255. 
Start master, then start consumer.

After a couple of minutes...

-rw-rw-r--  1 root root 51013293 Aug  5 21:13 /var/log/ldap
throw:/tmp # ldapsearch -LLL -x uid=adam -h localhost
morrisoninternalrate;ldapsearch -LLL -x uid=adam -h littleboy
morrisoninternalrate
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 16
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 16

Then change the morrisoninternalrate attribute...

throw:/tmp # ldapsearch -LLL -x uid=adam -h localhost
morrisoninternalrate;ldapsearch -LLL -x uid=adam -h littleboy
morrisoninternalrate
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 16
dn: cn=Adam Williams,ou=People,o=Morrison Industries,c=US
morrisoninternalrate: 17

Out of sync.
Comment 7 Kurt Zeilenga 2004-09-03 16:25:48 UTC
moved from Incoming to Software Bugs
Comment 8 adam@morrison-ind.com 2004-10-04 19:17:27 UTC
> I'll get back to you with some news within a day or two.

We've updated to the latest 2.2.x release and still see the same
problem.  Sync occurs when the slave(s) start, but does not persist.

Comment 9 Jonghyuk Choi 2004-10-04 19:20:04 UTC
I've prepared a patch for a persistent-syncrepl consistency problem which 
is currently at the final testing stage.
I'll upload the patch for test as soon as it becomes ready for general 
testing.
- Jong-Hyuk

------------------------
Jong Hyuk Choi
IBM Thomas J. Watson Research Center - Enterprise Linux Group
P. O. Box 218, Yorktown Heights, NY 10598
email: jongchoi@us.ibm.com
(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979





Adam Tauno WIlliams <adam@morrison-ind.com>
2004-10-04 오후 03:17
Please respond to adam
 
        To:     Jonghyuk Choi/Watson/IBM@IBMUS
        cc:     openldap-its@OpenLDAP.org, adam@morrison-ind.com
        Subject:        Re: SyncREPL Behaves Inconsistently (ITS#3262)


> I'll get back to you with some news within a day or two.

We've updated to the latest 2.2.x release and still see the same
problem.  Sync occurs when the slave(s) start, but does not persist.


Comment 10 adam@morrison-ind.com 2004-10-14 12:45:02 UTC
On Mon, 2004-10-04 at 15:20, Jonghyuk Choi wrote:
> I've prepared a patch for a persistent-syncrepl consistency problem
> which is currently at the final testing stage.
> I'll upload the patch for test as soon as it becomes ready for general
> testing.
> - Jong-Hyuk

Any news on this?
Comment 11 Jonghyuk Choi 2004-10-14 15:12:34 UTC
Jonghyuk Choi
2004-10-14 오전 08:57

        To:     adam@morrison-ind.com
        cc: 
        From:   Jonghyuk Choi/Watson/IBM@IBMUS
        Subject:        Re: SyncREPL Behaves Inconsistently (ITS#3262)

We're preparing 2.2.18 release and it will be released most probably in 
this week.
You can also test the HEAD branch to test drive before it is released.
The 2.2.18 and HEAD branch includes a patch for consisntecy problem of the 
persistent-mode sync replication.
Best reagards,
- Jong-Hyuk

------------------------
Jong Hyuk Choi
IBM Thomas J. Watson Research Center - Enterprise Linux Group
P. O. Box 218, Yorktown Heights, NY 10598
email: jongchoi@us.ibm.com
(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979





Adam Tauno WIlliams <adam@morrison-ind.com>
2004-10-14 오전 08:45
Please respond to adam
 
        To:     Jonghyuk Choi/Watson/IBM@IBMUS
        cc:     openldap-its@OpenLDAP.org
        Subject:        Re: SyncREPL Behaves Inconsistently (ITS#3262)


On Mon, 2004-10-04 at 15:20, Jonghyuk Choi wrote:
> I've prepared a patch for a persistent-syncrepl consistency problem
> which is currently at the final testing stage.
> I'll upload the patch for test as soon as it becomes ready for general
> testing.
> - Jong-Hyuk

Any news on this?


#### signature.asc has been removed from this note on October 14, 2004 by 
Jonghyuk Choi
Comment 12 Jonghyuk Choi 2004-10-27 19:28:10 UTC
OpenLDAP 2.2.18 release has a patch which can potentially resolve this issue
integrated in it.
Wondering whether the patch is effective for this issue or not.
- Jong-Hyuk

Comment 13 adam@morrison-ind.com 2004-10-27 20:43:41 UTC
On Wed, 2004-10-27 at 15:28, Jong-Hyuk wrote:
> OpenLDAP 2.2.18 release has a patch which can potentially resolve this issue
> integrated in it.
> Wondering whether the patch is effective for this issue or not.
> - Jong-Hyuk

I have 2.2.18 up on two boxes.  It seems the same to me.  I can startup
the consumer with no database, and it pulls down an entire up-to-date
copy of the master (what would be about a 4.2Mb LDIF).  Then no further
syncronization occurs.  Changes on the master are not replicated to the
consumer.

I slapcat'd the 2.2.17 master database, wiped /var/lib/ldap,  and added
it back with the -p -w flags via slapadd.

The provider is also the NTP provider for the consumer, so the clocks
are in sync (don't know if that matters at all).

Comment 14 Jonghyuk Choi 2004-10-28 04:23:34 UTC
I misconceived this as a similar one to ITS#3296.

Recommendation is to increase the sizelimit parameter for

uid=turnstone,ou=Replication,ou=SubSystems,o=Morrison Industries,c=US
uid=throw,ou=Replication,ou=SubSystems,o=Morrison Industries,c=US
uid=syncrepl,ou=Replication,ou=SubSystems,o=Morrison Industries,c=US

- Jong-Hyuk

Comment 15 Jonghyuk Choi 2004-10-28 04:27:43 UTC
moved from Software Bugs to Documentation
Comment 16 Jonghyuk Choi 2004-11-01 15:38:04 UTC
Wondering whether it's okay to close this issue...
- Jong-Hyuk

----- Original Message ----- 
From: "Adam Tauno Williams" <adam@morrison-ind.com>
To: "Jong-Hyuk" <jongchoi@OpenLDAP.org>
Cc: <adam@morrison-ind.com>
Sent: Friday, October 29, 2004 8:28 AM
Subject: Re: SyncREPL Behaves Inconsistently (ITS#3262)


> > I misconceived this as a similar one to ITS#3296.
> > Recommendation is to increase the sizelimit parameter for
> > uid=turnstone,ou=Replication,ou=SubSystems,o=Morrison Industries,c=US
> > uid=throw,ou=Replication,ou=SubSystems,o=Morrison Industries,c=US
> > uid=syncrepl,ou=Replication,ou=SubSystems,o=Morrison Industries,c=US
>
> Right, with size limits and timeouts set to unlimited on OL2.2.18 syncrepl
seems
> to be working (for the last ~18 hours anyway).  It doesn't work even with
> raised limits under 2.2.16, so something got fixed.
>
> Thanks.

Comment 17 Kurt Zeilenga 2005-06-14 02:37:36 UTC
changed state Feedback to Closed
Comment 18 Howard Chu 2009-02-17 06:48:54 UTC
moved from Documentation to Archive.Documentation