Full_Name: Emmanuel L.charny Version: 2.4.45 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (90.92.161.247) Replication won't start if we set it up on running servers configured with slapd.d. The scenario is the following : o We have 2 servers, one will become the provider, the other the consumer o They are initially empty%2ususing MDB (see the attached configurations, converted from slapd.conf to slapd.d) o We start the 2 servers o Step 1 : On the provider, we inject the following entry : dn: cn=config changetype: modify add: olcServerId olcServerId: 1 -
Is it just me, or the ITS WebUI is removing part of the message I typed ? Anyway, here is the complete descripton : Replication won't start if we set it up on running servers configured with slapd.d. The scenario is the following : o We have 2 servers, one will become the provider, the other the consumer o They are initially empty%2ususing MDB (see the attached configurations, converted from slapd.conf to slapd.d) o We start the 2 servers o Step 1 : On the provider, we inject the following entry : dn: cn=config changetype: modify add: olcServerId olcServerId: 1 - o Step 2 : On the provider, we inject the following entries : # Context entry dn: dc=example,dc=com changetype: add objectClass: domain objectClass: top dc: example # LDAPRoles, dc=example,dc=com dn: ou=LDAPRoles,dc=example,dc=com objectClass: top objectClass: organizationalUnit ou: LDAPRoles dn: dc=users,dc=example,dc=com changetype: add dc: users objectClass: domain objectClass: top dn: cn=Johndoe,dc=users,dc=example,dc=com changetype: add objectClass: person objectClass: top sn: John Doe cn: Johndoe # replicator, LDAPRoles, dc=example, dc=com dn: cn=replicator,ou=LDAPRoles,dc=example,dc=com objectClass: top objectClass: simpleSecurityObject objectClass: organizationalRole userPassword: secret cn: replicator o Step 3 : On the consumer, we inject the following entry : # Context entry dn: dc=example,dc=com changetype: add objectClass: domain objectClass: top dc: example o Step 4 : On the provider, we inject the following entries : dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov - dn: olcOverlay=syncprov,olcDatabase={1}mdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcConfig objectClass: top objectClass: olcSyncprovConfig olcOverlay: syncprov olcSpSessionLog: 10000 olcSpCheckpoint: 100 10 dn: olcDatabase={1}mdb,cn=config changetype: modify add: olcLimits olcLimits: dn.exact="cn=replicator,ou=LDAPRoles,dc=example,dc=com" time.soft=unlimited time.h ard=unlimited size.soft=unlimited size.hard=unlimited - dn: olcDatabase={1}mdb,cn=config changetype: modify replace: olcAccess olcAccess: {0}to dn.subtree="dc=example,dc=com" by self write by dn.exact="cn= replicator,ou=LDAPRoles,dc=example,dc=com" read by anonymous auth by * read - o Step 5 : On the consumer, we inject the following entry : dn: olcDatabase={1}mdb,cn=config changetype: modify add: olcSyncrepl olcSyncrepl: rid=1 provider=ldap://10.61.155.18 bindmethod=simple bi nddn="cn=replicator,ou=LDAPRoles,dc=example,dc=com" credentials=secret type=refreshAndPersis t searchbase="dc=example,dc=com" filter="(objectclass=*)" scope=sub schemacheck ing=on retry="5 10 60 +" sizeLimit=unlimited timelimit=unlimited - At this point, one would expect the replication to kick on, and having the entries flowing from the producer to the consumer, but nothing happens. Ignoring the first step (ServerID setting), and applying the other steps, just work fine. It seems that setting the ServerID blocks everything (FTR, it does not help either to setup the consumer's ServerID). This is problematic in a scenario where we would try to make 2 servers being replicated in a MMR typology with MirrorMode set, as the ServerID wil be mandatory. Here is the provider configuratio (this is in slapd.conf format for convenience, it is being converted to slapd.d before the server is started) : ################################################################################# ## Provider configuration ################################################################################## # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # # Schema files. Note that not all of these schemas co-exist peacefully. # Use only those you need and leave the rest commented out. include "/opt/symas/etc/openldap/schema/core.schema" include "/opt/symas/etc/openldap/schema/cosine.schema" include "/opt/symas/etc/openldap/schema/inetorgperson.schema" include "/opt/symas/etc/openldap/schema/misc.schema" # TLSCipherSuite <cipher-suite-spec> # Permits configuring what ciphers will be accepted and the # preference order. <cipher-suite-spec> should be a cipher # specification for the TLS library in use (OpenSSL, GnuTLS, or # Mozilla NSS). TLSCipherSuite HIGH:MEDIUM # Files in which to store the process id and startup arguments. # These files are needed by the init scripts, so only change # these if you are prepared to edit those scripts as well. pidfile "/var/symas/run/slapd.pid" argsfile "/var/symas/run/slapd.args" # Choose the directory for loadable modules. modulepath "/opt/symas/lib64/openldap" # Uncomment the moduleloads as needed to enable additional # functionalityi when configured. NOTE: We package many # more modules options than those found below. moduleload back_mdb.la moduleload back_monitor.la # Sample access control policy: # Allow read access of root DSE # Allow self write access # Allow authenticated users read access # Allow anonymous users to authenticate # Directives needed to implement policy: access to dn="" by * read access to * by self write by users read by anonymous auth #----------------------------------------------------------------------- # LOGGING loglevel stats sync ####################################################################### # config database ####################################################################### database config rootdn "cn=Directory Manager,cn=config" rootpw secret access to * by * none ####################################################################### # Sample LMDB database definitions ####################################################################### database mdb suffix "dc=example,dc=com" rootdn "cn=Directory Manager,dc=example,dc=com" # Cleartext passwords, especially for the rootdn, should # be avoided. See slappasswd(8) and slapd.conf(5) for details describing # the creation of encrypted passwords. rootpw secret # Indices to maintain # index default sets the basic type of indexing to perform if there isn't any indexing specified for a given attribute index default eq index objectClass index cn # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd/tools. Mode 700 recommended. # One directory will be needed for each backend, so you should # create a subdirectory beneath /var/symas/openldap-data for each # new backend. This is also where the DB_CONFIG file needs to be # placed. directory "/var/symas/openldap-data/example" # Here we specify the maximum on-disk size of the database. It is # Recommended to set this near the expected free-space availability # for the machine. This paramiter is not pre-allocated and simply # represents the upward limit to which the database will be allowed # to grow. Note: Specified in *bytes*. Here, we set it to 1gb. maxsize 10485760 ####################################################################### # Monitor database ####################################################################### database monitor access to dn.subtree="cn=monitor" by * read And here is the consumer configuration : ################################################################################# ## Consumer configuration ################################################################################## # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # # Schema files. Note that not all of these schemas co-exist peacefully. # Use only those you need and leave the rest commented out. include "/opt/symas/etc/openldap/schema/core.schema" include "/opt/symas/etc/openldap/schema/cosine.schema" include "/opt/symas/etc/openldap/schema/inetorgperson.schema" include "/opt/symas/etc/openldap/schema/misc.schema" # # TLSCipherSuite <cipher-suite-spec> # Permits configuring what ciphers will be accepted and the # preference order. <cipher-suite-spec> should be a cipher # specification for the TLS library in use (OpenSSL, GnuTLS, or # Mozilla NSS). TLSCipherSuite HIGH:MEDIUM # Files in which to store the process id and startup arguments. # These files are needed by the init scripts, so only change # these if you are prepared to edit those scripts as well. pidfile "/var/symas/run/slapd.pid" argsfile "/var/symas/run/slapd.args" # Choose the directory for loadable modules. modulepath "/opt/symas/lib64/openldap" # Uncomment the moduleloads as needed to enable additional # functionalityi when configured. NOTE: We package many # more modules options than those found below. moduleload back_mdb.la moduleload back_monitor.la # Sample access control policy: # Allow read access of root DSE # Allow self write access # Allow authenticated users read access # Allow anonymous users to authenticate # Directives needed to implement policy: access to dn="" by * read access to * by self write by users read by anonymous auth #----------------------------------------------------------------------- # LOGGING loglevel stats sync ####################################################################### # config database ####################################################################### database config rootdn "cn=Directory Manager,cn=config" rootpw secret access to * by * none ####################################################################### # Sample LMDB database definitions ####################################################################### database mdb suffix "dc=example,dc=com" rootdn "cn=Directory Manager,dc=example,dc=com" # Cleartext passwords, especially for the rootdn, should # be avoided. See slappasswd(8) and slapd.conf(5) for details describing # the creation of encrypted passwords. rootpw secret # Indices to maintain # index default sets the basic type of indexing to perform if there isn't any indexing specified for a given attribute index default eq index objectClass index cn # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd/tools. Mode 700 recommended. # One directory will be needed for each backend, so you should # create a subdirectory beneath /var/symas/openldap-data for each # new backend. This is also where the DB_CONFIG file needs to be # placed. directory "/var/symas/openldap-data/example" # Here we specify the maximum on-disk size of the database. It is # Recommended to set this near the expected free-space availability # for the machine. This paramiter is not pre-allocated and simply # represents the upward limit to which the database will be allowed # to grow. Note: Specified in *bytes*. Here, we set it to 1gb. maxsize 10485760 ####################################################################### # Monitor database ####################################################################### database monitor access to dn.subtree="cn=monitor" by * read Le 21/10/16 à 23:45, openldap-its@OpenLDAP.org a écrit : > *** THIS IS AN AUTOMATICALLY GENERATED REPLY *** > > Thanks for your report to the OpenLDAP Issue Tracking System. Your > report has been assigned the tracking number ITS#8521. > > One of our support engineers will look at your report in due course. > Note that this may take some time because our support engineers > are volunteers. They only work on OpenLDAP when they have spare > time. > > If you need to provide additional information in regards to your > issue report, you may do so by replying to this message. Note that > any mail sent to openldap-its@openldap.org with (ITS#8521) > in the subject will automatically be attached to the issue report. > > mailto:openldap-its@openldap.org?subject=(ITS#8521) > > You may follow the progress of this report by loading the following > URL in a web browser: > http://www.OpenLDAP.org/its/index.cgi?findid=8521 > > Please remember to retain your issue tracking number (ITS#8521) > on any further messages you send to us regarding this report. If > you don't then you'll just waste our time and yours because we > won't be able to properly track the report. > > Please note that the Issue Tracking System is not intended to > be used to seek help in the proper use of OpenLDAP Software. > Such requests will be closed. > > OpenLDAP Software is user supported. > http://www.OpenLDAP.org/support/ > > -------------- > Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved. > -- Emmanuel Lecharny Symas.com directory.apache.org
moved from Incoming to Software Bugs
--On Friday, October 21, 2016 11:02 PM +0000 elecharny@gmail.com wrote: > o Step 3 : On the consumer, we inject the following entry : > ># Context entry > dn: dc=3Dexample,dc=3Dcom > changetype: add > objectClass: domain > objectClass: top > dc: example This step is not valid. The consumer should have no pre-existing data that differs from the provider's database. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
The problem is that if we already have a ServerID on the producer, it will never generate a contextCSN, waiting for the consumer to send one, or for an update to occur : static int syncprov_db_open( BackendDB *be, ConfigReply *cr ) { ... /* Didn't find a contextCSN, should we generate one? */ if ( !si->si_ctxcsn ) { char csnbuf[ LDAP_PVT_CSNSTR_BUFSIZE ]; struct berval csn; if ( slap_serverID || SLAP_SYNC_SHADOW( op->o_bd )) { /* If we're also a consumer, then don't generate anything. * Wait for our provider to send it to us, or for a local * modify if we have multimaster. */ goto out; } At this point, nothing happens when teh consumer connects for teh very first time, because it is expected to send a contextCSN, which it obviously don't have. I do think that the provider should always generate a contextCSN for its own data, when the syncprov overlay is added. that should solve the problem. Now, I don't know (yet) what would be the impact on replication if we do so. Still investigating... -- Emmanuel Lecharny Symas.com directory.apache.org
FTR, the check on slapd_serverID presence has been added in ITS#8281. Le 21/10/16 à 23:45, openldap-its@OpenLDAP.org a écrit : > *** THIS IS AN AUTOMATICALLY GENERATED REPLY *** > > Thanks for your report to the OpenLDAP Issue Tracking System. Your > report has been assigned the tracking number ITS#8521. > > One of our support engineers will look at your report in due course. > Note that this may take some time because our support engineers > are volunteers. They only work on OpenLDAP when they have spare > time. > > If you need to provide additional information in regards to your > issue report, you may do so by replying to this message. Note that > any mail sent to openldap-its@openldap.org with (ITS#8521) > in the subject will automatically be attached to the issue report. > > mailto:openldap-its@openldap.org?subject=(ITS#8521) > > You may follow the progress of this report by loading the following > URL in a web browser: > http://www.OpenLDAP.org/its/index.cgi?findid=8521 > > Please remember to retain your issue tracking number (ITS#8521) > on any further messages you send to us regarding this report. If > you don't then you'll just waste our time and yours because we > won't be able to properly track the report. > > Please note that the Issue Tracking System is not intended to > be used to seek help in the proper use of OpenLDAP Software. > Such requests will be closed. > > OpenLDAP Software is user supported. > http://www.OpenLDAP.org/support/ > > -------------- > Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved. > -- Emmanuel Lecharny Symas.com directory.apache.org
Actually, it's an invalid issue. The problem is all in teh sequence of updates on the provider. There are 6 possible scenario, depending on which order we inject the serverID, teh syncprov overlay and the data : 1) data, serverID, syncprov : invalid. All the data will have a wrong entryCSN 2) data, syncprov,serverID : invalid, same reason 3) serverID, syncprov, data : All work properly 4) syncprov, serverID, data : All work properly 5) serverID, data, syncprov : the corner case scenario... Syncprov has no way to know which contextCSN to generate. The only way for the server to replicate is to do an update, so that the contextCSN is properly generated. 6) syncprov, data, serverID : invalid. All the data will have a wrong entryCSN Bottom line, *always* inject the serverID and syncprov *before injecting some data, or be ready to do an update after having injected syncprov and serverID. Le 21/10/16 à 23:45, openldap-its@OpenLDAP.org a écrit : > *** THIS IS AN AUTOMATICALLY GENERATED REPLY *** > > Thanks for your report to the OpenLDAP Issue Tracking System. Your > report has been assigned the tracking number ITS#8521. > > One of our support engineers will look at your report in due course. > Note that this may take some time because our support engineers > are volunteers. They only work on OpenLDAP when they have spare > time. > > If you need to provide additional information in regards to your > issue report, you may do so by replying to this message. Note that > any mail sent to openldap-its@openldap.org with (ITS#8521) > in the subject will automatically be attached to the issue report. > > mailto:openldap-its@openldap.org?subject=(ITS#8521) > > You may follow the progress of this report by loading the following > URL in a web browser: > http://www.OpenLDAP.org/its/index.cgi?findid=8521 > > Please remember to retain your issue tracking number (ITS#8521) > on any further messages you send to us regarding this report. If > you don't then you'll just waste our time and yours because we > won't be able to properly track the report. > > Please note that the Issue Tracking System is not intended to > be used to seek help in the proper use of OpenLDAP Software. > Such requests will be closed. > > OpenLDAP Software is user supported. > http://www.OpenLDAP.org/support/ > > -------------- > Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved. > -- Emmanuel Lecharny Symas.com directory.apache.org
not a bug
changed notes changed state Open to Closed