[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: (ITS#3939) min/max function extension to LDAP protocol

--On Friday, August 19, 2005 11:11 PM -0700 "Kurt D. Zeilenga" 
<Kurt@OpenLDAP.org> wrote:

> Well, what I noted was that there was an existing protocol
> mechanism to request return the entry with the lowest/highest
> CSN.
> I pretty much ignored most of rest of your post as it didn't
> make much sense to me at the time (and still doesn't).

Okay, maybe this will explain it more:

When the consumer connects to the provider, it tries to determine if it 
needs to synch any data or not.  To do that, it uses the value of its 
cookie, and compares that value to the value of the cookie on the master. 
If its cookie value is not equivalent to what is on the master, it then 
looks for *all* values <= to its value.  This works fine in a very small 
database (say 10k entries).  It takes around 20-30 minutes in my 400k 
database.  It would take even longer on a very large database (say 50 
million entries).  The more consumers you have, the worse it gets, as well. 
By giving the consumer a way to immediately get the smallest cookie value 
that the provider has, the consumer can immediately know whether or not its 
cookie is valid, there by skipping the <= search.  If its cookie isn't 
valid, it does a full resynch of data.  If its cookie is valid, then it 
only looks for entries that have been modified since the value of its 
cookie.  That was the point of the recent CSN checking change done between 
2.3.5 and 2.3.6, which took care of some cases to allow an immediate 
equality check (before it defaulted to always doing <=).  But there are 
still times when the master and replica can differ in value, and for 
syncrepl to be worthwhile, there needs to be a way to get an immediate 
yes/no answer as to whether or not it needs to do a full resync.


Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html

"These censorship operations against schools and libraries are stronger
than ever in the present religio-political climate. They often focus on
fantasy and sf books, which foster that deadly enemy to bigotry and blind
faith, the imagination." -- Ursula K. Le Guin