Re: Checkpointing and BDB

[Matthew Backes <lucca@csun.edu>]

On Friday, Apr 25, 2003, at 08:31 US/Pacific, Quanah Gibson-Mount wrote:

> So I have a basic question as to the usefulness of checkpointing.  The 
> stated purpose of checkpointing is to make sure that received writes 
> are flushed to disk.  How does this help you if slapd crashes between 
> writes? At Stanford, we have about 14,500 changes a day.
>
> If there is one feature I appreciated about Netscape, it was that it 
> knew what updates the slaves had or had not received.  Therefore, to 
> recover a slave, all you had to do was load in a DB dump from a 
> previous time, and that slave would be caught up by the master.  That 
> does not appear possible with OpenLDAP.  The only thing I can see to 
> guarantee consistency with OpenLDAP when slapd dies on a slave is to 
> export the database on the master, copy it to the slave, and then copy 
> it in.  Which makes checkpointing once again worthless.  I suppose it 
> could be helpful for the master, in that you know you only have to go 
> back say 10 minutes in time of your changes if the master crashes, to 
> make sure it has received everything.
>
> Is there a more elegant solution I'm missing here?

Seems more like a replication issue.  I've had good luck with telling 
slapd to generate replog entries for fake servers that slurpd will 
ignore, so that the changes remain in the replog and then using the 
preserved changes to replay things later.

Slurpd isn't designed for that manner of abuse though, so it tends to 
hold the changes in memory until you halt slurpd and rotate replication 
logs.  Automated rotation of that requires you examine timestamps of 
the replication entries wrt the slurpd status file.

The nice thing is then you Have all the changes for whatever span of 
time you want, and can do exactly what you need.  Also handy for 
auditing purposes.

Ultimately this is probably the wrong direction, but has the virtue of 
already existing.

Matthew Backes
lucca@csun.edu