[Date Prev][Date Next] [Chronological] [Thread] [Top]

back_mdb: does rtxnsize affect slapcat live backups?



Hi!

I am running OpenLDAP 2.4.47 with significant write activity (in the
hundreds of modify/add/del ops per second), and a large volume of data
compared to available RAM (about 20G)

For backups, I am trying to do a live slapcat dump to maintain
availability, and encountered the growth in MDB database size described
in those ITS:

http://www.openldap.org/its/index.cgi/Software%20Bugs?id=7904
http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8226


I am however a little puzzled over the rtxnsize option. My understanding
is that this option splits large read transactions (such as backups)
into smaller transactions, allowing freed pages to be reused before the
end of the full operation. This is critical for me because backups take
more than 10 minutes and a significant portion of my MDB file (100G)
gets filled up during that time.

However, setting rtxnsize to a smaller value from the default does not
seem to have any effect on the growth of my MDB file. I have made
several tries with different rtxnsizes from 1M to 100, starting from a
freshly rebuilt MDB database every time.

I saw that the rtxnsize option was added to handle an issue with
replication, so perhaps it was never supposed to help in my particular
case. For the whole duration of a slapcat backup, I see the same
transaction ID from the slapcat process in mdb_stat -r.

I noticed that using ldapsearch for backups behaves much better (MDB
only grows by a few thousand pages in my workload), but takes a while
longer, regardless of the rtxnsize setting

Is it normal that rtxnsize does not affect slapcat dumps while slapd is
running, or am I missing something?

How would you handle live backups of a database with lots of write activity?

-- 
Maxime Besson
Systems Engineer - Worteks
maxime.besson@worteks.com

Attachment: pEpkey.asc
Description: application/pgp-keys