RE: slapadding a large bdb database

["Howard Chu" <hyc@symas.com>]

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of John Morrissey

> I have one more question, though. I slapadded the database in two parts
> because the transaction logs got so large that they filled up the
> filesystem. I split the ldif, slapadded the first one then slapadded the
> second one. After the second slapadd had finished its recovery, I removed
> the old transaction logs (db_archive listed them as freeable). The
> bdb files
> themselves get to be about 1GB (roughly the same size as our existing ldbm
> database, maybe a little bigger) Does this sound sane?

The sizes should be comparable to the back-ldbm sizes, yes.
>
> The problem comes when I start slapd for the first time with the new bdb
> database. slapd starts and runs recovery just fine. However, I can't find
> entries I know should be in the directory (they were in the ldif). I tried
> searching on different attrs (guessing at possible index corruption), but
> that didn't help. Downed slapd, reindexed with slapindex, brought
> slapd back
> up. Same results. Of the 458,904 entries in the ldif, I can only "find"
> 31,729 of them.

Not sure what to make of that. If you slapcat the database does the output
match the LDIF files that you fed into slapadd?
>
> Also, I was getting lots of:
> Aug  3 16:08:17 oh slapindex: bdb(o=frontier): Duplicate data items are not
> supported with sorted data
>
> interspersed with some:
>
> Aug  3 16:08:17 oh slapindex: => bdb_dn2id_add: put failed: DB_KEYEXIST:
> Key/data pair already exists -30996
>
> while running slapindex.
>
> Did I do something wrong? Have any idea what would cause this?

This is to be expected. In back-bdb the slapindex command also recreates the
dn2id
index, whereas older code left that database untouched. slapindex doesn't
empty the
indices before operating on them, so you'll see a lot of KEYEXIST errors when
re-indexing an existing database. If you suspect index corruption you should
delete all the index database files before running slapindex. The only file
that should *NOT* be deleted is the id2entry.bdb file.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support