Full_Name: Sumit A. Vakil Version: 1.2 OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (205.138.236.250) I have OpenLDAP version 1.2 compiled on a FreeBSD 2.2.x system. I'm using the Berkeley DB 2.3.16 database. ldbmcat does not work. Running ldbmcat does not produce any ldif output. It also complains about chunks of memory being freed again. The problem is easy to reproduce. Simply run ldbmcat (with or without the -n option) on an existing db. The details of the problem and a suggested fix are as follows: The problem is that ldbm_firstkey() sets the key flags to DB_DBT_MALLOC whereas the DB functions called by ldbm_fetch() expect the key flags to be zero. I fixed the problem by setting the key flags to zero before calling ldbm_fetch() and then setting them back to DB_DBT_MALLOC. Note that the key flags should be set back to DB_DBT_MALLOC before calling ldbm_nextkey(). Another problem I noticed was that some chunks of memory were being free twice. The problem there is that ldbmcat tries to free key.data. This memory is already freed by ldbm_nextkey(). Removing the if ( last.dptr != NULL ) ldbm_datum_free( dbp, last ); lines fixes the problem. Updated main() in ldbmcat.c: int main( int argc, char **argv ) { Datum key, last, data; LDBM dbp; int rc, type; long id; char *file, *s; int printid = 1; #ifdef HAVE_BERKELEY_DB2 DBC *cursorp; #endif ldbm_datum_init( key ); ldbm_datum_init( last ); ldbm_datum_init( data ); if ( argc < 2 || argc > 3 || ( argc == 3 && strcmp( argv[1], "-n" ) != 0 )) { usage( argv[0] ); } if ( argc == 3 && strcmp( argv[1], "-n" ) == 0 ) { printid = 0; file = argv[2]; } else { file = argv[1]; } if ( (dbp = ldbm_open( file, LDBM_READER, 0, 0 )) == NULL ) { perror( file ); exit ( 1 ); } last.dptr = NULL; #ifdef HAVE_BERKELEY_DB2 for ( key = ldbm_firstkey( dbp, &cursorp ); key.dptr != NULL; key = ldbm_nextkey( dbp, last, cursorp ) ) #else for ( key = ldbm_firstkey( dbp ); key.dptr != NULL; key = ldbm_nextkey( dbp, last ) ) #endif { /* Begin Internet Devices mods. */ #ifdef HAVE_BERKELEY_DB2 key.flags = 0; data = ldbm_fetch( dbp, key ); key.flags = DB_DBT_MALLOC; #else if ( last.dptr != NULL ) ldbm_datum_free( dbp, last ); data = ldbm_fetch( dbp, key ); #endif HAVE_BERKELEY_DB2 /* End Internet Devices mods. */ if (( s = data.dptr ) != NULL ) { if ( !printid && isdigit( *s )) { if (( s = strchr( s, '\n' )) != NULL ) { ++s; } } if ( s != NULL ) { puts( s ); } if ( data.dptr != NULL ) { ldbm_datum_free( dbp, data ); } } last = key; } /* Begin Internet Devices mods. */ #ifndef HAVE_BERKELEY_DB2 if ( last.dptr != NULL ) ldbm_datum_free( dbp, last ); #endif HAVE_BERKELEY_DB2 /* End Internet Devices mods. */ ldbm_close( dbp ); exit( 0 ); }
moved from Incoming to Software Bugs
changed notes
Hi, I have just fixed the ldbmcat problem. The problem was twofold, in that db_appinit was not invoked, although needed by ldbm_open, and in that key data were free'ed twice, causing a segmentation fault. Thanks for reporting the bugs. Regards, Kurt ksp@openldap.org On Thu, 25 Feb 1999 svakil@internetdevices.com wrote: > Date: Thu, 25 Feb 1999 18:04:53 GMT > From: svakil@internetdevices.com > To: openldap-its@OpenLDAP.org > Subject: Release 1.2 ldbmcat problem with Berkeley DB (ITS#85) > > Full_Name: Sumit A. Vakil > Version: 1.2 > OS: FreeBSD > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (205.138.236.250) > > > I have OpenLDAP version 1.2 compiled on a FreeBSD 2.2.x system. I'm using the > Berkeley DB 2.3.16 database. ldbmcat does not work. Running ldbmcat does not > produce any ldif output. It also complains about chunks of memory being freed > again. > > The problem is easy to reproduce. Simply run ldbmcat (with or without the -n > option) on an existing db. > > The details of the problem and a suggested fix are as follows: > > The problem is that ldbm_firstkey() sets the key flags to DB_DBT_MALLOC whereas > the DB functions called by ldbm_fetch() expect the key flags to be zero. I > fixed the problem by setting the key flags to zero before calling ldbm_fetch() > and then setting them back to DB_DBT_MALLOC. Note that the key flags should be > set back to DB_DBT_MALLOC before calling ldbm_nextkey(). > > Another problem I noticed was that some chunks of memory were being free twice. > The problem there is that ldbmcat tries to free key.data. This memory is > already freed by ldbm_nextkey(). Removing the > if ( last.dptr != NULL ) > ldbm_datum_free( dbp, last ); > lines fixes the problem. > > Updated main() in ldbmcat.c: > > int > main( int argc, char **argv ) > { > Datum key, last, data; > LDBM dbp; > int rc, type; > long id; > char *file, *s; > int printid = 1; > > #ifdef HAVE_BERKELEY_DB2 > DBC *cursorp; > #endif > > ldbm_datum_init( key ); > ldbm_datum_init( last ); > ldbm_datum_init( data ); > > if ( argc < 2 || argc > 3 || ( argc == 3 && strcmp( argv[1], "-n" ) > != 0 )) { > usage( argv[0] ); > } > if ( argc == 3 && strcmp( argv[1], "-n" ) == 0 ) { > printid = 0; > file = argv[2]; > } else { > file = argv[1]; > } > > if ( (dbp = ldbm_open( file, LDBM_READER, 0, 0 )) == NULL ) { > perror( file ); > exit ( 1 ); > } > > last.dptr = NULL; > > #ifdef HAVE_BERKELEY_DB2 > for ( key = ldbm_firstkey( dbp, &cursorp ); key.dptr != NULL; > key = ldbm_nextkey( dbp, last, cursorp ) ) > #else > for ( key = ldbm_firstkey( dbp ); key.dptr != NULL; > key = ldbm_nextkey( dbp, last ) ) > #endif > { > /* Begin Internet Devices mods. */ > #ifdef HAVE_BERKELEY_DB2 > key.flags = 0; > data = ldbm_fetch( dbp, key ); > key.flags = DB_DBT_MALLOC; > #else > if ( last.dptr != NULL ) > ldbm_datum_free( dbp, last ); > data = ldbm_fetch( dbp, key ); > #endif HAVE_BERKELEY_DB2 > /* End Internet Devices mods. */ > > if (( s = data.dptr ) != NULL ) { > > if ( !printid && isdigit( *s )) { > > if (( s = strchr( s, '\n' )) != NULL ) { > ++s; > } > } > if ( s != NULL ) { > puts( s ); > } > > if ( data.dptr != NULL ) { > ldbm_datum_free( dbp, data ); > } > > } > > last = key; > } > > /* Begin Internet Devices mods. */ > #ifndef HAVE_BERKELEY_DB2 > if ( last.dptr != NULL ) > ldbm_datum_free( dbp, last ); > #endif HAVE_BERKELEY_DB2 > /* End Internet Devices mods. */ > > ldbm_close( dbp ); > > exit( 0 ); > } > > ----------==========#########>>>>>ZDV<<<<<#########==========---------- X.500: Tel.: Kurt Spanier, Zentrum fuer Datenverarbeitung, +49 7071 29-70334 Universitaet Tuebingen, DE SMTP-Mail: FAX.: kurt.spanier@zdv.uni-tuebingen.de +49 7071 29-5912 Snail-Mail: Dr. Kurt Spanier, Zentrum fuer Datenverarbeitung, Universitaet Tuebingen, Waechterstrasse 76, D-72074 Tuebingen PGP-Public-Key: finger "Kurt Spanier"@x500.uni-tuebingen.de ----------==========##########>>>>>@<<<<<##########==========----------
changed notes changed state Open to Feedback
Though I believe Ksp is working on a long term solution, I am going to apply this patch to rel eng 1.2. It appears to resolve the problem the reported problems. Kurt At 06:04 PM 2/25/99 GMT, svakil@internetdevices.com wrote: >Full_Name: Sumit A. Vakil >Version: 1.2 >OS: FreeBSD >URL: ftp://ftp.openldap.org/incoming/ >Submission from: (NULL) (205.138.236.250) > > >I have OpenLDAP version 1.2 compiled on a FreeBSD 2.2.x system. I'm using the >Berkeley DB 2.3.16 database. ldbmcat does not work. Running ldbmcat does not >produce any ldif output. It also complains about chunks of memory being freed >again. > >The problem is easy to reproduce. Simply run ldbmcat (with or without the -n >option) on an existing db. > >The details of the problem and a suggested fix are as follows: > >The problem is that ldbm_firstkey() sets the key flags to DB_DBT_MALLOC whereas >the DB functions called by ldbm_fetch() expect the key flags to be zero. I >fixed the problem by setting the key flags to zero before calling ldbm_fetch() >and then setting them back to DB_DBT_MALLOC. Note that the key flags should be >set back to DB_DBT_MALLOC before calling ldbm_nextkey(). > >Another problem I noticed was that some chunks of memory were being free twice. >The problem there is that ldbmcat tries to free key.data. This memory is >already freed by ldbm_nextkey(). Removing the > if ( last.dptr != NULL ) > ldbm_datum_free( dbp, last ); >lines fixes the problem. > >Updated main() in ldbmcat.c: > >int >main( int argc, char **argv ) >{ > Datum key, last, data; > LDBM dbp; > int rc, type; > long id; > char *file, *s; > int printid = 1; > >#ifdef HAVE_BERKELEY_DB2 > DBC *cursorp; >#endif > > ldbm_datum_init( key ); > ldbm_datum_init( last ); > ldbm_datum_init( data ); > > if ( argc < 2 || argc > 3 || ( argc == 3 && strcmp( argv[1], "-n" ) > != 0 )) { > usage( argv[0] ); > } > if ( argc == 3 && strcmp( argv[1], "-n" ) == 0 ) { > printid = 0; > file = argv[2]; > } else { > file = argv[1]; > } > > if ( (dbp = ldbm_open( file, LDBM_READER, 0, 0 )) == NULL ) { > perror( file ); > exit ( 1 ); > } > > last.dptr = NULL; > >#ifdef HAVE_BERKELEY_DB2 > for ( key = ldbm_firstkey( dbp, &cursorp ); key.dptr != NULL; > key = ldbm_nextkey( dbp, last, cursorp ) ) >#else > for ( key = ldbm_firstkey( dbp ); key.dptr != NULL; > key = ldbm_nextkey( dbp, last ) ) >#endif > { >/* Begin Internet Devices mods. */ >#ifdef HAVE_BERKELEY_DB2 > key.flags = 0; > data = ldbm_fetch( dbp, key ); > key.flags = DB_DBT_MALLOC; >#else > if ( last.dptr != NULL ) > ldbm_datum_free( dbp, last ); > data = ldbm_fetch( dbp, key ); >#endif HAVE_BERKELEY_DB2 >/* End Internet Devices mods. */ > > if (( s = data.dptr ) != NULL ) { > > if ( !printid && isdigit( *s )) { > > if (( s = strchr( s, '\n' )) != NULL ) { > ++s; > } > } > if ( s != NULL ) { > puts( s ); > } > > if ( data.dptr != NULL ) { > ldbm_datum_free( dbp, data ); > } > > } > > last = key; > } > >/* Begin Internet Devices mods. */ >#ifndef HAVE_BERKELEY_DB2 > if ( last.dptr != NULL ) > ldbm_datum_free( dbp, last ); >#endif HAVE_BERKELEY_DB2 >/* End Internet Devices mods. */ > > ldbm_close( dbp ); > > exit( 0 ); >} > > >
Please test 1.2.1
changed notes changed state Feedback to Closed
changed notes changed state Closed to Feedback
I've ressurrected ITS#85 and closed #130 as duplicate. Some of the changes made to LDBM 1.2.X code appear to be incompatible with DB 2.3.16. Until this is fixed, you can workaround the problem by: 1) using an earlier version of ldbmcat (from 1.1.x) 2) experiment with BerkeleyDB 2.7.3 Kurt At 07:42 PM 4/16/99 GMT, asparks@nss.harris.com wrote: >Full_Name: Alan Sparks >Version: 1.2.1 >OS: HP/UX 10.20 >URL: ftp://ftp.openldap.org/incoming/ >Submission from: (NULL) (151.114.2.18) > > >ldbmcat running against an existing database produces no LDIF output, >or error messages. > >Same apparent behavior as in ITS #85. > >Compiled with db-2.3.16 on HP/UX 10.20 with GCC 2.8.1. > >
I tried running an older ldbmcat, and recompiling an older ldbmcat.c using the 1.2.1 env as a base. Neither one works. Now I don't know how to get a usable dump out of the system to go back to a previous release, or even to have an LDIF backup... :-( -Alan On Fri, 16 Apr 1999, Kurt D. Zeilenga wrote: > I've ressurrected ITS#85 and closed #130 as duplicate. > > Some of the changes made to LDBM 1.2.X code appear to be > incompatible with DB 2.3.16. Until this is fixed, > you can workaround the problem by: > 1) using an earlier version of ldbmcat (from 1.1.x) > 2) experiment with BerkeleyDB 2.7.3 > > Kurt > > At 07:42 PM 4/16/99 GMT, asparks@nss.harris.com wrote: > >Full_Name: Alan Sparks > >Version: 1.2.1 > >OS: HP/UX 10.20 > >URL: ftp://ftp.openldap.org/incoming/ > >Submission from: (NULL) (151.114.2.18) > > > > > >ldbmcat running against an existing database produces no LDIF output, > >or error messages. > > > >Same apparent behavior as in ITS #85. > > > >Compiled with db-2.3.16 on HP/UX 10.20 with GCC 2.8.1. > > > >
Sorry, I really didn't mean to whine (but sure reads like it). I went back and noted the -L option to ldapsearch, which looks good for doing the database dump for import into ldif2ldbm. -Alan On Fri, 16 Apr 1999, Alan Sparks wrote: > I tried running an older ldbmcat, and recompiling an older ldbmcat.c > using the 1.2.1 env as a base. Neither one works. > Now I don't know how to get a usable dump out of the system to go back > to a previous release, or even to have an LDIF backup... :-( > -Alan
Alan Sparks wrote: > I tried running an older ldbmcat, and recompiling an older ldbmcat.c > using the 1.2.1 env as a base. Don't mix and match source files between versions. An ldbmcat built from 1.1.x sources should work fine. If it doesn't, you could experiment with db 2.7.3. (I say experiement as we have not fully tested 2.7.3 yet). Kurt
Yes... I have rebuild a version of ldbmcat from OpenLDAP 1.2.1 with a copy of Sleepycat 2.7.4 (latest on site). This works apparently fine. Excellent enough. Thanks for the hints. -Alan -----Original Message----- From: Kurt D. Zeilenga <Kurt@OpenLDAP.Org> To: asparks@nss.harris.com <asparks@nss.harris.com> Cc: openldap-its@OpenLDAP.Org <openldap-its@OpenLDAP.Org> Date: Friday, April 16, 1999 1:02 PM Subject: Re: ldbmcat still broken as per (ITS#85) >I've ressurrected ITS#85 and closed #130 as duplicate. > >Some of the changes made to LDBM 1.2.X code appear to be >incompatible with DB 2.3.16. Until this is fixed, >you can workaround the problem by: > 1) using an earlier version of ldbmcat (from 1.1.x) > 2) experiment with BerkeleyDB 2.7.3 > >Kurt > >At 07:42 PM 4/16/99 GMT, asparks@nss.harris.com wrote: >>Full_Name: Alan Sparks >>Version: 1.2.1 >>OS: HP/UX 10.20 >>URL: ftp://ftp.openldap.org/incoming/ >>Submission from: (NULL) (151.114.2.18) >> >> >>ldbmcat running against an existing database produces no LDIF output, >>or error messages. >> >>Same apparent behavior as in ITS #85. >> >>Compiled with db-2.3.16 on HP/UX 10.20 with GCC 2.8.1. >> >>
Please try OpenLDAP 1.2.1 with BerkeleyDB 2.7.3 (or 2.7.4). Support both 2.7.3 and 2.3.16 appears to be difficult. As 2.7.4 is newer and appears to be stable, we're considerring changing our recommended BerkeleyDB version to 2.7.4. Kurt
changed notes changed state Feedback to Release
changed notes changed state Release to Closed
1.2.1 appears to incompatible with DB 2.3.16. Advised user to upgrade to DB 2.7.3 See Also: ITS#106