[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#7491) mdb write bus error before adding 5 million keys
Thank you. I did not consider the transaction size.
On 1/11/13, Howard Chu <hyc@symas.com> wrote:
> clayton.stangeland@gmail.com wrote:
>> Full_Name: Clayton Stangeland
>> Version: git mdb.master last commit
>> fed573cb86ed99f37bd062908ec814ee0ca47053
>> OS: Fedora 17 64 bit
>> URL: ftp://ftp.openldap.org/incoming/mtest-big.c
>> Submission from: (NULL) (192.94.73.31)
>>
>>
>> Building and running mdb from git branch mdb.master commit
>> fed573cb86ed99f37bd062908ec814ee0ca47053 on Fedora 17 64 bit I get a bus
>> error.
>
> Thanks for the report. The bus error is now fixed in git mdb.master. We
> assume
> that most users will not batch so many operations into a single
> transaction.
>>
>> When running the test below, which is a modification of mtest.c, I get a
>> Bus
>> error. This happens while trying to write the file. gdb reports line 3936
>> of
>> mdb.c as the erroring line inside mdb_page_search_root: while
>> (IS_BRANCH(mp)) {
>>
>> If I set count to 4 million instead of 5 million it works, and adding back
>> in
>> all the other mtest.c tests works as well then.
>>
>> Here is the code:
>> /* mtest.c - memory-mapped database tester/toy */
>> /*
>> * Copyright 2011 Howard Chu, Symas Corp.
>> * All rights reserved.
>> *
>> * Redistribution and use in source and binary forms, with or without
>> * modification, are permitted only as authorized by the OpenLDAP
>> * Public License.
>> *
>> * A copy of this license is available in the file LICENSE in the
>> * top-level directory of the distribution or, alternatively, at
>> * <http://www.OpenLDAP.org/license.html>.
>> */
>> #define _XOPEN_SOURCE 500 /* srandom(), random() */
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <time.h>
>> #include "lmdb.h"
>>
>> int main(int argc,char * argv[])
>> {
>> int i = 0, j = 0, rc;
>> MDB_env *env;
>> MDB_dbi dbi;
>> MDB_val key, data;
>> MDB_txn *txn;
>> MDB_stat mst;
>> MDB_cursor *cursor;
>> int count;
>> int *values;
>> char sval[32];
>>
>> srandom(time(NULL));
>>
>> count = (5000000) + 64;
>> values = (int *)malloc(count*sizeof(int));
>>
>> for(i = 0;i<count;i++) {
>> values[i] = random()%1000000000;
>> }
>>
>> rc = mdb_env_create(&env);
>> rc = mdb_env_set_mapsize(env, 10485760000);
>> rc = mdb_env_open(env, "./testdb", MDB_FIXEDMAP
>> /*|MDB_NOSYNC*/,
>> 0664);
>> rc = mdb_txn_begin(env, NULL, 0, &txn);
>> rc = mdb_open(txn, NULL, 0, &dbi);
>>
>> key.mv_size = sizeof(sval);
>> key.mv_data = sval;
>> data.mv_size = sizeof(sval);
>> data.mv_data = sval;
>>
>> for (i=0;i<count;i++) {
>> sprintf(sval, "%08x %d foo bar", values[i],
>> values[i]);
>> rc = mdb_put(txn, dbi, &key, &data,
>> MDB_NOOVERWRITE);
>> if (rc) j++;
>> }
>> if (j) printf("%d duplicates skipped\n", j);
>> rc = mdb_txn_commit(txn);
>> rc = mdb_env_stat(env, &mst);
>>
>> free(values);
>> mdb_close(env, dbi);
>> mdb_env_close(env);
>>
>> return 0;
>>
>> }
>>
>>
>
>
> --
> -- Howard Chu
> CTO, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
>