[Date Prev][Date Next]
Re: (ITS#8392) Opening multiple databases from the same process
- To: openldap-its@OpenLDAP.org
- Subject: Re: (ITS#8392) Opening multiple databases from the same process
- From: firstname.lastname@example.org
- Date: Sat, 26 Mar 2016 18:44:56 +0000
- Auto-submitted: auto-generated (OpenLDAP-ITS)
> Full_Name: Matthew Sackman
> Version: LMDB 0.9.18
> OS: Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (18.104.22.168)
> Linux koba 4.4.0-1-amd64 #1 SMP Debian 4.4.6-1 (2016-03-17) x86_64 GNU/Linux
> LMDB 0.9.18 (compiled locally)
> I'm currently writing a program that iterates through multiple LMDB databases in
> order to check differences between them. On each database I have one
> long-running read-only txn with a cursor, and I'm walking through all the
> databases at the same time.
> I'm seeing some very odd behaviour. For example, cursor.Get with FIRST is not
> always returning the first item in the table (which I happen to know exactly
> what it should be). The behaviour is non-deterministic - different runs produce
> different results.
> I've searched through the docs and I can't find any statements regarding opening
> multiple dbs from the same process. Is this supported?
> I'm writing this in Go, using my own bindings. I'm pretty certain that I'm doing
> all the threading correctly and I certainly get this same issue regardless of
> whether I'm using NOTLS or not (with my bindings, it shouldn't be necessary
> because I'm locking go-routines to os-threads), plus these are the same bindings
> that I've used throughout GoshawkDB and I've not come across this problem until
> I can guarantee that each DB that I open will be opened in a different OS thread
> and that no single OS thread will ever access different DBs.
Is this entire task read-only, nobody is modifying any of the DBs while these
readers are iterating thru the DBs?
I've only used Go very briefly. I would be quite suspicious of that goroutine
behavior but don't have any insight there.
You might consider compiling LMDB with MDB_DEBUG defined, enabling debug
output, and then watching the cursor activity. Of course, the act of writing
debug messages will impact thread scheduling, so it may not reveal what's
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/