Issue 7448 - "assertion failed" in MDB
Summary: "assertion failed" in MDB
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-11-20 23:53 UTC by artemciy@gmail.com
Modified: 2013-05-23 22:28 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description artemciy@gmail.com 2012-11-20 23:53:29 UTC
Full_Name: Artem Grinblat
Version: git, commit be781ab8081d7e9ad1eb89da81077c7f936a5604
OS: Debian Wheezy
URL: 
Submission from: (NULL) (213.108.212.44)


MDB works okay for some time, then crashes with

mdb.c:3778: mdb_page_search_root: Assertion `(((mp)->mp_pb.pb.pb_lower -
((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1' failed.

(gdb) bt
#0  WriteCoreDump (file_name=0x7f1f1f705fa0 "/tmp/tntnet.5541.core") at
src/coredumper.c:192
#1  0x00007f1f4b163da9 in img2::writeCore (path=0x7f1f1f705fa0
"/tmp/tntnet.5541.core") at src/conf.cpp:49
#2  0x00007f1f4b16145d in img2::custom_sigaction (signum=6) at src/conf.cpp:89
#3  <signal handler called>
#4  0x00007f1f469ba475 in *__GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
#5  0x00007f1f469bd6f0 in *__GI_abort () at abort.c:92
#6  0x00007f1f469b3621 in *__GI___assert_fail (
    assertion=assertion@entry=0x7f1f4155cdb0 "(((mp)->mp_pb.pb.pb_lower -
((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1", file=<optimized
out>,
    file@entry=0x7f1f4155cb30 "mdb.c", line=line@entry=3778,
function=function@entry=0x7f1f4155d360 "mdb_page_search_root") at assert.c:81
#7  0x00007f1f41555ccd in mdb_page_search_root (mc=mc@entry=0x7f1f1f708500,
key=key@entry=0x0, modify=modify@entry=0) at mdb.c:3778
#8  0x00007f1f4155467a in mdb_page_search (mc=mc@entry=0x7f1f1f708500,
key=key@entry=0x0, flags=flags@entry=0) at mdb.c:3908
#9  0x00007f1f41554a07 in mdb_cursor_first (mc=0x7f1f1f708500,
key=key@entry=0x7f1f1f7070f0, data=data@entry=0x0) at mdb.c:4354
#10 0x00007f1f4155517e in mdb_cursor_set (mc=mc@entry=0x7f1f1f708370,
key=key@entry=0x7f1f1f708770, data=data@entry=0x7f1f1f7070f0,
op=op@entry=MDB_SET,
    exactp=exactp@entry=0x7f1f1f7070d4) at mdb.c:4307
#11 0x00007f1f41559c8d in mdb_cursor_put (mc=0x7f1f1f708370, key=0x7f1f1f708770,
data=0x7f1f1f708790, flags=0) at mdb.c:4672
#12 0x00007f1f4155bdc9 in mdb_put (txn=<optimized out>, dbi=<optimized out>,
key=0x7f1f1f708770, data=0x7f1f1f708790, flags=<optimized out>) at mdb.c:6599

Also under Valgrind:
==7503== Warning: set address range perms: large range [0x3959d000, 0x7959d000)
(defined)
**7503** custom_sigaction, signal 6   at 0x51B6EEC:
VALGRIND_PRINTF_BACKTRACE(char const*, ...) (valgrind.h:3771)
==7503==    by 0x51B72A1: img2::custom_sigaction(int) (conf.cpp:77)
==7503==    by 0x985B4EF: ??? (in /lib/x86_64-linux-gnu/libc-2.13.so)
==7503==    by 0x985B474: raise (raise.c:64)
==7503==    by 0x985E6EF: abort (abort.c:92)
==7503==    by 0x9854620: __assert_fail (assert.c:81)
==7503==    by 0xEDD9CCC: mdb_page_search_root (mdb.c:3778)
==7503==    by 0xEDD8679: mdb_page_search (mdb.c:3908)
==7503==    by 0xEDD8F78: mdb_cursor_set (mdb.c:4268)
==7503==    by 0xEDD8EB0: mdb_cursor_set (mdb.c:4316)
==7503==    by 0xEDDFCA8: mdb_del (mdb.c:6165)
Comment 1 Howard Chu 2012-11-21 18:15:42 UTC
More information is needed than this. Your stack traces aren't even complete. 
What was the workload leading up to this? Give us a sample configuration that 
reproduces the assert.

artemciy@gmail.com wrote:
> Full_Name: Artem Grinblat
> Version: git, commit be781ab8081d7e9ad1eb89da81077c7f936a5604
> OS: Debian Wheezy
> URL:
> Submission from: (NULL) (213.108.212.44)
>
>
> MDB works okay for some time, then crashes with
>
> mdb.c:3778: mdb_page_search_root: Assertion `(((mp)->mp_pb.pb.pb_lower -
> ((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1' failed.
>
> (gdb) bt
> #0  WriteCoreDump (file_name=0x7f1f1f705fa0 "/tmp/tntnet.5541.core") at
> src/coredumper.c:192
> #1  0x00007f1f4b163da9 in img2::writeCore (path=0x7f1f1f705fa0
> "/tmp/tntnet.5541.core") at src/conf.cpp:49
> #2  0x00007f1f4b16145d in img2::custom_sigaction (signum=6) at src/conf.cpp:89
> #3  <signal handler called>
> #4  0x00007f1f469ba475 in *__GI_raise (sig=<optimized out>) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> #5  0x00007f1f469bd6f0 in *__GI_abort () at abort.c:92
> #6  0x00007f1f469b3621 in *__GI___assert_fail (
>      assertion=assertion@entry=0x7f1f4155cdb0 "(((mp)->mp_pb.pb.pb_lower -
> ((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1", file=<optimized
> out>,
>      file@entry=0x7f1f4155cb30 "mdb.c", line=line@entry=3778,
> function=function@entry=0x7f1f4155d360 "mdb_page_search_root") at assert.c:81
> #7  0x00007f1f41555ccd in mdb_page_search_root (mc=mc@entry=0x7f1f1f708500,
> key=key@entry=0x0, modify=modify@entry=0) at mdb.c:3778
> #8  0x00007f1f4155467a in mdb_page_search (mc=mc@entry=0x7f1f1f708500,
> key=key@entry=0x0, flags=flags@entry=0) at mdb.c:3908
> #9  0x00007f1f41554a07 in mdb_cursor_first (mc=0x7f1f1f708500,
> key=key@entry=0x7f1f1f7070f0, data=data@entry=0x0) at mdb.c:4354
> #10 0x00007f1f4155517e in mdb_cursor_set (mc=mc@entry=0x7f1f1f708370,
> key=key@entry=0x7f1f1f708770, data=data@entry=0x7f1f1f7070f0,
> op=op@entry=MDB_SET,
>      exactp=exactp@entry=0x7f1f1f7070d4) at mdb.c:4307
> #11 0x00007f1f41559c8d in mdb_cursor_put (mc=0x7f1f1f708370, key=0x7f1f1f708770,
> data=0x7f1f1f708790, flags=0) at mdb.c:4672
> #12 0x00007f1f4155bdc9 in mdb_put (txn=<optimized out>, dbi=<optimized out>,
> key=0x7f1f1f708770, data=0x7f1f1f708790, flags=<optimized out>) at mdb.c:6599
>
> Also under Valgrind:
> ==7503== Warning: set address range perms: large range [0x3959d000, 0x7959d000)
> (defined)
> **7503** custom_sigaction, signal 6   at 0x51B6EEC:
> VALGRIND_PRINTF_BACKTRACE(char const*, ...) (valgrind.h:3771)
> ==7503==    by 0x51B72A1: img2::custom_sigaction(int) (conf.cpp:77)
> ==7503==    by 0x985B4EF: ??? (in /lib/x86_64-linux-gnu/libc-2.13.so)
> ==7503==    by 0x985B474: raise (raise.c:64)
> ==7503==    by 0x985E6EF: abort (abort.c:92)
> ==7503==    by 0x9854620: __assert_fail (assert.c:81)
> ==7503==    by 0xEDD9CCC: mdb_page_search_root (mdb.c:3778)
> ==7503==    by 0xEDD8679: mdb_page_search (mdb.c:3908)
> ==7503==    by 0xEDD8F78: mdb_cursor_set (mdb.c:4268)
> ==7503==    by 0xEDD8EB0: mdb_cursor_set (mdb.c:4316)
> ==7503==    by 0xEDDFCA8: mdb_del (mdb.c:6165)
>
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 2 artemciy@gmail.com 2012-11-21 19:07:34 UTC
On Wed, Nov 21, 2012 at 10:15 PM, Howard Chu <hyc@symas.com> wrote:

> More information is needed than this. Your stack traces aren't even
> complete.


This isn't relevant.


> What was the workload leading up to this?


I can try to make an ltrace log, would that suffice?

Give us a sample configuration that reproduces the assert.
>

Don't have time to make a test case now.


> artemciy@gmail.com wrote:
>
>> Full_Name: Artem Grinblat
>> Version: git, commit be781ab8081d7e9ad1eb89da81077c**7f936a5604
>> OS: Debian Wheezy
>> URL:
>> Submission from: (NULL) (213.108.212.44)
>>
>>
>> MDB works okay for some time, then crashes with
>>
>> mdb.c:3778: mdb_page_search_root: Assertion `(((mp)->mp_pb.pb.pb_lower -
>> ((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1' failed.
>>
>> (gdb) bt
>> #0  WriteCoreDump (file_name=0x7f1f1f705fa0 "/tmp/tntnet.5541.core") at
>> src/coredumper.c:192
>> #1  0x00007f1f4b163da9 in img2::writeCore (path=0x7f1f1f705fa0
>> "/tmp/tntnet.5541.core") at src/conf.cpp:49
>> #2  0x00007f1f4b16145d in img2::custom_sigaction (signum=6) at
>> src/conf.cpp:89
>> #3  <signal handler called>
>> #4  0x00007f1f469ba475 in *__GI_raise (sig=<optimized out>) at
>> ../nptl/sysdeps/unix/sysv/**linux/raise.c:64
>> #5  0x00007f1f469bd6f0 in *__GI_abort () at abort.c:92
>> #6  0x00007f1f469b3621 in *__GI___assert_fail (
>>      assertion=assertion@entry=**0x7f1f4155cdb0
>> "(((mp)->mp_pb.pb.pb_lower -
>> ((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1",
>> file=<optimized
>> out>,
>>      file@entry=0x7f1f4155cb30 "mdb.c", line=line@entry=3778,
>> function=function@entry=**0x7f1f4155d360 "mdb_page_search_root") at
>> assert.c:81
>> #7  0x00007f1f41555ccd in mdb_page_search_root (mc=mc@entry
>> =0x7f1f1f708500,
>> key=key@entry=0x0, modify=modify@entry=0) at mdb.c:3778
>> #8  0x00007f1f4155467a in mdb_page_search (mc=mc@entry=0x7f1f1f708500,
>> key=key@entry=0x0, flags=flags@entry=0) at mdb.c:3908
>> #9  0x00007f1f41554a07 in mdb_cursor_first (mc=0x7f1f1f708500,
>> key=key@entry=0x7f1f1f7070f0, data=data@entry=0x0) at mdb.c:4354
>> #10 0x00007f1f4155517e in mdb_cursor_set (mc=mc@entry=0x7f1f1f708370,
>> key=key@entry=0x7f1f1f708770, data=data@entry=**0x7f1f1f7070f0,
>> op=op@entry=MDB_SET,
>>      exactp=exactp@entry=**0x7f1f1f7070d4) at mdb.c:4307
>> #11 0x00007f1f41559c8d in mdb_cursor_put (mc=0x7f1f1f708370,
>> key=0x7f1f1f708770,
>> data=0x7f1f1f708790, flags=0) at mdb.c:4672
>> #12 0x00007f1f4155bdc9 in mdb_put (txn=<optimized out>, dbi=<optimized
>> out>,
>> key=0x7f1f1f708770, data=0x7f1f1f708790, flags=<optimized out>) at
>> mdb.c:6599
>>
>> Also under Valgrind:
>> ==7503== Warning: set address range perms: large range [0x3959d000,
>> 0x7959d000)
>> (defined)
>> **7503** custom_sigaction, signal 6   at 0x51B6EEC:
>> VALGRIND_PRINTF_BACKTRACE(char const*, ...) (valgrind.h:3771)
>> ==7503==    by 0x51B72A1: img2::custom_sigaction(int) (conf.cpp:77)
>> ==7503==    by 0x985B4EF: ??? (in /lib/x86_64-linux-gnu/libc-2.**13.so<http://libc-2.13.so>
>> )
>> ==7503==    by 0x985B474: raise (raise.c:64)
>> ==7503==    by 0x985E6EF: abort (abort.c:92)
>> ==7503==    by 0x9854620: __assert_fail (assert.c:81)
>> ==7503==    by 0xEDD9CCC: mdb_page_search_root (mdb.c:3778)
>> ==7503==    by 0xEDD8679: mdb_page_search (mdb.c:3908)
>> ==7503==    by 0xEDD8F78: mdb_cursor_set (mdb.c:4268)
>> ==7503==    by 0xEDD8EB0: mdb_cursor_set (mdb.c:4316)
>> ==7503==    by 0xEDDFCA8: mdb_del (mdb.c:6165)
>>
>>
>>
>
> --
>   -- Howard Chu
>   CTO, Symas Corp.           http://www.symas.com
>   Director, Highland Sun     http://highlandsun.com/hyc/
>   Chief Architect, OpenLDAP  http://www.openldap.org/**project/<http://www.openldap.org/project/>
>
Comment 3 artemciy@gmail.com 2012-11-26 12:00:43 UTC
On Mon, Nov 26, 2012 at 4:55 AM, Howard Chu <hyc@symas.com> wrote:

> We will investigate this when a reproducible test case is available.
>

You can close the issue then, because I won't be producing one.
Comment 4 Quanah Gibson-Mount 2012-11-26 21:54:06 UTC
--On Monday, November 26, 2012 12:01 PM +0000 artemciy@gmail.com wrote:

> --047d7b339025cf973e04cf64ae11
> Content-Type: text/plain; charset=ISO-8859-1
>
> On Mon, Nov 26, 2012 at 4:55 AM, Howard Chu <hyc@symas.com> wrote:
>
>> We will investigate this when a reproducible test case is available.
>>
>
> You can close the issue then, because I won't be producing one.

I don't understand why you would take the time to report a bug, but then 
refuse to provide the information necessary to fix it.  That is not 
particularly helpful. :/

--Quanah



--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Comment 5 artemciy@gmail.com 2012-11-26 22:06:14 UTC
On Tue, Nov 27, 2012 at 1:54 AM, Quanah Gibson-Mount <quanah@zimbra.com>wrote:

> You can close the issue then, because I won't be producing one.
>>
>
> I don't understand why you would take the time to report a bug, but then
> refuse to provide the information necessary to fix it.  That is not
> particularly helpful. :/
>
> --Quanah
>

Quanah, from past experience I know that making a test case is a research
endeavor that can be quite costly. IMO, refusing to investigate a bug
without a test case is not a sane support politics.
For now I have found switching to Leveldb easier than taking another
library for maintance (e.g. http://lambda-the-ultimate.org/node/4636 ->
http://akkartik.name/blog/libraries).
BTW, here is an example of good support:
http://tracker.newdream.net/issues/3495
Comment 6 Howard Chu 2012-11-26 23:16:21 UTC
artemciy@gmail.com wrote:
> --e89a8f3b55c652009c04cf6d24b9
> Content-Type: text/plain; charset=ISO-8859-1
>
> On Tue, Nov 27, 2012 at 1:54 AM, Quanah Gibson-Mount <quanah@zimbra.com>wrote:
>
>> You can close the issue then, because I won't be producing one.
>>>
>>
>> I don't understand why you would take the time to report a bug, but then
>> refuse to provide the information necessary to fix it.  That is not
>> particularly helpful. :/
>>
>> --Quanah
>>
>
> Quanah, from past experience I know that making a test case is a research
> endeavor that can be quite costly. IMO, refusing to investigate a bug
> without a test case is not a sane support politics.

Chasing "bugs" in 3rd party code that you haven't shared with us is not a sane 
policy. If we can't see the code, we certainly can't identify the root cause 
of the issue. Given that our library is working in our primary application 
(slapd) as well as multiple other projects, the burden of proof is on you to 
demonstrate that the bug is not caused by your C++ application.

> For now I have found switching to Leveldb easier than taking another
> library for maintance (e.g. http://lambda-the-ultimate.org/node/4636 ->
> http://akkartik.name/blog/libraries).
> BTW, here is an example of good support:
> http://tracker.newdream.net/issues/3495


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 7 artemciy@gmail.com 2012-11-26 23:19:41 UTC
On Tue, Nov 27, 2012 at 3:16 AM, Howard Chu <hyc@symas.com> wrote:

> Quanah, from past experience I know that making a test case is a research
>> endeavor that can be quite costly. IMO, refusing to investigate a bug
>> without a test case is not a sane support politics.
>>
>
> Chasing "bugs" in 3rd party code that you haven't shared with us is not a
> sane policy. If we can't see the code, we certainly can't identify the root
> cause of the issue. Given that our library is working in our primary
> application (slapd) as well as multiple other projects, the burden of proof
> is on you to demonstrate that the bug is not caused by your C++ application.


If you think it is acceptable for your library to die with "assert" because
of a 3rd party code then I have no further questions about the
"reliability" of your software.
Comment 8 artemciy@gmail.com 2012-11-27 07:13:53 UTC
On Tue, Nov 27, 2012 at 1:54 AM, Quanah Gibson-Mount <quanah@zimbra.com>wrote:

> You can close the issue then, because I won't be producing one.
>>
>
> I don't understand why you would take the time to report a bug, but then
> refuse to provide the information necessary to fix it.


Indeed, you might stress on http://www.openldap.org/its/index.cgi that bugs
without a test case wouldn't be investigated so that people won't waste
their time.

 That is not particularly helpful. :/
>
> --Quanah
>
Comment 9 Howard Chu 2012-11-27 09:04:30 UTC
artemciy@gmail.com wrote:
> --f46d042fddcee1dc8704cf74ca2b
> Content-Type: text/plain; charset=ISO-8859-1
>
> On Tue, Nov 27, 2012 at 1:54 AM, Quanah Gibson-Mount <quanah@zimbra.com>wrote:
>
>> You can close the issue then, because I won't be producing one.
>>>
>>
>> I don't understand why you would take the time to report a bug, but then
>> refuse to provide the information necessary to fix it.
>
>
> Indeed, you might stress on http://www.openldap.org/its/index.cgi that bugs
> without a test case wouldn't be investigated so that people won't waste
> their time.

http://www.openldap.org/faq/data/cache/1122.html


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 10 artemciy@gmail.com 2012-11-27 09:08:45 UTC
On Tue, Nov 27, 2012 at 1:04 PM, Howard Chu <hyc@symas.com> wrote:

>
> Indeed, you might stress on http://www.openldap.org/its/**index.cgi<http://www.openldap.org/its/index.cgi>that bugs
>> without a test case wouldn't be investigated so that people won't waste
>> their time.
>>
>
> http://www.openldap.org/faq/**data/cache/1122.html<http://www.openldap.org/faq/data/cache/1122.html>


"In case the first aim doesn't succeed, and the programmer *can't* see it
failing themselves, the second aim of a bug report is to describe what went
wrong. Describe everything in detail. State what you saw, and also state
what you expected to see. Write down the error messages, *especially* if
they have numbers in". -
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

Stack trace and proposal to do ltrace is the best detail you've got in a
normal bug report.
Good luck.
Comment 11 Howard Chu 2012-11-27 09:53:59 UTC
Artem Grinblat wrote:
> On Tue, Nov 27, 2012 at 1:04 PM, Howard Chu <hyc@symas.com
> <mailto:hyc@symas.com>> wrote:
>
>
>         Indeed, you might stress on http://www.openldap.org/its/__index.cgi
>         <http://www.openldap.org/its/index.cgi> that bugs
>         without a test case wouldn't be investigated so that people won't waste
>         their time.
>
>
>     http://www.openldap.org/faq/__data/cache/1122.html
>     <http://www.openldap.org/faq/data/cache/1122.html>
>
>
> "In case the first aim doesn't succeed, and the programmer /can't/ see it
> failing themselves, the second aim of a bug report is to describe what went
> wrong. Describe everything in detail. State what you saw, and also state what
> you expected to see. Write down the error messages, /especially/ if they have
> numbers in". - http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
>
> Stack trace and proposal to do ltrace is the best detail you've got in a
> normal bug report.
> Good luck.

A complete stack trace is usually sufficient, when we have complete source 
code in hand. In this case your source code is not present.

Ultimately the question is "do you want help to make this work?" Clearly 
you're not interested in helping. ITS closed.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 12 Howard Chu 2012-11-27 09:54:33 UTC
changed state Open to Closed
Comment 13 Quanah Gibson-Mount 2012-11-27 20:06:30 UTC
--On Tuesday, November 27, 2012 9:09 AM +0000 artemciy@gmail.com wrote:

> http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

I would suggest you actually take the time to read what you link to. 
Clearly, in this case, you have failed to do that.  For example, under the 
section:

"Show me how to show myself."

 This is the era of the Internet. This is the era of worldwide 
communication. This is the era in which I can send my software to somebody 
in Russia at the touch of a button, and he can send me comments about it 
just as easily. But if he has a problem with my program, he can't have me 
standing in front of it while it fails. "Show me" is good when you can, but 
often you can't.

If you have to report a bug to a programmer who can't be present in person, 
the aim of the exercise is to enable them to reproduce the problem. You 
want the programmer to run their own copy of the program, do the same 
things to it, and make it fail in the same way. When they can see the 
problem happening in front of their eyes, then they can deal with it.

So tell them exactly what you did. If it's a graphical program, tell them 
which buttons you pressed and what order you pressed them in. If it's a 
program you run by typing a command, show them precisely what command you 
typed. Wherever possible, you should provide a verbatim transcript of the 
session, showing what commands you typed and what the computer output in 
response.

Give the programmer all the input you can think of. If the program reads 
from a file, you will probably need to send a copy of the file. If the 
program talks to another computer over a network, you probably can't send a 
copy of that computer, but you can at least say what kind of computer it 
is, and (if you can) what software is running on it.


We're asking you to "Show us" what you did.  Provide your code that caused 
an issue, since mdb is working fine with numerous projects.  You are 
refusing to do provide code that shows what the issue is, and then 
expecting the developers to magically fix things with no relevant code path 
to examine.  I don't understand your attitude in this at all.

--Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Comment 14 artemciy@gmail.com 2012-11-27 20:20:39 UTC
On Wed, Nov 28, 2012 at 12:06 AM, Quanah Gibson-Mount <quanah@zimbra.com>wrote:

> --On Tuesday, November 27, 2012 9:09 AM +0000 artemciy@gmail.com wrote:
>
>  http://www.chiark.greenend.**org.uk/~sgtatham/bugs.html<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
>>
>
> I would suggest you actually take the time to read what you link to.
> Clearly, in this case, you have failed to do that.


Yeah, too many letters.
Comment 15 David Hawes 2013-05-22 21:38:42 UTC
On Wed, Nov 21, 2012 at 1:16 PM,  <hyc@symas.com> wrote:
> More information is needed than this. Your stack traces aren't even complete.
> What was the workload leading up to this? Give us a sample configuration that
> reproduces the assert.

I came across what looks like this issue today. Let's see if I can
provide more info.

I noticed one of my test machines (2.4.35) had crashed overnight.
Unfortunately, I had neglected to set ulimit -c, so I had no core dump
to work with. Upon restarting, however, I got the following:

slapd: ./../../../libraries/liblmdb/mdb.c:4025: mdb_page_search_root:
Assertion `(((mp)->mp_pb.pb.pb_lower - ((unsigned) __builtin_offsetof
(MDB_page, mp_ptrs))) >> 1) > 1' failed.

A backtrace shows that this occurred during an accesslog_purge(). The
backtrace is at the end of this email.

The setup I have is a simple accesslog that records binds:

##### slapd.conf #####
...
database mdb
suffix "cn=authstats"
directory /apps/local/openldap-2.4.35/var/openldap-data/authstats
maxsize 17179869184
dbnosync

rootdn cn=manager,dc=vt,dc=edu
index default eq
index entryCSN,objectClass,reqEnd,reqResult,reqStart

checkpoint 1024 10

overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
...
# dc=vt,dc=edu
overlay accesslog
logdb "cn=authstats"
logops bind
logpurge 01+00:00 0+08:00
...
#####

If I comment out the logpurge line, slapd starts without incident,
cn=authstats is queryable, and new entries are logged.

If I slapcat cn=authstats, remove the cn=authstats .mdb files, and
slapadd everything back, upon startup the logpurge works as expected.
If I move the original .mdb files back, the assert fails again.

I'm attempting to reproduce this starting with an empty accesslog. I
have so far been unsuccessful. I do have the original .mdb files if
they may be helpful (though they are around 1G).

One thing to note with the following backtrace is that the
accesslog_purge() line number will be incorrect, as I have patched it
to record connection info (like ITS#7345).  None of this affects
accesslog_purge(), though I can run these tests without the patch if
necessary.

##### backtrace #####
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ff7f1722700 (LWP 1576)]
0x00007ffff66b28a5 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install
db4-4.7.25-17.el6.x86_64 glibc-2.12-1.107.el6.x86_64
keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.2.x86_64
libcom_err-1.41.12-14.el6.x86_64 libselinux-2.0.94-5.3.el6.x86_64
libtool-ltdl-2.2.6-15.5.el6.x86_64
nss-softokn-freebl-3.12.9-11.el6.x86_64
openssl-1.0.0-27.el6_4.2.x86_64 zlib-1.2.3-29.el6.x86_64
#0  0x00007ffff66b28a5 in raise () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007ffff66b4085 in abort () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007ffff66aba1e in __assert_fail_base () from /lib64/libc.so.6
No symbol table info available.
#3  0x00007ffff66abae0 in __assert_fail () from /lib64/libc.so.6
No symbol table info available.
#4  0x0000000000511836 in mdb_page_search_root (mc=0x7ff7e01023f8, key=0x0,
    modify=0) at ./../../../libraries/liblmdb/mdb.c:4025
        node = 0x8
        i = 0
        mp = 0x7ff7e03c9000
        rc = 0
        __PRETTY_FUNCTION__ = "mdb_page_search_root"
#5  0x0000000000517839 in mdb_node_move (csrc=0x7ff7f1720750,
    cdst=0x7ff7e01023f8) at ./../../../libraries/liblmdb/mdb.c:5988
        snum = 3
        s2 = 0x7ffff31270c0
        bkey = {mv_size = 4294967304, mv_data = 0x0}
        srcnode = 0x7ff7e03ccc18
        key = {mv_size = 80, mv_data = 0x7ff7e03ccc20}
        data = {mv_size = 111622, mv_data = 0x7ff7e03ccc70}
        srcpg = 111622
        mn = {mc_next = 0x7ff7f1720530, mc_orig = 0x51cb38,
          mc_xcursor = 0x7ff7f1720740, mc_txn = 0x7ffff31270c0,
          mc_dbi = 4050782896, mc_db = 0x800000008, mc_dbx = 0x7ff700000063,
          mc_dbflag = 0x40 <Address 0x40 out of bounds>, mc_snum = 8,
          mc_top = 0, mc_flags = 0, mc_pg = {0x1030, 0x0, 0x0, 0x1,
            0x7000000101, 0x44ab4, 0x7ffff31270c0, 0x7ff7f1720860,
            0x800000c12, 0x100000009, 0x0, 0x7ff7f17205d0, 0x51cb38,
            0x7ff7f17207e0, 0x7ffff31270c0, 0xc12, 0x900000009,
            0x7ff7f17208b0, 0x50ca2c, 0x300000003, 0x7ff7f17208e0,
            0x11013f9a8, 0x7ff7f1720a00, 0x7ff7e01025f0, 0x0, 0x7ff7e03c0b48,
            0x50cb83, 0x24188, 0x7ff7f1720ab0, 0x7ffc2a2f6000, 0xe03bfd40,
            0x1}, mc_ki = {64832, 57403, 32759, 0, 1696, 61810, 32759, 0,
            5899, 81, 1, 0, 9648, 57360, 32759, 0, 2736, 61810, 32759, 0,
            16776, 2, 0, 0, 57296, 57403, 32759, 0, 64832, 57403, 32759, 0}}
        rc = 0
        flags = 0
        __PRETTY_FUNCTION__ = "mdb_node_move"
#6  0x00000000005191b9 in mdb_rebalance (mc=0x7ff7e01023f8)
    at ./../../../libraries/liblmdb/mdb.c:6391
        node = 0x7ff7e03c8fe8
        rc = 0
        minkeys = 2
        ptop = 1
        mn = {mc_next = 0x7ff7e03bdfd0, mc_orig = 0x7ff7e03ca010,
          mc_xcursor = 0x0, mc_txn = 0x7ff7e03bdfd0, mc_dbi = 4,
          mc_db = 0x7ff7e0102580, mc_dbx = 0x7ff7e01025b0,
          mc_dbflag = 0x7ff7f1720e70 "", mc_snum = 3, mc_top = 2,
          mc_flags = 5, mc_pg = {0x7ff7e03c6fe0, 0x7ff7e03c7ff0,
            0x7ff7e03cc030, 0x7ff7e01025f0, 0x0, 0x7ff7e03cabda, 0x50cb83,
            0x0, 0x100000000000, 0x44ab4, 0x7ff7e03cb020, 0x7ff7e03bdfd0,
            0x7ff7e03cb020, 0x44ab4, 0x0, 0x7ff7f17208a0, 0x50cc73,
            0x7ff7f1720dcc, 0x7ff7e01023f8, 0x7ffc37bf9000, 0x7ff7e03c5fd0,
            0x7ff7e03ca010, 0x7ff7e03ca010, 0x7ff7e01025f0, 0xf1720b00,
            0x7ff7e03cbc58, 0x50cb83, 0x3f17208b0, 0x100000024188,
            0x7ffc34e0d000, 0x7ff7e03cb020, 0x7ff7e03cb884}, mc_ki = {0, 0,
            12, 0, 12616, 185, 0, 0, 3090, 0, 0, 0, 2368, 61810, 0, 0, 0, 0,
            0, 0, 2400, 61810, 32759, 0, 2416, 61810, 32759, 0, 0, 0, 21, 0}}
        __PRETTY_FUNCTION__ = "mdb_rebalance"
#7  0x000000000051880d in mdb_page_merge (csrc=0x7ff7e01023f8,
    cdst=0x7ff7f1720a00) at ./../../../libraries/liblmdb/mdb.c:6220
        rc = 0
        i = 11
        j = 22
        srcnode = 0x7ff7e03cac8c
        key = {mv_size = 81, mv_data = 0x7ff7e03cac94}
        data = {mv_size = 0, mv_data = 0x7ff7e03cace5}
        nkeys = 11
        __PRETTY_FUNCTION__ = "mdb_page_merge"
#8  0x000000000051920f in mdb_rebalance (mc=0x7ff7e01023f8)
    at ./../../../libraries/liblmdb/mdb.c:6396
        node = 0x7ff7e03c9c3c
        rc = 0
        minkeys = 1
        ptop = 2
        mn = {mc_next = 0x7ffc2ff89ff8, mc_orig = 0x5114df, mc_xcursor = 0x0,
          mc_txn = 0x7ff7e03bdfd0, mc_dbi = 4, mc_db = 0x7ff7e0102580,
          mc_dbx = 0x7ff7e01025b0, mc_dbflag = 0x7ff7e03bfd40 "\210A\002",
          mc_snum = 4, mc_top = 3, mc_flags = 5, mc_pg = {0x7ff7e03c6fe0,
            0x7ff7e03c7ff0, 0x7ff7e03c9000, 0x7ff7e03cb020, 0x51117c,
            0x7ff7f1720c70, 0x511cf4, 0x7ff7f1720ae0, 0x200b5ca30, 0x0,
            0x7ff7e01025f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ff7e03bdfd0,
            0x7ff700000001, 0x7ff7e03be068, 0xb5c9d0, 0x7ff7e03bfca9, 0x1,
            0x7ff7e03bfd40, 0x7ff7f1720cb0, 0x0, 0x0, 0x7ff7e03bdfd0,
            0x7ff700000001, 0x7ff7e03be068, 0xb5c9d0, 0x7ff7e03bfca9, 0x1},
          mc_ki = {0, 1, 0, 10, 3072, 61810, 32759, 0, 44982, 57404, 32759, 0,
            45072, 57404, 32759, 0, 44084, 57404, 32759, 0, 988, 0, 0, 0,
            15065, 63088, 32767, 0, 3008, 61810, 32759, 0}}
        __PRETTY_FUNCTION__ = "mdb_rebalance"
#9  0x00000000005193cb in mdb_cursor_del0 (mc=0x7ff7e01023f8,
    leaf=0x7ff7e03cafb6) at ./../../../libraries/liblmdb/mdb.c:6424
        rc = 15
#10 0x0000000000515b10 in mdb_cursor_del (mc=0x7ff7e01023f8, flags=0)
    at ./../../../libraries/liblmdb/mdb.c:5305
        leaf = 0x7ff7e03cafb6
        rc = 0
#11 0x00000000005159d1 in mdb_cursor_del (mc=0x7ff7e0102270, flags=0)
    at ./../../../libraries/liblmdb/mdb.c:5278
        leaf = 0x7ff7e03c6f24
        rc = 0
#12 0x000000000056e5ee in mdb_dn2id_delete (op=0x7ff7f17213c0,
    mc=0x7ff7e0102270, id=1087351, nsubs=1) at dn2id.c:217
        nid = 73
        ptr = 0x7ffc37bf9f0a ""
        rc = 0
#13 0x000000000056d058 in mdb_delete (op=0x7ff7f17213c0, rs=0x7ff7f1721350)
    at delete.c:335
        mdb = 0x7ffff7f2a010
        pdn = {bv_len = 12, bv_val = 0x7ff7e010347f "cn=authstats"}
        e = 0x7ff7e0000be0
        p = 0x7ff7e00009b0
        manageDSAit = 0
        children = 0x935970
        entry = 0x9356d0
        txn = 0x7ff7e03bdfd0
        mc = 0x7ff7e0102270
        opinfo = {moi_oe = {oe_next = {sle_next = 0x0},
            oe_key = 0x7ffff7f2a010}, moi_txn = 0x7ff7e03bdfd0, moi_ref = 1,
          moi_flag = 0 '\000'}
        moi = 0x7ff7f1720e70
        preread_ctrl = 0x0
        ctrls = {0x0, 0x7ff7f172127e, 0x7ff7e0000908, 0x7ff7f1721281,
          0x7ff7f172127e, 0x88}
        num_ctrls = 0
        parent_is_glue = 0
        parent_is_leaf = 0
        __PRETTY_FUNCTION__ = "mdb_delete"
#14 0x00000000004d8b1d in overlay_op_walk (op=0x7ff7f17213c0,
    rs=0x7ff7f1721350, which=op_delete, oi=0x9b7e60, on=0x0) at backover.c:671
        func = 0x85d558
        rc = 32768
#15 0x00000000004d8d34 in over_op_func (op=0x7ff7f17213c0, rs=0x7ff7f1721350,
    which=op_delete) at backover.c:723
        oi = 0x9b7e60
        on = 0x9b8040
        be = 0x9b7190
        db = {bd_info = 0x85d500, bd_self = 0x9b7190,
          be_ctrls =
"\000\001\001\001\000\001\000\000\001\000\000\001\001\000\001\000\000\001",
'\000' <repeats 14 times>, "\001", be_flags = 2312,
          be_restrictops = 0, be_requires = 0, be_ssf_set = {sss_ssf = 0,
            sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0,
            sss_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0,
            sss_simple_bind = 128}, be_suffix = 0x9b79e0,
          be_nsuffix = 0x9b7a10, be_schemadn = {bv_len = 0, bv_val = 0x0},
          be_schemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 23,
            bv_val = 0x9b7c50 "cn=manager,dc=vt,dc=edu"}, be_rootndn = {
            bv_len = 23, bv_val = 0x9b7ca0 "cn=manager,dc=vt,dc=edu"},
          be_rootpw = {bv_len = 0, bv_val = 0x0}, be_max_deref_depth = 15,
          be_def_limit = {lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = 500,
            lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0,
            lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0x9b7cf0,
          be_acl = 0x9b84c0, be_dfltaccess = ACL_READ, be_extra_anlist = 0x0,
          be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0,
          be_pending_csn_list = 0xb5f2d0, be_pcl_mutex = {__data = {
              __lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
              __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
            __size = '\000' <repeats 39 times>, __align = 0},
          be_syncinfo = 0x0, be_pb = 0x0, be_cf_ocs = 0x864500,
          be_private = 0x7ffff7f2a010, be_next = {stqe_next = 0x9b8760}}
        cb = {sc_next = 0x7ffff35e5fe0,
          sc_response = 0x4d7afa <over_back_response>, sc_cleanup = 0,
          sc_private = 0x9b7e60}
        sc = 0x7ff7f1721354
        rc = 32768
        __PRETTY_FUNCTION__ = "over_op_func"
#16 0x00000000004d8eee in over_op_delete (op=0x7ff7f17213c0, rs=0x7ff7f1721350)
    at backover.c:780
No locals.
#17 0x00007ffff33dcb46 in accesslog_purge (ctx=0x7ff7f1721aa0, arg=0xa1e4e0)
    at accesslog.c:707
        i = 0
        rtask = 0xa1e4e0
        li = 0xa1f300
        conn = {c_struct_state = SLAP_C_UNINITIALIZED,
          c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0,
          c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __count = 0,
              __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {
                __prev = 0x0, __next = 0x0}},
            __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0,
          c_starttime = 0, c_activitytime = 0,
          c_connid = 18446744073709551615, c_peer_domain = {bv_len = 0,
            bv_val = 0x5d1370 ""}, c_peer_name = {bv_len = 0,
            bv_val = 0x5d1370 ""}, c_listener = 0x5d92a0, c_sasl_bind_mech = {
            bv_len = 0, bv_val = 0x0}, c_sasl_dn = {bv_len = 0, bv_val = 0x0},
          c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0,
          c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {
              bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 0, bv_val = 0x0},
            sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0,
            sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0},
          c_protocol = 0, c_ops = {stqh_first = 0x0, stqh_last = 0x0},
          c_pending_ops = {stqh_first = 0x0, stqh_last = 0x0},
          c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0,
              __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0,
                __next = 0x0}}, __size = '\000' <repeats 39 times>,
            __align = 0}, c_write1_cv = {__data = {__lock = 0, __futex = 0,
              __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0,
              __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0},
            __size = '\000' <repeats 47 times>, __align = 0},
          c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0,
              __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0,
                __next = 0x0}}, __size = '\000' <repeats 39 times>,
            __align = 0}, c_write2_cv = {__data = {__lock = 0, __futex = 0,
              __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0,
              __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0},
            __size = '\000' <repeats 47 times>, __align = 0},
          c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000',
          c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000',
          c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000',
          c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000',
          c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0,
          c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0,
            ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {
              bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0,
          c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0,
          c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0,
          c_clientfunc = 0, c_clientarg = 0x0,
          c_send_ldap_result = 0x45be00 <slap_send_ldap_result>,
          c_send_search_entry = 0x45cb08 <slap_send_search_entry>,
          c_send_search_reference = 0x45ebb8 <slap_send_search_reference>,
          c_send_ldap_extended = 0x45c667 <slap_send_ldap_extended>,
          c_send_ldap_intermediate = 0x45c8e5 <slap_send_ldap_intermediate>}
        opbuf = {ob_op = {o_hdr = 0x7ff7f1721530, o_tag = 74,
            o_time = 1369246130, o_tincr = 2, o_bd = 0x7ff7f1720fe0,
            o_req_dn = {bv_len = 44,
              bv_val = 0x7ff7e0103420
"reqStart=20130520202434.000180Z,cn=authstats"}, o_req_ndn = {bv_len =
43,
              bv_val = 0x7ff7e0103460
"reqStart=20130520202434.00018Z,cn=authstats"}, o_request = {oq_add =
{rs_modlist = 0x1, rs_e = 0xffffffffffffffff},
              oq_bind = {rb_method = 1, rb_cred = {
                  bv_len = 18446744073709551615, bv_val = 0x0}, rb_edn = {
                  bv_len = 1, bv_val = 0x861740 "\003"}, rb_ssf = 4050785040,
                rb_mech = {bv_len = 27,
                  bv_val = 0x7ff7e0000908 "\300\017r\361\367\177"}},
              oq_compare = {rs_ava = 0x1}, oq_modify = {rs_mods = {
                  rs_modlist = 0x1, rs_no_opattrs = -1 '\377'},
                rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x1,
                  rs_no_opattrs = -1 '\377'}, rs_deleteoldrdn = 0,
                rs_newrdn = {bv_len = 1, bv_val = 0x861740 "\003"},
                rs_nnewrdn = {bv_len = 140702884434704,
                  bv_val = 0x1b <Address 0x1b out of bounds>},
                rs_newSup = 0x7ff7e0000908, rs_nnewSup = 0x0}, oq_search = {
                rs_scope = 1, rs_deref = 0, rs_slimit = -1, rs_tlimit = -1,
                rs_limit = 0x0, rs_attrsonly = 1, rs_attrs = 0x861740,
                rs_filter = 0x7ff7f1721310, rs_filterstr = {bv_len = 27,
                  bv_val = 0x7ff7e0000908 "\300\017r\361\367\177"}},
              oq_abandon = {rs_msgid = 1}, oq_cancel = {rs_msgid = 1},
              oq_extended = {rs_reqoid = {bv_len = 1,
                  bv_val = 0xffffffffffffffff <Address
0xffffffffffffffff out of bounds>}, rs_flags = 0, rs_reqdata = 0x1},
oq_pwdexop = {rs_extended = {
                  rs_reqoid = {bv_len = 1,
                    bv_val = 0xffffffffffffffff <Address
0xffffffffffffffff out of bounds>}, rs_flags = 0, rs_reqdata = 0x1},
rs_old = {bv_len = 8787776,
                  bv_val = 0x7ff7f1721310 "\246"}, rs_new = {bv_len = 27,
                  bv_val = 0x7ff7e0000908 "\300\017r\361\367\177"},
                rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0,
            o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000',
            o_is_auth_check = 0 '\000', o_dont_replicate = 1 '\001',
            o_acl_priv = ACL_NONE, o_nocaching = 0 '\000',
            o_delete_glue_parent = 0 '\000', o_no_schema_check = 0 '\000',
            o_no_subordinate_glue = 0 '\000',
            o_ctrlflag = '\000' <repeats 31 times>,
            o_controls = 0x7ff7f1721678, o_authz = {sai_method = 0,
              sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 23,
                bv_val = 0x9b7c50 "cn=manager,dc=vt,dc=edu"}, sai_ndn = {
                bv_len = 23, bv_val = 0x9b7ca0 "cn=manager,dc=vt,dc=edu"},
              sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0,
              sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0,
            o_callback = 0x7ff7e0000908, o_ctrls = 0x0, o_csn = {bv_len = 40,
              bv_val = 0x7ff7f1721260
"20130521180706.125179Z#000000#000#000000"}, o_private = 0x0, o_extra
= {slh_first = 0x7ff7f1720e70}, o_next = {
              stqe_next = 0x0}}, ob_hdr = {oh_opid = 0,
            oh_connid = 18446744073709551615, oh_conn = 0x7ff7f1721780,
            oh_msgid = 0, oh_protocol = 0, oh_tid = 140702884439808,
            oh_threadctx = 0x7ff7f1721aa0, oh_tmpmemctx = 0x7ff7e00008c0,
            oh_tmpmfuncs = 0x861ce0, oh_counters = 0x8b7d40,
            oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>},
          ob_controls = {0x0 <repeats 32 times>}}
        op = 0x7ff7f17213c0
        rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = -30798,
          sr_matched = 0x0, sr_text = 0x0, sr_ref = 0x0, sr_ctrls = 0x0,
          sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0,
              r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0,
              r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = {
              r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0}
        cb = {sc_next = 0x0, sc_response = 0x7ffff33dc4f0 <log_old_lookup>,
          sc_cleanup = 0, sc_private = 0x7ff7f17212c0}
        f = {f_choice = 166, f_un = {f_un_result = -244182288,
            f_un_desc = 0x7ff7f17212f0, f_un_ava = 0x7ff7f17212f0,
            f_un_ssa = 0x7ff7f17212f0, f_un_mra = 0x7ff7f17212f0,
            f_un_complex = 0x7ff7f17212f0}, f_next = 0x0}
        ava = {aa_desc = 0x9ad600, aa_value = {bv_len = 15,
            bv_val = 0x7ff7f17212a0 "20130521180850Z"}}
        pd = {slots = 22700, used = 22689, dn = 0x7ff7e04cab30,
          ndn = 0x7ff7e0523600, csn = {bv_len = 40,
            bv_val = 0x7ff7f1721260 "20130521180706.125179Z#000000#000#000000"}}
        timebuf = "20130521180850Z\000\000\000\000\000\000"
        csnbuf = "20130521180706.125179Z#000000#000#000000", '\000'
<repeats 23 times>
        old = 1369159730
        __PRETTY_FUNCTION__ = "accesslog_purge"
#18 0x0000000000595639 in ldap_int_thread_pool_wrapper (xpool=0x93a270)
    at tpool.c:688
        pool = 0x93a270
        task = 0x7ff7ec000a20
        work_list = 0x93a308
        ctx = {ltu_id = 140702884439808, ltu_key = {{ltk_key = 0x4b8d8b,
              ltk_data = 0x7ff7e00008c0,
              ltk_free = 0x4b8bb0 <slap_sl_mem_destroy>}, {
              ltk_key = 0x7ffff3127010, ltk_data = 0x7ff7e0100910,
              ltk_free = 0x5746dc <mdb_reader_free>}, {ltk_key = 0x525df8,
              ltk_data = 0x7ff7ea7fe010,
              ltk_free = 0x525dd5 <search_stack_free>}, {ltk_key = 0x52329d,
              ltk_data = 0x7ff7f0420010,
              ltk_free = 0x523255 <scope_chunk_free>}, {ltk_key = 0x0,
              ltk_data = 0x0, ltk_free = 0} <repeats 25 times>, {
              ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x80}, {ltk_key = 0x0,
              ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0,
              ltk_free = 0}}}
        kctx = 0x0
        i = 32
        keyslot = 672
        hash = 284303008
        __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper"
#19 0x00007ffff764a851 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#20 0x00007ffff676890d in clone () from /lib64/libc.so.6
No symbol table info available.
#####

Comment 16 Quanah Gibson-Mount 2013-05-22 21:49:15 UTC
--On Wednesday, May 22, 2013 9:39 PM +0000 dhawes@vt.edu wrote:

> On Wed, Nov 21, 2012 at 1:16 PM,  <hyc@symas.com> wrote:
>> More information is needed than this. Your stack traces aren't even
>> complete. What was the workload leading up to this? Give us a sample
>> configuration that reproduces the assert.
>
> I came across what looks like this issue today. Let's see if I can
> provide more info.
>
> I noticed one of my test machines (2.4.35) had crashed overnight.
> Unfortunately, I had neglected to set ulimit -c, so I had no core dump
> to work with. Upon restarting, however, I got the following:
>
> slapd: ./../../../libraries/liblmdb/mdb.c:4025: mdb_page_search_root:
> Assertion `(((mp)->mp_pb.pb.pb_lower - ((unsigned) __builtin_offsetof
> (MDB_page, mp_ptrs))) >> 1) > 1' failed.
>
> A backtrace shows that this occurred during an accesslog_purge(). The
> backtrace is at the end of this email.

This is ITS#7536.  There were more fixes for it in RE24 after 2.4.35 was 
released.

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Comment 17 David Hawes 2013-05-23 22:28:03 UTC
On Wed, May 22, 2013 at 5:49 PM, Quanah Gibson-Mount <quanah@zimbra.com> wrote:
> This is ITS#7536.  There were more fixes for it in RE24 after 2.4.35 was
> released.

I just wanted to verify that RE24 does indeed fix this issue.