Issue 7487 - memberof & mirrormode with delta-syncrepl crashing when using dynamic configuration
Summary: memberof & mirrormode with delta-syncrepl crashing when using dynamic configu...
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: 2.4.33
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-08 15:16 UTC by afrunning@gmail.com
Modified: 2014-08-01 21:04 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description afrunning@gmail.com 2013-01-08 15:16:11 UTC
Full_Name: Al F
Version: 2.4.33
OS: Redhat 6.3 x64
URL: ftp://ftp.openldap.org/incoming/openldap-af.zip
Submission from: (NULL) (74.123.146.10)


I'm having an issue with mirrormode replication (via delta syncrepl) and the
memberof overlay when using dynamic configuration.  When a group member is
deleted on the first instance, slapd on the second instance is crashing with the
following error (as seen from running slapd with -d-1):

slapd: memberof.c:1465: memberof_res_modify: Assertion `0' failed.

If I disable either memberof or mirrormode, the change replicates correctly.  If
I use an identical static configuration file instead of a dynamic configuration,
the change replicates correctly.  This behavior happens on both bdb and mdb
backends. 

I have setup a very basic test and have included the configurations for server1
and server2 in both static and dynamic configuration (both running on the same
machine, just different ports), ldif file to populate tree (ldifs/basic.ldif),
as well as the ldif file to crash the second node (ldifs/crash.ldif).

the zip file contents (ftp://ftp.openldap.org/incoming/openldap-af.zip) are:
openldap-af/
openldap-af/bt full.txt  (output from gdb bt full)
openldap-af/config.txt (output from source compilation)
openldap-af/debug level logging.txt (output of the crash while running debug
logs)
openldap-af/ldifs/basic.ldif (the basic tree)
openldap-af/ldifs/crash.ldif (ldif file applied to cause the failure)
openldap-af/server1slapd.conf (server1's  static configuration)
openldap-af/server1slapd.d/* (server1's dynamic configuration - produced
directly from static configuration with slaptest)
openldap-af/server2slapd.conf (server2's  static configuration)
openldap-af/server2slapd.d/* (server2's dynamic configuration - produced
directly from static configuration with slaptest)
openldap-af/thread apply all bt.txt (output from gdb thread apply all bt)

version & config information:

OL_PACKAGE="OpenLDAP"
OL_MAJOR=2
OL_MINOR=4
OL_PATCH=33
OL_API_INC=20433
OL_API_LIB_RELEASE=2.4
OL_API_LIB_VERSION=10:5:8
OL_VERSION=2.4.33
OL_TYPE=Release
OL_STRING="OpenLDAP 2.4.33-Release"
OL_RELEASE_DATE="2012/10/10"

Red Hat Enterprise Linux Server release 6.3 (Santiago)

kernel 2.6.32-279.19.1.el6.x86_64

running on a vmware esx host, with a single AMD Opteron(tm) Processor 6134. 4Gig
RAM.

please let me know if you need any more information. This is my first time using
gdb, so I appologize if I missed something critical.
Comment 1 ando@openldap.org 2013-01-08 16:05:58 UTC
> Full_Name: Al F
> Version: 2.4.33
> OS: Redhat 6.3 x64
> URL: ftp://ftp.openldap.org/incoming/openldap-af.zip
> Submission from: (NULL) (74.123.146.10)
>
>
> I'm having an issue with mirrormode replication (via delta syncrepl) and
> the
> memberof overlay when using dynamic configuration.  When a group member is
> deleted on the first instance, slapd on the second instance is crashing
> with the
> following error (as seen from running slapd with -d-1):
>
> slapd: memberof.c:1465: memberof_res_modify: Assertion `0' failed.

If you have the core file handy, or if you can quickly reproduce the
problem, could you print the value of ml->sml_mod, i.e. from within gdb:

(to see the backtrace)
$ bt
(where <x> is the number of the frame of function memberof_res_modify)
$ f <x>
(to see the value)
$ print ml->sml_mod

Thanks, p.

-- 
Pierangelo Masarati
Associate Professor
Dipartimento di Ingegneria Aerospaziale
Politecnico di Milano

Comment 2 afrunning@gmail.com 2013-01-08 16:34:55 UTC
> > slapd: memberof.c:1465: memberof_res_modify: Assertion `0' failed.
>
> If you have the core file handy, or if you can quickly reproduce the
> problem, could you print the value of ml->sml_mod, i.e. from within gdb:
>
> (to see the backtrace)
> $ bt
> (where <x> is the number of the frame of function memberof_res_modify)
> $ f <x>
> (to see the value)
> $ print ml->sml_mod

(gdb) bt
#0  0x00000032b04328a5 in raise () from /lib64/libc.so.6
#1  0x00000032b0434085 in abort () from /lib64/libc.so.6
#2  0x00000032b042ba1e in __assert_fail_base () from /lib64/libc.so.6
#3  0x00000032b042bae0 in __assert_fail () from /lib64/libc.so.6
#4  0x00000000004fc989 in memberof_res_modify (op=0x7fffebffe330,
rs=<value optimized out>) at memberof.c:1465
#5  0x000000000042f91e in slap_response_play (op=0x7fffebffe330,
rs=0x7fffebffd620) at result.c:507
#6  0x00000000004304a9 in send_ldap_response (op=0x7fffebffe330,
rs=0x7fffebffd620) at result.c:582
#7  0x000000000043119c in slap_send_ldap_result (op=0x7fffebffe330,
rs=0x7fffebffd620) at result.c:860
#8  0x00000000004b4035 in mdb_modify (op=0x7fffebffe330,
rs=0x7fffebffd620) at modify.c:656
#9  0x00000000004845a7 in overlay_op_walk (op=0x7fffebffe330,
rs=0x7fffebffd620, which=op_modify, oi=0x963b20, on=0x0) at
backover.c:671
#10 0x0000000000484f87 in over_op_func (op=0x7fffebffe330, rs=<value
optimized out>, which=<value optimized out>) at backover.c:723
#11 0x000000000047738d in syncrepl_message_to_op (si=0x9636f0,
op=0x7fffebffe330, msg=<value optimized out>) at syncrepl.c:2318
#12 0x000000000047b1bf in do_syncrep2 (op=0x7fffebffe330, si=0x9636f0)
at syncrepl.c:986
#13 0x00000000004809c2 in do_syncrepl (ctx=<value optimized out>,
arg=0x963ee0) at syncrepl.c:1523
#14 0x00000000004210e1 in connection_read_thread (ctx=0x7fffebffeab0,
argv=<value optimized out>) at connection.c:1288
#15 0x00000000005230a0 in ldap_int_thread_pool_wrapper
(xpool=0x8b7640) at tpool.c:688
#16 0x00000032b0807851 in start_thread () from /lib64/libpthread.so.0
#17 0x00000032b04e811d in clone () from /lib64/libc.so.6
(gdb)  f 4
#4  0x00000000004fc989 in memberof_res_modify (op=0x7fffebffe330,
rs=<value optimized out>) at memberof.c:1465
1465 assert( 0 );
(gdb)  print ml->sml_mod
$1 = {sm_desc = 0x9640f0, sm_values = 0x7fffe00009b0, sm_nvalues =
0x7fffe00009d0, sm_numvals = 1, sm_op = 4097, sm_flags = 0, sm_type =
{bv_len = 48,
    bv_val = 0x70756f72673d6e63 <Address 0x70756f72673d6e63 out of bounds>}}
(gdb)

Regards,

Al

Comment 3 ando@openldap.org 2013-01-08 16:47:02 UTC
>> > slapd: memberof.c:1465: memberof_res_modify: Assertion `0' failed.
>>
>> If you have the core file handy, or if you can quickly reproduce the
>> problem, could you print the value of ml->sml_mod, i.e. from within gdb:
>>
>> (to see the backtrace)
>> $ bt
>> (where <x> is the number of the frame of function memberof_res_modify)
>> $ f <x>
>> (to see the value)
>> $ print ml->sml_mod
>
> (gdb) bt
> #0  0x00000032b04328a5 in raise () from /lib64/libc.so.6
> #1  0x00000032b0434085 in abort () from /lib64/libc.so.6
> #2  0x00000032b042ba1e in __assert_fail_base () from /lib64/libc.so.6
> #3  0x00000032b042bae0 in __assert_fail () from /lib64/libc.so.6
> #4  0x00000000004fc989 in memberof_res_modify (op=0x7fffebffe330,
> rs=<value optimized out>) at memberof.c:1465
> #5  0x000000000042f91e in slap_response_play (op=0x7fffebffe330,
> rs=0x7fffebffd620) at result.c:507
> #6  0x00000000004304a9 in send_ldap_response (op=0x7fffebffe330,
> rs=0x7fffebffd620) at result.c:582
> #7  0x000000000043119c in slap_send_ldap_result (op=0x7fffebffe330,
> rs=0x7fffebffd620) at result.c:860
> #8  0x00000000004b4035 in mdb_modify (op=0x7fffebffe330,
> rs=0x7fffebffd620) at modify.c:656
> #9  0x00000000004845a7 in overlay_op_walk (op=0x7fffebffe330,
> rs=0x7fffebffd620, which=op_modify, oi=0x963b20, on=0x0) at
> backover.c:671
> #10 0x0000000000484f87 in over_op_func (op=0x7fffebffe330, rs=<value
> optimized out>, which=<value optimized out>) at backover.c:723
> #11 0x000000000047738d in syncrepl_message_to_op (si=0x9636f0,
> op=0x7fffebffe330, msg=<value optimized out>) at syncrepl.c:2318
> #12 0x000000000047b1bf in do_syncrep2 (op=0x7fffebffe330, si=0x9636f0)
> at syncrepl.c:986
> #13 0x00000000004809c2 in do_syncrepl (ctx=<value optimized out>,
> arg=0x963ee0) at syncrepl.c:1523
> #14 0x00000000004210e1 in connection_read_thread (ctx=0x7fffebffeab0,
> argv=<value optimized out>) at connection.c:1288
> #15 0x00000000005230a0 in ldap_int_thread_pool_wrapper
> (xpool=0x8b7640) at tpool.c:688
> #16 0x00000032b0807851 in start_thread () from /lib64/libpthread.so.0
> #17 0x00000032b04e811d in clone () from /lib64/libc.so.6
> (gdb)  f 4
> #4  0x00000000004fc989 in memberof_res_modify (op=0x7fffebffe330,
> rs=<value optimized out>) at memberof.c:1465
> 1465 assert( 0 );
> (gdb)  print ml->sml_mod
> $1 = {sm_desc = 0x9640f0, sm_values = 0x7fffe00009b0, sm_nvalues =
> 0x7fffe00009d0, sm_numvals = 1, sm_op = 4097, sm_flags = 0, sm_type =
> {bv_len = 48,
>     bv_val = 0x70756f72673d6e63 <Address 0x70756f72673d6e63 out of
> bounds>}}
> (gdb)

Indeed, sm_op == SLAP_MOD_SOFTDEL, as I inferred from syncrepl code,
thanks for the feedback.  Internal special modification types should be
handled now in HEAD code.  Could you pull it from git and test, please? 
Instructions here <http://www.openldap.org/software/repo.html>.

Thanks, p.

-- 
Pierangelo Masarati
Associate Professor
Dipartimento di Ingegneria Aerospaziale
Politecnico di Milano

Comment 4 ando@openldap.org 2013-01-08 16:54:19 UTC
changed notes
changed state Open to Test
moved from Incoming to Software Bugs
Comment 5 afrunning@gmail.com 2013-01-08 17:24:04 UTC
> Indeed, sm_op == SLAP_MOD_SOFTDEL, as I inferred from syncrepl code,
> thanks for the feedback.  Internal special modification types should be
> handled now in HEAD code.  Could you pull it from git and test, please?
> Instructions here <http://www.openldap.org/software/repo.html>.

Looks good!  I can add and remove users to groups in my setup without
issue.  Thanks for your quick response.

Al

Comment 6 Quanah Gibson-Mount 2013-01-09 20:36:17 UTC
changed notes
changed state Test to Release
Comment 7 Quanah Gibson-Mount 2013-03-05 02:25:39 UTC
changed notes
changed state Release to Closed
Comment 8 OpenLDAP project 2014-08-01 21:04:45 UTC
slapo-memberof does not handle internal operations
fixed in HEAD
fixed in RE24