[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#6453) OpenLDAP memory leak on LDAP_TIMEOUT
--0-1691785669-1266069835=:3216
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Hi Serghei,
=A0
=A0=A0 I am using OpenLdap 2.4.19. By examining the change log i don't see =
any behaviour=A0 change for ldap_search functions against 2.4.16.
=A0
=A0 My application is already multithreaded. Different search requests come=
in the same time=A0and multiple threads are usedfor this. There is an ldap=
connection pool in place, a thread will pick up a free connection, will us=
e it for search and return it back to the pool.
=A0
=A0 I used the sync api because I looked over the OpenLdap source code and =
the sync api is based on the async one.=A0While waiting for a meesage, the =
async api will examine the abandoned results and destroy them. However it l=
ooks like this behaviour doesn't work very well.=A0
=A0
=A0 You mentioned "But you can filter itby using ldap_msgtype() and killing=
the results for searches only." Can you provide a piece of code for that?
=A0
Thanks,
Alin
=A0
--- On Sat, 2/13/10, linamat@telco.md <linamat@telco.md> wrote:
From: linamat@telco.md <linamat@telco.md>
Subject: Re: (ITS#6453) OpenLDAP memory leak on LDAP_TIMEOUT
To: alinachegalati@yahoo.com
Date: Saturday, February 13, 2010, 4:44 AM
>
> Sorry, it is my fault here. I missed the trailling "s"...
>
> I am using the OpenLDAP api only for searching for ldap entries on a
different LDAP serve (I am not aware of its type).=A0 So the only
libraries linked to my app are libldap_r.so and liblber.so.
>
And what version of library?
I am using 2.4.16.
> I don' have access to the source code now but I'll post a section of it
soon.
>
You are welcome;)
> The idea of using ldap_result with LDAP_RES_ANY is interesting. Do you
>know what is the difference between LDAP_RES_ANY and
LDAP_RES_UNSOLICITED?
The answer is on
http://www.openldap.org/lists/openldap-technical/200912/msg00007.html
As I know LDAP_RES_ANY includes search results too. But you can filter it
by using ldap_msgtype() and killing the results for searches only.
>I'm thinking to create a separate daemon thread that takes
> ldap_connections from the pool and makes this call. My main concern is
not >to clash with the existing results.
>
Good aproach. But if you should switch to asyncronous version of calls,
you could beep the horn during periods of waiting for results withing one
thread.
> I'll try this solution and let you know how it worked.
>
> Regards,
> Alin
>
>
> --- On Sat, 2/13/10, linamat@telco.md <linamat@telco.md> wrote:
>
>
> From: linamat@telco.md <linamat@telco.md>
> Subject: Re: (ITS#6453) OpenLDAP memory leak on LDAP_TIMEOUT
> To: alinachegalati@yahoo.com
> Date: Saturday, February 13, 2010, 12:52 AM
>
>
>>
>>=A0 =A0 =A0 For the first point, the function ldap_msgfree=A0 will check =
for
> NULL, > so if the check is missing, no error occurs.
>>
>
> I believe the pointer should be checked for NULL just to be safe in
cases when implementation suddenly changes.
>
>>=A0 =A0=A0=A0I am using the ldap_search_ext_s function (the synchronous o=
ne) so
> the >second and third point doesn't apply as i don't have access to the
msgid. >If the memory leak is high maybe will switch to the
> async queries.
>> Regards,
>> Alin
>
> You wrote "We have a memory leak LDAP_TIMEOUT is encountered on
> ldap_search_ext_invocation". It not clear if sync or async primitive is
used. If ldap_search_ext_s is used then general timeout set per
> LDAPConnection is aplied. May be ldap_search_ext_st is better for use.
>
> As to point 3 it can be applied in your case for checking and destroying
pending or timed out operations.
>
> I am currently using openldap in different applications on Solaris
x86/x64/sparc=A0=A0=A0both 32-bit and 64 without any troubles. Also we need=
to
mention that you can try libldap for version 2.4.16 as it seems to be
rock stable and free of this issues. Also if you are using dbd backend,
dbd must be 4.7.25 + 4 patches recommended by Oracle.
>
> You could post the code block for verifying.
>
> With respect, Serghei.
>
>>
>>
>>
>>
>> --- On Fri, 2/12/10, linamat@telco.md <linamat@telco.md> wrote:
>>
>>
>> From: linamat@telco.md <linamat@telco.md>
>> Subject: Re: (ITS#6453) OpenLDAP memory leak on LDAP_TIMEOUT
>> To: openldap-its@OpenLDAP.org
>> Cc: alinachegalati@yahoo.com
>> Date: Friday, February 12, 2010, 1:34 PM
>>
>>
>> Buna ziua, domnule.
>>
>> 1. The pointer must be checked before operation, e.g.
>> if (res !=3D NULL) {
>>=A0 =A0=A0=A0ldap_msgfree (res);
>>=A0 =A0=A0=A0res =3D NULL;
>> }
>>
>> 2. Also while processing the ext query, you should call the functions in
> the following manner:
>> 2.1 ldap_search_ext() which has pointer to msgid
>> 2.2 ldap_result() which gives the LDAPMessage structure for the msgid
> 2.3 if ok - ldap_parse_result()
>> 2.4 on any stage if something is not good and you have the msgid filled
> with value (integer, typicaly more then 1) you should call
>> ldap_abandon_ext() for that msgid
>> 2.5 Finish with ldap_msgfree()
>>
>> 3. Periodically between the asynchronous searches (after 100 for
> example)you should call ldap_result() with short timer and LDAP_RES_ANY
argue for clearing stalled or nonfreed calls. The obtained LDAPMessage
can be freed with ldap_msgfree(). Of course, you can count the asynch
calls which hadn't been completed or were timed out and you can make the
corresponding cycle for checking all iterations.
>>
>> Please try and let me know.
>>
>> With respect, Serghei.
>>
>>
>>
>>
>>
>
>
>
>
>
>
=0A=0A=0A
--0-1691785669-1266069835=:3216
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
<table cellspacing=3D"0" cellpadding=3D"0" border=3D"0" ><tr><td valign=3D"=
top" style=3D"font: inherit;"><DIV><BR>Hi Serghei,</DIV>
<DIV> </DIV>
<DIV> I am using OpenLdap 2.4.19. By examining the change log i=
don't see any behaviour change for ldap_search functions against 2.4=
.16.</DIV>
<DIV> </DIV>
<DIV> My application is already multithreaded. Different search reque=
sts come in the same time and multiple threads are usedfor this. There=
is an ldap connection pool in place, a thread will pick up a free connecti=
on, will use it for search and return it back to the pool.</DIV>
<DIV> </DIV>
<DIV> I used the sync api because I looked over the OpenLdap source c=
ode and the sync api is based on the async one. While waiting for a me=
esage, the async api will examine the abandoned results and destroy them. H=
owever it looks like this behaviour doesn't work very well. </DIV>
<DIV> </DIV>
<DIV> You mentioned "But you can filter itby using ldap_msgtype() and=
killing the results for searches only." Can you provide a piece of code fo=
r that?</DIV>
<DIV> </DIV>
<DIV>Thanks,</DIV>
<DIV>Alin<BR></DIV>
<DIV> </DIV>
<DIV>--- On <B>Sat, 2/13/10, linamat@telco.md <I><linamat@telco.md></=
I></B> wrote:<BR></DIV>
<BLOCKQUOTE style=3D"BORDER-LEFT: rgb(16,16,255) 2px solid; PADDING-LEFT: 5=
px; MARGIN-LEFT: 5px"><BR>From: linamat@telco.md <linamat@telco.md><B=
R>Subject: Re: (ITS#6453) OpenLDAP memory leak on LDAP_TIMEOUT<BR>To: alina=
chegalati@yahoo.com<BR>Date: Saturday, February 13, 2010, 4:44 AM<BR><BR>
<DIV class=3DplainMail>><BR>> Sorry, it is my fault here. I missed th=
e trailling "s"...<BR>><BR>> I am using the OpenLDAP api only for sea=
rching for ldap entries on a<BR>different LDAP serve (I am not aware of its=
type). So the only<BR>libraries linked to my app are libldap_r.so an=
d liblber.so.<BR>><BR><BR>And what version of library?<BR>I am using 2.4=
.16.<BR><BR>> I don' have access to the source code now but I'll post a =
section of it<BR>soon.<BR>><BR><BR>You are welcome;)<BR><BR>> The ide=
a of using ldap_result with LDAP_RES_ANY is interesting. Do you<BR>>know=
what is the difference between LDAP_RES_ANY and<BR>LDAP_RES_UNSOLICITED?<B=
R><BR>The answer is on<BR><A href=3D"http://www.openldap.org/lists/openldap=
-technical/200912/msg00007.html" target=3D_blank>http://www.openldap.org/li=
sts/openldap-technical/200912/msg00007.html</A><BR><BR>As I know LDAP_RES_A=
NY includes search results too. But you can filter it<BR>by using
ldap_msgtype() and killing the results for searches only.<BR><BR>>I'm t=
hinking to create a separate daemon thread that takes<BR>> ldap_connecti=
ons from the pool and makes this call. My main concern is<BR>not >to cla=
sh with the existing results.<BR>><BR><BR>Good aproach. But if you shoul=
d switch to asyncronous version of calls,<BR>you could beep the horn during=
periods of waiting for results withing one<BR>thread.<BR><BR>> I'll try=
this solution and let you know how it worked.<BR>><BR>> Regards,<BR>=
> Alin<BR>><BR>><BR>> --- On Sat, 2/13/10, <A href=3D"http://us=
.mc507.mail.yahoo.com/mc/compose?to=3Dlinamat@telco.md" ymailto=3D"mailto:l=
inamat@telco.md">linamat@telco.md</A> <<A href=3D"http://us.mc507.mail.y=
ahoo.com/mc/compose?to=3Dlinamat@telco.md" ymailto=3D"mailto:linamat@telco.=
md">linamat@telco.md</A>> wrote:<BR>><BR>><BR>> From: <A href=
=3D"http://us.mc507.mail.yahoo.com/mc/compose?to=3Dlinamat@telco.md";
ymailto=3D"mailto:linamat@telco.md";>linamat@telco.md</A> <<A href=3D"ht=
tp://us.mc507.mail.yahoo.com/mc/compose?to=3Dlinamat@telco.md" ymailto=3D"m=
ailto:linamat@telco.md">linamat@telco.md</A>><BR>> Subject: Re: (ITS#=
6453) OpenLDAP memory leak on LDAP_TIMEOUT<BR>> To: <A href=3D"http://us=
.mc507.mail.yahoo.com/mc/compose?to=3Dalinachegalati@yahoo.com" ymailto=3D"=
mailto:alinachegalati@yahoo.com";>alinachegalati@yahoo.com</A><BR>> Date:=
Saturday, February 13, 2010, 12:52 AM<BR>><BR>><BR>>><BR>>&=
gt; For the first point, the function ldap_msgfree =
; will check for<BR>> NULL, > so if the check is missing, no error oc=
curs.<BR>>><BR>><BR>> I believe the pointer should be checked f=
or NULL just to be safe in<BR>cases when implementation suddenly changes.<B=
R>><BR>>> I am using the ldap_search_ext_s=
function (the synchronous one) so<BR>> the >second and third point d=
oesn't
apply as i don't have access to the<BR>msgid. >If the memory leak is hi=
gh maybe will switch to the<BR>> async queries.<BR>>> Regards,<BR>=
>> Alin<BR>><BR>> You wrote "We have a memory leak LDAP_TIMEOUT=
is encountered on<BR>> ldap_search_ext_invocation". It not clear if syn=
c or async primitive is<BR>used. If ldap_search_ext_s is used then general =
timeout set per<BR>> LDAPConnection is aplied. May be ldap_search_ext_st=
is better for use.<BR>><BR>> As to point 3 it can be applied in your=
case for checking and destroying<BR>pending or timed out operations.<BR>&g=
t;<BR>> I am currently using openldap in different applications on Solar=
is<BR>x86/x64/sparc both 32-bit and 64 without any trouble=
s. Also we need to<BR>mention that you can try libldap for version 2.4.16 a=
s it seems to be<BR>rock stable and free of this issues. Also if you are us=
ing dbd backend,<BR>dbd must be 4.7.25 + 4 patches recommended by
Oracle.<BR>><BR>> You could post the code block for verifying.<BR>&g=
t;<BR>> With respect, Serghei.<BR>><BR>>><BR>>><BR>>&g=
t;<BR>>><BR>>> --- On Fri, 2/12/10, <A href=3D"http://us.mc507.=
mail.yahoo.com/mc/compose?to=3Dlinamat@telco.md" ymailto=3D"mailto:linamat@=
telco.md">linamat@telco.md</A> <<A href=3D"http://us.mc507.mail.yahoo.co=
m/mc/compose?to=3Dlinamat@telco.md" ymailto=3D"mailto:linamat@telco.md";>lin=
amat@telco.md</A>> wrote:<BR>>><BR>>><BR>>> From: <A h=
ref=3D"http://us.mc507.mail.yahoo.com/mc/compose?to=3Dlinamat@telco.md"; yma=
ilto=3D"mailto:linamat@telco.md";>linamat@telco.md</A> <<A href=3D"http:/=
/us.mc507.mail.yahoo.com/mc/compose?to=3Dlinamat@telco.md" ymailto=3D"mailt=
o:linamat@telco.md">linamat@telco.md</A>><BR>>> Subject: Re: (ITS#=
6453) OpenLDAP memory leak on LDAP_TIMEOUT<BR>>> To: <A href=3D"http:=
//us.mc507.mail.yahoo.com/mc/compose?to=3Dopenldap-its@OpenLDAP.org"
ymailto=3D"mailto:openldap-its@OpenLDAP.org";>openldap-its@OpenLDAP.org</A>=
<BR>>> Cc: <A href=3D"http://us.mc507.mail.yahoo.com/mc/compose?to=3D=
alinachegalati@yahoo.com" ymailto=3D"mailto:alinachegalati@yahoo.com";>alina=
chegalati@yahoo.com</A><BR>>> Date: Friday, February 12, 2010, 1:34 P=
M<BR>>><BR>>><BR>>> Buna ziua, domnule.<BR>>><BR>&g=
t;> 1. The pointer must be checked before operation, e.g.<BR>>> if=
(res !=3D NULL) {<BR>>> ldap_msgfree (res);<=
BR>>> res =3D NULL;<BR>>> }<BR>>>=
<BR>>> 2. Also while processing the ext query, you should call the fu=
nctions in<BR>> the following manner:<BR>>> 2.1 ldap_search_ext() =
which has pointer to msgid<BR>>> 2.2 ldap_result() which gives the LD=
APMessage structure for the msgid<BR>> 2.3 if ok - ldap_parse_result()<B=
R>>> 2.4 on any stage if something is not good and you have the msgid
filled<BR>> with value (integer, typicaly more then 1) you should call<=
BR>>> ldap_abandon_ext() for that msgid<BR>>> 2.5 Finish with l=
dap_msgfree()<BR>>><BR>>> 3. Periodically between the asynchron=
ous searches (after 100 for<BR>> example)you should call ldap_result() w=
ith short timer and LDAP_RES_ANY<BR>argue for clearing stalled or nonfreed =
calls. The obtained LDAPMessage<BR>can be freed with ldap_msgfree(). Of cou=
rse, you can count the asynch<BR>calls which hadn't been completed or were =
timed out and you can make the<BR>corresponding cycle for checking all iter=
ations.<BR>>><BR>>> Please try and let me know.<BR>>><BR>=
>> With respect, Serghei.<BR>>><BR>>><BR>>><BR>>=
><BR>>><BR>><BR>><BR>><BR>><BR>><BR>><BR><BR><BR=
></DIV></BLOCKQUOTE></td></tr></table><br>=0A=0A
--0-1691785669-1266069835=:3216--