[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#7642) make test for AIX fails on test060-mt-hot
--001a11c37b0a23a7cd04e2943a0d
Content-Type: text/plain; charset=ISO-8859-1
I tried recompiling the Berkeley db (6.0.20) with -D_LARGE_THREADS, no
difference.
I modified the test to use THR=50 and INNER ...*4 so it looks like this:
=====
# Perform a basic multi-threaded search on a single connection
THR=100
THR=50
OUTER=5
INNER=`expr $TESTLOOPS \* 4`
echo "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) loops..."
echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \
-e "$MONITORDN" \
-m $THR -L $OUTER -l $INNER
$SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \
-e "$MONITORDN" -f "(objectclass=*)" \
-m $THR -L $OUTER -l $INNER >> $MTREADOUT 2>&1
RC=$?
if test $RC != 0 ; then
echo "slapd-mtread failed ($RC)!"
test $KILLSERVERS != no && kill -HUP $KILLPIDS
exit $RC
fi
=======
and the tests complete. Not sure how arbitrary the THR=100 is, but the 50k
transactions is maintained
(50 * 5 * 50 * 4) versus (100 * 5 * 50 * 2)
====
In short, using THR=50, and doubled INNER factor test060 succeeds.
Now - dies this break the test, or just show a different issue?
Michael
On Sun, Jul 28, 2013 at 3:57 PM, Michael Felt <mamfelt@gmail.com> wrote:
> After the new configure with -D_LARGE_THREADS still same status:
> root@x094:[/data/prj/openldap-2.4.35/tests]./run test060
> Cleaning up test run directory leftover from previous run.
> Running ./scripts/test060-mt-hot for bdb...
>
> running defines.sh
> Running slapadd to build slapd database...
> Running slapindex to index slapd database...
> Starting slapd on TCP/IP port 9011...
> /data/prj/openldap-2.4.35/tests/../servers/slapd/slapd -s0 -f
> /data/prj/openldap-2.4.35/tests/testrun/slapd.1.conf -h
> ldap://localhost:9011/ -d stats
> Testing basic monitor search...
> Monitor searches
> Testing basic mt-hot search: 1 threads (1 x 50000) loops...
> ./progs/slapd-mtread -H ldap://localhost:9011/ -D
> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000
> Testing basic mt-hot search: 5 threads (1 x 10000) loops...
> ./progs/slapd-mtread -H ldap://localhost:9011/ -D
> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000
> Testing basic mt-hot search: 100 threads (5 x 100) loops...
> ./progs/slapd-mtread -H ldap://localhost:9011/ -D
> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 100 -L 5 -l 100
> slapd-mtread failed (1)!
> root@x094:[/data/prj/openldap-2.4.35/tests]echo $?
> 1
>
>
> On Sun, Jul 28, 2013 at 2:23 PM, Michael Felt <mamfelt@gmail.com> wrote:
>
>> Note: tried nofiles=5000 and nofiles=10000 - no change.
>>
>> I am beginning to think it could be related to this:
>>
>> *Limits and Default Values*
>> The threads library has some implementation-dependent limits and default
>> values. These limits and default values can be retrieved by symbolic
>> constants to enhance the portability of programs:
>>
>> - The maximum number of threads per process is 512. The maximum
>> number of threads can be retrieved at compilation time using the *
>> PTHREAD_THREADS_MAX* symbolic constant defined in the *pthread.h*header file. If an application is compiled with the
>> *-D_LARGE_THREADS* flag, the maximum number of threads per process is
>> 32767.
>>
>> ===
>>
>> Made this small change to the test:
>>
>> # Perform a basic multi-threaded search on a single connection
>> echo Perform a basic multi-threaded search on a single connection
>> THR=100
>> OUTER=5
>> INNER=`expr $TESTLOOPS \* 2`
>> for THR in 60 62 64 65 66 67 68 69 70 80 90 100
>> do
>> echo "Testing basic mt-hot search: $THR threads ($OUTER x $INNER)
>> loops..."
>> echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \
>> -e "$MONITORDN" \
>> -m $THR -L $OUTER -l $INNER
>> $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \
>> -e "$MONITORDN" -f "(objectclass=*)" \
>> -m $THR -L $OUTER -l $INNER >> $MTREADOUT 2>&1
>> RC=$?
>> if test $RC != 0 ; then
>> echo "slapd-mtread failed ($RC)!"
>> test $KILLSERVERS != no && kill -HUP $KILLPIDS
>> exit $RC
>> fi
>> done
>>
>> Got this as output:
>>
>> Testing basic monitor search...
>> Monitor searches
>> Testing basic mt-hot search: 1 threads (1 x 50000) loops...
>> ./progs/slapd-mtread -H ldap://localhost:9011/ -D
>> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000
>> Testing basic mt-hot search: 5 threads (1 x 10000) loops...
>> ./progs/slapd-mtread -H ldap://localhost:9011/ -D
>> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000
>> Perform a basic multi-threaded search on a single connection
>> Testing basic mt-hot search: 60 threads (5 x 100) loops...
>> ./progs/slapd-mtread -H ldap://localhost:9011/ -D
>> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 60 -L 5 -l 100
>> Testing basic mt-hot search: 62 threads (5 x 100) loops...
>> ./progs/slapd-mtread -H ldap://localhost:9011/ -D
>> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 62 -L 5 -l 100
>> Testing basic mt-hot search: 64 threads (5 x 100) loops...
>> ./progs/slapd-mtread -H ldap://localhost:9011/ -D
>> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 64 -L 5 -l 100
>> slapd-mtread failed (1)!
>> root@x054:[/data/prj/openldap-2.4.35/tests]
>>
>> ====
>>
>> slapd-mtread PID=6750246: MT Test Start: conns: 1 (ldap://localhost:9011/)
>> slapd-mtread PID=6750246: Threads: RO: 64 RW: 0
>> slapd-mtread PID=6750246: RO thread 0 pass=500 fail=0
>> slapd-mtread PID=6750246: RO thread 1 pass=500 fail=0
>> slapd-mtread PID=6750246: RO thread 2 pass=500 fail=0
>>
>> ...
>>
>> slapd-mtread PID=6750246: RO thread 59 pass=500 fail=0
>> slapd-mtread PID=6750246: RO thread 60 pass=500 fail=0
>> slapd-mtread PID=6750246: RO thread 61 pass=500 fail=0
>> slapd-mtread PID=6750246: RO thread 62 pass=500 fail=0
>> slapd-mtread PID=6750246: RO thread 63 pass=0 fail=0
>> slapd-mtread PID=6750246: FAIL RO thread 63
>> slapd-mtread PID=6750246: MT Test complete
>>
>> ===
>>
>> Back to original test, mtread.out tail is:
>>
>> slapd-mtread PID=5242956: RO thread 60 pass=500 fail=0
>> slapd-mtread PID=5242956: RO thread 61 pass=500 fail=0
>> slapd-mtread PID=5242956: RO thread 62 pass=500 fail=0
>> slapd-mtread PID=5242956: RO thread 63 pass=0 fail=0
>> slapd-mtread PID=5242956: FAIL RO thread 63
>> slapd-mtread PID=5242956: RO thread 64 pass=0 fail=0
>> slapd-mtread PID=5242956: FAIL RO thread 64
>> slapd-mtread PID=5242956: RO thread 65 pass=0 fail=0
>> ....
>>
>> slapd-mtread PID=5242956: FAIL RO thread 97
>> slapd-mtread PID=5242956: RO thread 98 pass=0 fail=0
>> slapd-mtread PID=5242956: FAIL RO thread 98
>> slapd-mtread PID=5242956: RO thread 99 pass=0 fail=0
>> slapd-mtread PID=5242956: FAIL RO thread 99
>> slapd-mtread PID=5242956: MT Test complete
>>
>> ===
>>
>> Shall test after new build with CFLAGS=-D_LARGE_THREADS
>>
>>
>> On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <hyc@symas.com> wrote:
>>
>>> Michael Felt wrote:
>>>
>>>> The log file did not show any errors.
>>>>
>>>
>>> If there's no server side errors then I suppose it's a problem in
>>> mtread. Check the testrun/mtread.out file.
>>>
>>>
>>>
>>> --
>>> -- Howard Chu
>>> CTO, Symas Corp. http://www.symas.com
>>> Director, Highland Sun http://highlandsun.com/hyc/
>>> Chief Architect, OpenLDAP http://www.openldap.org/**project/<http://www.openldap.org/project/>
>>>
>>
>>
>
--001a11c37b0a23a7cd04e2943a0d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
I tried recompiling the Berkeley db (6.0.20) with -D_LARGE_THREADS, no diff=
erence.<br><br>I modified the test to use THR=3D50 and INNER ...*4 so it lo=
oks like this:<br>=3D=3D=3D=3D=3D<br># Perform a basic multi-threaded searc=
h on a single connection<br>
THR=3D100<br>THR=3D50<br>OUTER=3D5<br>INNER=3D`expr $TESTLOOPS \* 4`<br>ech=
o "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) loops..=
."<br>echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD =
\<br>
=A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" \<br>=A0=A0=A0=A0=A0=A0=A0 =
-m $THR -L $OUTER -l $INNER<br>$SLAPDMTREAD -H $URI1 -D "$MANAGERDN&qu=
ot; -w $PASSWD \<br>=A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" -f &quo=
t;(objectclass=3D*)" \<br>=A0=A0=A0=A0=A0=A0=A0 -m $THR -L $OUTER -l $=
INNER >> $MTREADOUT 2>&1<br>
RC=3D$?<br>if test $RC !=3D 0 ; then<br>=A0=A0=A0=A0=A0=A0=A0 echo "sl=
apd-mtread failed ($RC)!"<br>=A0=A0=A0=A0=A0=A0=A0 test $KILLSERVERS !=
=3D no && kill -HUP $KILLPIDS<br>=A0=A0=A0=A0=A0=A0=A0 exit $RC<br>=
fi<br>=3D=3D=3D=3D=3D=3D=3D<br>and the tests complete. Not sure how arbitra=
ry the THR=3D100 is, but the 50k transactions is maintained<br>
(50 * 5 * 50 * 4) versus (100 * 5 * 50 * 2)<br>=3D=3D=3D=3D<br>In short, us=
ing THR=3D50, and doubled INNER factor test060 succeeds.<br><br>Now - dies =
this break the test, or just show a different issue?<br><br>Michael<br><div=
class=3D"gmail_quote">
On Sun, Jul 28, 2013 at 3:57 PM, Michael Felt <span dir=3D"ltr"><<a href=
=3D"mailto:mamfelt@gmail.com"; target=3D"_blank">mamfelt@gmail.com</a>></=
span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e=
x;border-left:1px #ccc solid;padding-left:1ex">
After the new configure with -D_LARGE_THREADS still same status:<br><span s=
tyle=3D"font-family:courier new,monospace">root@x094:[/data/prj/openldap-2.=
4.35/tests]./run test060<br>Cleaning up test run directory leftover from pr=
evious run.<br>
Running ./scripts/test060-mt-hot for bdb...<div class=3D"im"><br>running de=
fines.sh<br>Running slapadd to build slapd database...<br>Running slapindex=
to index slapd database...<br>Starting slapd on TCP/IP port 9011...<br>
/data/prj/openldap-2.4.35/tests/../servers/slapd/slapd -s0 -f /data/prj/ope=
nldap-2.4.35/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d stats<=
br></div><div class=3D"im">
Testing basic monitor search...<br>Monitor searches<br>Testing basic mt-hot=
search: 1 threads (1 x 50000) loops...<br>./progs/slapd-mtread -H ldap://l=
ocalhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMoni=
tor -m 1 -L 1 -l 50000<br>
Testing basic mt-hot search: 5 threads (1 x 10000) loops...<br>./progs/slap=
d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w=
secret -e cn=3DMonitor -m 5 -L 1 -l 10000<br></div>Testing basic mt-hot se=
arch: 100 threads (5 x 100) loops...<br>
./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample=
,dc=3Dcom -w secret -e cn=3DMonitor -m 100 -L 5 -l 100<br>slapd-mtread fail=
ed (1)!<br>root@x094:[/data/prj/openldap-2.4.35/tests]echo $?<br>1<br></spa=
n><div class=3D"HOEnZb">
<div class=3D"h5"><br>
<br><div class=3D"gmail_quote">On Sun, Jul 28, 2013 at 2:23 PM, Michael Fel=
t <span dir=3D"ltr"><<a href=3D"mailto:mamfelt@gmail.com"; target=3D"_bla=
nk">mamfelt@gmail.com</a>></span> wrote:<br><blockquote class=3D"gmail_q=
uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e=
x">
Note: tried nofiles=3D5000 and nofiles=3D10000 - no change.<br><br>I am beg=
inning to think it could be related to this:<br><p><b>Limits and Default Va=
lues</b></p>
The threads library has some implementation-dependent limits and
default values. These limits and default values can be retrieved by
symbolic constants to enhance the portability of programs: <ul><li>The maxi=
mum number of threads per process is 512. The maximum
number of threads can be retrieved at compilation time using the <b>PTHREAD=
_THREADS_MAX</b> symbolic
constant defined in the <b>pthread.h</b> header file. If an application
is compiled with the <b>-D_LARGE_THREADS</b> flag, the maximum number
of threads per process is 32767.</li></ul><p>=3D=3D=3D</p><p>Made this smal=
l change to the test:</p><p><font face=3D"courier new,monospace"># Perform =
a basic multi-threaded search on a single connection<br>echo Perform a basi=
c multi-threaded search on a single connection<br>
THR=3D100<br>OUTER=3D5<br>INNER=3D`expr $TESTLOOPS \* 2`<br>for THR in 60 6=
2 64 65 66 67 68 69 70 80 90 100<br>do<br>echo "Testing basic mt-hot s=
earch: $THR threads ($OUTER x $INNER) loops..."<br>echo $SLAPDMTREAD -=
H $URI1 -D "$MANAGERDN" -w $PASSWD \<br>
=A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" \<br>=A0=A0=A0=A0=A0=A0=A0 =
-m $THR -L $OUTER -l $INNER<br>$SLAPDMTREAD -H $URI1 -D "$MANAGERDN&qu=
ot; -w $PASSWD \<br>=A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" -f &quo=
t;(objectclass=3D*)" \<br>=A0=A0=A0=A0=A0=A0=A0 -m $THR -L $OUTER -l $=
INNER >> $MTREADOUT 2>&1<br>
RC=3D$?<br>if test $RC !=3D 0 ; then<br>=A0=A0=A0=A0=A0=A0=A0 echo "sl=
apd-mtread failed ($RC)!"<br>=A0=A0=A0=A0=A0=A0=A0 test $KILLSERVERS !=
=3D no && kill -HUP $KILLPIDS<br>=A0=A0=A0=A0=A0=A0=A0 exit $RC<br>=
fi<br>done<br></font></p><p>Got this as output:</p>
<p></p><div>Testing basic monitor search...<br>Monitor searches<br>Testing =
basic mt-hot search: 1 threads (1 x 50000) loops...<br>./progs/slapd-mtread=
-H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret =
-e cn=3DMonitor -m 1 -L 1 -l 50000<br>
Testing basic mt-hot search: 5 threads (1 x 10000) loops...<br>./progs/slap=
d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w=
secret -e cn=3DMonitor -m 5 -L 1 -l 10000<br></div>Perform a basic multi-t=
hreaded search on a single connection<br>
Testing basic mt-hot search: 60 threads (5 x 100) loops...<br>./progs/slapd=
-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w =
secret -e cn=3DMonitor -m 60 -L 5 -l 100<br>Testing basic mt-hot search: 62=
threads (5 x 100) loops...<br>
./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample=
,dc=3Dcom -w secret -e cn=3DMonitor -m 62 -L 5 -l 100<br>Testing basic mt-h=
ot search: 64 threads (5 x 100) loops...<br>./progs/slapd-mtread -H ldap://=
localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMon=
itor -m 64 -L 5 -l 100<br>
slapd-mtread failed (1)!<br>root@x054:[/data/prj/openldap-2.4.35/tests]<br>=
<p></p><p>=3D=3D=3D=3D</p><p>slapd-mtread PID=3D6750246: MT Test Start: con=
ns: 1 (ldap://localhost:9011/)<br>slapd-mtread PID=3D6750246: Threads: RO: =
64 RW: 0<br>
slapd-mtread PID=3D6750246: RO thread 0 pass=3D500 fail=3D0<br>slapd-mtread=
PID=3D6750246: RO thread 1 pass=3D500 fail=3D0<br>slapd-mtread PID=3D67502=
46: RO thread 2 pass=3D500 fail=3D0<br></p><p>...</p><p>slapd-mtread PID=3D=
6750246: RO thread 59 pass=3D500 fail=3D0<br>
slapd-mtread PID=3D6750246: RO thread 60 pass=3D500 fail=3D0<br>slapd-mtrea=
d PID=3D6750246: RO thread 61 pass=3D500 fail=3D0<br>slapd-mtread PID=3D675=
0246: RO thread 62 pass=3D500 fail=3D0<br>slapd-mtread PID=3D6750246: RO th=
read 63 pass=3D0 fail=3D0<br>
slapd-mtread PID=3D6750246: FAIL RO thread 63<br>slapd-mtread PID=3D6750246=
: MT Test complete<br></p><p>=3D=3D=3D</p><p>Back to original test, mtread.=
out tail is:</p><p>slapd-mtread PID=3D5242956: RO thread 60 pass=3D500 fail=
=3D0<br>slapd-mtread PID=3D5242956: RO thread 61 pass=3D500 fail=3D0<br>
slapd-mtread PID=3D5242956: RO thread 62 pass=3D500 fail=3D0<br>slapd-mtrea=
d PID=3D5242956: RO thread 63 pass=3D0 fail=3D0<br>slapd-mtread PID=3D52429=
56: FAIL RO thread 63<br>slapd-mtread PID=3D5242956: RO thread 64 pass=3D0 =
fail=3D0<br>slapd-mtread PID=3D5242956: FAIL RO thread 64<br>
slapd-mtread PID=3D5242956: RO thread 65 pass=3D0 fail=3D0<br>....</p><p>sl=
apd-mtread PID=3D5242956: FAIL RO thread 97<br>slapd-mtread PID=3D5242956: =
RO thread 98 pass=3D0 fail=3D0<br>slapd-mtread PID=3D5242956: FAIL RO threa=
d 98<br>slapd-mtread PID=3D5242956: RO thread 99 pass=3D0 fail=3D0<br>
slapd-mtread PID=3D5242956: FAIL RO thread 99<br>slapd-mtread PID=3D5242956=
: MT Test complete<br></p><p>=3D=3D=3D</p><p>Shall test after new build wit=
h CFLAGS=3D-D_LARGE_THREADS<br></p><div><div><br><br><div class=3D"gmail_qu=
ote">
On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <span dir=3D"ltr"><<a href=
=3D"mailto:hyc@symas.com"; target=3D"_blank">hyc@symas.com</a>></span> wr=
ote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div>Michael Felt wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
The log file did not show any errors.<br>
</blockquote>
<br></div>
If there's no server side errors then I suppose it's a problem in m=
tread. Check the testrun/mtread.out file.<span><font color=3D"#888888"><br>
<br>
<br>
<br>
-- <br></font></span><div><div>
=A0 -- Howard Chu<br>
=A0 CTO, Symas Corp. =A0 =A0 =A0 =A0 =A0 <a href=3D"http://www.symas.com"; t=
arget=3D"_blank">http://www.symas.com</a><br>
=A0 Director, Highland Sun =A0 =A0 <a href=3D"http://highlandsun.com/hyc/"; =
target=3D"_blank">http://highlandsun.com/hyc/</a><br>
=A0 Chief Architect, OpenLDAP =A0<a href=3D"http://www.openldap.org/project=
/" target=3D"_blank">http://www.openldap.org/<u></u>project/</a><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br>
--001a11c37b0a23a7cd04e2943a0d--