[Date Prev][Date Next] [Chronological] [Thread] [Top]
Re: Solaris 32bit File Descriptor Limit (ITS#2636)

To: openldap-its@OpenLDAP.org
Subject: Re: Solaris 32bit File Descriptor Limit (ITS#2636)
From: joseph.tingiris@sdc.cox.net
Date: Sat, 19 Jul 2003 21:17:01 GMT
Ok, I've been doing some testing and have a question I hope someone can
answer.  I've compiled 2.1.22 64bit for Solaris expecting to solve this 256
simulataneous connection issue I'm having, but something's still not right.
After loading the databases, I re-run the same test on the 64bit binary and
when I get 256 descriptors open, I get almost the exact same log message,
but now it logs at 266 instead of 256:

slapd[28075]: [ID 759906 local6.debug] daemon: 266 beyond descriptor table
size 256

The ldapsearh client produces this:

ldap_simple_bind_s: Can't contact LDAP server

It seems that the other threads block all connections over 256 until there
is a free descriptor <256 at which point the query is returned.  Is this
right?  Why?  Is there a limit in the code or something?

I've done a lot of testing and checked the process with plimit and can see
it is not an OS barrier (unlimited).  I don't really see a need to debug,
yet, because it's not coring or anything like that.  But, I'm sure the
search will eventually abort and/or time out (tested) if I don't free a
descriptor <256 so I think this is a bug or something I'm just completely
missing.  Threads are set to 32 and cachesize, dbcachesize, etc are all
default.  I'm using back-ldbm and back-glue because my tree is really large
and I needed to split the db up (and these are replicas).  Here is an output
of the pmap and for reference, none of the libraries are 32bit:

28075:  /usr/local/openldap64/libexec/slapd -f
/openldap/backup/replica64/etc/
0000000100000000   2608K read/exec
/usr/local/openldap64/libexec/slapd
000000010038A000     32K read/write/exec
/usr/local/openldap64/libexec/slapd
0000000100392000  22760K read/write/exec     [ heap ]
FFFFFFFF7B3FC000     16K read/write          [ anon ]
FFFFFFFF7BBFC000     16K read/write          [ anon ]
FFFFFFFF7D300000     16K read/write          [ anon ]
FFFFFFFF7D7F8000     32K read/write          [ anon ]
FFFFFFFF7D900000    112K read/write          [ anon ]
FFFFFFFF7DA00000     16K read/write          [ anon ]
FFFFFFFF7DB00000     16K read/write          [ anon ]
FFFFFFFF7DC00000      8K read/write/exec     [ anon ]
FFFFFFFF7DD00000     96K read/exec
/usr/lib/lwp/sparcv9/libthread.so.1
FFFFFFFF7DE18000     16K read/write/exec
/usr/lib/lwp/sparcv9/libthread.so.1
FFFFFFFF7DF00000      8K read/write/exec/shared   [ anon ]
FFFFFFFF7E000000     16K read/exec         /usr/lib/sparcv9/libmp.so.2
FFFFFFFF7E104000      8K read/write/exec   /usr/lib/sparcv9/libmp.so.2
FFFFFFFF7E200000      8K read/write/exec     [ anon ]
FFFFFFFF7E300000    720K read/exec         /usr/lib/sparcv9/libc.so.1
FFFFFFFF7E4B4000     56K read/write/exec   /usr/lib/sparcv9/libc.so.1
FFFFFFFF7E4C2000      8K read/write/exec   /usr/lib/sparcv9/libc.so.1
FFFFFFFF7E500000     24K read/exec         /usr/lib/sparcv9/libpthread.so.1
FFFFFFFF7E606000      8K read/write/exec   /usr/lib/sparcv9/libpthread.so.1
FFFFFFFF7E700000      8K read/exec         /usr/lib/sparcv9/libmtmalloc.so.1
FFFFFFFF7E802000      8K read/write/exec   /usr/lib/sparcv9/libmtmalloc.so.1
FFFFFFFF7E900000      8K read/exec
/usr/platform/sun4u-us3/lib/sparcv9/libc_psr.so.1
FFFFFFFF7EA00000     56K read/exec         /usr/lib/sparcv9/libsocket.so.1
FFFFFFFF7EB0E000     16K read/write/exec   /usr/lib/sparcv9/libsocket.so.1
FFFFFFFF7EC00000      8K read/write/exec     [ anon ]
FFFFFFFF7ED00000    664K read/exec         /usr/lib/sparcv9/libnsl.so.1
FFFFFFFF7EEA6000     56K read/write/exec   /usr/lib/sparcv9/libnsl.so.1
FFFFFFFF7EEB4000     40K read/write/exec   /usr/lib/sparcv9/libnsl.so.1
FFFFFFFF7EF00000     32K read/exec         /usr/lib/sparcv9/libgen.so.1
FFFFFFFF7F008000      8K read/write/exec   /usr/lib/sparcv9/libgen.so.1
FFFFFFFF7F100000    232K read/exec         /usr/lib/sparcv9/libresolv.so.2
FFFFFFFF7F23A000     32K read/write/exec   /usr/lib/sparcv9/libresolv.so.2
FFFFFFFF7F300000      8K read/exec         /usr/lib/sparcv9/libdl.so.1
FFFFFFFF7F400000      8K read/shared       dev:238,6 ino:493389
FFFFFFFF7F500000      8K read/write/exec     [ anon ]
FFFFFFFF7F600000    144K read/exec         /usr/lib/sparcv9/ld.so.1
FFFFFFFF7F722000     16K read/write/exec   /usr/lib/sparcv9/ld.so.1
FFFFFFFF7FFFA000     24K read/write          [ stack ]
         total    27976K

----- Original Message ----- 
From: <joseph.tingiris@sdc.cox.net>
To: <openldap-its@OpenLDAP.org>
Sent: Tuesday, July 08, 2003 2:13 PM
Subject: Re: Solaris 32bit File Descriptor Limit (ITS#2636)


> Some more information.
>
> I can reproduce the problem fairly easily in my lab.  I just telnet to the
> port (255 times), background the processes, and then try a real ldap
> operation (ldapsearch, modify, etc) and I get LDAP_SERVER_DOWN returned.
> During this time, lsof or pfiles shows 256 descriptors in use.  I've tried
> ulimit, plimit, and all of the /etc/system settings to no avail.
>
> I found this thread
> (http://www.openldap.org/lists/openldap-software/200111/msg00324.html)
> regarding OpenLDAP 2.0.11 and it appears to be the same issue.  I found
the
> post about fsio intriguing and I may attempt to compile the 32bit version
> with these libraries instead of Sun's.  Has anyone tried this, since?
>
> Sun's Response:
>
> "For a 32-bit application, a stdio library FILE structure represents the
> underlying file descriptor as an unsigned char, limiting the range of fds
> which can be opened as FILE's to 0-255 inclusive. A common, known problem
is
> that when the 32-bit stdio is used for a large server application, the 255
> limit is frequently exceeded. Although this limitation does not exist for
> 64-bit applications, this problem will always remain for 32-bit
> applications."
>
> Sun also suggests using open() instead of fopen() ... would a fix of this
> nature be possible for OpenLDAP?
>
> I've got a valid OpenLDAP 2.1.16 64bit build working and I'm testing it,
> now.  I expect this will be my final resolution, but until then I will
> reserve further comments and settle for yet another question.  Who else is
> using 64bit OpenLDAP on Solaris 2.8 and what are your experiences,
good/bad?
>
> Thanks!
>
> Joseph
>
> ----- Original Message ----- 
> From: <joseph.tingiris@cox.net>
> To: <openldap-its@OpenLDAP.org>
> Sent: Tuesday, July 08, 2003 11:32 AM
> Subject: Solaris 32bit File Descriptor Limit (ITS#2636)
>
>
> > Full_Name: Joseph Tingiris
> > Version: 2.1.16
> > OS: Solaris 8
> > URL: ftp://ftp.openldap.org/incoming/
> > Submission from: (NULL) (206.157.230.254)
> >
> >
> > Under high load, I get the following log message:
> >
> > Jul  6 23:49:34 lakeldap01 slapd[10327]: [ID 759906 local5.debug]
daemon:
> 256
> > beyond descriptor table size 256
> >
> > I've experienced this with Apache and understand that it is a 32bit
limit
> on
> > descriptors.  Again, 32bit Apache has a workaround for this that allows
> the
> > connection table to use non-file type descriptors.
> >
> > Does anyone know of a simple workaround for this, other than going to a
> 64bit
> > compiled version?
> >
> > Thanks!
> >
> >
Prev by Date: Fix to aclocal.m4 (ITS#2659)
Next by Date: Re: Solaris 32bit File Descriptor Limit (ITS#2636)
Index(es):
- Chronological
- Thread