Full_Name: Randolf Werner Version: 2.3.35 OS: Windows XP URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (155.56.68.221) Hi, i just have found the following compatibilty issue when using the 1.2.840.113556.1.4.319 control (http://www.ietf.org/rfc/rfc2696.txt) supported by slapd. According to Microsoft and IBM LDAP API ldap_create_page_control http://msdn2.microsoft.com/en-us/library/aa366547.aspx http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp?topic=/apis/ldap_create_page_control.htm the page size parameter should be an 32 Bit unsigned integer, however the RFC specifies it as "(INTEGER (0..maxInt)". Therefore our LDAP client uses 2**32-1 as default page size value. This works fine with Microsoft Active Directroy for many years but fails with "Protocol Error" (Error 2) when using slapd. Our workaround is to use 2**31-1 as default value in our client. I am not sure which implementation is correct, anyway it is a compatibility issue for ldap clients and might be "fixed" to improve compatibility. Best regards Randolf
On Aug 15, 2007, at 2:21 PM, randolf.werner@sap.com wrote: > Full_Name: Randolf Werner > Version: 2.3.35 > OS: Windows XP > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (155.56.68.221) > > > Hi, > i just have found the following compatibilty issue when using the > 1.2.840.113556.1.4.319 control (http://www.ietf.org/rfc/ > rfc2696.txt) supported > by slapd. According to Microsoft and IBM LDAP API > ldap_create_page_control > > http://msdn2.microsoft.com/en-us/library/aa366547.aspx > http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp? > topic=/apis/ldap_create_page_control.htm > > the page size parameter should be an 32 Bit unsigned integer, > however the RFC > specifies it as "(INTEGER (0..maxInt)". Therefore our LDAP client > uses 2**32-1 > as default page size value. This works fine with Microsoft Active > Directroy for > many years but fails with "Protocol Error" (Error 2) when using > slapd. Our > workaround is to use 2**31-1 as default value in our client. I am > not sure which > implementation is correct, anyway it is a compatibility issue for > ldap clients > and might be "fixed" to improve compatibility. Given maxInt is defined in RFC 4511 as: maxInt INTEGER ::= 2147483647 -- (2^^31 - 1) -- this doesn't appear to be a bug in OpenLDAP Software. -- Kurt
randolf.werner@sap.com wrote: > Full_Name: Randolf Werner > Version: 2.3.35 > OS: Windows XP > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (155.56.68.221) > > > Hi, > i just have found the following compatibilty issue when using the > 1.2.840.113556.1.4.319 control (http://www.ietf.org/rfc/rfc2696.txt) supported > by slapd. According to Microsoft and IBM LDAP API ldap_create_page_control > > http://msdn2.microsoft.com/en-us/library/aa366547.aspx > http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp?topic=/apis/ldap_create_page_control.htm > > the page size parameter should be an 32 Bit unsigned integer, however the RFC > specifies it as "(INTEGER (0..maxInt)". Therefore our LDAP client uses 2**32-1 > as default page size value. This works fine with Microsoft Active Directroy for > many years but fails with "Protocol Error" (Error 2) when using slapd. Our > workaround is to use 2**31-1 as default value in our client. I am not sure which > implementation is correct, anyway it is a compatibility issue for ldap clients > and might be "fixed" to improve compatibility. I suggest you file bug reports with Microsoft and IBM to fix their broken implementations then. RFC4511 clearly defines maxInt to be 2^31-1. That value has not changed since RFC2251 either. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
> randolf.werner@sap.com wrote: >> Full_Name: Randolf Werner >> Version: 2.3.35 >> OS: Windows XP >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (155.56.68.221) >> >> >> Hi, >> i just have found the following compatibilty issue when using the >> 1.2.840.113556.1.4.319 control (http://www.ietf.org/rfc/rfc2696.txt) >> supported >> by slapd. According to Microsoft and IBM LDAP API >> ldap_create_page_control >> >> http://msdn2.microsoft.com/en-us/library/aa366547.aspx >> http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp?topic=/apis/ldap_create_page_control.htm >> >> the page size parameter should be an 32 Bit unsigned integer, however >> the RFC >> specifies it as "(INTEGER (0..maxInt)". Therefore our LDAP client uses >> 2**32-1 >> as default page size value. This works fine with Microsoft Active >> Directroy for >> many years but fails with "Protocol Error" (Error 2) when using slapd. >> Our >> workaround is to use 2**31-1 as default value in our client. I am not >> sure which >> implementation is correct, anyway it is a compatibility issue for ldap >> clients >> and might be "fixed" to improve compatibility. > > I suggest you file bug reports with Microsoft and IBM to fix their broken > implementations then. RFC4511 clearly defines maxInt to be 2^31-1. That > value > has not changed since RFC2251 either. Or, fix your clients so that they don't use pagedResults when not necessary. 2**31-1 is a pretty large number. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
Hi, thanks for your replies. Anyway i did not opened this message to discuss who implented RFC 2696 correctly and who not. The point is something else: 1) I tried to share my experience (and solution) with other LDAP application programmers, since i am pretty sure there are/will be more clients running into this issue (e.g. ldp.exe from Windows support tools seems to have the same problem when enabling 1.2.840.113556.1.4.319 / pages search type again slapd). 2) Try to point out how it can easily happen to write a client working with Active Directory but not with slapd and suggested an easy but perhaps not 100% RFC 2696 compliant slapd change. Perhaps i should explain 2) a little bit more from my application develeopers point of view. Here is my rough application history: a) In ~1999 we started building a LDAP V3 client for SAP that had to work on a very wide variety of OS platforms and as much as possible LDAP V3 compliant directories. b) It was first shipped in 2000/2001 and customers were happy. c) Data volume at customer side increazed and it stopped working for Active Directory customers. d) We learned about size limits, page size and the fact you simply have to use 1.2.840.113556.1.4.319 to get more than 1000 entries from Active Directroy. We implemented it and everybody was happy again. Implenting ldap_create_page_control for all OS platforms was not that trivial at that time. e) In ~2004 the first customer liked to use slapd. We noticed some minor issues (e.g. setting V3 protocol version explicitly) fixed it and once more everybody was happy. f) Yesterday a slapd 2.3.35 slapd customer reported a problem similar to e) ("Protocol Error"). So we analyzed it and fixed it again by reducing default page size to 2**31-1. Here is why i believe others might run into the same issue. If you need to create a ldap client dealing with big search results for various LDAP directories you had to use 1.2.840.113556.1.4.319 even if you are only interested in collecting the entire result otherwise you would fail with Active Directory. You look at the ldap_create_page_control API spec (developers oftern use API specs and not protocol RFCs) and notice page size to be an usinged integer. Since you are not really interested in pages you specify the largest possible value 2**32-1. This way you end up with client working with Active Directory, all directories not supporting 1.2.840.113556.1.4.319 but not with slapd supporting 1.2.840.113556.1.4.319. I am not interested in discussing who is right and who is wrong, i am just interested in making as much as possible working together in the real world with as little as possible customer pain. Therefore i am almost 100% sure Active Directory will not break existing clients by using signed instead of unsigned integer page size just to be more RFC 2696 compliant.On the other hand slapd has a chance to detect negavtive page sizes and assume some reasonable page size in that situation instead of "Protocol Error" without breaking any existing client. For my customers it will not make much difference since the patch will be rolled out soon, but perhaps others can benefit. Best regards Randolf
changed notes
randolf.werner@sap.com writes: > Active Directory. You look at the ldap_create_page_control API spec > (developers oftern use API specs and not protocol RFCs) and notice page > size to be an usinged integer. Since you are not really interested in > pages you specify the largest possible value 2**32-1. So that API spec is bogus, it should have specified the range. Which data type is being used is not relevant to what the protocol accepts, in particular since the width of data types sometimes change. E.g. if you move from a 32-bit to a 64-bit host. > I am not interested in discussing who is right and who is wrong, i am > just interested in making as much as possible working together in the > real world with as little as possible customer pain. Notice however that your first problem happened because AD breaks the standard, and your second problem because you rewrote it by bogus Microsoft documentation and it worked because of another way AD breaks the standard. (Though less seriously in the second case, it follows the "be liberal in what you accept" way.) If OpenLDAP is changed to accept wider limits, it might also encourage more people to write bogus code like you did. OpenLDAP has been going in the other direction over the years, doing ever more strict error checking. That said, pretending that Microsoft does not exist isn't always a winning strategy, so it might make sense to implement a "relaxed" mode which accepts common ways to break the standard. But I don't think it should be turned on by default. > Therefore i am almost 100% sure Active Directory will not break > existing clients by using signed instead of unsigned integer page size > just to be more RFC 2696 compliant. In addition to fixing the doc, they they could fix that in API by reducing a passed value >= 2**31 to the max valid value. Not that they seem to care much about standards anyway. One note - I'm curious why you did not use page size 0, which the Microsoft API pages you posted say overrides such page sizes. Though maybe I misunderstood and that doesn't actually help. -- Regards, Hallvard
randolf.werner@sap.com wrote: > Hi, > =20 > thanks for your replies. Anyway i did not opened this message to discuss > who implented RFC 2696 correctly and who not. The point is something > else: The point is that there is a specific definition of the protocol. When vendors implement the protocol incorrectly, interoperability suffers. The fix for those interoperability problems is to take steps to ensure that vendors don't deviate from the published specifications. Likewise, application developers must pay attention to the published specs and write compliant code. Otherwise the entire concept of published standards is meaningless. > Here is why i believe others might run into the same issue. If you need > to create a ldap client dealing with big search results for various LDAP > directories you had to use 1.2.840.113556.1.4.319 even if you are only > interested in collecting the entire result otherwise you would fail with > Active Directory. Nonsense. You simply ask your AD administrator to raise the default sizelimit, the same way you would ask an OpenLDAP administrator to do the same thing. > I am not interested in discussing who is right and who is wrong, i am > just interested in making as much as possible working together in the > real world with as little as possible customer pain. In the real world, if you ignore the specs, your programs break. If you adhere to the specs, your programs keep working. What you're asking for is the equivalent of this: "My Microsoft building has a nail sticking out on the floor and I keep tripping over it when I walk by. Your OpenLDAP building is missing this nail; please pull a nail out so I can trip over it the same way." -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
h.b.furuseth@usit.uio.no wrote: > That said, pretending that Microsoft does not exist isn't always a > winning strategy, so it might make sense to implement a "relaxed" mode > which accepts common ways to break the standard. But I don't think it > should be turned on by default. We don't pretend Microsoft doesn't exist. But we *do* expect them to mean what they say; since 3 Microsoft employees worked on RFC2696 you would sort of expect them to get it right. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Hi, oh my god what have i started here. Please don't try to convince me that standards are important and slapd pricecly implents RFC 2696, i was already convinced of that before i started the discussion. Anyway i like to resond to Howard Chu because he mentioned something dangerous and wrong regarding Active Directory (somewhat of topic here) but i like to avaoid people following his suggestion: Howard Chu <hyc@symas.com> wrote: >> Here is why i believe others might run into the same issue. If you need >> to create a ldap client dealing with big search results for various LDAP >> directories you had to use 1.2.840.113556.1.4.319 even if you are only >> interested in collecting the entire result otherwise you would fail with >> Active Directory. > >Nonsense. You simply ask your AD administrator to raise the default sizelimit, >the same way you would ask an OpenLDAP administrator to do the same thing. Well about 6 years ago when i first noticed this problem, i asked Microsoft about that limit and yes you can use ntdsutil.exe to set MaxPageSize to a higher limit: http://support.microsoft.com/kb/315071/en-us But it is strongly recommend to not change it!!!! Over the years i have seen quite some Active Directories customers and many of them are very afraid when you ask them you change some configuration or extending the schema. In 2001 SAP's IT set MaxPageSize=10000 for a few weeks as a temporary workaround until i was able to provide a real solution. That was a long time ago and today SAP's and probably most other companies Active Directories are running with default values. 1.2.840.113556.1.4.319 and MaxPageSize had not been invented to make our life more difficult. A good article about it is e.g. http://searchwinit.techtarget.com/tip/0,289483,sid1_gci1265206,00.html. I like to close with Gary Olsen's conclusion "The best practice for setting MaxPageSize is to leave it alone." Best regards Randolf
randolf.werner@sap.com wrote: > Over the years i have > seen quite some Active Directories customers and many of them are very > afraid when you ask them you change some configuration or extending the > schema. Doesn't it seem odd to you that people would voluntarily use software that they are afraid of? Never mind, this discussion has gone far enough off-topic. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
changed state Open to Suspended
changed notes moved from Incoming to Historical
not ours proposed change rejected
changed notes changed state Suspended to Closed