Full_Name: Rich Megginson Version: CVS HEAD (2.4.16+) OS: Fedora URL: ftp://ftp.openldap.org/incoming/openldap-2.4.16-libldif-20090702.patch Submission from: (NULL) (76.113.59.19) This patch allows liblutil to provide the LDIF reading and writing functions as public APIs in a libldif, and exposes ldif.h to the public API. This patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not assigned rights and/or interest in this work to any party. I, Richard Megginson am authorized by Red Hat, Inc., my employer, to release this work under the following terms. Red Hat, Inc. hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.
rmeggins@redhat.com wrote: > Full_Name: Rich Megginson > Version: CVS HEAD (2.4.16+) > OS: Fedora > URL: ftp://ftp.openldap.org/incoming/openldap-2.4.16-libldif-20090702.patch > Submission from: (NULL) (76.113.59.19) > > > This patch allows liblutil to provide the LDIF reading and writing functions as > public APIs in a libldif, and exposes ldif.h to the public API. See also ITS#4033; ldif.h only parses LDIF into chunks/records, it's still up to the caller to turn these records into something meaningful. In fact libldif used to be a library of its own, but in RE22 we made the decision to merge it into liblutil because its functionality is so limited. I don't expect we'll reverse this decision unless/until it gets enough parsing added to it to make it more generally useful. > This patch file is derived from OpenLDAP Software. All of the > modifications to OpenLDAP Software represented in the following > patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not > assigned rights and/or interest in this work to any party. I, Richard > Megginson am authorized by Red Hat, Inc., my employer, to release this > work under the following terms. > > Red Hat, Inc. hereby place the following modifications to OpenLDAP > Software (and only these modifications) into the public domain. Hence, > these modifications may be freely used and/or redistributed for any > purpose with or without attribution and/or other notice. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Howard Chu wrote: > rmeggins@redhat.com wrote: >> Full_Name: Rich Megginson >> Version: CVS HEAD (2.4.16+) >> OS: Fedora >> URL: >> ftp://ftp.openldap.org/incoming/openldap-2.4.16-libldif-20090702.patch >> Submission from: (NULL) (76.113.59.19) >> >> >> This patch allows liblutil to provide the LDIF reading and writing >> functions as >> public APIs in a libldif, and exposes ldif.h to the public API. > > See also ITS#4033; ldif.h only parses LDIF into chunks/records, it's > still up to the caller to turn these records into something > meaningful. In fact libldif used to be a library of its own, but in > RE22 we made the decision to merge it into liblutil because its > functionality is so limited. I don't expect we'll reverse this > decision unless/until it gets enough parsing added to it to make it > more generally useful. I think it's very useful as it is - the mozldap libldif (which is almost identical in functionality to my proposed openldap libldif) is used in several different applications in the 389 directory server suite. What more does it need? Some sort of higher level "entry" and "attribute" abstraction? > >> This patch file is derived from OpenLDAP Software. All of the >> modifications to OpenLDAP Software represented in the following >> patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not >> assigned rights and/or interest in this work to any party. I, Richard >> Megginson am authorized by Red Hat, Inc., my employer, to release this >> work under the following terms. >> >> Red Hat, Inc. hereby place the following modifications to OpenLDAP >> Software (and only these modifications) into the public domain. Hence, >> these modifications may be freely used and/or redistributed for any >> purpose with or without attribution and/or other notice. >
Any ideas about what it would take to make a public LDIF API?
One thing that a general purpose LDIF parser needs is support for change records - for example, the ability to read in a changetype: modify and parse the changes into an LDAPMod ** or something like that. There's already code in ldapmodify to do this, it would need to be refactored into libldif. Since the library should be able to parse any sort of LDIF, the parse function would have to read in the first couple of lines of each LDIF record to determine if it is a plain entry or a change record. The parser would return a struct with a union for the different sets of parameters. The function that parses the LDIF could take a flag (or bitmask) that specifies which types of records it expects (e.g. the LDIF parser usage in slapadd would want to ignore/error on change records).
rmeggins@redhat.com wrote: > One thing that a general purpose LDIF parser needs is support for change > records - for example, the ability to read in a changetype: modify and > parse the changes into an LDAPMod ** or something like that. There's > already code in ldapmodify to do this, it would need to be refactored > into libldif. There are not only add/modify operations possible in a LDIF file. Personally I'm banging my head about adding full support for change records in python-ldap's LDIF parser. At the end one would have to trigger all LDAP operations including processing of LDAPv3 extended controls. So I guess for a C API this would mean a call-back interface full-featured like libldap itself. (So the next question would be whether to stick with this LDAP C API...) Ciao, Michael.
moved from Incoming to Software Enhancements
The ldif code does not currently have knowledge of any of the higher level LDAP data types such as LDAPMod or LDAPControl - some of the proposals would have the ldif API return these objects - Do we really want to introduce this higher level knowledge into the relatively low level libldif, along with the dependency on libldap? I think in order to do this properly, we would have to introduce a new LDAP API e.g. ldap_mods_from_ldif(const char *ldifstr, LDAPMod***, LDAPControl***) or something like that - something that would parse the given ldif record into those higher level data types.
I think this could be accomplished in one of two ways: 1) Just have libldif return lists of struct berval* for the various data parsed. The caller would be responsible for turning these into LDAPMod or LDAPControl structures - the advantage is that libldif doesn't have to know about any of these higher level structures 2) Have libldif create LDAPMod and LDAPControl - I think this could be accomplished by having ldif.c #include <ldap.h> to pull in the definitions of LDAPMod and LDAPControl - would this be ok?
rmeggins@redhat.com wrote: > I think this could be accomplished in one of two ways: > 1) Just have libldif return lists of struct berval* for the various data > parsed. The caller would be responsible for turning these into LDAPMod > or LDAPControl structures - the advantage is that libldif doesn't have > to know about any of these higher level structures > 2) Have libldif create LDAPMod and LDAPControl - I think this could be > accomplished by having ldif.c #include<ldap.h> to pull in the > definitions of LDAPMod and LDAPControl - would this be ok? Let's move this discussion to the openldap-devel mailing list. I'm thinking (2) is OK but I'd like to hear from other developers / potential users of this library. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
rmeggins@redhat.com wrote: > The ldif code does not currently have knowledge of any of the higher > level LDAP data types such as LDAPMod or LDAPControl - some of the > proposals would have the ldif API return these objects - Do we really > want to introduce this higher level knowledge into the relatively low > level libldif, along with the dependency on libldap? I think in order > to do this properly, we would have to introduce a new LDAP API e.g. > ldap_mods_from_ldif(const char *ldifstr, LDAPMod***, LDAPControl***) or > something like that - something that would parse the given ldif record > into those higher level data types. Should have mentioned this sooner; this ITS needs to be merged with ITS#4033. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Rich Megginson Version: CVS HEAD (2.4.21+) OS: Fedora URL: ftp://ftp.openldap.org/incoming/openldap-2.4.21-libldif-and-new-ldif-api-20100409.patch This patch allows liblutil to provide the LDIF reading and writing functions as public APIs in a libldif, and exposes ldif.h to the public API. This patch also creates a new LDAP API for parsing raw LDIF records into higher level LDAP structures such as LDAPMod and LDAPControl - ldap_parse_ldif_record() - and changes ldapmodify.c to use the new API. This patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not assigned rights and/or interest in this work to any party. I, Richard Megginson am authorized by Red Hat, Inc., my employer, to release this work under the following terms. Red Hat, Inc. hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.
> Full_Name: Rich Megginson > Version: CVS HEAD (2.4.21+) > OS: Fedora > URL: > ftp://ftp.openldap.org/incoming/openldap-2.4.21-libldif-and-new-ldif-api-20100409.patch > > This patch allows liblutil to provide the LDIF reading and writing > functions as > public APIs in a libldif, and exposes ldif.h to the public API. This > patch also creates > a new LDAP API for parsing raw LDIF records into higher level LDAP > structures > such as LDAPMod and LDAPControl - ldap_parse_ldif_record() - and changes > ldapmodify.c to use the new API. > > This patch file is derived from OpenLDAP Software. All of the > modifications to OpenLDAP Software represented in the following > patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not > assigned rights and/or interest in this work to any party. I, Richard > Megginson am authorized by Red Hat, Inc., my employer, to release this > work under the following terms. > > Red Hat, Inc. hereby place the following modifications to OpenLDAP > Software (and only these modifications) into the public domain. Hence, > these modifications may be freely used and/or redistributed for any > purpose with or without attribution and/or other notice. Kurt, if you give IPR clearance I can start (evaluating and) integrating this. p.
changed notes
> Full_Name: Rich Megginson > Version: CVS HEAD (2.4.21+) > OS: Fedora > URL: > ftp://ftp.openldap.org/incoming/openldap-2.4.21-libldif-and-new-ldif-api-20100409.patch Rich, ldifutil.c seems to be missing in that file. Can you provide it? Either a separate file, or a new version of the patch is fine. Thanks, p. > This patch allows liblutil to provide the LDIF reading and writing > functions as > public APIs in a libldif, and exposes ldif.h to the public API. This > patch also creates > a new LDAP API for parsing raw LDIF records into higher level LDAP > structures > such as LDAPMod and LDAPControl - ldap_parse_ldif_record() - and changes > ldapmodify.c to use the new API. > > This patch file is derived from OpenLDAP Software. All of the > modifications to OpenLDAP Software represented in the following > patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not > assigned rights and/or interest in this work to any party. I, Richard > Megginson am authorized by Red Hat, Inc., my employer, to release this > work under the following terms. > > Red Hat, Inc. hereby place the following modifications to OpenLDAP > Software (and only these modifications) into the public domain. Hence, > these modifications may be freely used and/or redistributed for any > purpose with or without attribution and/or other notice. > > > > >
changed notes changed state Open to Test moved from Software Enhancements to Development
On Sun, Apr 11, 2010 at 10:07 AM, <masarati@aero.polimi.it> wrote: >> Full_Name: Rich Megginson >> Version: CVS HEAD (2.4.21+) >> OS: Fedora >> URL: >> ftp://ftp.openldap.org/incoming/openldap-2.4.21-libldif-and-new-ldif-api-20100409.patch > > Rich, > > ldifutil.c seems to be missing in that file. Can you provide it? Either > a separate file, or a new version of the patch is fine. Sorry about that. You can find it here: http://rmeggins.fedorapeople.org/ldifutil.c > > Thanks, p. > > >> This patch allows liblutil to provide the LDIF reading and writing >> functions as >> public APIs in a libldif, and exposes ldif.h to the public API. This >> patch also creates >> a new LDAP API for parsing raw LDIF records into higher level LDAP >> structures >> such as LDAPMod and LDAPControl - ldap_parse_ldif_record() - and changes >> ldapmodify.c to use the new API. >> >> This patch file is derived from OpenLDAP Software. All of the >> modifications to OpenLDAP Software represented in the following >> patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not >> assigned rights and/or interest in this work to any party. I, Richard >> Megginson am authorized by Red Hat, Inc., my employer, to release this >> work under the following terms. >> >> Red Hat, Inc. hereby place the following modifications to OpenLDAP >> Software (and only these modifications) into the public domain. Hence, >> these modifications may be freely used and/or redistributed for any >> purpose with or without attribution and/or other notice. >> >> >> >> >> > > > > > > >
> Sorry about that. You can find it here: > http://rmeggins.fedorapeople.org/ldifutil.c Gotit, thanks. I have one first comment: the public API does not generally expose the memory context. I'm renaming ldap_parse_ldif_record() as ldap_parse_ldif_record_x(), and eliminating the void *ctx arg from ldap_parse_ldif_record(), if you don't mind. p.
> >> Sorry about that. You can find it here: >> http://rmeggins.fedorapeople.org/ldifutil.c > > Gotit, thanks. I have one first comment: the public API does not > generally expose the memory context. I'm renaming > ldap_parse_ldif_record() as ldap_parse_ldif_record_x(), and eliminating > the void *ctx arg from ldap_parse_ldif_record(), if you don't mind. tested and applied to HEAD; please test. Thanks, p.
>> >>> Sorry about that. You can find it here: >>> http://rmeggins.fedorapeople.org/ldifutil.c >> >> Gotit, thanks. I have one first comment: the public API does not >> generally expose the memory context. I'm renaming >> ldap_parse_ldif_record() as ldap_parse_ldif_record_x(), and eliminating >> the void *ctx arg from ldap_parse_ldif_record(), if you don't mind. > > tested and applied to HEAD; please test. Thanks, p. More comments: 1) perhaps it may be worth providing a function that converts a LDIFRecord structure into a string, and one that sends it to a stream 2) I note you put LDAPMod **lr_mods; /* list of mods for LDAP_REQ_MODIFY, LDAP_REQ_ADD */ struct berval lr_newrdn; /* LDAP_REQ_MODDN, LDAP_REQ_MODRDN, LDAP_REQ_RENAME */ struct berval lr_newsuperior; /* LDAP_REQ_MODDN, LDAP_REQ_MODRDN, LDAP_REQ_RENAME */ int lr_deleteoldrdn; /* LDAP_REQ_MODDN, LDAP_REQ_MODRDN, LDAP_REQ_RENAME */ /* the following are for future support */ struct berval lr_extop_oid; /* LDAP_REQ_EXTENDED */ struct berval lr_extop_data; /* LDAP_REQ_EXTENDED */ struct berval lr_cmp_attr; /* LDAP_REQ_COMPARE */ struct berval lr_cmp_bvalue; /* LDAP_REQ_COMPARE */ in the structure; the last four are not used right now. I think it would make sense to group struct members by op in substructures, and then put them in a union, to stress the fact that they're mutually exclusive. Much like Howard did for the corresponding substructures in the Operation struct in slapd. p.
masarati@aero.polimi.it wrote: >>>> Sorry about that. You can find it here: >>>> http://rmeggins.fedorapeople.org/ldifutil.c >>>> >>> Gotit, thanks. I have one first comment: the public API does not >>> generally expose the memory context. I'm renaming >>> ldap_parse_ldif_record() as ldap_parse_ldif_record_x(), and eliminating >>> the void *ctx arg from ldap_parse_ldif_record(), if you don't mind. >>> >> tested and applied to HEAD; please test. Thanks, p. >> > > More comments: > > 1) perhaps it may be worth providing a function that converts a LDIFRecord > structure into a string, and one that sends it to a stream > For debugging purposes, or something like that? > 2) I note you put > > LDAPMod **lr_mods; /* list of mods for LDAP_REQ_MODIFY, > LDAP_REQ_ADD */ > struct berval lr_newrdn; /* LDAP_REQ_MODDN, LDAP_REQ_MODRDN, > LDAP_REQ_RENAME */ > struct berval lr_newsuperior; /* LDAP_REQ_MODDN, LDAP_REQ_MODRDN, > LDAP_REQ_RENAME */ > int lr_deleteoldrdn; /* LDAP_REQ_MODDN, LDAP_REQ_MODRDN, > LDAP_REQ_RENAME */ > /* the following are for future support */ > struct berval lr_extop_oid; /* LDAP_REQ_EXTENDED */ > struct berval lr_extop_data; /* LDAP_REQ_EXTENDED */ > struct berval lr_cmp_attr; /* LDAP_REQ_COMPARE */ > struct berval lr_cmp_bvalue; /* LDAP_REQ_COMPARE */ > > in the structure; the last four are not used right now. I think it would > make sense to group struct members by op in substructures, and then put > them in a union, to stress the fact that they're mutually exclusive. Much > like Howard did for the corresponding substructures in the Operation > struct in slapd. > Please check out this patch - ftp://ftp.openldap.org/incoming/openldap-2.4.21-ldifrecord-union-20100412.patch > p. > > > > >
IPR Okay applied to HEAD
*** Issue 4033 has been marked as a duplicate of this issue. ***