[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: [ldapext] UTF-8 full support in LDIF / LDIF v2
Thanks Kurt, sorry for my asking an answer in another email, it seems we were writing at the same time. 


Kurt Zeilenga wrote:


-the directory is broken
-you export to LDIF
-compare this LDIF with a previous one from when the directory was working.
You don't need UTF-8 for this. A simple text diff tool will tell you that the base64 differs.
True, diff will tell you that they are different, and where, but then you need to decode base64 to find what the text is in the two file to help you understand why they are different.
If there were no base64 encryption, then you would know right away, in most case, making it a much faster process.
But now you assume you'll be able to read them. This is a bad assumption.
I still don't understand why you are saying this. Can you give precise examples ?
A simple diff tool might show two DIFFERENT values the same way, leading the human to believe there is no difference when there is a significant difference.
So, for example, one file contains U+2026 (ellipsis, "…") while the other contains three U_002E (three times the full stop character, "..."), and the issue of duplication in Unicode (http://en.wikipedia.org/wiki/Duplicate_characters_in_Unicode) ?
Well, what do we do today when diff shows us that there is a difference in two ascii files, but our eyes can't see it ? We hex dump of the offending line(s), and go "ahah, I've got <CR> here, but <CR><LF> there). On the other hand what's the percentage of time you diff files and run into this problem ?
Having those base64 encoded will make the fact that they are different more obvious, but won't help us when we're trying to understand why they are different, we will still have to go base64 --> UTF-8 --> hex values.
Data can be different because it comes from different sources, in which case it is likely that people will use different duplicate of Unicode point to express the same thing, but data can also be different because it was changed, which is the case I am more concerned with (backup of LDAP servers etc...), and in this case, I doubt somebody will change the mu symbol to the micro symbol or something similar. It might happen for malicious reasons, but I expect that to be a small percentage.
The key phrase here is "Unicode text". And most such display tools not only require "well-formed" text, but often cannot display all "well-formed" text. But removing the ASCII restriction does not make a LDIF file "Unicode text". It makes it a series of Unicode code points and hence display of it as text will be quite problematic.
I don't understand what you are saying here. Could you give an example of a problematic case, or a link to an explanation ? 

Why is a series of Unicode point not text ?
What do you mean by "well-formed text", or actually by text that is not well-formed ?
And even it's displayable, you have the problem that two values might display in the same way, making visual diff'ing problematic. 

This is the issue above (Unicode duplication) ? Right ?


>> Other case: People have mentioned scripts that build LDIF file from
>> other source, and have mentioned that encoding the values in base64 is
>> an overhead they could do without.
>
> While base64 data is an additional step, it's an additional step that
> well supported today.  If we lift the ASCII restriction now, we'll have
> some implementations that do support it and some that don't, and that
> will cause interop problems.   I cannot support inducing such interop
> problems without a strong justification.
Adding a new version of the standard, does not remove the old one. You can tell people that you only work with version 1 LDIF files. The advantage of having a standard, is that the tools will slowly adopt it and will be able to deal with it. If we don't create a standard, we run into the situation where everybody creates their own format, and create their own tools to transform their format to the current LDIF one. If I remember right, Jim mentioned that he already uses his version of extended LDIF. 


--
Yves.
http://www.sollers.ca/

_______________________________________________
Ldapext mailing list
Ldapext@ietf.org
https://www.ietf.org/mailman/listinfo/ldapext