Re: Using libtool -release versus -version-info breaks packages (ITS#3035)

[kean@armory.com]

> It was (is) allowing incompatible versions of the libraries
> to be picked up dynamically (which is causing the usual
> problems).
There are ways to address that. See below.

>         c) ensures the Project is able to update the ABIs as
>         needed (including applying fixes that effect the ABI
>         as patch releases, including on older branches).
I think that this requirement is almost impossible to meet even using 
the -release, in any sensible way, without some effort. I also think 
you're using the term ABI slightly incorrectly, and really mean API. 
Consider this. Suppose you have release 2.1.30. It has a defined API, 
that is expressed in header files and implemented in the shared library. 
That implementation defines the ABI, as it is what applications that 
link against that version of the library will expect. Lets say this 
library was installed with an SONAME of libldap.so.2.1. For argument's 
sake, lets say there was a function int some_ldap_fn (int). Now you are 
producing version 2.1.31, and you find that this function absolutely 
must take an extra argument, lets say it now becomes int some_ldap_fn 
(int, const char *).

You are now faced with two choices. When fixing this, you can address it 
at the API level and leave the ABI untouched (requires some extra work) 
or you can 'full steam ahead and f*** the icebergs' and change both the 
API and the ABI (requires no work beyond the change itself). The first 
approach is, obviously, preferable. Here's one way of doing it. Suppose 
this function was declared in ldap.h. Instead of changing:
    extern int some_ldap_fn(int);
to
    extern int some_ldap_fn(int, const char *);

you would instead do something like:
    extern int some_ldap_fn_2(int, const char *);
    #define some_ldap_fn(A,B) some_ldap_fn_2((A),(B))

In which ever file implements some_ldap_fn you have to of course still have:
int some_ldap_fn (int x)
{
   return old_behaviour();
}

but I bet that in most cases this can simply be a stub that calls the 
new one, passing in the extra argument as a NULL, perhaps as:
int some_ldap_fn (int x)
{
   return some_ldap_fn_2 (x, NULL);
}

The details will vary from case to case, of course.

This ensures that for those users that are recompiling code, that they 
get the new API definition, and will result in their application using 
the new ABI. But for those applications that are not being recompiled, 
and are simply linked against the library, nothing will change, because 
at the time they were compiled the function did just take one argument, 
and that is preserved.

This approach means that through the lifetime of, say, OpenLDAP 2.1.x, 
you can keep the same shared library name. When you change the API and 
ABI in ways that cannot practically be handled above, you then change to 
OpenLDAP 2.2.x, and change the shared library to libopenldap.so.2.2.

This has a distinct advantage for OS vendors and package maintainers. It 
means that for the lifetime of release 2.1, they do not have to 
recompile every application in the OS distribution that uses OpenLDAP. 
They can continue to update OpenLDAP itself, and it will continue to 
serve all the applications that were linked against it. It also means 
that for backwards compatiblity, the package maintainers only need to 
provide one version of the shared library for each major release cycle 
of OpenLDAP. Consider the alternative.

Suppose you stick with the -release mechanism. Today, the release is 
2.2.7. So anything I link against OpenLDAP will have a dependency (at 
the ELF level) on a shared library clled libldap-2.2.7.so. Lets say that 
today I aolso compile PHP, and it uses OpenLDAP. I now package both and 
release them to the world. 2 weeks later there is a new version of 
OpenLDAP, but PHP has unchanged. I would like to release just a new 
OpenLDAP. However, I can no longer do that. Well, I can, but I have to 
preserve the old libldap-2.2.7.so. So now my OpenLDAP package contains 
botht hat version and libldap-2.2.8.so. And so on and so on ... you will 
end up in the situation where OS vendors will need to ship 20 or 30 
versions of the same library, all very subtly different. The only 
alternative is to track down absolutely every package that may have used 
it, and recompile them and redistribute them. And yet you will *STILL* 
have to provide all the old versions becuase you have no idea what the 
end user has done, and which versions they have linked against. This is 
the *HUGE* problem with using -release.

> encode the version (e.g., 20207).  This was rejected as
> libtool only supports sonames as large as 999.  It is
> also questionable if all dynamic linkers support sonames
> larger than 255.
Thats a libtool limitation then, but not required. You can (and really 
should try to) keep one shared library version per OpenLDAP series. For 
example, keep libldap.so.2.1 as the soname throughout the lifetime of 
the OpenLDAP 2.1.x series, and libldap.so.2.2 through the lifetime of 
OpenLDAP 2.2.x etc. Also note that the version number that appears in a 
DT_SONAME is numeric by convention only, and the RTLD's (on all the OSes 
Ive ever worked on) pay no attention to it except as a string value. Its 
how its defined in the ELF standards. There is absolutely no 
significance to the name whatsoever - it either matches what a binary 
has in its DT_NEEDED or it doesn't. If it does, the RTLD will use it. 
But the point is moot if you use smaller, more human readable versions 
like 2.1 and 2.2 that reflect the core software the library is implementing.

> If you know of some other approach which meets the
> project requirements, please feel free to suggest it on
> the developers list.  However, reverting to the old broken
> way of doing things is not an option.
For the record, the old was was only broken for how it was being used, 
not intrinsically. libtool has a sophisticated (read: obtuse and 
convoluted) algorithm for determining the SONAME to use, and that is 
controlled by the arguments to -version-info. I think you were using 
that option incorrectly in the last. If you changed the ABI one of the 
three values (I forget which) should have changed too. BVut that is a 
much worse implementation than what I described above.

Kean