Re: (ITS#4607) openldap doesn't compile in 64bit/Solaris10

[jlick@jameslick.com]

I have also encountered this bug when trying to upgrade openldap on my 
systems.

This bug seems to first appear on systems running Solaris 10 update 
6/06, or patched with the corresponding kernel patch.  For Solaris x86 
systems this is patch 118855-14, and for Solaris sparc systems it 
appears to be 118833-17 (I don't have a sparc system running 6/06 so 
someone will have to verify the correct patch level, however I do have a 
system with that patch which also has the problem). 

With these releases of the OS/kernel, libavl is added along with 
associated support files.  The problem occurs because 
/usr/include/sys/avl.h is added, and various system include files have 
been modified to include avl.h.  openldap's liblutil library's avl 
implementation uses similar names to the Solaris version, but the 
implementations do not appear to be compatible. 

As the system avl.h is referenced in 14 separate system includes, it 
would be impractical to avoid including it.  However, avl.h is only 
included if the symbol _AVL_H is not defined.  This provides the 
opportunity to define this at compile time to avoid the conflicting 
definitions.  Fortunately the openldap avl.h uses a different symbol 
(_AVL) to avoid duplicate inclusion.

End user workaround: Add "-D_AVL_H" to the CFLAGS environment variable 
before running configure.  (openldap compiles and 'make test' completes 
successfully with this change on my systems.)

openldap developers:

easy fix: add -D_AVL_H to CFLAGS within configure on affected Solaris 
systems (e.g. if OS is solaris and /usr/include/sys/avl.h exists).

hard fix: as mentioned earlier, modify all symbols in openldap's avl 
implementation to be non-conflicting.  Probably want to rename avl.h too 
just in case some brain dead cpp can't figure out which one to use.

As Solaris 10 update 6/06 contains the highly anticipated ZFS support 
and these kernel patches are marked as recommended and security patches, 
it should be expected that lots of people will be running into this problem.

-- 
James Lick -- �建溥 -- jlick@jameslick.com -- http://jameslick.com/