|OpenLDAP Faq-O-Matic : OpenLDAP Developer's FAQ : Porting OpenLDAP : Porting to IBM OS/390|
|Notes on building OpenLDAP on OS/390 with the V2R9.0 C Compiler
This is a very rough description of a very rough build process.
At some point we may fix things up in configure to handle this all automatically. Sorry for the crude nature of this build, it's certainly not for the faint of heart. [The build process has been updated, please read this entire topic for the full details.]
Overview --------OS/390 has a fairly comprehensive Unix Support Services layer, so most of the functions required by OpenLDAP exist. However, because the mainframe uses EBCDIC for its native character encoding, and the OpenLDAP source code is designed to work with ASCII/UTF8, either the character-manipulation code must be rewritten for EBCDIC or the character data being manipulated must be translated between EBCDIC and ASCII.
For this port, I chose to keep all of the LDAP internals in ASCII, and only deal with translation of EBCDIC from the host interfaces - reading config files, parsing command line arguments, and writing log files. The V2R9 C Compiler tools have two features to assist in this approach; a CONVLIT option which lets you specify the character set to use when compiling character and string literals, and a __LIBASCII mode for the C library, which replaces a number of library functions with code that accepts arguments in ASCII instead of EBCDIC.
With the combination of these two features, most of the porting effort is taken care of, but there are still some issues.
The configure script will trip up on detecting the Pthreads support. This is because the OS/390 implementation of Pthreads uses function signatures that are different from other published standards. The correct thing to do here is fix the configure script to recognize this IBM variant. I haven't fixed the configure script yet, instead I bypassed the test by setting some environment variables.
Some of the __LIBASCII functions don't work correctly, the getopt() function always segfaulted, and the regcomp() never generated working regular expression matches. So, I used the supplied liblutil/getopt.c and a specially compiled copy of Henry Spencer's regex library instead of the __LIBASCII functions.
In order to handle the ASCII/EBCDIC translation for config file reading and log message writing, I defined several replacement functions (in liblutil/stdio.c) and use macros to make the regular source use these functions. I haven't found a clean location to define these macros, so at present I manually insert them into include/portable.h after configure has run.
There's a lot of other manual tweaks in this process. In time this will all get
absorbed into the automated configure script.
Build Prerequisites -------------------You must have a working gdbm and regex library already installed, and their header files should be in your include path. The Henry Spencer regex.h must come before the system's regex.h in your include path, to make sure you get the right definitions.
The regex library must be compiled in ASCII mode. This is done by invoking c89 as:
c89 -Wc,CONVLIT(ISO8859-1)For ease of use, I put this command into a shell script ("acc" is a good name) and set the environment variable CC = acc before configuring anything. The shell script is simple:
#! /bin/sh exec /bin/c89 -Wc,CONVLIT\(ISO8859-1\) -Wc,LANGLVL\(EXTENDED\) -D__LIBASCII $*Setting LANGLVL to EXTENDED is for the OpenLDAP build, it enables the "long long" data type. This type is only used in liblutil/csn.c. This option is not needed for the gdbm and regex builds.
Defining __LIBASCII activates the definitions for the ASCII-compatibility library.
Note that you must use this script approach, because libtool will strip most of these options and that will give you an unusable build.
The gdbm library should be built in the default EBCDIC mode. The libldbm library has been patched to translate incoming ASCII database names into EBCDIC so the files can be opened properly.
Running Configure -----------------The OpenLDAP configure script will run into problems in several places. You will need these environment variables set, at a minimum:
CC=acc (the shell script mentioned above, it must be on your $PATH) CPP="c89 -E" (all preprocessing is done thru the C compiler driver) LD=c89 (all linking is done thru the C compiler driver) _C89_CCMODE=1 (allows arbitrary order of commandline arguments) CPPFLAGS="-I/usr/local/include -D_ALL_SOURCE" LDFLAGS=-L/usr/local/lib LIBS=-lregex ol_cv_pthread_create=yes ol_cv_pthread_create_works=yes ac_cv_func_getopt=noThe "_ALL_SOURCE" feature macro is used to activate the Pthreads definitions. There may be a more specific macro you could use instead, but I didn't take the time to narrow things down, there may be other needed features as well.
Since the configure script doesn't handle the OS/390 version of Pthreads yet, I used the "ol_cv_pthread_XXX" macros to bypass those parts of the tests. The OS/390 library does provide getopt but the __LIBASCII version crashes so we force the use of the liblutil version instead.
For this build, I only had gdbm and not Berkeley DB. I also did not have SASL, or OpenSSL or Kerberos. I ran configure with --enable-ldbm --disable-bdb.
Note that configure will not detect support for shared libraries. OS/390 does support shared libraries, but libtool doesn't know about it yet. Perhaps in the future this support will be added.
I found no "makedepend" or equivalent tool, so I skipped that step after configure completed.
When configure finishes, you still need to perform some manual edits before you can
run make. In include/portable.h, add these definitions:
#ifdef HAVE_EBCDIC #define fputs lutil_fputs #define fgets lutil_fgets #define printf lutil_printf #define fprintf lutil_fprintf #define vfprintf lutil_vfprintf #define vsprintf lutil_vsprintf #define BOTH_STRINGS_H 1 #define HAVE_PTHREADS_OS390 1 #endifIn libraries/libldap_r/Makefile, add "-DLDAP_PVT_THREAD_STACK_SIZE=0" to AC_DEFS. Otherwise, the size that is used (16M) is too big and pthread_create will always fail with ENOMEM. (You could have added this def to CPPFLAGS before running configure, and avoided this editing step.)
In libraries/liblutil/Makefile, add stdio.c to the UNIX_SRCS macro and add stdio.o
to the UNIX_OBJS macro. Eventually the configure script will be fixed to do this
automatically, but for now you need to do this by hand.
Conclusion ----------I believe that wraps up the necessary tweaks. You should be able to type "make" and get a clean build. Then you can "make test" and the entire test suite should complete without error.
2002-Aug-21: More notes - you should probably add the flag "-Wc,ros" to the acc script as well. This tells the compiler to place string literals into read-only memory. The "-Wc,roc" flag may also be useful, to make const variables read-only. This is mainly an issue when compiling DLL code, which currently isn't supported, so it's not a big deal.
The OS/390 runtime environment allocates stacks in the 24-bit address space by default. This is referred to as memory "below the 16M line." This is the reason why thread creation gets ENOMEM using the default 16M stack size that OpenLDAP uses. Instead of setting LDAP_PVT_THREAD_STACK_SIZE=0 as mentioned above, change the runtime STACK option to allow stack space to be allocated anywhere in the 31-bit address space. This is done by setting the _CEE_RUNOPT environment variable:
_CEE_RUNOPT="STACK(,,ANY)"The back-ldbm backend is not particularly stack-intensive, so things will work fine using LDAP_PVT_THREAD_STACK_SIZE=0, which just tells libldap_r to use the system's default values. However, back-bdb is very stack-intensive, and needs at least 2MB of stack per thread. This will quickly consume all of the memory "below the line" so you must set the runtime option as above if you plan to use back-bdb. (See next Answer for info on building BDB.)
See the OS/390 Language Environment Programming Reference for more details on the STACK option.
It appears that the OpenLDAP default LDAP_PVT_THREAD_STACK_SIZE of 16MB is rather excessive. 2MB is a good starting point, and the stack can still grow incrementally as needed. Also, instead of relying on an environment variable, you can compile in your desired settings using #pragma runopts. This pragma must be specified only once for any given application, and generally goes in the source file where main() is defined. One way or another, it's critical to set the STACK runtime option for slapd and slurpd since they use threads and thus make heavy use of the stack.
|2002-08-30 - Integration with OpenSSL 0.9.6g was successfully completed. See the answer below for details. Some minor patches are needed, and as usual the build process is not entirely automated.
|2002-09-04 - Partial integration with Cyrus SASL 2.1.7. SASL/EXTERNAL works, allowing certificate-based secure authentication. DIGEST-MD5 works with in-directory secrets, has not been tested with sasldb. A significant amount of patching is needed to make this code work, but most of it should be included in Cyrus 2.1.8 whenever that gets released.
2002-09-08 - The configure script and POSIX thread detection has been updated so the build process is a bit smoother now:
CC=acc (the shell script mentioned above, it must be on your $PATH) CPP="c89 -E" (all preprocessing is done thru the C compiler driver) LD=c89 (all linking is done thru the C compiler driver) _C89_CCMODE=1 (allows arbitrary order of commandline arguments) CPPFLAGS="-I/usr/local/include -D_ALL_SOURCE" LDFLAGS=-L/usr/local/lib LIBS="-lregex -lcrypto" MKDEP=c89 MKDEP_FLAGS=-EC _C89_ELINES=1 (needed by the mkdep process)With these settings configure should run without any problems. After you take care of the default thread stack size, you can type "make depend" and then "make" and everything should build properly.
2003-04-24 - If you want to build with XPLINK on the 2.10 or z/OS compiler, you must add "-Wc,xplink,nocsect -Wl,xplink" to the c89 options in the acc script. All of the code that you use must be built with the same XPLINK options for any of it to work - OpenSSL, SASL, regex, BDB, etc.
|Answers in this category:|
Building Berkeley DB 4
Building Cyrus SASL 2
|[New Answer in "Porting to IBM OS/390"]|
|Previous:||Porting to MinGW|
|Next:||Porting to new (unix-like) platforms|