Configurations of OpenSSL target platforms ========================================== Target configurations are a collection of facts that we know about different platforms and their capabilities. We organise them in a hash table, where each entry represent a specific target. In each table entry, the following keys are significant: inherit_from => Other targets to inherit values from. Explained further below. [1] template => Set to 1 if this isn't really a platform target. Instead, this target is a template upon which other targets can be built. Explained further below. [1] sys_id => System identity for systems where that is difficult to determine automatically. cc => The compiler command, usually one of "cc", "gcc" or "clang". This command is normally also used to link object files and libraries into the final program. cflags => Flags that are used at all times when compiling. defines => As an alternative, macro definitions may be present here instead of in `cflags'. If given here, they MUST be as an array of the string such as "MACRO=value", or just "MACRO" for definitions without value. shared_cflag => Extra compilation flags used when compiling for shared libraries, typically something like "-fPIC". (linking is a complex thing, see [3] below) ld => Linker command, usually not defined (meaning the compiler command is used instead). (NOTE: this is here for future use, it's not implemented yet) lflags => Flags that are used when linking apps. shared_ldflag => Flags that are used when linking shared or dynamic libraries. plib_lflags => Extra linking flags to appear just before the libraries on the command line. ex_libs => Extra libraries that are needed when linking. ar => The library archive command, the default is "ar". (NOTE: this is here for future use, it's not implemented yet) arflags => Flags to be used with the library archive command. ranlib => The library archive indexing command, the default is 'ranlib' it it exists. unistd => An alternative header to the typical ''. This is very rarely needed. shared_extension => File name extension used for shared libraries. obj_extension => File name extension used for object files. On unix, this defaults to ".o" (NOTE: this is here for future use, it's not implemented yet) exe_extension => File name extension used for executable files. On unix, this defaults to "" (NOTE: this is here for future use, it's not implemented yet) thread_scheme => The type of threads is used on the configured platform. Currently known values are "(unknown)", "pthreads", "uithreads" (a.k.a solaris threads) and "winthreads". Except for "(unknown)", the actual value is currently ignored but may be used in the future. See further notes below [2]. dso_scheme => The type of dynamic shared objects to build for. This mostly comes into play with engines, but can be used for other purposes as well. Valid values are "DLFCN" (dlopen() et al), "DLFCN_NO_H" (for systems that use dlopen() et al but do not have fcntl.h), "DL" (shl_load() et al), "WIN32" and "VMS". perlasm_scheme => The perlasm method used to created the assembler files used when compiling with assembler implementations. shared_target => The shared library building method used. This is a target found in Makefile.shared. build_scheme => The scheme used to build up a Makefile. In its simplest form, the value is a string with the name of the build scheme. The value may also take the form of a list of strings, if the build_scheme is to have some options. In this case, the first string in the list is the name of the build scheme. Currently recognised build scheme is "unified". For the "unified" build scheme, this item *must* be an array with the first being the word "unified" and the second being a word to identify the platform family. multilib => On systems that support having multiple implementations of a library (typically a 32-bit and a 64-bit variant), this is used to have the different variants in different directories. bn_ops => Building options (was just bignum options in the earlier history of this option, hence the name). This a string of words that describe properties on the designated target platform, such as the type of integers used to build up the bitnum, different ways to implement certain ciphers and so on. To fully comprehend the meaning, the best is to read the affected source. The valid words are: BN_LLONG use 'unsigned long long' in some bignum calculations. This has no value when SIXTY_FOUR_BIT or SIXTY_FOUR_BIT_LONG is given. RC4_CHAR makes the basic RC4 unit of calculation an unsigned char. SIXTY_FOUR_BIT processor registers are 64 bits, long is 32 bits, long long is 64 bits. SIXTY_FOUR_BIT_LONG processor registers are 64 bits, long is 64 bits. THIRTY_TWO_BIT processor registers are 32 bits. EXPORT_VAR_AS_FN for shared libraries, export vars as accessor functions. apps_extra_src => Extra source to build apps/openssl, as needed by the target. cpuid_asm_src => assembler implementation of cpuid code as well as OPENSSL_cleanse(). Default to mem_clr.c bn_asm_src => Assembler implementation of core bignum functions. Defaults to bn_asm.c ec_asm_src => Assembler implementation of core EC functions. des_asm_src => Assembler implementation of core DES encryption functions. Defaults to 'des_enc.c fcrypt_b.c' aes_asm_src => Assembler implementation of core AES functions. Defaults to 'aes_core.c aes_cbc.c' bf_asm_src => Assembler implementation of core BlowFish functions. Defaults to 'bf_enc.c' md5_asm_src => Assembler implementation of core MD5 functions. sha1_asm_src => Assembler implementation of core SHA1, functions, and also possibly SHA256 and SHA512 ones. cast_asm_src => Assembler implementation of core CAST functions. Defaults to 'c_enc.c' rc4_asm_src => Assembler implementation of core RC4 functions. Defaults to 'rc4_enc.c rc4_skey.c' rmd160_asm_src => Assembler implementation of core RMD160 functions. rc5_asm_src => Assembler implementation of core RC5 functions. Defaults to 'rc5_enc.c' wp_asm_src => Assembler implementation of core WHIRLPOOL functions. cmll_asm_src => Assembler implementation of core CAMELLIA functions. Defaults to 'camellia.c cmll_misc.c cmll_cbc.c' modes_asm_src => Assembler implementation of cipher modes, currently the functions gcm_gmult_4bit and gcm_ghash_4bit. padlock_asm_src => Assembler implementation of core parts of the padlock engine. This is mandatory on any platform where the padlock engine might actually be built. [1] as part of the target configuration, one can have a key called 'inherit_from' that indicate what other configurations to inherit data from. These are resolved recursively. Inheritance works as a set of default values that can be overridden by corresponding key values in the inheriting configuration. Note 1: any configuration table can be used as a template. Note 2: pure templates have the attribute 'template => 1' and cannot be used as build targets. If several configurations are given in the 'inherit_from' array, the values of same attribute are concatenated with space separation. With this, it's possible to have several smaller templates for different configuration aspects that can be combined into a complete configuration. instead of a scalar value or an array, a value can be a code block of the form 'sub { /* your code here */ }'. This code block will be called with the list of inherited values for that key as arguments. In fact, the concatenation of strings is really done by using 'sub { join(" ",@_) }' on the list of inherited values. An example: "foo" => { template => 1, haha => "ha ha", hoho => "ho", ignored => "This should not appear in the end result", }, "bar" => { template => 1, haha => "ah", hoho => "haho", hehe => "hehe" }, "laughter" => { inherit_from => [ "foo", "bar" ], hehe => sub { join(" ",(@_,"!!!")) }, ignored => "", } The entry for "laughter" will become as follows after processing: "laughter" => { haha => "ha ha ah", hoho => "ho haho", hehe => "hehe !!!", ignored => "" } [2] OpenSSL is built with threading capabilities unless the user specifies 'no-threads'. The value of the key 'thread_scheme' may be "(unknown)", in which case the user MUST give some compilation flags to Configure. [3] OpenSSL has three types of things to link from object files or static libraries: - shared libraries; that would be libcrypto and libssl. - shared objects (sometimes called dynamic libraries); that would be the engines. - applications; those are apps/openssl and all the test apps. Very roughly speaking, linking is done like this (words in braces represent the configuration settings documented at the beginning of this file): shared libraries: {ld} $(CFLAGS) {shared_ldflag} -shared -o libfoo.so \ -Wl,--whole-archive libfoo.a -Wl,--no-whole-archive \ {plib_lflags} -lcrypto {ex_libs} shared objects: {ld} $(CFLAGS) {shared_ldflag} -shared -o libeng.so \ blah1.o blah2.o {plib_lflags} -lcrypto {ex_libs} applications: {ld} $(CFLAGS) {lflags} -o app \ app1.o utils.o {plib_lflags} -lssl -lcrypto {ex_libs} Historically, the target configurations came in form of a string with values separated by colons. This use is deprecated. The string form looked like this: "target" => "{cc}:{cflags}:{unistd}:{thread_cflag}:{sys_id}:{lflags}:{bn_ops}:{cpuid_obj}:{bn_obj}:{ec_obj}:{des_obj}:{aes_obj}:{bf_obj}:{md5_obj}:{sha1_obj}:{cast_obj}:{rc4_obj}:{rmd160_obj}:{rc5_obj}:{wp_obj}:{cmll_obj}:{modes_obj}:{padlock_obj}:{perlasm_scheme}:{dso_scheme}:{shared_target}:{shared_cflag}:{shared_ldflag}:{shared_extension}:{ranlib}:{arflags}:{multilib}" Build info files ================ The build.info files that are spread over the source tree contain the minimum information needed to build and distribute OpenSSL. It uses a simple and yet fairly powerful language to determine what needs to be built, from what sources, and other relationships between files. For every build.info file, all file references are relative to the directory of the build.info file for source files, and the corresponding build directory for built files if the build tree differs from the source tree. When processed, every line is processed with the perl module Text::Template, using the delimiters "{-" and "-}". The hashes %config and %target are passed to the perl fragments, along with $sourcedir and $builddir, which are the locations of the source directory for the current build.info file and the corresponding build directory, all relative to the top of the build tree. To begin with, things to be built are declared by setting specific variables: PROGRAMS=foo bar LIBS=libsomething ENGINES=libeng SCRIPTS=myhack EXTRA=file1 file2 Note that the files mentioned for PROGRAMS, LIBS and ENGINES *must* be without extensions. The build file templates will figure them out. For each thing to be built, it is then possible to say what sources they are built from: PROGRAMS=foo bar SOURCE[foo]=foo.c common.c SOURCE[bar]=bar.c extra.c common.c It's also possible to tell some other dependencies: DEPEND[foo]=libsomething DEPEND[libbar]=libsomethingelse (it could be argued that 'libsomething' and 'libsomethingelse' are source as well. However, the files given through SOURCE are expected to be located in the source tree while files given through DEPEND are expected to be located in the build tree) For some libraries, we maintain files with public symbols and their slot in a transfer vector (important on some platforms). It can be declared like this: ORDINALS[libcrypto]=crypto The value is not the name of the file in question, but rather the argument to util/mkdef.pl that indicates which file to use. One some platforms, shared libraries come with a name that's different from their static counterpart. That's declared as follows: SHARED_NAME[libfoo]=cygfoo-{- $config{shlibver} -} The example is from Cygwin, which has a required naming convention. Sometimes, it makes sense to rename an output file, for example a library: RENAME[libfoo]=libbar That lines has "libfoo" get renamed to "libbar". While it makes no sense at all to just have a rename like that (why not just use "libbar" everywhere?), it does make sense when it can be used conditionally. See a little further below for an example. In some cases, it's desirable to include some source files in the shared form of a library only: SHARED_SOURCE[libfoo]=dllmain.c For any file to be built, it's also possible to tell what extra include paths the build of their source files should use: INCLUDE[foo]=include In some cases, one might want to generate some source files from others, that's done as follows: GENERATE[foo.s]=asm/something.pl $(CFLAGS) GENERATE[bar.s]=asm/bar.S The value of each GENERATE line is a command line or part of it. Configure places no rules on the command line, except the the first item muct be the generator file. It is, however, entirely up to the build file template to define exactly how those command lines should be handled, how the output is captured and so on. Sometimes, the generator file itself depends on other files, for example if it is a perl script that depends on other perl modules. This can be expressed using DEPEND like this: DEPEND[asm/something.pl]=../perlasm/Foo.pm There may also be cases where the exact file isn't easily specified, but an inclusion directory still needs to be specified. INCLUDE can be used in that case: INCLUDE[asm/something.pl]=../perlasm NOTE: GENERATE lines are limited to one command only per GENERATE. As a last resort, it's possible to have raw build file lines, between BEGINRAW and ENDRAW lines as follows: BEGINRAW[Makefile(unix)] haha.h: {- $builddir -}/Makefile echo "/* haha */" > haha.h ENDRAW[Makefile(unix)] The word withing square brackets is the build_file configuration item or the build_file configuration item followed by the second word in the build_scheme configuration item for the configured target within parenthesis as shown above. For example, with the following relevant configuration items: build_file => "build.ninja" build_scheme => [ "unified", "unix" ] ... these lines will be considered: BEGINRAW[build.ninja] build haha.h: echo "/* haha */" > haha.h ENDRAW[build.ninja] BEGINRAW[build.ninja(unix)] build hoho.h: echo "/* hoho */" > hoho.h ENDRAW[build.ninja(unix)] Should it be needed because the recipes within a RAW section might clash with those generated by Configure, it's possible to tell it not to generate them with the use of OVERRIDES, for example: SOURCE[libfoo]=foo.c bar.c OVERRIDES=bar.o BEGINRAW[Makefile(unix)] bar.o: bar.c $(CC) $(CFLAGS) -DSPECIAL -c -o $@ $< ENDRAW[Makefile(unix)] See the documentation further up for more information on configuration items. Finally, you can have some simple conditional use of the build.info information, looking like this: IF[1] something ELSIF[2] something other ELSE something else ENDIF The expression in square brackets is interpreted as a string in perl, and will be seen as true if perl thinks it is, otherwise false. For example, the above would have "something" used, since 1 is true. Together with the use of Text::Template, this can be used as conditions based on something in the passed variables, for example: IF[{- $disabled{shared} -}] LIBS=libcrypto SOURCE[libcrypto]=... ELSE LIBS=libfoo SOURCE[libfoo]=... ENDIF or: # VMS has a cultural standard where all libraries are prefixed. # For OpenSSL, the choice is 'ossl_' IF[{- $config{target} =~ /^vms/ -}] RENAME[libcrypto]=ossl_libcrypto RENAME[libssl]=ossl_libssl ENDIF Build-file programming with the "unified" build system ====================================================== "Build files" are called "Makefile" on Unix-like operating systems, "descrip.mms" for MMS on VMS, "makefile" for nmake on Windows, etc. To use the "unified" build system, the target configuration needs to set the three items 'build_scheme', 'build_file' and 'build_command'. In the rest of this section, we will assume that 'build_scheme' is set to "unified" (see the configurations documentation above for the details). For any name given by 'build_file', the "unified" system expects a template file in Configurations/ named like the build file, with ".tmpl" appended, or in case of possible ambiguity, a combination of the second 'build_scheme' list item and the 'build_file' name. For example, if 'build_file' is set to "Makefile", the template could be Configurations/Makefile.tmpl or Configurations/unix-Makefile.tmpl. In case both Configurations/unix-Makefile.tmpl and Configurations/Makefile.tmpl are present, the former takes precedence. The build-file template is processed with the perl module Text::Template, using "{-" and "-}" as delimiters that enclose the perl code fragments that generate configuration-dependent content. Those perl fragments have access to all the hash variables from configdata.pem. The build-file template is expected to define at least the following perl functions in a perl code fragment enclosed with "{-" and "-}". They are all expected to return a string with the lines they produce. generatesrc - function that produces build file lines to generate a source file from some input. It's called like this: generatesrc(src => "PATH/TO/tobegenerated", generator => [ "generatingfile", ... ] generator_incs => [ "INCL/PATH", ... ] generator_deps => [ "dep1", ... ] generator => [ "generatingfile", ... ] incs => [ "INCL/PATH", ... ], deps => [ "dep1", ... ], intent => one of "libs", "dso", "bin" ); 'src' has the name of the file to be generated. 'generator' is the command or part of command to generate the file, of which the first item is expected to be the file to generate from. generatesrc() is expected to analyse and figure out exactly how to apply that file and how to capture the result. 'generator_incs' and 'generator_deps' are include directories and files that the generator file itself depends on. 'incs' and 'deps' are include directories and files that are used if $(CC) is used as an intermediary step when generating the end product (the file indicated by 'src'). 'intent' indicates what the generated file is going to be used for. src2obj - function that produces build file lines to build an object file from source files and associated data. It's called like this: src2obj(obj => "PATH/TO/objectfile", srcs => [ "PATH/TO/sourcefile", ... ], deps => [ "dep1", ... ], incs => [ "INCL/PATH", ... ] intent => one of "lib", "dso", "bin" ); 'obj' has the intended object file *without* extension, src2obj() is expected to add that. 'srcs' has the list of source files to build the object file, with the first item being the source file that directly corresponds to the object file. 'deps' is a list of explicit dependencies. 'incs' is a list of include file directories. Finally, 'intent' indicates what this object file is going to be used for. obj2lib - function that produces build file lines to build a static library file ("libfoo.a" in Unix terms) from object files. called like this: obj2lib(lib => "PATH/TO/libfile", objs => [ "PATH/TO/objectfile", ... ]); 'lib' has the intended library file name *without* extension, obj2lib is expected to add that. 'objs' has the list of object files (also *without* extension) to build this library. libobj2shlib - function that produces build file lines to build a shareable object library file ("libfoo.so" in Unix terms) from the corresponding static library file or object files. called like this: libobj2shlib(shlib => "PATH/TO/shlibfile", lib => "PATH/TO/libfile", objs => [ "PATH/TO/objectfile", ... ], deps => [ "PATH/TO/otherlibfile", ... ], ordinals => [ "word", "/PATH/TO/ordfile" ]); 'lib' has the intended library file name *without* extension, libobj2shlib is expected to add that. 'shlib' has the corresponding shared library name *without* extension. 'deps' has the list of other libraries (also *without* extension) this library needs to be linked with. 'objs' has the list of object files (also *without* extension) to build this library. 'ordinals' MAY be present, and when it is, its value is an array where the word is "crypto" or "ssl" and the file is one of the ordinal files util/libeay.num or util/ssleay.num in the source directory. This function has a choice; it can use the corresponding static library as input to make the shared library, or the list of object files. obj2dso - function that produces build file lines to build a dynamic shared object file from object files. called like this: obj2dso(lib => "PATH/TO/libfile", objs => [ "PATH/TO/objectfile", ... ], deps => [ "PATH/TO/otherlibfile", ... ]); This is almost the same as libobj2shlib, but the intent is to build a shareable library that can be loaded in runtime (a "plugin"...). The differences are subtle, one of the most visible ones is that the resulting shareable library is produced from object files only. obj2bin - function that produces build file lines to build an executable file from object files. called like this: obj2bin(bin => "PATH/TO/binfile", objs => [ "PATH/TO/objectfile", ... ], deps => [ "PATH/TO/libfile", ... ]); 'bin' has the intended executable file name *without* extension, obj2bin is expected to add that. 'objs' has the list of object files (also *without* extension) to build this library. 'deps' has the list of library files (also *without* extension) that the programs needs to be linked with. in2script - function that produces build file lines to build a script file from some input. called like this: in2script(script => "PATH/TO/scriptfile", sources => [ "PATH/TO/infile", ... ]); 'script' has the intended script file name. 'sources' has the list of source files to build the resulting script from. In all cases, file file paths are relative to the build tree top, and the build file actions run with the build tree top as current working directory. Make sure to end the section with these functions with a string that you thing is appropriate for the resulting build file. If nothing else, end it like this: ""; # Make sure no lingering values end up in the Makefile -}