Difference between revisions of "Build Bootstrapping"

Revision as of 15:37, 16 March 2012

Before you can build Apache OpenOffice you have to set up the so called build environment. One part of this is to install prerequisite software like compiler and certain libraries. The second part, the one that is described here, takes care of platform detection, feature selection, and download of missing files.

A hands on building guide can be found here. This page focuses on what is going on behind the scenes and addresses developers that want to improve the build bootstrapping.

Autoconf

Both the platform detection as well as the feature selection is handled by the autoconf package. However, the related automake is not used. AOO uses its own set of dmake and GNU makefiles. Autoconf is a prerequisite and not part of AOO.

autoconf translates the main/configure.in template, written in the M4 macro language, into the main/configure shell script.

Configure

The configure script serves two purposes:

• Detect OS and platform specific programs, tools, and features. Most of this is done automatically. In some cases, especially on Windows (keep in mind that autoconf has been designed for Unix like operating systems), configure needs help. For example the home of the windows compiler has to be specified via the --with-cl-home=<path> switch.
• Let the developer choose to activate or deactivate optional features. One example is the --with-lang="<white space separated list of language codes>" switch that controls the set of supported languages.

All this results in environment variables whose definitions are written to the main/set_soenv perl script. This is created by the line near the end in configure.in (you run configure but you edit configure.in)

   AC_CONFIG_FILES([set_soenv Makefile])

from the main/set_soenv.in template.

configure then runs main/set_seoenv script in order to

• write a platform and operating system specific (tcsh) shell script file that contains all the relevant definitions of environment variables. The name of the script depends on platform and os, like LinuxX86Env.Set for unxlngi6 or winenv.set for wntmsci12. For bash scripts a '.sh' suffix is appended.

• translate the main/bootstrap.1 template into the main/bootstrap bash script file.

Bootstrap

The main/bootstrap script has to be executed manually after main/configure has finished. It chains main/fetch_tarballs.sh that reads main/ooo.lst (its file format is described below) and loads the missing external libraries to ext_libraries/. The category A licensed libraries are part of the source tree and therefore should be already present. The category B licensed libraries are not included for legal reasons and have to be downloaded.

When the respective switches have been given to configure then main/fetch_tarballs.sh also loads the source code packages for dmake and epm. After main/fetch_tarballs.sh returns main/bootstrap checks if the dmake executable already exists somewhere in PATH. If not the dmake.executable is built and installed into main/solver/340/<platform>/bin.

Extensions

Finally, bootstrap chains solenv/bin/download_missing_extensions.pl that determines the set of extensions to include into the installation set and downloads the missing files.

The content of main/extensions.lst is read and interpreted at two times:

1. Once at the end of the bootstrap process. At this time the extensions, that are not part of the AOO source code, are downloaded from their home servers to xt_sources/ (when they are not already present, e.g. from a previous run of main/bootstrap.) Local extensions (those with file:// protocol) are ignored at this stage.

   The language selectors in main/extensions.lst are ignored at this time and evaluate to true. That means that extensions are only rejected at this time, ie not downloaded, when another condition is not met.

2. The second time is when the installation sets are created. Then the languages are taken into account. Only extensions that match the installation language(s) are included into installation sets.

Note that the

   --with-bundled-extension-blobs

configure switch overrides the set of remote extensions defined by main/extensions.lst.

Extensions are treated differently depending on their protocol in main/extensions.lst:

http

Remote extensions are included as blob, without being unpacked. They are installed on first office start. This has legal reasons.

file

Locally built extensions are unpacked and pre-registered. Their installation is finished on first office start.

File formats

ooo.lst

The file format of main/ooo.lst is line based:

• Comments start with # and go to the end of the line.
• Lines that contain only white space and comments are ignored.
• Lines that contain a URL that ends with '/' are interpreted as prefixes. It is added to all following lines that do not contain a URL protocol (like protocol://) until the next URL prefix is found.
• URLs whose base name starts with an MD5 checksum are verified against that checksum after they are downloaded.

extensions.lst

The file format of main/extensions.lst is line based:

• Comments start with # and go to the end of the line.
• Lines that contain only white space and comments are ignored.
• Other lines contain either a selector or the URL of an extension.

Selectors

Selectors contain simple logical expressions enclosed in brackets. If the expression evaluates to true then the following URLs are included. Otherwise they are ignored.

• A logical expression can be empty (which always evaluates to true) or can contain an arbitrary number of terms that are separated by the logical operators || or &&. No parentheses are allowed at the moment.
• A term has the form <left-hand-side> <operator> <right-hand-side>.

The left hand side can have three forms:

• The keyword "language" which, during installation is replaced by the current target language.
• The keyword "platform" is replaced with the platform id (strings defined in main/set_seoenv, e.g. "wntmsci12" or "unxlngi6").
• Words that consist only of uppercase letters, digits, and "_" are replaced with the value of the corresponding environment value.

There is basically only one supported operator but it can be written as "=", "==", or "eq". Whatever feels best. All of them are mapped to perl's "=~".

The right hand side can be any regular expression that is accepted by perl.

Examples of selectors:

[ ], [ language=all ], [ platform=all] are always true.

[ language=de ] is true only when a German installation set is build. de-AT or de-CH do not match.

[ language=de.* ] matches any variant of German (de, de-AT, de-CH)

[ language=de-.* ] matches the German variants (de-AT, de-CH, but not de)

URLs

A URL can have the protocols file:// for extensions whose source code is in the SVN repository. Their path is relative to the path into which extensions are delivered (main/solver/340/<platform>/bin/).

Use http:// protocol for external extensions. These are downloaded during the bootstrap process. They can (and should) be prefixed by MD5 checksums. It is an error if the file referenced by the URL does not match the checksum.

Example

A file like this

[ language=all && ENABLE_PRESENTER_SCREEN=YES ]
    file://presenter/presenter-screen.oxt

[ language=en.* ]
   b7ce02d25eb302e5b23572cdccaea461 http://numbertext.org/tmp/dict-en.oxt

[ language=de || language=de-DE ]
   eb3d3397b8034c7fce6e0d78daf914ca http://extensions.services.openoffice.org/e-files/1075/10/dict-de_DE-frami_2011-06-03.

would add the local presenter console extension for all languages (but only when the --with-presenter-console switch was given to configure), the dict-en.oxt for all English variants, and dict-de_DE-frami_2011-06-03 for de or de-DE.