ODF Toolkit/Efforts/Three-Layer OOo

From Apache OpenOffice Wiki
Jump to: navigation, search

After OOo without URE this is the next step in the broader Packaging Modularization project. The goal is to replace the old, monolithic OOo(-derived) products with new, layered ones that share common packages for OOo 3.

  • The lowest layer: the URE. Ideally installs into the directory the existing URE product already installs into.
  • The middle layer: the “OOo-Basis” contains the bulk, brand-independent OOo functionality. Installs into some directory whose name has to be chosen based on technical grounds (i.e., with version numbers to keep different OOo versions apart). Does not contain any client-facing parts (i.e., is only referenced from the top layer).
  • The top layer: brand specific parts. Every product (e.g., “OpenOffice.org”, “BrOffice”, or “StarOffice-Brand”) has its own brand layer, with directory naming as done for prior versions (e.g., /opt/openoffice.org 3.0, /opt/sun/staroffice 9). Multiple brand layers can be installed in parallel and used concurrently (and ideally share commonly installed lower layer packages).
Documentation note.png The below information is no longer totally accurate. It needs updating.

♦ Files that moved from plain OOo (i.e., the new OOo-Basis layer) into the new Brand layer:

  • program/soffice (Unix) resp. program/soffice.exe (Windows)
  • program/soffice.bin (only moved here on Mac)
  • program/unopkg (Unix) resp. program/unopkg.com and program/unopkg.exe (Windows)
  • program/unopkg.bin (only moved here on Mac)
  • program/about.bmp and program/intro.bmp
  • program/bootstraprc (Unix/Mac) resp. program/bootstrap.ini (Windows):
    • ProductKey, ProductPatch, ErrorReportServer, ErrorReportPort, InstallMode, and UserInstallation are potentially brand specific.
    • BaseInstallation=${OOO_BASE_DIR} appears not to be brand specific but is just retained in this file.
  • program/sofficerc (Unix/Mac) resp. program/soffice.ini (Windows):
    • ProgressBarColor, ProgressSize, ProgressPosition, ProgressFrameColor, Logo, and HideEula are potentially brand specific.
    • MacOSXIntegrationUserFile=${$ORIGIN/bootstraprc:UserInstallation}/user/macosxrc.txt and MacOSXIntegrationDefaultFile=${$ORIGIN/bootstraprc:BaseInstallation}/presets/macosxrc.txt appear not to be brand specific but are just retained in this file.
  • program/versionrc (Unix/Mac) resp. program/version.ini (Windows):
    • buildid, ProductPatch, ProductSource, ProductCode, UpgradeCode, ProductMajor, ProductMinor, ProductBuildid, AllLanguages, MsiProductVersion, UpdateURL, UpdateID, UpdateUserAgent, and ExtensionUpdateURL appear to be potentially brand specific. (TODO: are they all? It has to be clarified what exactly their semantics are, and whether they are indeed brand specific, or maybe need to be split in two—like a ProductBuildid for the OOo-Basis layer and the Brand layer each.)
    • OOOBaseVersion is not brand specific but is just retained in this file.
  • share/registry/data/org/openoffice/Office/Common.xcu:
    • Help.Registration.URL is potentially brand specific.
    • View, Menus, Forms, Startup, Save, and Security appear not to be brand specific but are just retained in this file.
  • share/registry/data/org/openoffice/Office/Compatibility.xcu:
    • WriterCompatibilityVersion.OOo11 is potentially brand specific.
    • AllFileFormats appears not to be brand specific but is just retained in this file.
  • share/registry/data/org/openoffice/Office/UI.xcu:
    • ColorScheme.CurrentColorScheme (and thus the name of the single ColorScheme.ColorSchemes group) is potentially brand specific.
    • FilterClassification appears not to be brand specific but is just retained in this file.
  • share/registry/data/org/openoffice/Setup.xcu:
    • Product.ooName, Product.ooSetupVersion, Product.ooSetupVersionAboutBox, Product.ooSetupExtension, Product.ooXMLFileFormatVersion, Product.ooXMLFileFormatName, and Product.ooOpenSourceContext appear to be potentially brand specific.
    • Office, L10N, Configuration, and Migration appear not to be brand specific but are just retained in this file.
  • the complete share/extension (containing install sub-directory) and share/uno_packages (containing cache sub-directory) directories:
    • The share/extension/install directory can contain brand specific Extensions that are deployed shared at installation time.
    • This implies that the shared Extension layer (share/uno_packages) is in the Brand layer, not the OOo-Basis layer. (TODO: a future extension might be to add a third Extension layer in the OOo-Basis layer.)
  • the complete share/xdg directory, containing base.desktop, calc.desktop, draw.desktop, extension.desktop, impress.desktop, math.desktop, printeradmin.desktop, qstart.desktop, writer.desktop
    • math.desktop is not present in StarOffice and StarOffice-based products
    • TODO: in general, how will desktop integration be handled (e.g., there can only be one /usr/bin/soffice symlink)?

Newly added files in the Brand layer:

  • LICENSE and README
  • basis-link
    • On Unix, this is a symbolic link to ../openoffice.orgbasis2.4. TODO: the version number must not be fixed in the .scp file as it is done for now. TODO: instead of having a hard-coded (relative) path installed, the symbolic link could be computed at installation time, based on package information (where are the OOo-Basis packages installed to?).
    • On Windows, this is a plain text file containing ..\OpenOffice.org Basis VERSION, where VERSION is replaced with 2.4 in code>desktop/scripts/makefile.mk:1.11.184.1. TODO: the version number must not be fixed in the makefile.mk as it is done for now.
    • On Mac, this is a directory containing the OOo-Basis part (which in turn contains ure-link as a directory containing the URE; i.e., on Mac for now product installation sets are self contained and do not share common parts).
  • program/fundamentalrc (Unix/Mac) resp. program/fundamental.ini (Windows):
    • BRAND_BASE_DIR=$ORIGIN/.. points at the Brand layer base directory (it is used to locate UNO_SHARED_PACKAGES=$BRAND_BASE_DIR/share/uno_packages and UNO_USER_PACKAGES=${$BRAND_BASE_DIR/program/bootstraprc:UserInstallation}/user/uno_packages in the OOo-Basis layer program/unorc, and to locate $BRAND_BASE_DIR/program/bootstraprc and $BRAND_BASE_DIR/program/versionrc in the OOo-Basis layer program/configmgrrc).
    • CustomDataUrl=${BRAND_BASE_DIR} activates the Brand layer share/registry as an additional configuration layer.
    • OOO_BASE_DIR=${BRAND_BASE_DIR}/basis-link (Unix/Mac) resp. OOO_BASE_DIR=${.link:${BRAND_BASE_DIR}/basis-link} (Windows) fixes the OOO_BASE_DIR already introduced in OOo without URE. See below for the ${.link:...} hack.
    • UNO_SHARED_PACKAGES_CACHE, UNO_USER_PACKAGES_CACHE, URE_MORE_TYPES, URE_MORE_SERVICES, URE_MORE_JAVA_TYPES, URE_MORE_JAVA_CLASSPATH_URLS, and URE_BIN_DIR forward to program/fundamentalbasisrc (Unix/Mac) resp. program/fundamentalbasis.ini (Windows), an adapted version of the fundamentalrc/fundamental.ini already introduced in OOo without URE.
  • program/redirectrc (Unix) resp. program/redirect.ini (Windows):
    • URE_BOOTSTRAP=${ORIGIN}/fundamentalrc (Unix) resp. URE_BOOTSTRAP=${ORIGIN}/fundamental.ini (Windows) used as the UNO ini-file (via -env:INIFILEPATH command line argument) of Basis layer javaldx (Unix) and unopkg.bin.

♦ The executables that make up the client interface of a (branded) office suite need to be in the Brand layer:

  • They are expected at well-defined places so that clients can interact with them.
  • They are potentially brand specific (e.g., on Windows they will contain brand specific application icons).

TODO: it is unclear what executables exactly make up the interface, see The interface of OOo; for now, I just take soffice and unopkg.

♦Places in the code have been identified where “look next to me” mechanisms had to be replaced with mechanisms to explicitly address the OOo-Basis or the Brand layer:

  • In framework::SubstitutePathVariables, compute $(prog)$ as $(inst)/program (making both $(inst) and $(prog) point into the OOo-Basis layer).
  • In module shell ShellExec, the URL launchers are now looked for in the $OOO_BASE_DIR/program directory, instead of next to the executable.
  • tools/source/rc/resmgr.cxx:1.49.8.1 ResMgrContainer::init now looks for resource files in both $BRAND_BASE_DIR/program/resource and $OOO_BASE_DIR/program/resource.
  • In many places, uses of $SYSBINDIR, $ORIGIN, and osl_getExecutableFile etc. had to be replaced with explicit uses of $OOO_BASE_DIR or $BRAND_BASE_DIR.
  • TODO: Some code in vcl extracts window icons directly from the executable, which has to be changed, see Issue 85730 .

♦ What shall be the paths where the tree layers are installed, and is it necessary to encode any version numbers in those paths?

  • URE only ever changes compatibly, so no version number is required in its path (i.e., multiple applications can share a single URE installation, as long as its minor version number is at least as large as the largest one required by any of the applications).
  • There is no such compatibility guarantee for the interface between the OOo-Basis layer and the Brand layer:
    • Especially with various binary executables (like soffice.bin) in the Brand layer, controlling a compatible interface between the two layers seems extremely hard and error prone.
    • When various Brand layers (of different minor versions) are installed in parallel on top of a single OOo-Basis layer (which needs to be at least of the highest required minor version), it might be unexpected by clients that Brand products of lower minor version nevertheless much behave like products of a higher minor version (e.g., suddenly offer features at the UI that were only introduced in later OOo minors).
    • The above points suggest that the major.minor OOo version should be encoded in the OOo-Brand layer path. However, that has the drawback that a software artefact used by a client to update an installed branded office suite probably cannot be kept small (by only including data that has really changed), as the paths of all files in the OOo-Brand layer change (even if their content does not change).
    • TODO: A related problem is that for example a StarOffice-Brand layer distributed by Sun will not work on top of an OOo-Basis layer (and maybe not even on top of a URE layer) provided by some Linux distribution, as they have been built from (nominally compatible) OOo sources using different configure switches that potentially break compatibility at the interfaces among the three layers. See also additional comments from cmc to issue 97054.
  • Encoding any version numbers in the Brand layer paths is only necessary if different versions of the same brand shall be installable in parallel, or there shall be small software artefacts to update between different versions (see above). It is a responsibility of the respective brand owners to define policies here. (TODO: define such a policy at least for OpenOffice.org itself.)
  • For now, I use the paths:
    • /opt/openoffice.org/ure, /opt/openoffice.orgbasis2.4, and /opt/openoffice.org2.4 (and /opt/staroffice8) on Unix;
    • Program Files\OpenOffice.org\URE, Program Files\OpenOffice.org Basis 2.4, and Program Files\OpenOffice.org 2.4 (and Program Files\StarOffice 8) on Windows.
    • As described above, on Mac the URE and Basis parts are contained within the branded product, anyway (so this is irrelevant there for now).

♦ After dropping support for Windows 98/ME, delayloading of uwinapi.dll is not needed at all any longer (the executables in the brand layer must not link against uwinapi.dll).

♦ On Windows, the loaders add both ..\→basis-link\program and ..\→basis-link\→ure-link\bin to the front of PATH, failing catastrophically (with a MessageBox based on GetLastError) if the links do not work. TODO: to avoid failure in plain OOo, a fake (.) basis-link and a (non-functional) ure-link are added to plain OOo for now.

♦ Completely removed deprecated pkgchk tool; use unopkg instead.

♦ Start scripts (desktop/scripts/soffice.sh:1.28.34.3, desktop/scripts/soffice_lean.sh:1.9.34.3, desktop/scripts/unopkg.sh:1.6.34.2) look for javaldx in $sd_prog (TODO: to be removed once there is no plain OOo any more) and $sd_prog/../basis-link/ure-link/bin, and for pagein in $sd_prog/../basis-link/program (which also works in plain OOo as a trivial basis-link is also included there for now).

♦ For the APPnRPATH/SHLnRPATH mechanism, added BRAND case ($ORIGIN:$ORIGIN/../basis-link/program:$ORIGIN/../basis-link/ure-link/lib).

♦ To reduce differences among brands, include PyUNO in StarOffice and StarOffice-based products.

♦ New rtl_convertStringToUString converts a byte string to a Unicode string, signalling failure (analoguous to existing rtl_convertUStringToString). Unit tests in sal/qa/rtl/oustring/rtl_OUString2.cxx:1.13.8.1.

♦ Added internal ${.link:...} hack to rtl/bootstrap.h macro expansion to simulate symbolic links under Windows. Unit tests in sal/qa/rtl/bootstrap/rtl_Bootstrap.cxx:1.7.10.2.

♦ In .scp files, after preprocessing, adjacent string literals within a line are merged (analoguous to how C and C++ treat adjacent string literals).

♦ The Linux and Solaris user install scripts now support an additional -a switch to add products into existing trees (so that products for all three layers can be installed into a common, non-default root).

♦ Compared to traditional OOo, the locations where information about a specific JRE to use are stored changed from per–office-installation (optional .../share/config and UserInstallation/user/config) to per–URE-installation (optional /etc/opt/ure and ~/.ure, where multiple Brand layers would share a single set of settings). This caused problems, see issue 84985 and has been changed back to the traditional behavior:

  • URE has been extended with URE_OVERRIDE_JAVA_JFW_SHARED_DATA and URE_OVERRIDE_JAVA_JFW_USER_DATA deployment variables, which, if set, override the default shared/per-user locations.
  • In fundamentalbasisrc/fundamentalrc, URE_OVERRIDE_JAVA_JFW_SHARED_DATA is set to a potential Brand layer share/config entry (i.e., shared settings would be per–Brand-layer), and URE_OVERRIDE_JAVA_JFW_USER_DATA is set to a UserInstallation/user/config entry.
  • So that javaldx called from the start scripts behaves correctly, a program/javaldxrc is added to the Brand layer (on those platforms that actually use javaldx), containing a definition of URE_BOOTSTRAP and thus linking to the fundamentalbasisrc/fundamentalrc settings.
  • The sal bootstrap code is extended with an internal feature to allow INIFILEPATH to contain a system path to the ini-file, similar to how INIFILENAME can contain the URL of the ini-file. This is used to pass the system path of the javaldxrc to javaldx (as computing a URL from a system path is non-trivial).

♦ On Mac, a mechanism was still missing how executables and dynamic libraries in one layer can access dynamic libraries in another layer. This has now been fixed.

  • Building on the existing SHLnRPATH stuff, the install name of every dynamic library being built is set to a “@” followed by fifty “_” followed by the expansion of the relevant SHLnRPATH (i.e., either URELIB or OOO). The resulting very long name containing fifty underscores is needed so that some install_name_tool -change call later on does not complain that “larger updated load commands do not fit.” (TODO: ld feature to leave room in generated libraries, so that the very long name hack is not needed?)
  • Then, building an executable or dynamic library:
    • a re-written solenv/bin/macosx-dylib-link-list.pl scans the preliminary linker command line for any -L and -l switches and adds to the final linker command line the necessary -dylib_file arguments for the transitive hull of all (indirectly) included dynamic libraries;
    • a new solenv/bin/macosx-change-install-names.pl post-processes the generated executable or dynamic library, replacing the recorded install names of linked-against dynamic libraries with install names that will work in the final, installed products. To do so, it uses three pieces of information for every changed install name: whether the entity being post-processed is an executable (app) or a dynamic library (shl), the APP/SHLnRPATH of the entity being post-processed (UREBIN, URELIB, OOO, or BRAND), and the SHLnRPATH of the linked-against dynamic library (recoverd from the recorded install name, URELIB or OOO). This gives a matrix of rules how to rewrite the path prefixes of install names (missing entries are invalid):
      • app/UREBIN/URELIB → @executable_path/../lib
      • app/OOO/URELIB → @executable_path/../ure-link/lib
      • app/OOO/OOO → @executable_path
      • app/BRAND/URELIB → @executable_path/../basis-link/ure-link/lib
      • app/BRAND/OOO → @executable_path/../basis-link/program
      • shl/URELIB/URELIB → @loader_path
      • shl/OOO/URELIB → @loader_path/../ure-link/lib
      • shl/OOO/OOO → @loader_path
    • (The good old monolithic OOo is kept alive for now by adding a fake ure-link/lib → ../program symbolic link to its installation set.)
  • Mac OS X 10.5 reportedly also supports an RPATH similar to ELF, but using it instead of fiddling with the install names would violate the Mac OS X 10.4 baseline.
  • Using install names this way complicates external libraries (that are built with their own mechanisms):
    • For STLport, this is circumvented by making --without-stlport the default for Mac OS X (TODO: and probably completely removing --with-stlport for that platform), see also CWSs newportstl and stl4leopardppc.
    • Similarly, libxml2 and libxslt are always taken from the system. TODO: are the system versions good enough? TODO: adapt configure.
    • Calls to solenv/bin/macosx-dylib-link-list.pl had to be injected into external module makefiles where applicable.
    • solenv/inc/tg_ext.mk:1.81.4.2 calls solenv/bin/macosx-change-install-names.pl to adapt their install names in a post-process step, using a new EXTRPATH (analogous to APP/SHLnRPATH; set to a default of OOO for all modules; TODO: needed at all?).
Retrieved from "https://wiki.openoffice.org/w/index.php?title=ODF_Toolkit/Efforts/Three-Layer_OOo&oldid=250194"
Personal tools