Difference between revisions of "ODF Toolkit/Efforts/OOo without URE"

Revision as of 14:47, 27 April 2007

This is one part of the broader Packaging Modularization project. The goal is to have a special OOo product (“OOo-wo-URE”) that behaves like the normal OOo product but re-uses the existing URE product. That is, installing OOo-wo-URE will require URE to be installed as a prerequisite, and will use the functionality of the installed URE instead of duplicating it as the normal OOo does. For now, OOo-wo-URE is just a special additional product, but if things work out fine, it should replace the normal OOo in the future (like for OOo 3). (See also the OOo w/o URE and OOo w/o URE mail threads for past discussions on this topic.)

Work is done on CWS sb71, which is currently based on SRC680m208. Work is done in multiple steps. The first step is to create a OOo-wo-URE that can be installed manually and successfully smoketest on unxsoli4.pro. The following list gives the details of how this step was reached.

♦ Using the scp2 mechanism to undefine items of an existing product to define a new product, OOo-wo-URE has easily been set up as a small delta to normal OOo in scp2. All items that are already available in the URE have been unset, and nothing else has been added, except for a few exceptions. This means that the resulting OOo-wo-URE installation sets are structured essentially the same as the existing normal OOo installation sets; this should definitely be revisited (when normal OOo is one day replaced by OOo-wo-URE, defining OOo-wo-URE in terms of normal OOo is no longer a good idea, anyway; but for now it greatly helps keep things in sync). The exceptions are as follows:

• LICENSE, THIRDPARTYLICENSEREADME.html, and README (with different contents) are available in both URE and OOo-wo-URE (where the OOo-wo-URE versions have the same content as the normal OOo versions for now).
• types.rdb is in URE and a newly introduced offapi.rdb, containing the difference between normal OOo types.rdb and URE types.rdb, in OOo-wo-URE replaces the normal OOo types.rdb (see below).
• services.rdb are available in both URE and OOo-wo-URE, where the OOo-wo-URE services.rdb contains the difference between normal OOo services.rdb and URE services.rdb (see below).
• unorc (with different contents) is available in both URE and OOo-wo-URE (where the OOo-wo-URE version also has different content from the normal OOo version, see below).

♦ The installed OOo-wo-URE will need to find the installed URE. At least on Solaris and Linux, URE is by default installed to /opt/openoffice.org/ure, and OOo-wo-URE is by default installed to /opt/openoffice.org2.3 (same as normal OOo). However, these paths can be changed (at deployment time and also later on). For now, assume there is a soft link named ure-link in the base directory of OOo-wo-URE that points to the base directory of URE. It is still unclear how this link is generated at deployment time, and how it stays in sync when moving URE and/or OOo-wo-URE around. It is still unclear what to do on Windows (the soft link solution will work for all Unix derivates). (Maybe ure instead of ure-link would be a better name, thinking about the possibility to potentially move the URE installation into the OOo-wo-URE installation so that ure would not be a soft link but a true directory.)

♦ Executables and shared libraries in the OOo-wo-URE program directory need to find shared libraries in the URE lib directory. This has been solved by changing LINKFLAGSRUNPATH (solenv/inc/platform.mk) from just $ORIGIN to $ORIGIN and $ORIGIN/../ure-link/lib. This should be refined: For one, executables and libraries in the URE do not need it (some already explicitly overwrite LINKFLAGSRUNPATH anyway and are thus already fine). For another, there might be executables or libraries in OOo-wo-URE that do not directly link to URE libraries. (URE README states: “C++ UNO components run from within the uno executable can depend on an environment in which the public C++ UNO runtime dynamic libraries (cppu, cppuhelper, sal, salhelper, stlport) are already available (that is, on Linux x86, Solaris x86, and Solaris SPARC, a component dynamic library need not make sure that the UNO runtime dynamic libraries it needs can be found on its RPATH).” The question is whether this should also pertain to UNO components or arbitrary other libraries started from elsewhere.)

♦ Native (C++) services in the OOo services.rdb are currently registered without a path, and the loader finds them next to itself. Java services in the OOo services.rdb are registered with a vnd.sun.star.expand:$UNO_JAVA_COMPONENT_PATH/ prefix, and this works because regcomp gets passed a -env:UNO_JAVA_COMPONENT_PATH that points to the location of the services at build time. Native services (there were no Java services until now) in the URE services.rdb are currently registered in a hacky way directly in instsetoo_native/util/makefile.mk instead of with the general mechanisms in solenv/bin/modules/installer/servicesfile.pm (with a vnd.sun.star.expand:$URE_INTERNAL_LIB_DIR/ prefix, using the regcomp -env trick). For OOo-wo-URE we need native services to be registered with a vnd.sun.star.expand:$ORIGIN/ prefix. The regcomp -env trick does not work here, as you must not mess up regcomp's idea of $ORIGIN. regcomp has recently been extended with -wop, stripping the path from registered services. I improved that to -wop=prefix, exchanging the path of registered services with the given prefix. I added build variables NATIVESERVICESURLPREFIX (defaulting to vnd.sun.star.expand:$ORIGIN/) and JAVASERVICESURLPREFIX (defaulting to vnd.sun.star.expand:$UNO_JAVA_COMPONENT_PATH/) and now always use regcomp -wop=prefix (getting rid of the hacky build of URE services.rdb in instsetoo_native/util/makefile.mk). Registration of Python services (if any) in solenv/bin/modules/installer/servicesfile.pm has not been adapted for now.

♦ There are two unorc files, the URE one at opt/openoffice.org/ure/lib/unorc and the OOo-wo-URE one at opt/openoffice.org2.3/program/unorc. All code running in an OOo-wo-URE context, whether located in the URE or the OOo-wo-URE tree, should use the OOo-wo-URE unorc, but normally the unorc next to the cppuhelper shared library (i.e., the URE unorc) is used. Introduced URE_UNORC bootstrap variable (a URL pointing to the unorc) to override the location; explicitly set in the OOo-wo-URE (and normal OOo) start scripts (soffice, spadmin, unopkg, pkgchk; normal OOo uno). The URE unorc has four entries:

• The two public entries are UNO_TYPES and UNO_SERVICES. The OOo-wo-URE unorc should include in its respective UNO_TYPES and UNO_SERVICES either just the paths to the URE types.rdb and services.rdb (the solution taken for now) or the complete values of the URE UNO_TYPES and UNO_SERVICES (including rdb files at /etc/opt/ure and ~/.ure).
• The two private entries are URE_INTERNAL_LIB_DIR and URE_INTERNAL_JAVA_DIR. First of all, they should not be used at all in OOo-wo-URE (as they are private to URE). Thus, they should not be mentioned in OOo-wo-URE unorc, not even as trivial forwards as is done for now (URE_INTERNAL_LIB_DIR=${ORIGIN/../ure-link/lib/unorc:URE_INTERNAL_LIB_DIR}). A better solution would be to allow syntax in an rc file to include another rc file and then modify or replace some of its entries.

♦ The normal OOo jvmfwk3rc contains different data than the URE jvmfwk3rc (at opt/openoffice.org/ure/lib/jvmfwk3rc): For one, the normal OOo has UNO_JAVA_JFW_SHARED_DATA and UNO_JAVA_JFW_USER_DATA pointing at javasettings.xml files in the normal OOo shared and per-user configuration trees, while the URE has those variables pointing at javasettings.xml files in the URE-specific /etc/opt/ure and ~/.ure directories. This means that OOo-wo-URE now also uses the URE locations—which in turn means that different installations of OOo-wo-URE share those javasettings.xml files, whether that is considered good or bad. For another, the normal OOo sets UNO_JAVA_JFW_CLASSPATH_URLS to contain certain jars taken from the system where applicable (see lines 820–848 of scp2/source/ooo/profileitem_ooo.scp:1.44). This has been solved by introducing the public deployment variable URE_MORE_JAVA_CLASSPATH_URLS in the URE's client interface, setting UNO_JAVA_JFW_CLASSPATH_URLS to $URE_MORE_JAVA_CLASSPATH_URLS in the URE jvmfwk3rc, and setting URE_MORE_JAVA_CLASSPATH_URLS as appropriate in the OOo-wo-URE fundamentalrc.

♦ The javavendors.xml (at opt/openoffice.org/ure/share/misc/javavendors.xml, used by both URE and OOo-wo-URE, referenced by UNO_JAVA_JFW_VENDOR_SETTINGS in jvmfwk3rc) lists the names (without paths) of shared libraries to access various JVM implementations. Those libraries (currently just the single sunjavaplugin.so) are located at opt/openoffice.org/ure/lib, but searched for in a way that breaks for OOo-wo-URE. The solution is to use complete URLs (namely vnd.sun.star.expand URLs) in javavendors.xml: vnd.sun.star.expand:$URE_INTERNAL_LIB_DIR/sunjavaplugin.so. A new internal C++ helper function cppu::bootstrap_expandUri has been added to cppuhelper (also used in stoc/source/loader/dllcomponentloader.cxx). Also, with this change, URE and (normal) OOo no longer need different versions of javavendors.xml files generated in jvmfwk. In turn, normal OOo now also needs to define URE_INTERNAL_LIB_DIR in its unorc (maybe another fresh variable is needed that is not URE-internal).

♦ The above change implies that build-time calls of tools like regcomp now also need to find a definition of URE_INTERNAL_LIB_DIR. Instead of always passing it as an -env command line argument, a minimal unorc/uno.ini has been added to the build environment (delivered from cppuhelper to the solver lib (Unix) resp. bin (Windows) directory; in both worlds, both unorc and uno.ini are delivered, which should be harmless). The unorc only contains a definition for URE_INTERNAL_LIB_DIR (just $ORIGIN, as the unorc/uno.ini is located in the same directory as the cppuhelper shared library on the solver). It would be nice to also include a definition for URE_INTERNAL_JAVA_DIR (and not have to pass it via -env in solenv/bin/modules/installer/servicesfile.pm, for example), but that would have to be $ORIGIN/../binext on Unix, where ext can vary over a build's time (at Sun Hamburg for example, MWS builds by release engineering are done with ext empty, then solver is copied so that ext is the milestone number, and developer CWS builds spawned from that milestond also have a mileston number ext). So I unfortunately had to leave URE_INTERNAL_JAVA_DIR alone.

♦ The extension manager (within soffice, as well as the standalone unopkg and deprecated pkgchk) spawns uno processes. Until now it located the uno start script in the “program directory” (opt/openoffice.org2.3/program). To find the opt/openoffice.org/ure/bin/uno (and have a normal OOo still working, too), the URE_BIN_DIR bootstrap variable is introduced. It is set in the OOo-wo-URE (and normal OOo) start scripts (soffice, spadmin, unopkg, pkgchk; normal OOo uno). URE_BIN_DIR is also used directly in the various start scripts to locate the javaldx executable that was similarly expected in the “program directory” until now (and that makes it more convenient to have URE_BIN_DIR as an environment variable than in unorc).

♦ Until now the uno executable had been started from extension manager with UNO_SERVICES set to just the services.rdb in the “program directory” and an empty INIFILENAME. It is unclear what exactly this was supposed to be good for, and has been removed for now. Similarly, until now the unopkg executable had bootstrapped UNO in a nonstandard way, using just the types.rdb and services.rdb in the “program directory.” Again, it is unclear what exactly this was supposed to be good for, and has been replaced with a standard UNO bootstrapping mechanism for now.

♦ Manual installation for now: For testing purposes, I have a little shell script that uses the user install scripts from setup_native to install URE and OOo-wo-URE into some local directory tree (the argument given to the script) and create the symbolic ure-link. This setup does not catch the fact that for now a dependency on the Solaris package level is missing from (some core package of) the OOo-wo-URE installation set to (the single package of) the URE installation set.

rm -rf $1 ~/.ure
mkdir $1 || exit 1
$SOLARVER/$GVERDIR/bin$UPDMINOREXT/userscripts/install \
  $SOLARSRC/instsetoo_native/$GVERDIR/URE/[pr]*/install/en-US/[pR]* \
  $1 || exit 1
if [ ! -d $1/opt ] ; then
  mkdir $1/.TMP || exit 1
  mv $1/* $1/.TMP || exit 1
  mkdir -p $1/opt/openoffice.org || exit 1
  mv $1/.TMP $1/opt/openoffice.org/ure || exit 1
fi
$SOLARVER/$GVERDIR/bin$UPDMINOREXT/userscripts/install \
  $SOLARSRC/instsetoo_native/$GVERDIR/OpenOffice_woURE/[pr]*/install/en-US/[pR]* \
  $1/.TMP || exit 1
if [ ! -d $1/.TMP/opt ] ; then
  mkdir $1/.TMP/.TMP || exit 1
  mv $1/.TMP/* $1/.TMP/.TMP || exit 1
  mkdir $1/.TMP/opt || exit 1
  mv $1/.TMP/.TMP $1/.TMP/opt/openoffice.org2.3 || exit 1
fi
mv $1/.TMP/opt/openoffice.org2.3 $1/opt || exit 1
rm -r $1/.TMP || exit 1
ln -s ../openoffice.org/ure $1/opt/openoffice.org2.3/ure-link || exit 1

♦ Some parts of URE that are currently classified as private by ure/source/README:1.11 are referenced from OOo-wo-URE:

• The jvmaccess and jvmfwk shared libraries (referenced from various OOo-wo-URE shared libraries).
• unorc (referenced from the OOo-wo-URE unorc).
• On Linux, the compiler support libraries (libgcc_s.so.1 and libstdc++.so.6, referenced from various OOo-wo-URE executables and shared libraries).