ARM
Cross Compiling OOo for ARM Key to build OOo for ARM reasonably fast is the user space QEMU, leading to build times of about 24h, which may be pushed to under 5h, if applying some "afterburner" :-). Only left problem so far is with Java technology not successfully being executable in user space QEMU, but in system QEMU only, means that we need both.
Contents
Hosting
Ideally you have a fast ARM machine, where you install the distro of choice, and where you build OOo just as usual. Unfortunately I did not had any hardware, so I had to build in emulation. The host as well as the target environment were both Ubuntu Jaunty Jackalope, this made it easy to improve build times later, after the basic approach did work.
ARM Root File System
The latest Ubuntu (9.04 - Jaunty Jackalope) is available for ARM CPUs. Just follow the instructions here and you get a tarred ARM root file system (e.g. armel-rootfs-200903170957.tgz). Untar that to a convenient location ...
> mkdir /local/jaunty_arm_rfs > cd /local/jaunty_arm_rfs > tar -xvzf armel-rootfs-200903170957.tgz
... and have a first look at it:
> file bin/ls bin/ls: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped
QEMU
"QEMU is a generic and open source machine emulator and virtualizer."
QEMU does not only allow system emulation, but also supports process granularity (user space) emulation. The former means, that a whole computer is emulated, including BIOS, user-land, kernel-land, hardware and everything. The latter means, that a program compiled e.g. for ARM (a "foreign" binary) may seamlessly be executed on a host system, e.g. x86 or x64. Important difference for building OOo in user space emulation and not in system emulation is the execution speed and available memory. Whole system emulation is slow and especially for ARM memory size is limited to 256MB, while user-space (or in-process) emulation is much faster, as it only emulates user land, the kernel and everything else run natively. This is interesting as it allows to mix foreign binaries with native ones, see "afterburner" in my next blog posting.
User Space Qemu
To utilize user space qemu, we need the target distribution as a directory on the host system (a "root file system"), so that we can enter it with "chroot". We need to register user space qemu (e.g. qemu-arm) for the foreign binaries and need to make it available in the root file system, as this is now executable we call that a "chroot environment". Unfortunately the user space qemu from the repository does not work, means we have to build our own, but see below ...
Building and Patching
Get the latest QEMU sources from the QEMU site:
> wget http://download.savannah.gnu.org/releases/qemu/qemu-0.10.3.tar.gz
... unpack it:
> tar -xvzf qemu-0.10.3.tar.gz
Next we need to get the ARM EABI patches from SourceForge and apply them, Heiner helped me with that, actually we had to delete some of the patches from the patch series, as they did not apply or disturbed otherwise:
> tar -xvjf qemu-arm-eabi-0.3.tar.bz2 > mv qemu-arm-eabi-0.3/patches . > export QUILT_PATCHES=../patches > cd qemu-0.10.3 > for patch in 02_fix_page_range_check.patch 04_shmat_strace.patch \ 10_signal_jobs.patch 19_zero_null.4.acct.patch 19_zero_null.2.read.patch \ 22_IPCOP_msg.patch 23_msg_syscalls.patch 24_IPCOP_sem.patch 25_sem_syscalls.patch \ 26_IPCOP_shm.patch 27_shm_syscalls.patch 33_fix_getdents_syscalls.patch \ 36_fix_iovec.patch 38_fix_recvmsg_return_value.patch 39_fix_exit_syscall.patch \ 99_sbox_proc.patch ; do quilt delete -r $patch ; done
Unfortunately utimensat has a bug in user space qemu, so we need another patch to fix this:
> cat syscall_utimensat.patch
--- qemu/linux-user/syscall.c.org 2009-05-08 16:04:11.908901048 +0200
+++ qemu/linux-user/syscall.c 2009-05-08 16:05:49.229947790 +0200
@@ -6203,9 +6203,15 @@
#if defined(TARGET_NR_utimensat) && defined(__NR_utimensat)
case TARGET_NR_utimensat:
{
- struct timespec ts[2];
- target_to_host_timespec(ts, arg3);
- target_to_host_timespec(ts+1, arg3+sizeof(struct target_timespec));
+ struct timespec * ts = NULL;
+
+ if (arg3) {
+ struct timespec ts_[2];
+ ts = ts_;
+ target_to_host_timespec(ts, arg3);
+ target_to_host_timespec(ts+1, arg3+sizeof(struct target_timespec));
+ }
+
if (!arg2)
ret = get_errno(sys_utimensat(arg1, NULL, ts, arg4));
else {
... add this patch to the quilt patch series:
> quilt import syscall_utimensat.patch
... and apply all patches:
> quilt push -fa
If quilt stops complaining that a refresh is needed, just do
> quilt refresh
.. and start again
> quilt push -fa
.. until it says, that all patches have been applied:
> quilt applied
Next we configure and build the user space qemu ...
> ./configure --target-list=arm-linux-user --static > make -j 4 > ./arm-linux-user/qemu-arm -v qemu-arm version 0.10.3, Copyright (c) 2003-2008 Fabrice Bellard usage: qemu-arm [options] program [arguments...] Linux CPU emulator (compiled for arm emulation) ...
Phew!, that was quite some work :-)
Register User Space Qemu
Now we need to register user space qemu ("qemu-arm"), which effectively is the "interpreter" or "CPU" for all ARM binares, to be invoked in case an ARM binary becomes invoked. The following line has been copied from the qemu-binfmt-conf.sh script, which is part of QEMU 0.10.3. Important difference is the appended "C", which ensures that the ARM binaries are executed on behalf of the executing user, respectively taking any suid into account:
> echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm:C' \ >> /proc/sys/fs/binfmt_misc/register
We need to workaround a common problem with user space qemus mmap schema:
> echo 4 > /proc/sys/vm/mmap_min_addr
From now on, we may execute any ARM binary by simply invoking it - unfortunately library resolution and placement has not been designed with mixed binaries in mind, so we likely get an error for non matching libraries if we try so. This should be better in the chroot environment ...
Chroot Environment
Your distribution may bring a user space qemu, though I recommend to build one from scratch (please see above), as we need to patch it slightly. Copy this (ideally statically) linked qemu-arm into your brand new ARM root file system:
> cp qemu-arm /local/jaunty_arm_rfs/usr/bin copying ..
... and see what happens when we chroot into it now:
> chroot /local/jaunty_arm_rfs/ /bin/su - ubuntu > uname -m armv5tel
... so now we do have a "chroot environment" :-)
System Qemu
To use system qemu, we may create a disk image and everything, which may than be booted by system qemu ... or we try to re-use our chroot environment and to boot a kernel in system qemu with the chroot environment as its root file system, mounted via NFS.
So, let's export the chroot environment via NFS ...
> echo '/local/jaunty_arm_rfs/ *(rw,no_root_squash,async,insecure)' >> /etc/exports > exportfs -ra ... > showmount -e localhost Export list for localhost: /local/jaunty_arm_rfs *
... make the network available:
> cp /etc/resolv.conf /local/jaunty_arm_rfs/etc
... install a kernel:
> mount -o bind /proc /local/jaunty_arm_rfs/proc > chroot /local/jaunty_arm_rfs > apt-get update > apt-get install linux-image-2.6.28-11-versatile
... prepare the initrd.img to enable NFS root by replacing "BOOT=local" with "BOOT=nfs" in /etc/initramfs-tools/initramfs.conf and update it:
> update-initramfs -uv
... disable the network manager as it interferes with NFS root:
> apt-get purge network-manager
I did experience problems with gdm, so I disabled it:
> update-rc.d -f gdm remove
Finally boot the kernel in system qemu:
> qemu-system-arm -M versatilepb -m 256 \ -kernel /local/jaunty_arm_rfs/boot/vmlinuz-2.6.28-11-versatile \ -initrd /local/jaunty_arm_rfs/boot/initrd.img-2.6.28-11-versatile \ -append "root=/dev/nfs ip=dhcp nfsroot=10.0.2.2:/local/jaunty_arm_rfs,timeo=14 rw mem=256M"
... and enjoy Ubuntu Jaunty form ARM :-)
Building OOo
Utilizing the chroot environment, OOo is now mostly buildable as usual, just check out the sources and configure them. Problematic is everything relying on Java technology and Mozilla. Java technology currently does not work well in user space qemu, but in system qemu only. You may want to build the related modules in system qemu, or for a first try just disable Java technology support during configuration. The used Mozilla version (1.7.5) is not buildable under ARM, AFAIK this will not change. So, you may either want to use system Mozilla or to disable Mozilla completely. Using the system Mozilla works fine as long as you don't need Address Book integration. This is actually how I did configure my build:
> ./configure --with-system-mozilla --with-openldap --without-java
And not to forget, you certainly need to install the prerequisites, so slow I do recommend doing that in system qemu, as you otherwise likely end up with system daemons running twice, one from the host and one from the chroot environment :^):
> apt-get update > apt-get dist-upgrade > apt-get install gnome-devel libcups2-dev tcsh
Caolan McNamara did fix sal/typesconfig/typesconfig.c for ARMel in cmcfixes56, for convenience I give that patch here:
> cat typeconfig.c.patch
Index: typesconfig.c
===================================================================
--- typesconfig.c (revision 270430)
+++ typesconfig.c (working copy)
@@ -163,20 +163,37 @@
|* Letzte Aenderung
|*
*************************************************************************/
-static int dummy(void* unused);
+#if defined(IA64) || defined(ARM32)
+int forceerror()
+{
+#if defined(ARM32)
+// workaround for qemu-user
+ hit = 1;
+#else
+ raise (SIGBUS);
+#endif
+ return 1;
+}
+
int GetAtAddress( Type eT, void* p )
{
-#if defined(IA64) || defined(ARM32)
switch ( eT )
{
case t_char: return *((char*)p);
- case t_short: if ((long)p % sizeof(short)) abort(); else return *((short*)p);
- case t_int: if ((long)p % sizeof(int)) abort(); else return *((int*)p);
- case t_long: if ((long)p % sizeof(long)) abort(); else return *((long*)p);
- case t_double: if ((long)p % sizeof(double)) abort(); else return *((double*)p);
+ case t_short: if ((long)p % sizeof(short)) return forceerror(); else return *((short*)p);
+ case t_int: if ((long)p % sizeof(int)) return forceerror(); else return *((int*)p);
+ case t_long: if ((long)p % sizeof(long)) return forceerror(); else return *((long*)p);
+ case t_double: if ((long)p % sizeof(double)) return forceerror(); else return *((double*)p);
}
+ abort();
+}
+
#else
+static int dummy(void* unused);
+
+int GetAtAddress( Type eT, void* p )
+{
switch ( eT )
{
case t_char: { char x = *(char*)p; return dummy(&x); }
@@ -185,7 +202,6 @@
case t_long: { long x = *(long*)p; return dummy(&x); }
case t_double: { double x = *(double*)p; return dummy(&x); }
}
-#endif
abort();
}
@@ -195,6 +211,7 @@
return 0;
}
+#endif
/*************************************************************************
|*
|* SetAtAddress()
Property changes on: typesconfig.c
___________________________________________________________________
Added: svn:mergeinfo
Merged /cws/cmcfixes56/sal/typesconfig/typesconfig.c:r269782-269948
The gcc has problems in ICU, dying with some "internal compiler" error, the workaround is to disable optimization, just apply:
> cat icu_makefile.mk.patch Index: makefile.mk =================================================================== --- makefile.mk (revision 270947) +++ makefile.mk (working copy) @@ -68,7 +68,7 @@ # add SYSBASE libraries and make certain that they are found *after* the # icu build internal libraries - in case that icu is available in SYSBASE # as well -icu_LDFLAGS+= -L../lib -L../../lib -L../stubdata -L../../stubdata -L$(SYSBASE)$/usr$/lib +icu_LDFLAGS+= -L../../lib -L../../stubdata -L$(SYSBASE)$/usr$/lib .ENDIF # "$(SYSBASE)"!="" .IF "$(OS)"=="MACOSX" @@ -85,8 +85,10 @@ icu_LDFLAGS+=-Wl,-z,noexecstack .ENDIF -icu_CFLAGS+=-O $(ARCH_FLAGS) $(EXTRA_CDEFS) -icu_CXXFLAGS+=-O $(ARCH_FLAGS) $(EXTRA_CDEFS) +icu_CFLAGS+= $(ARCH_FLAGS) $(EXTRA_CDEFS) +#icu_CFLAGS+=-O $(ARCH_FLAGS) $(EXTRA_CDEFS) +icu_CXXFLAGS+= $(ARCH_FLAGS) $(EXTRA_CDEFS) +#icu_CXXFLAGS+=-O $(ARCH_FLAGS) $(EXTRA_CDEFS) BUILD_ACTION_SEP=; # remove conversion and transliteration data to reduce binary size. @@ -116,7 +118,7 @@ # note the position of the single quotes. BUILD_DIR=$(CONFIGURE_DIR) -BUILD_ACTION=$(GNUMAKE) +BUILD_ACTION=$(GNUMAKE) -j 4 OUT2LIB= \ $(BUILD_DIR)$/lib$/libicudata$(DLLPOST).$(ICU_MAJOR)$(ICU_MINOR).$(ICU_MICRO) \ $(BUILD_DIR)$/lib$/libicudata$(DLLPOST).$(ICU_MAJOR)$(ICU_MINOR) \ Another patch is needed for xmlsecurity/source/xmlsec/nss/seinitializer_nssimpl.cxx as it has problems finding an include in case of building --with-system-mozilla: cat /usr/local/kr93794/xmlsecurity_source_xmlsec_nss.patch Index: seinitializer_nssimpl.cxx =================================================================== --- seinitializer_nssimpl.cxx (revision 270947) +++ seinitializer_nssimpl.cxx (working copy) @@ -68,7 +68,8 @@ #include "pk11func.h" #ifdef SYSTEM_MOZILLA #include "nssrenam.h" -#include "secmod.h" +#include "secmod.h" +#include "nss/nss.h" #endif #include "cert.h" #include "cryptohi.h"
Last but not least we have the problem, that the generated .deb have the wrong architecture, so here is a patch fixing that for the set_soenv.in script, for ARMel and AMD64:
# cat /usr/local/kr93794/set_soenv.in.patch
--- /usr/local/kr93794/Asus/xandros/SO_OOO310_m10/set_soenv.in 2009-04-16 12:45:37.000000000 +0200
+++ /usr/local/kr93794/set_soenv.in 2009-04-28 16:30:06.000000000 +0200
@@ -32,6 +32,19 @@
use strict; # pragma
use File::Basename;
+sub dpkg_architecture {
+ my @lines=split(/\n/,`/usr/bin/dpkg-architecture`);
+
+ my %data;
+ my @name_value;
+
+ foreach (@lines){
+ @name_value=split(/=/,$_);
+ $data{$name_value[0]}=$name_value[1];
+ }
+
+ return %data
+}
#
#--------------------------------------------------------
@@ -63,7 +76,7 @@
$GVER, $OS, $OSVERSION, $OUTPATH, $INPATH, $PATH_SEPERATOR,
$DYNAMIC_CRT, $SET_EXCEPTIONS, $use_shl_versions, $CDPATHx, $JRELIBDIR,
$JREEXTRALIBDIR, $JRETOOLKITDIR, $JRETHREADDIR,
- $FLIPCMD );
+ $FLIPCMD, $ABI );
#
#-------------------------------------------
# IIc. Declaring the environment variables.
@@ -373,6 +386,12 @@
$OS = "LINUX";
$PATH_SEPERATOR = $ps;
+ if (-e "/etc/debian_version")
+ {
+ my %data=dpkg_architecture();
+ $ABI=$data{"DEB_HOST_ARCH"};
+ }
+
#Set platform specific values:
if ($platform =~ m/^i[3456]86/)
{ print "Setting Linux x86 specific values... ";
@@ -409,11 +428,20 @@
}
elsif ($platform =~ m/^x86_64/)
{ print "Setting Linux x86-64 specific values... ";
- $outfile = "LinuxX86-64Env.Set";
+
$CPU = "X";
$CPUNAME = "X86_64";
$CVER = "C341";
- $OUTPATH = "unxlngx6";
+ if ($ABI =~ "") {
+ $outfile = "LinuxX86-64Env.Set";
+ $OUTPATH = "unxlngx6";
+ }
+ else {
+ $outfile = "Linux" . $ABI . "Env.Set";
+ $OUTPATH = "unxlng" . $ABI;
+ $EPM_FLAGS = "-a $ABI";
+ }
+
# Blackdown.org JDK porting project uses `amd64' and `server' in JDK 1.4.2 RC1
$JRELIBDIR = '$JAVA_HOME'.$ds."jre".$ds."lib".$ds."amd64";
# has both server and client
@@ -533,15 +561,24 @@
}
elsif ($platform =~ m/^arm.*?l-/)
{ print "Setting Linux ARM specific values... ";
- $outfile = "LinuxARMEnv.Set";
+
+ if ($ABI =~ "") {
+ $ABI="arm";
+ $outfile = "LinuxARMEnv.Set";
+ $OUTPATH = "unxlngr";
+ }
+ else {
+ $outfile = "Linux" . $ABI . "Env.Set";
+ $OUTPATH = "unxlng" . $ABI;
+ }
+
$CPU = "R";
$CPUNAME = "ARM";
- $OUTPATH = "unxlngr";
$JRELIBDIR = '$JAVA_HOME'.$ds."jre".$ds."lib".$ds."arm";
$JRETOOLKITDIR = '$JAVA_HOME'.$ds."jre".$ds."lib".$ds."arm".$ds."server";
$JRETHREADDIR = '$JAVA_HOME'.$ds."jre".$ds."lib".$ds."arm".$ds."native_threads";
$JREEXTRALIBDIR = '$JAVA_HOME'.$ds."jre".$ds."lib".$ds."arm".$ds."xawt";
- $EPM_FLAGS = "-a arm";
+ $EPM_FLAGS = "-a $ABI";
}
elsif ($platform =~ m/^mips/)
{ print "Setting Linux MIPS specific values... ";
After some clean ups I am going to put that into CWS as soon as I find the time ... that's it for now :-).
