Difference between revisions of "GNU Linux Sparc Porting"

Revision as of 11:39, 3 February 2006

GNU Linux Sparc Porting

The objective is to maintain the complete upstream sources so that the GNU/Linux sparc platform can be packaged for example by debian, gentoo and others. Also to provide an upstream "benchmark" released version that can be used for confirming who owns any bugs, while not interfering with the distribution installed packages.

For GNU/Linux sparc all building now is done with gcj. The gcj versions being used for upstream contributed builds are gcc4.0.2 and gcc4.1. Distribution packagers use the following versions:

Debian -  gcc4(?)/gcj
Gentoo?
any others?

Other projects that need more work:

- fix multimedia (sound was partly working in m137 but not since, and Java Media Framework does not run with any GNU/Linux with gcj)

- qa needs to be done

- very slow startup (depends on javaldx?)

The latest upstream release is on mirrors under pub/OpenOffice.org/contrib/linuxsparc

What is SPARC?

SPARC means Scalable Processor ARChitecture about SPARC

The most common hardware implementation is done by Sun Sun

Most GNU/Linux sparc/sparc64 users are running surplus (second hand) hardware which may be quite old and the installed user base and developer resources are relatively small.

There are 2 main flavours of SPARC hardware:

32 bit known as sparc (uname -m returns sparc)
64 bit known as sparc64 (uname -m returns sparc64)

GNU/Linux is ported to run on sparc and sparc64, there are separate kernels for sparc and sparc64 but the majority of user software is designed to run on both sparc and sparc64.

For example on my system I can find these files:

jim@sun:/boot$ file vmlinux-2.6.13
vmlinux-2.6.13: ELF 64-bit MSB executable, SPARC V9, version 1 (SYSV), statically linked, not stripped

jim@sun:~/o147/program$ file soffice.bin
soffice.bin: ELF 32-bit MSB executable, SPARC, version 1 (SYSV), for GNU/Linux 2.2.0, dynamically linked (uses shared libs), stripped

So the first issue for porting is, how to build the 32 bit product on a 64 bit system? This requires something like "cross compiling" because the product runs on a platform different from the build system, but for sparc/sparc64 this is normal.

Cross compiling ?

I read somewhere that Openoffice.org build tools do not support cross compiling (where?). But something like cross compiling is achieved when building with multiple compiler versions installed, or when the target platform is different from the build platform.

For GNU/Linux Sparc it can be done, with the main issues being:

configuration must be told what platform to use for the product, and to ignore the build platform, when setting the build environment

compiler flag "-m32" will tell gcc to build sparc and not sparc64

some modules (only in external?) will not obey the configured environment

installation has some hard-coded references to the target platform

The most common bad behaviour occurs when a module ignores the configured compiler path or compiler flags. For example:

configured to build with "/usr/local/4.1/bin/gcc -m32"
module builds with "gcc" which may be /usr/bin/gcc"

This will not be easily noticed unless all alternative possible compilers are excluded from the configured environment path. Still there remains a risk that a module will seek "/usr/local/bin/gcc" and find it. This sometimes seems to involve using libtool which may have some defaults configured on the build machine?

The other issue to investigate is that various tools are built and used during building. These tools need to be built for the building machine not for the target machine.

Specific code for GNU/Linux sparc

There are only a small number of places where GNU/Linux sparc requires different code than used by other platforms. The following list is not complete but it provides some examples and will be gradually increased. A full history of the sparc port can be found in issuezilla, just look for issues where sparcmoz has reported or commented.

Platform make file

Various compiler and other flags are set in the platform make file solenv/inc/unxlngs.mk. That file is kept as close as possible to solenv/inc/unxlngi6.mk but the following differences are important:

If the assembler is called directly, it must build for sparc v7 by default (could use v8 safely?) and not v8plus or v9

# mk file for unxlngs
ASM=$(CC)
AFLAGS=-Wa,-K,PIC -c $(CDEFS)

Code must be compiled with -fPIC, as -fpic will not do for sparc/sparc64

PICSWITCH:=-fPIC

Platform specific identifier for shared libs

DLLPOSTFIX=ls
DLLPRE=lib
DLLPOST=.so

Module sal

GNU/Linux sparc did not provide an implementation of frame.h that was needed for backtrace functions. It was also needed to provide the backtrace function without using the crash reporter. Some of this may be obsolete now and needs further investigation:

sal/osl/unx/backtrace.c
sal/osl/unx/backtrace.h

The following code provides runtime detection of the user hardware and loads some different code at start up.

sal/osl/unx/util.c
sal/osl/unx/asm/interlck_sparc.s

This file is modified to build the preceding files

sal/osl/unx/makefile.mk

Module sc

A number of runtime isssues (crash) were found with the early builds of the spreadsheet using gcc 3.2. A workaround was found to compile certain files with no optimisation. A review of the code suggests similar issues were found with other platforms, but in different files. The files were found fairly quickly by using a binary search pattern to locate the files that caused the crashes (build - test - crash - build half the files again without optimisation). Fortunately this type of work around is supported by the build system environment variables NOOPTFILES and EXCEPTIONSNOOPTFILES! It is possible some of these issues have gone away with later compilers, further investigation is needed. Refer to the source code as follows:

sc/source/core/data/makefile.mk:.IF "$(OS)$(COM)$(CPUNAME)"=="LINUXGCCSPARC"
sc/source/core/tool/makefile.mk:.IF "$(OS)$(COM)$(CPUNAME)"=="LINUXGCCSPARC"
sc/source/filter/excel/makefile.mk:.IF "$(OS)$(COM)$(CPUNAME)"=="LINUXGCCSPARC"
sc/source/ui/unoobj/makefile.mk:.IF "$(OS)$(COM)$(CPUNAME)"=="LINUXGCCSPARC"
sc/source/ui/view/makefile.mk:.IF "$(OS)$(COM)$(CPUNAME)"=="LINUXGCCSPARC"

A typical example follows from sc/source/core/data/makefile.mk:

.IF "$(OS)$(COM)$(CPUNAME)"=="LINUXGCCSPARC"
NOOPTFILES= \
               $(SLO)$/column2.obj \
               $(SLO)$/column3.obj \
               $(SLO)$/table3.obj \
               $(SLO)$/table4.obj  \
               $(SLO)$/documen4.obj \
               $(SLO)$/conditio.obj \
               $(SLO)$/validat.obj
EXCEPTIONSNOOPTFILES= \
       $(SLO)$/cell.obj
.ELSE
EXCEPTIONSFILES+= \
        $(SLO)$/cell.obj
.ENDIF

Module bridges

The bridges code in bridges/source/cpp_uno/gcc3_linux_sparc is copied exactly from the corresponding gcc3_linux_intel and only the assembler snippets have been changed.

TODO: configuration, installation

Building with GCC 4.1

There are 2 main issues listed below, and a few others yet...

i57855 Bad uses of getCppuType

Some patches for this isssue are in cws, for details see the dev@porting mail thread 
dev@porting

i58141 extra qualification on member

This affects many unrelated projects.

The entire product has been built on GNU/Linux Sparc using the patches mentioned at those issues and described in detail below. No modules are excluded by the configure step which is:

./configure --enable-crashdump=STATIC --enable-symbols=SMALL --with-java=gij --with-jdk-home=/usr/local/4.1

Building GNU/Linux Sparc SRC680_m148 with gcc-4.1/gcj

The following is becoming quickly out-of-date, while I am still building m148 already gcc41 is now resync to m148 and sb41 is integrated in m150. So gcc4-1 will soon build out of the box. But I will keep these notes here for a while yet and merge everything into a sparc linux building guide.

Well m148 never ended when I left for linux-conf.au so I will try again with gcc4.1, when m155 is ready. Currently building m154 with gcc4.0.

Building with gcc4.1 requires the patches prepared by sb and pmladek. This is work-in-progress since m142 and patches are moving into cws. This diary is mainly for my own benefit to reproduce my build on the next milestone. I hope I have described the following steps correctly, but let me know if there is any problem to reproduce this and I will fix it.

In my case GCC 4.1 is built using gcc.gnu.org sources checked out from gcc-4.1-branch and installed in /usr/local/4.1.

SRC_ROOT is jim@sun:~/build680 Everything is run from SRC_ROOT unless stated.

$ export CVSROOT=:pserver:sparcmoz@localhost:/cvs
$ cvs -z4 co -r SRC680_m148 OpenOffice2

Set the compiler variables. When building on sparc64, the -m32 is required to build for sparc:

$ export CC="/usr/local/4.1/bin/gcc -m32"
$ export CXX="/usr/local/4.1/bin/g++ -m32"

Adjust the PATH so there is no other gcc or gxx in the PATH, and that there is no fake java, so there is nothing returned by:

$ which gcc
$ which java

If gjdoc is not in the jdk_home then it will not be found so:

$ export JAVADOC=/usr/local/bin/gjdoc

Get the needed patches for gcc4.1:

$ cvs diff -u -r SRC680_m148 -r cws_src680_sb41 > sb41.diff
$ cvs diff -u -r SRC680_m148 -r cws_src680_gcc41 > gcc41.diff

Notice from the cvs log, some new files should be checked out for sb41:

cvs server: inc/cppu/unotype.hxx no longer exists, no comparison available
cvs server: qa/test_unotype.cxx no longer exists, no comparison available

jim@sun:~/build680/cppu$ cvs update -dP -r cws_src680_sb41 inc/cppu/unotype.hxx
U inc/cppu/unotype.hxx
jim@sun:~/build680/cppu$ cvs update -dP -r cws_src680_sb41  qa/test_unotype.cxx
U qa/test_unotype.cxx

For libwpd the fixes are in a new download version 0.8.4, to get this do:

$ cvs co -r cws-src680_fs05 libwpd

For jvmfwk need update to cmcfixes23 see issue 58896

$ cvs update -dP -r cws_src680_cmcfixes23 jvmfwk

Some patches for ant and gcj may be required, depending on the builders tools and environment. In my case the following are required (These are available consolidated in the m147 patch at issue 50599):

ant-gcj.diff: remove ant test from configure, set the ant flags for gcj, remember to run autoconf before configure
fastjar.diff: issue 57565, also need export JAR=/usr/local/4.1/bin/fastjar

For GNU/Linux Sparc it is also necessary to use the warnings01 patches for bridges with gcc4.1. This has not been resynced since m138, and after that we have sb41 and also unxlngs04 was integrated in bridges. Luckily sb41 is resynced to m148 already so the solution is to make a patch from m138 and merge that one after updating bridges to sb41. Some small changes were rejected and had to be applied by hand.

$ cvs diff -u -r SRC680_m138 -r cws_src680_warnings01 bridges > bridges-warnings01.diff
$ cvs update -dP -r cws_src680_sb41 bridges
$ patch -p0 < bridges-warnings01.diff

Some miscellaneous patches (see m147 patch, need to include issue references):

sparc.diff: issue xxxxxx GNU/Linux Sparc only, needs further investigation
register.diff: issue xxxxxx to preregister java components
xmerge.diff: issue xxxxxx needs a better solution
mozilla.diff: issue xxxxxx if building moz
libxml2.diff: issue xxxxxx

As only bridges was updated to warnings01, there is a dependency on sal, so a workaround is:

bridges: replace "sal::static_int_cast" by "static_cast"

TODO: build with all of warnings01

For berkeleydb a patch is needed, meanwhile:

$ export JAVA=/usr/local/4.1/bin/gij

Nothing to do for moz this time, using the pre-built mozilla packages in moz/zipped.

Out of Memory: qadevOOo with gcj will need about 800-1000mb of RAM, but I have only 256mb. I must remember to increase swap file to 1000mb to be safe.

To run dmake runtest in testtools/source/bridgetest/pyuno it was necessary to update PYTHONPATH and revert the makefile ( see issue 40091 ). This may be useful to remember later...

This is needed for some run time issues:

icu - issue 59722

Now run autoconf and configure:

$ cd config_office
$ autoconf
$ ./configure --with-java=gij --with-jdk-home=/usr/local/4.1 \
--with-epm=internal --enable-crashdump=STATIC --enable-symbols=SMALL \
--with-build-version="Built by Jim m148 gcc4.1/gcj" --with-package-format=rpm

Building with gcj

This is mainly intended as a diary to remind myself what i am doing. Building with gcj is not fully supported "out of the box" and anyone seeking to build this way will need to figure out some things for themselves. Recent patches for building GNU Linux Sparc with gcj are at i50599 but these will not be committed as they are specific to my building environment.

i50599 Build GNU Linux Sparc with gcj

The following describes how I build with gcj. I believe the work to make this possible was mostly done by Caolan and Rene showed me how to get Ant working.

Step 1: get a version of gcc with suitable gcj.

I think that gcc >= 4.0 is necessary, in my case I get the latest preview gcc4.1 sources and build gcc into /usr/local because gcj was for some time not working on debian/unstable. And it is easier to see what is happening without the various workarounds to provide commands such as java that do not really exist.

Step 2: set up the required commands for build tools

Note that the required compiler can be specified by setting environment variables CC and CXX before running configure, but the same is not possible for some related programs. In just a few places the code expects these commands to be available, normally in /usr/bin: java javac jar

But building gcc provides instead the corresponding commands: gij gcj fastjar (gcc 4.1)

Some distributions will arrange for the java etc commands to be available, but the method to provide these varies between distributions and over time. In my opion it will be better to let these be defined by the configuration step, but I am a long way from knowing how that should be done!

With /usr/local/bin in my PATH the configuration finds /usr/local/bin/gjdoc without being asked. In anycase it is possible to define the following for the configurator: JAVADOC=/usr/local/bin/gjdoc

Configuration --with-java=gij and --with-jdk-home=/usr/local takes care of gij and gcj in most cases. Patches are needed to workaround one case of hardcoding for each of java and jar.

Ant is more complicated, the basic idea is to set up the required command line and parameters to invoke Ant in solenv/inc/ant.mk depending on how Ant is installed, and is better explained by reviewing the patches here i50599 Build GNU Linux Sparc with gcj

Step 3: configure

In my case the configure commands may be like this:

$ export CC="/usr/local/bin/gcc -m32"

$ export CXX="/usr/local/bin/g++ -m32"

$ ./configure --with-jdk-home=/usr/local --with-java=gij

For my patches to avoid using java and jar commands I will also define the following:

$ export JAR=/usr/local/bin/fastjar

$ export JAVA=/usr/local/bin/gij <-- needed for berkeleydb

Step 4: packaging

Finally the build fails when preregistering java components. A workaround for gcc4.1 has been provided here: preregistering java components

Notes for Building with gcc4.0/gcj - m154

Issues marked (*) arise because I am using pure gcc/gcj in /usr/local while other builders use fake java commands such as provided by java-gjc-compat?

bean - see issue 53880

bean can not currently be built on debian/unstable due to issues with freetype. Issue 53880 references some upstream issues, but also note bean builds OK using --with-system-freetype. A not recommended workaround is to skip the failed test, by renaming the file bean/unxlngs.pro/lib/check_libofficebean.so to libofficebean.so then complete the build in the module and deliver

berkeleydb - see issue 54657

berkeleydb has hardcoded "java" command with fallback to JAVA but environment defines JAVAINTERPRETER. A workaround is export JAVA=/path/to/gij (*)

xmerge - see issue 52786

ant expects javadoc command and ignores environment JAVADOC=/path/to/gjdoc. A partial workaround is to make a fake command javadoc linked to gjdoc (*)

xmlhelp - needs this patch xmlhelp

This will be fixed when cws_src680_jaxpapi is integrated

filter - needs patches for this module from cws_src680_jaxpapi

xmlscript - postprocess complains this module is not built, workaround is to build and deliver it

No issue found for xmlscript - possible a missing dependency?

Installing from rpm without root access

The GNU Linux Sparc packages from contrib/linuxsparc should not interfere with installations provided by the various distribution packagers such as debian, gentoo etc. Therefore the supplied rpms can be installed in the user's home directory using the install_linux.sh script which is available on mirrors at pub/OpenOffice.org/developer/install_scripts/

The rpm command is required but root access is not required