Revision as of 15:57, 29 March 2010

Test Cleanup Project

See the mission statement mail.

User Interface

CppUnit

For tests in C++, plain CppUnit (at least version 1.12.1) is available. (OOo bundles CppUnit 1.12.1 as an external module; alternatively, you can configure --with-system-cppunit.)

Implement the test code as a dynamic library, following CppUnit conventions (i.e., using CPPUNIT_TEST_SUITE_REGISTRATION and CPPUNIT_PLUGIN_IMPLEMENT macros). The following settings are needed in the makefile.mk:

• ENABLE_EXCEPTIONS = TRUE
• CFLAGSCXX += $(CPPUNIT_CFLAGS)
• SHLnRPATH = NONE
• SHLnSTDLIBS += $(CPPUNITLIB)
• SHLnVERSIONMAP = version.map, where that file must contain the following (see #CppUnit_1.12.1 details below for improving this):

 UDK_3_0_0 {
   global:
     cppunitTestPlugIn;
   local:
     *;
 };

To run tests, sal/cppunittester delivers a cppunittester executable that has some advantages over the plain CppUnit DllPlugInTester:

• It uses SAL_IMPLEMENT_MAIN to properly set up the per-process settings required by URE. Test code as well as tested code based on URE will expect this and would fail in a process that is not set up in this way.
• Getting CppUnit to build as an external module within the Windows OOo build is a non-trivial task. The less parts of it that need to be built, the better.

Something like the following in makefile.mk will execute the tests:

 ALLTAR : test
 test .PHONY : $(SHL1TARGETN)
   $(CPPUNITTESTER) $(SHL1TARGETN)

$(CPPUNITTESTER) internally uses $(AUGMENT_LIBRARY_PATH_LOCAL) to favor library instances in the local output tree over ones in solver. Alternatively to the above, .INCLUDE: _cppunit.mk calls $(CPPUNITTESTER) on each of $(SHL1TARGETN), …, $(SHL10TARGETN) that is defined.

JUnit

For tests in Java, plain JUnit (at least version 4) is available. (OOo requires JUnit as a prerequisite, see junit.org for downloads; you can configure --with-junit=… if it is not automatically found; alternatively, you can configure --without-junit, disabling JUnit-based tests.)

Details

Working on CWS sb118  , CWS sb120  .

CppUnit 1.12.1

• Replace existing cppunit and testshl2 combo with latest CppUnit 1.12.1.
• OOo configure --with-system-cppunit (for now only if also --with-system-stl!); minimal required version for now arbitrarily 1.12.1.
• TODO: Upstream PATCH_FILES (windows.patch?!?).
• TODO: RPATH in generated libraries and executables.
• TODO: Improve CPPUNIT_PLUGIN_EXPORT and get rid of version maps (GCC exception problems?).
• TODO: cppunit/makefile.mk MY_LIBS = -lm Solaris hack.
• TODO: In general OOo build environment, no longer set CFLAGS etc. as environment variables; have package-specific OOO_package_CFLAGS etc. and combine those into CFLAGS etc. where necessary (see OOO_STLPORT_CXXFLAGS etc. in cppunit/makefile.mk).
• TODO: Why use cppunittester instead of DllPlugInTester? (Remove building/delivering DllPlugInTester.)
• TODO: Make CppUnit::assertion_traits<rtl::OUString> from smoketestoo_native/smoketest.cxx generally available.

testshl2

• solenv/inc_cppunit.mk/cppunit.mk: “adapted cppunit.mk to cppunittester and no longer require ‘dmake test’ to run tests”.
• TODO: DEV300_m68 introduced TESTSHL2LIB (which I removed again) and uses it in many qa makefiles (that I need to touch someday, anyway).

Unit Tests

• OOO_SUBSEQUENT_TESTS (build.lst lists local dependencies of those directories, so that modifying a module and then calling “OOO_SUBSEQUENT_TESTS=x build” rebuilds the module and executes the tests).
• TODO: If Build Environment 2.0 would build directly to solver (instead of local output tree/deliver), that would simplify tests that currently need to take care to test local libraries instead of solver ones (see, for example, OOO_TEST_PREFIX in the services.rdb of stoc/test/uriproc/makefile.mk).
• TODO: Constructs like “-env:UNO_…=$(my_file)…” would fail for problematic characters (spaces!) in paths.

Smoke Test

• Redesign smoketestoo_native to use CppUnit and pre-installed OOo.
• TODO: For smoketestoo_native/makefile.mk “-env:arg-env=$(OOO_LIBRARY_PATH_VAR)"$${{$(OOO_LIBRARY_PATH_VAR)+=$$$(OOO_LIBRARY_PATH_VAR)}}"” mechanism: osl_executeProcess ustrEnvironments members without “=”: all platforms; documentation; documentation of osl_executeProcess_WithRedicrectedIO.
• TODO: Dynamically create smoketestdoc.sxw?
• TODO: “changed tools/debug to never fall back from writing to shell to writing to file, and read DBGSV_INIT from rtl::Bootstrap; the latter causes problems if tools/debug is called from static code executing before SAL_MAIN has called osl_setCommandArgs → removed statics from code and added assertions to osl/process; various improvements to smoketests.cxx (incl. passing an appropriate dbgsv.ini for non-pro builds)”: Instead of -env:DBGSV_INIT= in smoketest.cxx (and supporting DBGSV_INIT as bootstrap variable), globally in the build environment let DBGSV_INIT environment variable point to a dbgsv.ini with error=coredump, to catch all assertions thrown during build time (incl. ones from tests like smoke test; see Issue 109142 ).
• TODO: -headless? (appears to still display windows on Windows and Mac OS X, however, see Issue 109052 )

Pre-Installed OOo

• For Sun Hamburg Release Engineering, take MWS archive installation sets from “S:” instead of solver: “take installation set from SHIPDRIVE if appropriate”.
• TODO: PKGFORMAT=installed instead of archive?

unoapi Tests

• Known issues: Issue 109517 , Issue 109518 , Issue 109643 , Issue 109917 .
• Missing tests: Issue 109564 .
• Sporadic failures: Issue 109728 , Issue 109770 , Issue 109916 , Issue 109939 , Issue 110253 .