Difference between revisions of "Test Refactor"

Revision as of 01:57, 16 July 2012

The page is created for tracking test refactor project.

Current Status

Currently, there are many modules for testing. Note, the mentioned testing here requires a running OpenOffice instances. That means OpenOffice must be installed before testing. It's not the unit test at the code level. The test code is organized in a mess and out of maintain. Even the most are not stable enough to give useful results. It's hard to get started. Three programming languages (Java, Basic and C++) are used. It's not necessary. The summary is here.

• test

Includes reusable code for UNO API test.

• testgraphical

Test tool to test documents by it's graphical representation.

• testautomation

All test scripts for the old VCL Testtool. Nobody maintains it now.

• smoketestdoc

It's used to generate the test documents required by smoke test.

• smoketestoo_native

A small test suite to verify if the basic functions of OpenOffice.org work.

• qadevOOo

UNO API test, complex test.

• {Module}/qa

UNO API test, complex test.
Some contributors have cleaned up the test code, see http://wiki.services.openoffice.org/wiki/Test_Cleanup, but it seems the project is not finished yet.

Target

The target is to reduce module count, organize the testing code clearer, make test code enough stable to integrate with buildbot, and provide more readable test report. Propose to write all test based on JUnit 4, because Java is easier than C/C++, more popular than Basic. Supporting multiple languages need more effort to maintain and leads duplicate more easily. Test code should be broken down into three categories.

1. purely GUI test. It requires graphical user interfaces enabled. This kind of test requires simulating user interaction (posting keyboard/mouse events and accessing GUI).So it can not be performed in headless mode and hard to run in parallel.
2. purely API test. Graphical user interfaces is not required. We can run this kind of test in parallel to save time.
3. Mixed test. This kind of test involves both GUI test and API test. It also can not be parallelized. Generally, I think it's not necessary to mix them.

Suggest to move all test stuff into test module and create several sub-projects in it:

1. test/testcommon. The reusable code for both GUI test and API test. It doesn't depend on any project.
2. test/testuno. API test should implemented in the sub-project.
3. test/testgui. GUI test should implemented in the sub-project.

Developers can still write test code under {MODULE}/qa.

Under test module, we provide Ant script to start all testing and generating test report:

 cd test
 ant

We can select a part of test classes to run (It's useful when we only want to test the impact area):

 ant -Dtest.classes=testcase.uno.*

TODO

1. Create testcommon. Done!
2. Create testgui. Done!
3. Create testuno.
4. Move the code under test module to testcommon.
5. Convert testgraphical to JUnit4 test case.
6. Convert the test scripts in testautomation to testgui.
7. Convert smoketestdoc & smoketestoo_native to JUnit 4 test cases. Done!
8. Convert the test scripts in qadevooo into testuno.
9. Convert the test scripts in ${MODULE}/qa to JUnit 4 test cases.
10. Remove the old code under test, testgraphical, testautomation, smoketestdoc, smoketestoo_native and qadevooo.