Difference between revisions of "QA/test automation guide"

Revision as of 07:21, 10 August 2012

Introduction

VCLAuto is a Java library for OpenOffice UI/functional testing like VCLTesttool. VCLAuto can be used with JUnit. The project is under developing and will be released in Apache OpenOffice 4.0.

There are a lot of testing codes in the OO project, e.g. unit test, qadevooo and smoketestoo_native. Why do we need the library?

Generally, unit test code (pure junit/cppunit test) is executed in the build process before OO is created. It's low-level and used to verify if the source code is right before they are linked to a real product. It's a white-box testing. If you want to verify a function/method/interface is correct, put your test code in unit test. qadevooo & smoketestoo_native uses UNO API/Macro to perform testing. To run them, a real OO product must be available. It's middle-level and used to verify if UNO api and business model work correctly. It's a gray-box testing. User interaction is not involved in this testing. It can't fully simulate a user behavior or check if GUI is correct. For example, check if user can draw a shape by dragging, check if OO automatically capitalize the first letter after typing a word, check if a button's checked when user change the selection, etc. Vclauto is high-level black-box testing. It performs testing more like a real user. It generates keyboard/mouse events, does GUI actions and gets information from the GUI to validate the function.

Background

Actually VCLAuto is a java version of VCLTesttool, which connects to the automation server (automation module) in OpenOffice with socket. With the "-enableautomation" argument, OpenOffice will start the server with listening on the port 12479 be default. VCLTesttool is heavily used by the QA team in SUN/Oracle, but many people doesn't like it because of its drawbacks.

• The poor IDE. (Java has many enough powerful IDEs to make writing code easily, e.g. Eclipse / Netbeans)
• Hard to debug.
• Hard to read the code and analyze the testing result. (A lot of code is written in Non-English. Maybe German)
• Too many errors
• Basic language is not popular.

I found Test_Cleanup project starts to unify all testing code to follow the standard xUnit style, so I think it's time to clean up GUI test as well.

Getting the source code

Anyone can checkout source code from our Subversion repository. Run the following command.

svn co https://svn.apache.org/repos/asf/incubator/ooo/trunk/main/test/ test

If you are not familiar with Subversion, see our Subversion Basics for more information.

Getting started with Eclipse

Prerequisites

• Eclipse Java IDE: Eclipse Juno(4.2) IDE for Java Developers or above.
  

Setup project

Step 1. Open Eclipse, click menu "File->Import...", and then select "General->Existing Projects into Workspace".
File:Vclauto guide 1.png
Step 2. Click next, set "Select root directory" to the source code directory, and then check the following projects.

testcommon: The project contains the common test utilities and low-level implementation to do GUI testing
testgui:  The project contains the GUI testing scripts. Test case should be written in this project.

File:Vclauto guide 2.png
Step 3. Click "Finish" to import the projects.

Run testing

Step 1. Set AOO's installation location firstly. If AOO is installed in the default directory, you can skip this step.
Otherwise, click menu "Window->Preferences" ("Eclipse->Preferences" on Mac), select "Java->Installed JREs" page and then select the checked JRE.
File:Vclauto guide 10.png
Click "Edit" button and then add the following line in "Default VM Arguments"

-Dopenoffice.home="Your OpenOffice installation directory which contains soffice.bin"

File:Vclauto guide 12.png
Click "Finish".
Step 2. Select one test class. e.g. testcase.gui.SayHelloToOO, and then click "Run" on the toolbar to start testing
File:Vclauto guide 14.png
When testing is finished, JUnit view will show the result.
File:Vclauto guide 16.png

Getting started with command line

Prerequisites

• JDK: JDK 1.5, JDK 1.6 or above.
  
• Ant: Apache Ant 1.8.2 or above.
• JUnit: JUnit 4.10 or above. For lazy people, if internet is available, it can be automatically installed during testing.

Run testing

How do I run testing on OpenOffice? Run the following commands.

cd test
ant -Dtest.classes="Test class list" -Dopenoffice.home="Your OpenOffice installation directory which contains soffice.bin"

e.g. Run testcase/gui/SayHelloToOO and testcase/gui/SmokeTest against OpenOffice installed in "/home/test/OpenOffice.org.app"

cd test
ant -Dtest.classes="testcase/gui/SayHelloToOO.class,testcase/gui/SmokeTest.class" -Dopenoffice.home="/home/test/OpenOffice.org.app/Contents/MacOS"

If test.classes is not specified, BVT will be executed. If openoffice.home is not specified, it will try to test OpenOffice installed in the default directory. e.g. Run BVT against OpenOffice in the default directory.

cd test
ant

Template:Documentation/Note

As a developer, how do I run testing against my own build?
It's easy for developers to run testing after building OpenOffice. One thing you need to do is to make sure you run the commands in the build environment. The script will automatically install your build and then start testing. e.g.

source LinuxX86Env.Set.sh
cd test && ant

Where to get the testing result?

By default, the testing result is stored in "test/testspace/output". There are several directories under it.

report/: Test result in HTML. Open "test/testspace/output/report/index.html" in your browser.
result/: Test result in XML.
screenshot/: Screenshot pictures when test assert is failed.
logs/: The detail log.

File:Vclautoreport.png

Development

You are interested in developing test scripts, read QA/dev_gui_testing to get more information.