QA/PVT

Introduction

Performance Verification Test, PVT for short, is a set of tests that run on new build to get the performance indicators of AOO. Performance indicators include consumed time/memory/handle after some operations.
PVT of AOO consists of two parts: GUI and UNO.

GUI PVT

GUI PVT is to simulate a common user to perform basic but core operations of each editor and count the consumed time/memory/handle after each operation.

UNO PVT

UNO PVT is to use UNO API to do basic documents operations, such as loading/saving/closing documents, then count the consumed time.

You can view daily PVT report here: [1]

Prerequisites

Platform

Test machine with OS in below list: WindowsXP, Windows7 32bit, Windows7 64bit,Ubuntu, Suse, Redhat,Mac

AOO Build

AOO4.0 build snapshots
AOO4.0.1 Development Build [2]

Environment tools

• SVN Client:

On Windows: Download and Install TortoiseSVN as SVN Client to checkout sourcecode.
http://tortoisesvn.net/downloads.html

• JDK: JDK 1.6 or above.
  
• Ant: Apache Ant 1.8.2 or above.

Soft Skills

• Basic skills about Windows Command line, Linux/Mac Terminal
• Basic knowledge of Java

How to run PVT automatically

Getting the source code

Anyone can checkout source code from our Subversion repository. Run the following command.

svn co https://svn.apache.org/repos/asf/openoffice/trunk/test/ test

On Windows, you can use TortoiseSVN GUI Client to checkout, URL of repository is https://svn.apache.org/repos/asf/openoffice/trunk/test/

Run testing

As a normal user, how to run testing against an installed OpenOffice?
Firstly compile the project with the command under test module.

ant -Dopenoffice.home="OpenOffice installation directory" compile

Then start testing with the command "run". e.g. Run the given test classes

run -Dopenoffice.home="/Applications/OpenOffice.org.app/Contents" -tc pvt.gui.Benchmark

Where to get the testing result?

By default, the testing output is stored in "test/testspace/output***". Open "test/testspace/output/" in your browser to see the testing report.

Special stuff in the output directory.

pvt.gui.Benchmark:Test result in XML. Every case Every repeat's detail result
screenshot/: Screenshot pictures when test assert is failed.

Sample on Windows

Running Platform: Windows 7 64bit

• Install OO, use default install directory. "C:/Program Files (x86)/OpenOffice.org 4 "
• Checkout sourcecode to "C:\test"
• Change CMD directory to "C:\test",then compile test use: ant -Dopenoffice.home="C:/Program Files (x86)/OpenOffice.org 4" compile
• Change CMD directory to "C:\test",then execute PVT: run -Dopenoffice.home="C:/Program Files (x86)/OpenOffice.org 4 " -tc pvt.uno.Conversion

FAQ

What is openoffice.home? The default installation directory of Apache OpenOffice: Mac OS X: "/Applications/OpenOffice.org.app/Contents" Linux: "/opt/openoffice.org4" Windows: "C:/Program Files/OpenOffice.org 4" Windows 64b: "C:\Program Files (x86)\OpenOffice 4"

Can I do other operations when PVT Gui is running? For PVT Gui cases wrote via GUI(Graphical User Interface) not UNO interface, little display offset may impact test result. So not do other operations when PVT is running.

Do I need compile pvt source code anytime? If there is not any change of pvt source code, there is no need to compile it anytime. That is "Java is compile once, run everywhere"