Difference between revisions of "QA/BVT"

From Apache OpenOffice Wiki
< QA
Jump to: navigation, search
Line 65: Line 65:
 
  Linux: "/opt/openoffice.org3"<br />
 
  Linux: "/opt/openoffice.org3"<br />
 
  Windows: "C:/Program Files/OpenOffice.org 3"<br />
 
  Windows: "C:/Program Files/OpenOffice.org 3"<br />
−
  Windows 64b: "C:/Program Files (x86)/OpenOffice.org 3"
+
  Windows 64b: "C:\Program Files (x86)\OpenOffice 4"
 
}}
 
}}
 
{{Template:Documentation/Note| '''Can I do other operations when BVT is running?'''<br />
 
{{Template:Documentation/Note| '''Can I do other operations when BVT is running?'''<br />

Revision as of 02:21, 16 April 2013

Introduction

Build Verification Testing, BVT for short, is a set of tests that run on new build to verify that whether the build is testable or not. It is done prior to its release to test team for further testing(So get latest build for test is more meaningful). This testing is done for Build Validation and Build Acceptance.

Template:Documentation/Tip

Prerequisites

Platform

Test machine with OS in below list: WindowsXP, Windows7 32bit, Windows7 64bit,Ubuntu, Suse, Redhat,Mac

AOO Build

AOO4.0 build snapshots
Below are builds for AOO4.0 trunk

  • Linux 32bit Nightly in buildbots table
  • Linux 64bit Nightly in buildbots table
  • Windows Nightly in buildbots table

Environment tools

  • SVN Client:

On Windows: Download and Install TortoiseSVN as SVN Client to checkout sourcecode.
http://tortoisesvn.net/downloads.html

Soft Skills

  • Basic skills about Windows Command line, Linux/Mac Terminal
  • Basic knowledge of Java

How to run BVT automatically

Getting the source code

Anyone can checkout source code from our Subversion repository. Run the following command. 

svn co https://svn.apache.org/repos/asf/openoffice/trunk/test/ test

On Windows, you can use TortoiseSVN GUI Client to checkout, URL of repository is https://svn.apache.org/repos/asf/openoffice/trunk/test/

Run testing

As a normal user, how to run testing against an installed OpenOffice?
Firstly compile the project with the command under test module. 

ant -Dopenoffice.home="OpenOffice installation directory" compile

Then start testing with the command "run". e.g. Run all test classes under package bvt 

run -Dopenoffice.home="/Applications/OpenOffice.org.app/Contents" -tp bvt

e.g. Run the given test classes 

run -Dopenoffice.home="/Applications/OpenOffice.org.app/Contents" -tc bvt.gui.BasicFunctionTest


Where to get the testing result?

By default, the testing output is stored in "test/testspace/output***". Open "test/testspace/output/result.html" in your browser to see the testing report.

Special stuff in the output directory. 

result.xml: Test result in XML. Every case's detail result, may include case error stack.
result.html: Test result in HTML. Collect all case's result, to report as HTML.
screenshot/: Screenshot pictures when test assert is failed.

Sample on Windows

Running Platform: Windows 7 64bit

  • Install OO, use default install directory. "C:/Program Files (x86)/OpenOffice.org 3"

"OOo-dev 3.5 (en-US) Installation Files" is directory for install files. not default installation directory.

  • Checkout sourcecode to "C:\test"
  • Change CMD directory to "C:\test",then compile test use: ant -Dopenoffice.home="C:/Program Files (x86)/OpenOffice.org 3" compile
  • Change CMD directory to "C:\test",then execute BVT: run -Dopenoffice.home="C:/Program Files (x86)/OpenOffice.org 3" -tp bvt

FAQ

Template:Documentation/Note Template:Documentation/Note Template:Documentation/Note

Task Process

1)Track your execution task on [BVT Report ]

  • Platform: Platform which you can run BVT on it.
  • Volunteer: Volunteer's name who take this execution task.
  • Revision: build version of AOO4.0
  • Status: Pass/Fail
  • Issue Number:Issue Number on BZ which failed BVT case .
Revision Platform Volunteer Status Issue Number
WindowsXP
Windows7 32bit
Windows7 64bit
Ubuntu
Suse
Redhat
Mac

2)Update your execution result on "Status" column, and add detail HTML result in new link(Record link on [BVT Report ] ).
3)If have fail/error case, you need check it manually. To confirm it's exact a bug. (For some BVT cases write via GUI(Graphical User Interface), it have some offset).
In this step, you need study to read cases. [Dev GUI Test] [A step by step example]
4)If there is a defect, you can open it on [Bugzilla],select specific editor as it's category, then record on [BVT Report ]
5)If you finish all steps above, send your result to QA mail list with your report link.

Appendix

BVT Case Detail

Currently, BVT include 2 classes. These test cases are based on VCLAuto, and verify main functionalities.

  1. Integrating smoke test from AOO.
  2. Test cases cover 5 products: word processor, spreadsheet, presentation, drawing, and equation editor.
  3. Testing main operations about each products: new, save, load, reopen.
  4. Objects operations: picture, chart, table, function.
  5. Other test points: export as pdf, print, about dialog, slide show.

Below is the detailed introduction of each case.

FileTypeTest.java

Test Point
Function Description

testSaveNewODT

testSaveNewOTT

testSaveNewSXW

testSaveNewSTW

testSaveNewDOC

testSaveNewTXT

  • Create a new text document.
  • Input some texts by keyboard.
  • Set the texts style.
  • Save the document. One case is corresponding to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the text still exists in the file.

testSaveNewODS

testSaveNewOTS

testSaveNewSXC

testSaveNewSTC

testSaveNewXLS

  • Create a new spreadsheet document.
  • Input some texts into a cell by keyboard.
  • Set style of the texts.
  • Save the document. One case is corresponding to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the text still exists in the file.

testSaveNewODP

testSaveNewOTP

testSaveNewPPT

testSaveNewPOT

testSaveNewSXI

testSaveNewSTI

  • Create a new presentation document.
  • Input some texts by keyboard.
  • Save the document. One case is corresponding to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the text still exists in the file.

testSaveNewODG

testSaveNewOTG

testSaveNewSXD

testSaveNewSTD

  • Create a new draw document.
  • Insert a picture from file.
  • Save the document. One case is corresponding *to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the picture still exists in the file.

testSaveNewODF

testSaveNewSXM

testSaveNewMML

  • Create a new formula document.
  • Insert a formula.
  • Save the document. One case is corresponding to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the formula still exists in the file.


BasicFunctionTest.java

Test Point

Function Description

SmokeTest

  • Create documents: word processor, spreadsheet, presentation,drawing, and equation editor.
  • Save above documents to supported types. e.g. Save wordprocessor as .odt, .sxw files.
  • Close above documents.
  • Open a database, insert/delete/seek data in the database,close the database.
  • Install and uninstall an extension.
  • Write test results in log files.

testExportAsPDF

  • Open an exist .odt file and export it as pdf by menu operation.
  • Use toolbar to export a file to pdf file.

testPrinter

Check if print file dialog exist

testRunMacro

Run basic Macro

testHelp

Test the About Dialog show

testInsertPictureInDocument

Test inserting a picture in text document

testInsertPictureInSpreadsheet

Test inserting a picture in spreadsheet document

testInsertPictureInPresentation

Test inserting a picture in presentation document

testSlideShow

Check slide show correctly

testFind

  • Open an existing .odt file.
  • Find a text in the file.
  • Replace a text in the file.

testFillInSpreadsheet

Test automatic filling function of spreadsheet.

Fill Down/Right/Up/Left/Series...

testSort

Test sort function of spreadsheet.

testInsertChartInDraw

Test inserting a chart in a draw document.

testInsertChartInDocument

Test inserting a chart in a text document

testInsertChartInSpreadsheet

Test inserting a chart in a spreadsheet document.

testInsertChartInPresentation

Test inserting a chart in a presentation document.

testInsertTableInDraw

Test inserting a table in a draw document.

testInsertTableInDocument

Test inserting a table in a text document

testInsertTableInPresentation

Test inserting a table in a presentation document

testSumInFormulaBar

Test inserting a function in a spreadsheet document via Sum button.

testInsertFunctionViaFormulaBar

Test inserting a function in a spreadsheet document via input line.

testFunctionWizardInFormulaBar

Test inserting a function in a spreadsheet document via Function Wizard Dialog



Reference

Retrieved from "https://wiki.openoffice.org/w/index.php?title=QA/BVT&oldid=217898"
Personal tools