Difference between revisions of "QA/Build Verification Test(BVT)"

From Apache OpenOffice Wiki
< QA
Jump to: navigation, search
(→‎TODO)
(→‎Introduction)
Line 20: Line 20:
 
Vclauto BVT is based on vclauto framework. It is GUI testing. Sometimes UNO API tests have no problem, but UI can not display correctly. Vclauto BVT can effectively avoid such mistakes. It performs testing like a real user. It generates keyboard/mouse events, does GUI actions and gets information from the GUI to validate the function.<br />
 
Vclauto BVT is based on vclauto framework. It is GUI testing. Sometimes UNO API tests have no problem, but UI can not display correctly. Vclauto BVT can effectively avoid such mistakes. It performs testing like a real user. It generates keyboard/mouse events, does GUI actions and gets information from the GUI to validate the function.<br />
 
Further more, vclauto BVT already integrated  smoketestoo_native test into it.<br />
 
Further more, vclauto BVT already integrated  smoketestoo_native test into it.<br />
 +
For more information, you can refer to[[QA/vclauto]]<br />.
  
 
== Framework ==
 
== Framework ==

Revision as of 06:54, 2 July 2012

Introduction

Build Verification Testing, BVT for short, is a set of tests that run on new build to verify that whether the build is testable or not. It is done prior to its release to test team for further testing. This testing is done for Build Validation and Build Acceptance.

Why do we need BVT?
BVT is a type of regression test and only take short time, like 30 - 60 minutes, to verify that whether the build is stable and can be tested thoroughly. BVT can save the efforts of testers to setup and test a build when major functionalities are having defects. If any of the tests fails, BVT will report to developers immediately to remind relevant developers to review source codes.

What's the difference between vclauto BVT and smoketestoo_native?
Currently, there are two modules in AOO can do build verification test.

  • smoketestdoc

It's used to generate the test documents required by smoke test.

  • smoketestoo_native

A small test suite to verify if the basic functions of OpenOffice.org work.

Smoketestoo_native uses UNO API/Macro to perform testing. It is based on basic language. It is hard to extend functionalities because of its drawbacks.

  • The poor IDE.
  • Hard to debug.
  • Hard to analyze the testing result.
  • Basic language is not popular.

Vclauto BVT is based on vclauto framework. It is GUI testing. Sometimes UNO API tests have no problem, but UI can not display correctly. Vclauto BVT can effectively avoid such mistakes. It performs testing like a real user. It generates keyboard/mouse events, does GUI actions and gets information from the GUI to validate the function.
Further more, vclauto BVT already integrated smoketestoo_native test into it.
For more information, you can refer toQA/vclauto
.

Framework

Currently, vclauto BVT include 4 classes, a total of 48 test cases. These test cases are based on VCLAuto, and verify main functionalities.
1. Integrating smoke test from AOO.
2. Test cases cover 5 products: word processor, spreadsheet, presentation, drawing, and equation editor.
3. Testing main operations about each products: new, save, load, reopen.
4. Objects operations: picture, chart, table, function.
5. Other test points: export as pdf, print, about dialog, slide show.

Below is the detailed introduction of each case.

SmokeTest

The original smoke test contains two test sample files, smoketestdoc.sxw and TestExtension.oxt. How to use them you can refer to http://wiki.services.openoffice.org/wiki/SmokeTest.
In vclauto BVT, smoke test is implemented in SmokeTest.java. Keeping two sample file, using UNO API to trigger the test instead of control event. Test points are following.
1. Create documents: word processor, spreadsheet, presentation, drawing, and equation editor.
2. Save above documents to supported types. e.g. Save word processor as .odt, .sxw files.
3. Close above documents.
4. Open a database, insert/delete/seek data in the database, close the database.
5. Install and uninstall an extension.
6. Write test results in log files.

BVTFileType

Test cases in BVTFileType.java are used to test main operations of supported file types. Test points are below.

Test Case Function Description

testSaveNewODT
testSaveNewOTT
testSaveNewSXW
testSaveNewSTW
testSaveNewDOC
testSaveNewTXT
  • Create a new text document.
  • Input some texts by keyboard.
  • Set the texts style.
  • Save the document. One case is corresponding to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the text still exists in the file.

testSaveNewODS
testSaveNewOTS
testSaveNewSXC
testSaveNewSTC
testSaveNewXLS
  • Create a new spreadsheet document.
  • Input some texts into a cell by keyboard.
  • Set style of the texts.
  • Save the document. One case is corresponding to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the text still exists in the file.

testSaveNewODP
testSaveNewOTP
testSaveNewPPT
testSaveNewPOT
testSaveNewSXI
testSaveNewSTI
  • Create a new presentation document.
  • Input some texts by keyboard.
  • Save the document. One case is corresponding to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the text still exists in the file.

testSaveNewODG
testSaveNewOTG
testSaveNewSXD
testSaveNewSTD
  • Create a new draw document.
  • Insert a picture from file.
  • Save the document. One case is corresponding *to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the picture still exists in the file.

testSaveNewODF
testSaveNewSXM
testSaveNewMML
  • Create a new formula document.
  • Insert a formula.
  • Save the document. One case is corresponding to one file type.
  • Close the document.
  • Reopen the saved file.
  • Verify if the formula still exists in the file.

BVTFunction

Test cases in BVTFunction.java are used to test basic functionalities of AOO. Test points are below.

Test Case Function Description
testExportAsPDF
  • Open an exist .odt file and export it as pdf by menu operation.
  • Use toolbar to export a file to pdf file.
testPrintDialog Check if file dialog exists.
testJavaDialog
  • Create an agenda by agenda wizard.
  • Check if the agenda template file can be created successfully.
testMacroToolsOrgDialog Check if “Basic Macor Organizer” dialog can be displayed successfully.
testAboutDialog Check if “About” dialog exists.
testInsertPictureInDocument Test inserting a picture in text document.
testInsertPictureInSpreadsheet Test inserting a picture in spreadsheet document.
testInsertPictureInPresentation Test inserting a picture in presentation document
testSlideShow
  • Open an existing .odp file.
  • Check if the slides show correctly.
testFind
  • Open an existing .odt file.
  • Find a text in the file.
  • Replace a text in the file.
testFillInSpreadsheet Test automatic filling function of spreadsheet.

Fill Down/Right/Up/Left/Series...

testSort Test sort funciton of spreadsheet.

BVTFunction2

Test Case Function Description
testInsertChartInDraw Test inserting a chart in a draw document.
testInsertChartInDocument Test inserting a chart in a text document.
testInsertChartInSpreadsheet Test inserting a chart in a spreadsheet document.
testInsertChartInPresentation Test inserting a chart in a presentation document.
testInsertTableInDraw Test inserting a table in a draw document.
testInsertTableInDocument Test inserting a table in a text document.
testInsertTableInPresentation Test inserting a table in a presentation document.
testInsertFunctionInSCViaSumButton Test inserting a function in a spreadsheet document via Sum button.
testInsertFunctionInSCViaInputbar Test inserting a function in a spreadsheet document via input line.
testInsertFunctionInSCViaFunctionWizard Test inserting a function in a spreadsheet document via Function Wizard Dialog.

TODO

1. Set up some machines to do BVT, including OS:windows 8, windows 7, windows xp, mac, ubuntu, redhat.
2. Send verification test result to community when daily/weekly build is ready, via mail list or wiki.
3. Develop more test cases, such as database cases.

Retrieved from "https://wiki.openoffice.org/w/index.php?title=QA/Build_Verification_Test(BVT)&oldid=202711"
Personal tools