|
|
| (6 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| | == Introduction == | | == Introduction == |
| − | Build Verification Testing, BVT for short, is a set of tests that run on new build to verify that whether the build is testable or not. It is done prior to its release to test team for further testing. This testing is done for Build Validation and Build Acceptance.<br /> | + | Build Verification Testing, BVT for short, is a set of tests that run on new build to verify that whether the build is testable or not. It is done prior to its release to test team for further testing. This testing is done for Build Validation and Build Acceptance. |
| | | | |
| − | '''Why do we need BVT?'''<br /> | + | {{Tip|'''Why do we need BVT?''' |
| − | BVT is a type of regression test and only take short time, like 30 - 60 minutes, to verify that whether the build is stable and can be tested thoroughly. BVT can save the efforts of testers to setup and test a build when major functionalities are having defects. If any of the tests fails, BVT will report to developers immediately to remind relevant developers to review source codes.<br /> | + | BVT is a type of regression test and only take short time, to verify that whether the build is stable and can be tested thoroughly. BVT can save the efforts of testers to setup and test a build when major functions are having defects. If any of the tests fails, BVT will report to developers immediately to remind relevant developers to review source codes. |
| | + | }} |
| | | | |
| − | '''What's the difference between vclauto BVT and smoketestoo_native?'''<br />
| + | All tests in BVT are automated, included in package bvt. Smoke test is included in BVT. It takes about 20 minutes to finish the test. Please refer to [[QA/test_automation_guide|Getting Started with Test Automation]] for further information. You can view daily BVT result on our [https://people.apache.org/~liuzhe/testdashboard/#bvt test dashboard]. |
| − | Currently, there are two modules in AOO can do build verification test.<br />
| + | |
| − | * smoketestdoc<br />
| + | |
| − | It's used to generate the test documents required by smoke test.<br /> | + | |
| − | * smoketestoo_native<br />
| + | |
| − | A small test suite to verify if the basic functions of OpenOffice.org work.<br />
| + | |
| | | | |
| − | Smoketestoo_native uses UNO API/Macro to perform testing. It is based on basic language. It is hard to extend functionalities because of its drawbacks. <br />
| + | [[Category:Quality Assurance]] |
| − | * The poor IDE.
| + | |
| − | * Hard to debug.
| + | |
| − | * Hard to analyze the testing result.
| + | |
| − | * Basic language is not popular.
| + | |
| − | | + | |
| − | Vclauto BVT is based on vclauto framework. It is GUI testing. Sometimes UNO API tests have no problem, but UI can not display correctly. Vclauto BVT can effectively avoid such mistakes. It performs testing like a real user. It generates keyboard/mouse events, does GUI actions and gets information from the GUI to validate the function.<br />
| + | |
| − | Further more, vclauto BVT already integrated smoketestoo_native test into it.<br />
| + | |
| − | For more information, you can refer to[[QA/vclauto]]<br />.
| + | |
| − | | + | |
| − | == Framework ==
| + | |
| − | Currently, vclauto BVT include 4 classes, a total of 48 test cases. These test cases are based on VCLAuto, and verify main functionalities.<br />
| + | |
| − | 1. Integrating smoke test from AOO.<br />
| + | |
| − | 2. Test cases cover 5 products: word processor, spreadsheet, presentation, drawing, and equation editor.<br />
| + | |
| − | 3. Testing main operations about each products: new, save, load, reopen.<br />
| + | |
| − | 4. Objects operations: picture, chart, table, function.<br />
| + | |
| − | 5. Other test points: export as pdf, print, about dialog, slide show.<br />
| + | |
| − | | + | |
| − | Below is the detailed introduction of each case.<br />
| + | |
| − | === SmokeTest ===
| + | |
| − | The original smoke test contains two test sample files, smoketestdoc.sxw and TestExtension.oxt. How to use them you can refer to http://wiki.services.openoffice.org/wiki/SmokeTest.<br />
| + | |
| − | In vclauto BVT, smoke test is implemented in SmokeTest.java. Keeping two sample file, using UNO API to trigger the test instead of control event. Test points are following.<br />
| + | |
| − | 1. Create documents: word processor, spreadsheet, presentation, drawing, and equation editor.<br />
| + | |
| − | 2. Save above documents to supported types. e.g. Save word processor as .odt, .sxw files.<br />
| + | |
| − | 3. Close above documents.<br />
| + | |
| − | 4. Open a database, insert/delete/seek data in the database, close the database.<br />
| + | |
| − | 5. Install and uninstall an extension.<br />
| + | |
| − | 6. Write test results in log files.<br />
| + | |
| − | | + | |
| − | === BVTFileType ===
| + | |
| − | Test cases in BVTFileType.java are used to test main operations of supported file types. Test points are below.<br />
| + | |
| − | {| class="wikitable"
| + | |
| − | |-
| + | |
| − | !Test Case
| + | |
| − | !Function Description
| + | |
| − | |-
| + | |
| − | |
| + | |
| − | testSaveNewODT<br />
| + | |
| − | testSaveNewOTT<br />
| + | |
| − | testSaveNewSXW<br />
| + | |
| − | testSaveNewSTW<br />
| + | |
| − | testSaveNewDOC<br />
| + | |
| − | testSaveNewTXT<br />
| + | |
| − | |
| + | |
| − | *Create a new text document.
| + | |
| − | *Input some texts by keyboard.
| + | |
| − | *Set the texts style.
| + | |
| − | *Save the document. One case is corresponding to one file type.
| + | |
| − | *Close the document.
| + | |
| − | *Reopen the saved file.
| + | |
| − | *Verify if the text still exists in the file.
| + | |
| − | |-
| + | |
| − | |
| + | |
| − | testSaveNewODS<br />
| + | |
| − | testSaveNewOTS<br />
| + | |
| − | testSaveNewSXC<br />
| + | |
| − | testSaveNewSTC<br />
| + | |
| − | testSaveNewXLS<br />
| + | |
| − | |
| + | |
| − | *Create a new spreadsheet document.
| + | |
| − | *Input some texts into a cell by keyboard.
| + | |
| − | *Set style of the texts.
| + | |
| − | *Save the document. One case is corresponding to one file type.
| + | |
| − | *Close the document.
| + | |
| − | *Reopen the saved file.
| + | |
| − | *Verify if the text still exists in the file.
| + | |
| − | |-
| + | |
| − | |
| + | |
| − | testSaveNewODP<br />
| + | |
| − | testSaveNewOTP<br />
| + | |
| − | testSaveNewPPT<br />
| + | |
| − | testSaveNewPOT<br />
| + | |
| − | testSaveNewSXI<br />
| + | |
| − | testSaveNewSTI<br />
| + | |
| − | |
| + | |
| − | *Create a new presentation document.
| + | |
| − | *Input some texts by keyboard.
| + | |
| − | *Save the document. One case is corresponding to one file type.
| + | |
| − | *Close the document.
| + | |
| − | *Reopen the saved file.
| + | |
| − | *Verify if the text still exists in the file.
| + | |
| − | |-
| + | |
| − | |
| + | |
| − | testSaveNewODG<br />
| + | |
| − | testSaveNewOTG<br />
| + | |
| − | testSaveNewSXD<br />
| + | |
| − | testSaveNewSTD<br />
| + | |
| − | |
| + | |
| − | *Create a new draw document.
| + | |
| − | *Insert a picture from file.
| + | |
| − | *Save the document. One case is corresponding *to one file type.
| + | |
| − | *Close the document.
| + | |
| − | *Reopen the saved file.
| + | |
| − | *Verify if the picture still exists in the file.
| + | |
| − | |-
| + | |
| − | |
| + | |
| − | testSaveNewODF<br />
| + | |
| − | testSaveNewSXM<br />
| + | |
| − | testSaveNewMML<br />
| + | |
| − | |
| + | |
| − | *Create a new formula document.
| + | |
| − | *Insert a formula.
| + | |
| − | *Save the document. One case is corresponding to one file type.
| + | |
| − | *Close the document.
| + | |
| − | *Reopen the saved file.
| + | |
| − | *Verify if the formula still exists in the file.
| + | |
| − | |}
| + | |
| − | | + | |
| − | === BVTFunction ===
| + | |
| − | Test cases in BVTFunction.java are used to test basic functionalities of AOO. Test points are below.<br />
| + | |
| − | {| class="wikitable"
| + | |
| − | |-
| + | |
| − | !Test Case
| + | |
| − | !Function Description
| + | |
| − | |-
| + | |
| − | |testExportAsPDF
| + | |
| − | |
| + | |
| − | *Open an exist .odt file and export it as pdf by menu operation.
| + | |
| − | *Use toolbar to export a file to pdf file.
| + | |
| − | |-
| + | |
| − | |testPrintDialog
| + | |
| − | |Check if file dialog exists.
| + | |
| − | |-
| + | |
| − | |testJavaDialog
| + | |
| − | |
| + | |
| − | *Create an agenda by agenda wizard.
| + | |
| − | *Check if the agenda template file can be created successfully.
| + | |
| − | |-
| + | |
| − | |testMacroToolsOrgDialog
| + | |
| − | |Check if “Basic Macor Organizer” dialog can be displayed successfully.
| + | |
| − | |-
| + | |
| − | |testAboutDialog
| + | |
| − | |Check if “About” dialog exists.
| + | |
| − | |-
| + | |
| − | |testInsertPictureInDocument
| + | |
| − | |Test inserting a picture in text document.
| + | |
| − | |-
| + | |
| − | |testInsertPictureInSpreadsheet
| + | |
| − | |Test inserting a picture in spreadsheet document.
| + | |
| − | |-
| + | |
| − | |testInsertPictureInPresentation
| + | |
| − | |Test inserting a picture in presentation document
| + | |
| − | |-
| + | |
| − | |testSlideShow
| + | |
| − | |
| + | |
| − | *Open an existing .odp file.
| + | |
| − | *Check if the slides show correctly.
| + | |
| − | |-
| + | |
| − | |testFind
| + | |
| − | |
| + | |
| − | *Open an existing .odt file.
| + | |
| − | *Find a text in the file.
| + | |
| − | *Replace a text in the file.
| + | |
| − | |-
| + | |
| − | |testFillInSpreadsheet
| + | |
| − | |Test automatic filling function of spreadsheet.
| + | |
| − | Fill Down/Right/Up/Left/Series...
| + | |
| − | |-
| + | |
| − | |testSort
| + | |
| − | |Test sort funciton of spreadsheet.
| + | |
| − | |}
| + | |
| − | | + | |
| − | === BVTFunction2 ===
| + | |
| − | {| class="wikitable"
| + | |
| − | |-
| + | |
| − | !Test Case
| + | |
| − | !Function Description
| + | |
| − | |-
| + | |
| − | |testInsertChartInDraw
| + | |
| − | |Test inserting a chart in a draw document.
| + | |
| − | |-
| + | |
| − | |testInsertChartInDocument
| + | |
| − | |Test inserting a chart in a text document.
| + | |
| − | |-
| + | |
| − | |testInsertChartInSpreadsheet
| + | |
| − | |Test inserting a chart in a spreadsheet document.
| + | |
| − | |-
| + | |
| − | |testInsertChartInPresentation
| + | |
| − | |Test inserting a chart in a presentation document.
| + | |
| − | |-
| + | |
| − | |testInsertTableInDraw
| + | |
| − | |Test inserting a table in a draw document.
| + | |
| − | |-
| + | |
| − | |testInsertTableInDocument
| + | |
| − | |Test inserting a table in a text document.
| + | |
| − | |-
| + | |
| − | |testInsertTableInPresentation
| + | |
| − | |Test inserting a table in a presentation document.
| + | |
| − | |-
| + | |
| − | |testInsertFunctionInSCViaSumButton
| + | |
| − | |Test inserting a function in a spreadsheet document via Sum button.
| + | |
| − | |-
| + | |
| − | |testInsertFunctionInSCViaInputbar
| + | |
| − | |Test inserting a function in a spreadsheet document via input line.
| + | |
| − | |-
| + | |
| − | |testInsertFunctionInSCViaFunctionWizard
| + | |
| − | |Test inserting a function in a spreadsheet document via Function Wizard Dialog.
| + | |
| − | |}
| + | |
| − | | + | |
| − | == TODO ==
| + | |
| − | 1. Set up some machines to do BVT, including OS:windows 8, windows 7, windows xp, mac, ubuntu, redhat.<br />
| + | |
| − | 2. Send verification test result to community when daily/weekly build is ready, via mail list or wiki.<br />
| + | |
| − | 3. Develop more test cases, such as database cases.<br />
| + | |
Build Verification Testing, BVT for short, is a set of tests that run on new build to verify that whether the build is testable or not. It is done prior to its release to test team for further testing. This testing is done for Build Validation and Build Acceptance.
