QAAutomation

From Apache OpenOffice Wiki
Revision as of 10:24, 29 June 2007 by Hde (Talk | contribs)

Jump to: navigation, search

The OOo QA Automation Team has the main responsibility to assure stability and quality of the whole OOo/SUN Staroffice. The automated GUI testing provides a test framework with test scripts and an application (VCL TestTool) to test almost the whole office application automatically. The VCL TestTool scripts are written in BASIC with some additional functions especially for the office. The VCL TestTool communicates via TCP/IP with the office application. To start the GUI testing it is recommended to download the scripts (TestTool Environment) and the VCL TestTool application for the platform where you want to run the tests. home of automation team for further details

Test cycles (category's)

Meanwhile there are a lot of automated tests available. With this number of tests it would last nearly 1 week to test 1 build completely. After all automated tests passed the testers have to verify the results...this will take also some time. Because of this conditions the QA Automation Team decided to create a test cycle that covers a meaningful number of tests run on each build. The outcome of this category's are 4 in numbers

  • 1 : Tests run on all builds and releases
  • 2 : Tests run on the first build (plus tests from category 1)
  • 3 : Tests run on the second build (plus tests from category 1)
  • 4 : Tests run on the third build (plus tests from category 1)

To make it easy for all involved the rules what type of test is included in what category is not strict documented. Except category 1 which holds all automated tests required for release testing (mostly resource or update-tests). For all other 3 category's it should be observed they have the approximate duration. This model shortens a test cycle for each build to a maximum of 2,5 days. A list of all automated tests and its category's can be found here => All current tests

QA Automation Alert Levels

On this page you'll find an explanation of 'Master-Quality-Meter' (not yet released on OOo) generated by automated tests running on every master workspace. This will be achieved by the VTTDI-Index defined by the QA Automation Team to make a simple understanding of quality status. The VTTDI-Index is calculated by the following formula: '(warnings * 1) + (errors * 3) / number of tests'

Automation Quality status: green
  • Alert level: 1 ( VTTDI <= 0.3 )
  • release and category tests have been finished
  • No critical issues have been found during automated tests
Automation Quality status: lookout
  • Alert level: 2 ( VTTDI >=0.3 and <= 0.5 )
  • release and category tests have been finished
  • Some errors and warnings have been found during automated testing
  • Testresults must be evaluated by QA
  • release might be at risk
Automation Quality status: moderate
  • Alert level: 3 ( VTTDI > 0.5 and <= 1 )
  • release and category tests have been finished
  • Some more errors and warnings have been found during automated testing
  • Testresults must be evaluated by QA
  • release is at risk
Automation Quality status: critical
  • Alert level: 4 ( VTTDI > 1 )
  • release and category tests have been finished
  • Loads of errors and warnings have been found during automated testing
  • Testresults must be evaluated by QA
  • release is not possible
Automation Quality status: severe
  • Alert level: 5 ( VTTDI of any size )
  • release tests have not been finished
  • Testresults must be evaluated by QA
  • release is not possible
Personal tools