QUASTe

- Page under construction -

- QUASTe FAQ -

Common

What is "QUASTe" ?

QUASTe is the abbreviation for 'Quality Assurance Statuspage'. The word 'Quaste' is german for 'tassle' so this name found out to be adequate as here 'all strings run together'. (See entry on Wikipedia about Quaste). QUASTe is collecting, validating and comparing all testresults created by VCL-Testtool while testing OpenOffice.org builds and creates easy to read resultspages. Everyone interested in quality of a specific build of OpenOffice.org has the ability to check the results of automated and of course manual tests. QUASTe has been developed by Helge Delfs in programming language PHP consisting of components and modules for the OpenSource Content-Management-System Joomla

QUASTe will be released soon on OpenOffice.org

IRC

There is a chat channel available where you can get support for QUASTe or to discuss features or bugs.
Server: irc.freenode.net
Channel: #quaste.openoffice.org

For Users

Definition of "Release Test"

Release tests are those which must run on every build. These tests should finish without any failure on every build and of course on every Child workspace

Definition of "Required Test"

Required tests are those which must run on a specific build. It depends on the category model what test are required.

Definition of "Optional Test"?

Optional tests are those test must not run on a specific build depending on the category model.

Errors?

Warnlogs?

QA Errorslogs?

Category model

Meanwhile there are a lot of automated tests available. With this number of tests it would last nearly 1 week to test 1 build completely. After all automated tests passed the testers have to verify the results...this will take also some time. Because of this conditions the QA Automation Team decided to create a test cycle that covers a meaningful number of tests run on each build. The outcome of this category's are 4 in numbers

0 : Tests run on all builds and releases
1 : Tests run on the first build (plus tests from category 0)
2 : Tests run on the second build (plus tests from category 0)
3 : Tests run on the third build (plus tests from category 0)

To make it easy for all involved the rules what type of test is included in what category is not strict documented. Except category 0 which holds all automated tests required for release testing (mostly resource or update-tests). For all other 3 category's it should be observed they have the approximate duration. This model shortens a test cycle for each build to a maximum of 2,5 days. A list of all automated tests and its category's can be found here => All current tests

What is the understanding of VTTDI?

See VTTDI-Page for further details on VTTDI

QUASTe: Error while adding testresults for testplan: xyz.bas

This is either because you are owner of this autotest or you were running this autotest and there some errors while adding the testresults to QUASTe database. Take a look into the content of this mail and see if you can find out some details about this error.

There is a testrun already available

'You can only update current testrun if you restart as user: '<username>' on machine: '<machinename>

If a testrun has been finished and is written into database there are some dependencies.

If there is no similar testrun found the testrun will be added.
If there is already a similar testrun it will be updated, but:
- only if the autotest was started on the same milestone depending on language, Machine and User. If these parameters not match the testresults will not be updated nor written to database. The tester will receive an errormail about this.

This behaviour found to be the best solution because testresults will not be overwritten by another tester. This avoids multiple testresults and saves space in database. Notice: On distributed testing one has to know what should be pass next and which test is already passed. This saves time.....

Incomplete testruns only allowed if complete previous testrun exists

Before it is possible to submit an incomplete testrun there must always exist a complete testrun in database. A complete testrun means all testcases belong to a test and checked in to QUASTe have to be run and submitted to QUASTe. Afterwards it is possible to start incomplete tests (e.g. contain only a collection of testcases) to submit changes in a test (see also There is a testrun already available). Normally QUASTe recognizes if a testcase has been renamed or removed from a test but in some cases the developer of a test has to fix changes via QUASTe-Backend. If you encounter problems with a test please contact the owner of a test.

Testcase: XYZ not found in resultfile but in database

This message could have different reasons, but it is easy to fix. If you have this testcase enabled for multiple autotests and f.E. excluded for this autotest you must 'disable' this testcase for this autotest in 'QUASTe Autotest-Mananger'. See 'How to disable testcases or tests for a specific Masterworkspace' (to be done) The other case could be that this testcase isn't called by bas-file. See if this testcase is excluded either in bas-file or in inc-file. If so and this testcase isn't needed anymore you have to remove this testcase from includefile and run 'Update test information' in 'QUASTe Autotest-Manager': How to update autotest information in QUASTe' (to be done) If you are not registered to QUASTe or you are not the owner of the file please contact file owner to commit these changes.

No valid mail-address or machine-name found

If you're starting automated tests and want them to be added to QUASTe it is required that you submit the name of the testing machine and a valid mail-address. Both are part of the testtool.ini (on unix .testtoolrc) Please check your testtool.ini (found in your home-directory) if the following entrys are filled with details:

[Others]

PCname={YOUR-PCNAME}

[Crashreporter]

ReturnAddress={vaild mail-address}

If these values not exist it is mandatory to create them. Re-run your test and results will be added to QUASTe

Helpful links

- the homepage of the QA project on OOo : http://qa.openoffice.org
- the mailing list of the QA project : http://qa.openoffice.org/servlets/SummarizeList?listName=dev
- query for existing bugs in IssueTracker : http://qa.openoffice.org/issues/query.cgi
- writing a new issue in IssueTracker : http://qa.openoffice.org/issue_handling/pre_submission.html
- the QA category on OOo wiki : Quality Assurance

Known issues

- Errormails about existing testresults send to CWS-Owner
If current tested Workspace is as CWS and someone wants to add results to database for a test that already exists the CWS Owner gets a mail in which he can decide whether testresults are updated or not !

- Add detailed testcase overview
Show overview of a selected testcase on what version it has been run

- Auto-Logout after unpublishing tests
When logged in to frontend and a test is unpublished by clicking the unpublish-button twice the user is logged out

- Global testcases are not visible in QUASTe
If a global testcase is called within an include file it is not visible in QUASTe. Seems I must read called testcases from header

- Possibility to compare results of cws/cws
It should be possible to compare testresults of 2 CWS'se like comparing of MWS and CWS testresults.

- Response-file for automated script update
If sourcetree is updated automatically there should be a response file created telling if there were errors or not !

- Handle testcase specifications
QUASTe should be able to handle Testcase specifications. Generating, viewing, editing of testcase specifications and evaluating of testresults should be possible