Difference between revisions of "SL/QUASTe"

Revision as of 17:15, 28 November 2009

Pogosto zastavljena vprašanja o QUASTe

Splošno

Kaj je 'QUASTe'?

QUASTe zbira, potrjuje in primerja vse rezultate preizkušanja z orodjem VCL-Testtool pri preizkušanju gradenj OpenOffice.org in tvori enostavno berljive strani z rezultati. Vsakdo, ki ga zanima kakovost posamezne gradnje paketa OpenOffice.org, ima možnost, da preveri rezultate samodejnih in seveda ročnih preizkusov. QUASTe je razvil Helge Delfs v programskem jeziku PHP, s komponentami in moduli za odprtokodni sistem upravljanja z vsebinami Joomla.

Kaj pomeni 'QUASTe'?

QUASTe je okrajšava za 'Quality Assurance Statuspage' ('stran stanja zagotavljanja kakovosti'), hkrati pa beseda 'Quaste' v nemškem jeziku pomeni 'čop(ek)', tako da se je to ime izkazalo za ustrezno, saj se tukaj/v tem primeru 'vse niti poganjajo skupaj/naenkrat'. (Oglejte si vnos v Wikipediji o Quaste).

Katero licenco uporablja 'QUASTe'?

Ker so moduli in komponente QUASTe napisane za Joomlo, bodo kmalu na voljo za prenos pod licenco GPL.

IRC

Na voljo je kanal za klepet IRC, kjer lahko pridobite podporo za QUASTe ali pa razpravljate o funkcionalnostih ali hroščih.
Strežnik: irc.freenode.net
Kanal: #quaste.openoffice.org

Za predstavnika za QA

Kako objavimo stanje kakovosti CWS v EIS

Normally if all required autotests created by VCLTesttool are finished and the results added QUASTe it then will sent its allover status (green or red) to EIS application. If it is necessary to re-run complete automated tests or parts of them for this CWS currently the status is not updated automatically in EIS. Therefor the QA-Representative must be logged in to QUASTe and has to update the status manually.

Prijavite se v sistem EIS (http://eis.services.openoffice.org)
sledite povezavam do *svojega* CWS
na strani overview => izberite 'Tests'
v vrstici 'status' click' select the link for TestName: 'AutomationCAT0'

you will then be forwarded to 'QUASTe Analyze page'

here the current details of automated tests compared to MWS are shown
check current quality status and choose if you've decided the results are 'ok' or 'failed'
By click on the relative button the status is submitted to EIS

The buttons (sent status 'green' or 'red' to EIS) give the possibility to overide the status in EIS or to sent a status manually if it could'nt be sent automatically due to e.g. network or other problems. Only the QA-Representative has the possibilty to submit this status (login required) that's the reason why it isn't visible to all browsing QUASTe.

Za uporabnike

Definicija pojma "Release Test"

Preizkusi izdaj ('release tests') so tisti, ki jih je potrebno izvesti za vsako gradnjo. Ti preizkusi se morajo končati brez napak za vsako gradnjo in seveda za vsak CWS ('child workspace').

Definicija pojma "Required Test"

Zahtevani preizkusi ('required tests') so tisti, ki jih je potrebno izvesti za določeno gradnjo. Od modela kategorije je odvisno, kateri preizkusi so zahtevani.

Definicija pojma "Optional Test"

Neobvezni preizkusi ('optional tests') so tisti, ki jih ni potrebno izvesti za določeno gradnjo glede na model kategorije.

Napake / opozorila / zapisniki napak QAErrorlogs

Errorlog

The Errorlog is generated by testtool if a general error occured. It is a possible hint to a crash, undeclared or not-found control. Errorlogs are red coloured.

Warnlog

A warnlog is a hint to a failure or bug occured in office. It is generated during a testrun by VCLTesttool if f.e. a feature does not work as expected by automated test. It is shown in orange color.

QAErrorlog

This is a string generated in Logfile to show specific problems like:

1. a bug has been found, issue has been written but it will not be fixed in currently tested version.

2. a reference to a condition that should be fulfiled to run test correctly. (Empty directorys in some case or a specific path or whatever)

Model kategorije

Meanwhile there are a lot of automated tests available. With this number of tests it would last nearly 1 week to test 1 build completely. After all automated tests passed the testers have to verify the results...this will take also some time. Because of this conditions the QA Automation Team decided to create a test cycle that covers a meaningful number of tests run on each build. The outcome of this category's are 4 in numbers

0 : Preizkusi se izvajajo na vseh gradnjah in izdajah
1 : Preizkusi se izvajajo na prvi gradnji (ter preizkusi iz kategorije 0)
2 : Preizkusi se izvajajo na drugi gradnji (ter preizkusi iz kategorije 0)
3 : Preizkusi se izvajajo na tretji gradnji (ter preizkusi iz kategorije 0)

To make it easy for all involved the rules what type of test is included in what category is not strict documented. Except category 0 which holds all automated tests required for release testing (mostly resource or update-tests). For all other 3 category's it should be observed they have the approximate duration. This model shortens a test cycle for each build to a maximum of 2,5 days. A list of all automated tests and its category's can be found here => All current tests

Opomba: This is not to be used on release-workspaces. Those are using another matrix that can be found in the OOoAutomationReleaseTestMatrix

What is the understanding of VTTDI?

See VTTDI-Page for further details on VTTDI

Kako objavim rezultate mojih preizkusov?

Before you add some testresults created by VCLTesttool on your local machine it is a good idea to check what testresults are missing. The best way to test a specific build would be to organize testing within your team to avoid double work. If you want to know what test for what platform is missing see 'Testresults grouped by application' page.

If you aren't familiar with automated testing please visit Home of automation Team and learn more about automated testing with 'VCL Testtool'

Assuming you've downloaded a 'VCL Testtool Binary' and checked out the qatesttool-project we can start.

1. Once an automated test has been finished its extracts for creating results on QUASTe will be found in the errorlog-directory in the form of txt-files.

2. Simply put these files in a zip-Archive

3. Pojdite na spletno stran QUASTe

4. Prijavite se s svojim uporabniškim imenom in geslom OpenOffice.org

5. Ko ste prijavljeni, you find some menu entrys on right part of QUASTe webpage

6. Kliknite menijsko izbiro 'Upload my testresults'

7. Prebrskajte do prej ustvarjene datoteke zip in pritisnite gumb 'Upload'

8. Čakajte ...

Once all files have been successfully added to QUASTe Database you will receive a message. If you find some error-messages please visit QUASTe-FAQ to get some help about these messages.

Kako odstranim rezultate preizkusov?

One can only delete testresults initially created by himself but of course testresults submitted by other users can be viewed.

Namig: Currently only testresults with errors occured can be removed.

If you want to remove your testresults from QUASTe-Database you must be logged in. Once logged in goto page 'Testresults grouped by application. This page now shows 2 additional columns by comparison if not logged in. First additional column holds the username who created the testresults and the second shows a checkbox. If you now want to remove a testresult check this checkbox and click the 'red-cross-icon' at the end of the page. If you want to remove more than one testresult simply check more than 1 checkboxes and click the 'red-cross-icon' After confirmation the testresults will be removed !

Možne napake

Error while adding testresults for testplan: xyz.bas

This is either because you are owner of this autotest or you were running this autotest and there some errors while adding the testresults to QUASTe database. Take a look into the content of this mail and see if you can find out some details about this error.

There is a testrun already available

'You can only update current testrun if you restart as user: '<username>' on machine: '<machinename>

If a testrun has been finished and is written into database there are some dependencies.

If there is no similar testrun found the testrun will be added.
If there is already a similar testrun it will be updated, but:
- only if the autotest was started on the same milestone depending on language, Machine and User. If these parameters not match the testresults will not be updated nor written to database. The tester will receive an errormail about this.

This behaviour found to be the best solution because testresults will not be overwritten by another tester. This avoids multiple testresults and saves space in database. Notice: On distributed testing one has to know what should be pass next and which test is already passed. This saves time.....

Incomplete testruns only allowed if complete previous testrun exists

Before it is possible to submit an incomplete testrun there must always exist a complete testrun in database. A complete testrun means all testcases belong to a test and checked in to QUASTe have to be run and submitted to QUASTe. Afterwards it is possible to start incomplete tests (e.g. contain only a collection of testcases) to submit changes in a test (see also There is a testrun already available). Normally QUASTe recognizes if a testcase has been renamed or removed from a test but in some cases the developer of a test has to fix changes via QUASTe-Backend. If you encounter problems with a test please contact the owner of a test.

Testcase: XYZ not found in resultfile but in database

This message could have different reasons, but it is easy to fix. If you have this testcase enabled for multiple autotests and f.E. excluded for this autotest you must 'disable' this testcase for this autotest in 'QUASTe Autotest-Mananger'. See 'How to disable testcases or tests for a specific Masterworkspace' (to be done) The other case could be that this testcase isn't called by bas-file. See if this testcase is excluded either in bas-file or in inc-file. If so and this testcase isn't needed anymore you have to remove this testcase from includefile and run 'Update test information' in 'QUASTe Autotest-Manager': How to update autotest information in QUASTe' (to be done) If you are not registered to QUASTe or you are not the owner of the file please contact file owner to commit these changes.

No valid mail-address or machine-name found

If you're starting automated tests and want them to be added to QUASTe it is required that you submit the name of the testing machine and a valid mail-address. Both are part of the testtool.ini (on unix .testtoolrc) Please check your testtool.ini (found in your home-directory) if the following entrys are filled with details:

[Others]

PCname={IME-VAŠEGA-RAČUNALNIKA}

[Crashreporter]

ReturnAddress={veljaven e-poštni naslov}

Če te vrednosti ne obstajajo, jih morate obvezno dodati.

Namig: Če ne želite ponovno izvesti preizkusov, lahko popravite povzetke rezultatov (datoteke .txt) v svoji mapi z zapisniki napak in dodate svoje podatke v naslednjih vrsticah:

hostname={IME-VAŠEGA-RAČUNALNIKA}

username={veljaven e-poštni naslov} => običajno vaš e-poštni naslov OpenOffice.org (uporabnik@openoffice.org)

Enostavno ponovno re-zip spremenjene datoteke in znova prenesite na strežnik!

QA-Representative: If you want to update previous testresults please click link 'Retry'

You will receive this email if you are going to add some testresults to QUASTe database if there are previous testresults already. This previous testresults will not be updated automatically by QUASTe if they were added by some other tester. However in this mail you receive is a link that leads you to a webpage were those results are listed in detail and you as a QA-representative can decide if previous results shall be updated with you data or should be leaved in the database.

No testcases found for testplan. Please contact script owner.

This email will be sent by QUASTe if there is a resultfile that seems to contain a full testrun but there are no testcases found for this autotest in QUASTe-database. Possibly it is a severe problem in original autotest-file. Please contact the owner of the script to investigate this problem. A list of responsibilities for testscripts can be found here.

Glejte tudi:

Prvi koraki - QUASTe

Kako uporabljati Childworkspaces

Koristne povezave

- domača stran projekta QA na OOo: http://qa.openoffice.org
- dopisni seznam projekta QA: http://qa.openoffice.org/servlets/SummarizeList?listName=dev
- poizvedba po obstoječih hroščih v sistemu IssueTracker: http://qa.openoffice.org/issues/query.cgi
- vpis nove zadeve v IssueTracker: http://qa.openoffice.org/issue_handling/pre_submission.html
- kategorija QA na wikiju OOo: Quality Assurance

Znane težave

- Errormails about existing testresults send to CWS-Owner
If current tested Workspace is as CWS and someone wants to add results to database for a test that already exists the CWS Owner gets a mail in which he can decide whether testresults are updated or not !

- Possibility to compare results of cws/cws
It should be possible to compare testresults of 2 CWS'se like comparing of MWS and CWS testresults.

- Response-file for automated script update
If sourcetree is updated automatically there should be a response file created telling if there were errors or not !

- Handle testcase specifications
QUASTe should be able to handle Testcase specifications. Generating, viewing, editing of testcase specifications and evaluating of testresults should be possible

@@ Line 6: / Line 6: @@
 === Kaj je 'QUASTe'? ===
-QUASTe zbira, potrjuje in primerja vse rezultate preizkušanja z orodjem VCL-Testtool pri preizkušanju gradenj/builds OpenOffice.org in tvori enostavno berljive strani z rezultati. Vsakdo, ki ga zanima kakovost posamezne gradnje/build paketa OpenOffice.org, ima možnost, da preveri rezultate samodejnih in seveda ročnih preizkusov.
+QUASTe zbira, potrjuje in primerja vse rezultate preizkušanja z orodjem VCL-Testtool pri preizkušanju gradenj OpenOffice.org in tvori enostavno berljive strani z rezultati. Vsakdo, ki ga zanima kakovost posamezne gradnje paketa OpenOffice.org, ima možnost, da preveri rezultate samodejnih in seveda ročnih preizkusov.
 QUASTe je razvil [http://wiki.services.openoffice.org/wiki/User:Hde Helge Delfs] v programskem jeziku PHP, s komponentami in moduli za odprtokodni sistem upravljanja z vsebinami [http://www.joomla.org Joomla].
 === Kaj pomeni 'QUASTe'? ===
-QUASTe je okrajšava za 'Quality Assurance Statuspage' ('stran stanja zagotavljanja kakovosti'), hkrati pa beseda 'Quaste' v nemškem jeziku pomeni 'tassle', tako da se je to ime izkazalo za adequate, saj se tukaj/v tem primeru 'all strings run together'. (Oglejte si vnos v Vikipediji o [http://de.wikipedia.org/wiki/Quaste Quaste]).
+QUASTe je okrajšava za 'Quality Assurance Statuspage' ('stran stanja zagotavljanja kakovosti'), hkrati pa beseda 'Quaste' v nemškem jeziku pomeni 'čop(ek)', tako da se je to ime izkazalo za ustrezno, saj se tukaj/v tem primeru 'vse niti poganjajo skupaj/naenkrat'. (Oglejte si vnos v Wikipediji o [http://de.wikipedia.org/wiki/Quaste Quaste]).
 === Katero licenco uporablja 'QUASTe'? ===
-Ker so moduli in komponente QUASTe napisane za Joomla CMS, bodo kmalu na voljo za prenos pod licenco GPL.
+Ker so moduli in komponente QUASTe napisane za Joomlo, bodo kmalu na voljo za prenos pod licenco GPL.
 == IRC ==
@@ Line 21: / Line 21: @@
-== For QA-Represenative ==
+== Za predstavnika za QA ==
-=== How to submit CWS's quality status to EIS ===
+=== Kako objavimo stanje kakovosti CWS v EIS ===
 Normally if all required autotests created by VCLTesttool are finished and the results added QUASTe it then will sent its allover status (green or red) to EIS application. If it is necessary to re-run complete automated tests or parts of them for this CWS currently the status is not updated automatically in EIS. Therefor the QA-Representative must be logged in to QUASTe and has to update the status manually.
-* Login to EIS application (http://eis.services.openoffice.org)
+* Prijavite se v sistem EIS (http://eis.services.openoffice.org)
-* follow the links to *your* CWS
+* sledite povezavam do *svojega* CWS
-* on overview page => choose 'Tests'
+* na strani overview => izberite 'Tests'
-* in row 'status' click' select the link for TestName: 'AutomationCAT0'
+* v vrstici 'status' click' select the link for TestName: 'AutomationCAT0'
 [[File:Quaste_eis_link.jpg]]
@@ Line 41: / Line 41: @@
 Only the QA-Representative has the possibilty to submit this status (login required) that's the reason why it isn't visible to all browsing QUASTe.
-== For Users ==
+== Za uporabnike ==
-=== Definition of "Release Test" ===
+=== Definicija pojma "Release Test" ===
-Release tests are those which must run on every build. These tests should finish without any failure on every build and of course on every Child workspace
+Preizkusi izdaj ('release tests') so tisti, ki jih je potrebno izvesti za vsako gradnjo. Ti preizkusi se morajo končati brez napak za vsako gradnjo in seveda za vsak CWS ('child workspace').
-=== Definition of "Required Test" ===
+=== Definicija pojma "Required Test" ===
-Required tests are those which must run on a specific build. It depends on the [http://wiki.services.openoffice.org/wiki/QUASTe#Category_model category model] what test are required.
+Zahtevani preizkusi ('required tests') so tisti, ki jih je potrebno izvesti za določeno gradnjo. Od [http://wiki.services.openoffice.org/wiki/QUASTe#Category_model modela kategorije] je odvisno, kateri preizkusi so zahtevani.
-=== Definition of "Optional Test" ===
+=== Definicija pojma "Optional Test" ===
-Optional tests are those test must not run on a specific build depending on the [http://wiki.services.openoffice.org/wiki/QUASTe#Category_model category model].
+Neobvezni preizkusi ('optional tests') so tisti, ki jih ni potrebno izvesti za določeno gradnjo glede na [http://wiki.services.openoffice.org/wiki/QUASTe#Category_model model kategorije].
-=== Errors / Warnings / QAErrorlogs ===
+=== Napake / opozorila / zapisniki napak QAErrorlogs ===
 {|
@@ Line 73: / Line 73: @@
 |}
-===  Category model ===
+===  Model kategorije ===
 Meanwhile there are a lot of automated tests available. With this number of tests it would last nearly 1 week to test 1 build completely. After all automated tests passed the testers have to verify the results...this will take also some time. Because of this conditions the QA Automation Team decided to create a test cycle that covers a meaningful number of tests run on each build.
 The outcome of this category's are 4 in numbers
-*0 : Tests run on all builds and releases
+*0 : Preizkusi se izvajajo na vseh gradnjah in izdajah
-*1 : Tests run on the first build  (plus tests from category 0)
+*1 : Preizkusi se izvajajo na prvi gradnji (ter preizkusi iz kategorije 0)
-*2 : Tests run on the second build (plus tests from category 0)
+*2 : Preizkusi se izvajajo na drugi gradnji (ter preizkusi iz kategorije 0)
-*3 : Tests run on the third build  (plus tests from category 0)
+*3 : Preizkusi se izvajajo na tretji gradnji (ter preizkusi iz kategorije 0)
 To make it easy for all involved the rules what type of test is included in what category is not strict documented. Except category 0 which holds all automated tests required for release testing (mostly resource or update-tests). For all other 3 category's it should be observed they have the approximate duration. This model shortens a test cycle for each build to a maximum of 2,5 days.
 A list of all automated tests and its category's can be found here => [http://quaste.services.openoffice.org/index.php?option=com_quaste&task=tests_overview&Itemid=11 All current tests]
-'''Note: This is not to be used on release-workspaces. Those are using another matrix that can be found in the [[OOoAutomationReleaseTestMatrix]]'''
+'''Opomba: This is not to be used on release-workspaces. Those are using another matrix that can be found in the [[OOoAutomationReleaseTestMatrix]]'''
 ===  What is the understanding of VTTDI? ===
@@ Line 93: / Line 93: @@
-=== How do I publish my testresults ? ===
+=== Kako objavim rezultate mojih preizkusov? ===
 Before you add some testresults created by VCLTesttool on your local machine it is a good idea to check what testresults are missing. The best way to test a specific build would be to organize testing within your team to avoid double work. If you want to know what test for what platform is missing see [http://wiki.services.openoffice.org/wiki/QUASTe_-_First_steps#Testresults_grouped_by_application 'Testresults grouped by application'] page.
@@ Line 105: / Line 105: @@
 . Simply put these files in a zip-Archive
-. goto [http://quaste.services.openoffice.org QUASTe] webpage
+. Pojdite na spletno stran [http://quaste.services.openoffice.org QUASTe]
-. Log in with your OpenOffice.org username and password
+. Prijavite se s svojim uporabniškim imenom in geslom OpenOffice.org
-. Once logged in, you find some menu entrys on right part of QUASTe webpage
+. Ko ste prijavljeni, you find some menu entrys on right part of QUASTe webpage
-. Click menu entry 'Upload my testresults'
+. Kliknite menijsko izbiro 'Upload my testresults'
-. Browse for previously created zip-file and press 'Upload' button
+. Prebrskajte do prej ustvarjene datoteke zip in pritisnite gumb 'Upload'
-. Wait.....
+. Čakajte ...
 Once all files have been successfully added to QUASTe Database you will receive a message. If you find some error-messages please visit [http://wiki.services.openoffice.org/wiki/QUASTe QUASTe-FAQ] to get some help about these messages.
-=== How do I remove testresults ? ===
+=== Kako odstranim rezultate preizkusov? ===
 One can only delete testresults initially created by himself but of course testresults submitted by other users can be viewed.
-Hint: Currently only testresults with errors occured can be removed.
+Namig: Currently only testresults with errors occured can be removed.
 If you want to remove your testresults from QUASTe-Database you must be logged in. Once logged in goto page  [http://wiki.services.openoffice.org/wiki/QUASTe%2C_First_steps#Testresults_grouped_by_application 'Testresults grouped by application]. This page now shows 2 additional columns by comparison if not logged in. First additional column holds the username who created the testresults and the second shows a checkbox. If you now want to remove a testresult check this checkbox and click the 'red-cross-icon' at the end of the page. If you want to remove more than one testresult simply check more than 1 checkboxes and click the 'red-cross-icon' After confirmation the testresults will be removed !
-== Possible errors ==
+== Možne napake ==
 === Error while adding testresults for testplan: xyz.bas ===
@@ Line 162: / Line 162: @@
 [Others]
-PCname={YOUR-PCNAME}
+PCname={IME-VAŠEGA-RAČUNALNIKA}
 [Crashreporter]
-ReturnAddress={valid mail-address}
+ReturnAddress={veljaven e-poštni naslov}
-If these values not exist it is mandatory to create them.
+Če te vrednosti ne obstajajo, jih morate obvezno dodati.
-'''Hint:''' If you don't want to re-run your tests you can edit the result-extracts (.txt-files) in your errorlog-directory and add your details to the following lines:
+'''Namig:''' Če ne želite ponovno izvesti preizkusov, lahko popravite povzetke rezultatov (datoteke .txt) v svoji mapi z zapisniki napak in dodate svoje podatke v naslednjih vrsticah:
-hostname={YOUR-PCNAME}
+hostname={IME-VAŠEGA-RAČUNALNIKA}
-username={valid mail-address} => normally your Openoffice mail-address (username@openoffice.org)
+username={veljaven e-poštni naslov} => običajno vaš e-poštni naslov OpenOffice.org (uporabnik@openoffice.org)
-Simply re-zip the changed files and upload again !
+Enostavno ponovno re-zip spremenjene datoteke in znova prenesite na strežnik!
@@ Line 189: / Line 189: @@
 == Glejte tudi: ==
-* [[QUASTe_-_First_steps|QUASTe first steps]]
+* [[QUASTe_-_First_steps|Prvi koraki - QUASTe]]
-* [[CWS|How to use Childworkspaces]]
+* [[CWS|Kako uporabljati Childworkspaces]]
-== <b>Helpful povezave</b><br> ==
+== <b>Koristne povezave</b><br> ==
-- the homepage of the QA project on OOo : http://qa.openoffice.org<br>
+- domača stran projekta QA na OOo: http://qa.openoffice.org<br>
-- the mailing list of the QA project : http://qa.openoffice.org/servlets/SummarizeList?listName=dev<br>
+- dopisni seznam projekta QA: http://qa.openoffice.org/servlets/SummarizeList?listName=dev<br>
-- query for existing bugs in IssueTracker : http://qa.openoffice.org/issues/query.cgi<br>
+- poizvedba po obstoječih hroščih v sistemu IssueTracker: http://qa.openoffice.org/issues/query.cgi<br>
-- writing a new issue in IssueTracker : http://qa.openoffice.org/issue_handling/pre_submission.html<br>
+- vpis nove zadeve v IssueTracker: http://qa.openoffice.org/issue_handling/pre_submission.html<br>
-- the QA category on OOo wiki : [[:Category:Quality_Assurance|Quality Assurance]]<br>
+- kategorija QA na wikiju OOo: [[:Category:Quality_Assurance|Quality Assurance]]<br>
-== Known issues ==
+== Znane težave ==
 - '''Errormails about existing testresults send to CWS-Owner'''<br>
 If current tested Workspace is as CWS and someone wants to add results to database for a test that already exists the CWS Owner gets a mail in which he can decide whether testresults are updated or not !