Difference between revisions of "UnoAPITest/project/cwsTests"
(→checkcws: add issue id) |
B michaelsen (Talk | contribs) m |
||
(17 intermediate revisions by one other user not shown) | |||
Line 5: | Line 5: | ||
The goal of the project is to set up a mechanism to find regression in the API on a cws. This should be done by run the [[UnoAPITest]]s on added modules to a cws. In mostly all modules which support [[UnoApi|UnoAPI]] exist a qa/unoapi folder. This folder contains a make file which starts the [[UnoAPITest]]s related to this module. | The goal of the project is to set up a mechanism to find regression in the API on a cws. This should be done by run the [[UnoAPITest]]s on added modules to a cws. In mostly all modules which support [[UnoApi|UnoAPI]] exist a qa/unoapi folder. This folder contains a make file which starts the [[UnoAPITest]]s related to this module. | ||
− | To make it easy a simple script, f.e [[ | + | To make it easy a simple script, f.e [[cwscheckapi]] (in coherence to [[checkapi]]) should do this job. |
== Project Plan == | == Project Plan == | ||
Line 15: | Line 15: | ||
|- | |- | ||
| develop concept and identify needing tools | | develop concept and identify needing tools | ||
− | | bgcolor="#00ff00" | | + | | bgcolor="#00ff00" | done |
| bgcolor="#00c0ff" | M1 | | bgcolor="#00c0ff" | M1 | ||
Line 25: | Line 25: | ||
|- | |- | ||
| query EIS for added modules | | query EIS for added modules | ||
− | | bgcolor="#00ff00" | | + | | bgcolor="#00ff00" | done |
| bgcolor="#ffc0ff" | M3 | | bgcolor="#ffc0ff" | M3 | ||
|- | |- | ||
− | | bgcolor="#e8e8e8" | | + | | bgcolor="#e8e8e8" | automatic Office installation |
− | | bgcolor="# | + | | bgcolor="#00ff00" | done |
| bgcolor="#ffc0c0" | M4 | | bgcolor="#ffc0c0" | M4 | ||
|- | |- | ||
− | | | + | | deliver status to EIS |
− | | | + | | bgcolor="#00ff00" | done |
| bgcolor="#c0ffc0" | M5 | | bgcolor="#c0ffc0" | M5 | ||
|- | |- | ||
| bgcolor="#e8e8e8" | Bugfixing | | bgcolor="#e8e8e8" | Bugfixing | ||
− | | bgcolor="# | + | | bgcolor="#00ff00" | started |
| bgcolor="#ffffc0" | M6 | | bgcolor="#ffffc0" | M6 | ||
Line 79: | Line 79: | ||
=== CWS is ready === | === CWS is ready === | ||
− | If the CWS is ready the developer has to run [[ | + | If the CWS is ready the developer has to run [[cwscheckapi]]. checkcws calls an [[UnoAPITest]] for all modules which are added to the cws. If the result is '''FAILED''' the developer has |
* to fix the failure inside the implementation | * to fix the failure inside the implementation | ||
* to get an [[#Exception|exception]] | * to get an [[#Exception|exception]] | ||
Line 93: | Line 93: | ||
TODO | TODO | ||
− | == | + | ==CwsCheckApi== |
− | + | cwscheckapi is an commandline script similar to [[checkapi]]. The runner query [[EIS]] for all added modules relayed to the CWS and execute the tests. | |
Issue [http://qa.openoffice.org/issues/show_bug.cgi?id=85368 85368] | Issue [http://qa.openoffice.org/issues/show_bug.cgi?id=85368 85368] | ||
Line 112: | Line 112: | ||
== Automatic Office installation == | == Automatic Office installation == | ||
− | * As the [[SmokeTest]] the [[ | + | === Notes === |
+ | * As the [[SmokeTest]] the [[cwscheckapi]] should be able to install the office automatically. | ||
** the smoke test has no separate installation routine. It's much easier to install the office via UserlandScripts. | ** the smoke test has no separate installation routine. It's much easier to install the office via UserlandScripts. | ||
* $SOURCE_ROOT could be the setup root entry | * $SOURCE_ROOT could be the setup root entry | ||
+ | * $SHIPDRIVE indicates the path to the seups of a master office | ||
+ | * $CWS_WORKSTAMP indicates a cws envirnment | ||
+ | * $SOL_TMP indicates a local environment | ||
* support of local environments | * support of local environments | ||
+ | |||
+ | === Install-Sets === | ||
+ | {| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #ffffff solid; border-collapse: collapse;" | ||
+ | |+ '''find install sets''' | ||
+ | | bgcolor="#cccccc" |<center>'''Platform'''</center> | ||
+ | | bgcolor="#cccccc" colspan="2" |<center>'''Solaris Sparc'''</center> | ||
+ | | bgcolor="#cccccc" colspan="2" |<center>'''Solaris x86'''</center> | ||
+ | | bgcolor="#cccccc" colspan="2" |<center>'''Linux'''</center> | ||
+ | | bgcolor="#cccccc" colspan="2" |<center>'''Windows'''</center> | ||
+ | |||
+ | |- | ||
+ | | bgcolor="#cccccc" |<center>'''Language'''</center> | ||
+ | | bgcolor="#00c0ff" |<center>en-US (default)</center> | ||
+ | | bgcolor="#00c0ff" | <center>other</center> | ||
+ | | bgcolor="#00c0ff" |<center>en-US (default)</center> | ||
+ | | bgcolor="#00c0ff" | <center>other</center> | ||
+ | | bgcolor="#00c0ff" |<center>en-US (default)</center> | ||
+ | | bgcolor="#00c0ff" | <center>other</center> | ||
+ | | bgcolor="#00c0ff" |<center>en-US (default)</center> | ||
+ | | bgcolor="#00c0ff" | <center>other</center> | ||
+ | |||
+ | |- | ||
+ | | bgcolor="#cccccc" |<center>'''Office-Version'''</center> | ||
+ | | bgcolor="#80ff80" |<center>StarOffice (default)</center> | ||
+ | | bgcolor="#80ff80" | <center>OpenOffice.org</center> | ||
+ | | bgcolor="#80ff80" |<center>StarOffice (default)</center> | ||
+ | | bgcolor="#80ff80" | <center>OpenOffice.org</center> | ||
+ | | bgcolor="#80ff80" |<center>StarOffice (default)</center> | ||
+ | | bgcolor="#80ff80" | <center>OpenOffice.org</center> | ||
+ | | bgcolor="#80ff80" |<center>StarOffice (default)</center> | ||
+ | | bgcolor="#80ff80" | <center>OpenOffice.org</center> | ||
+ | |||
+ | |- | ||
+ | | rowspan="2" | setsolar | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | <center>x</center> | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | <center>x</center> | ||
+ | | bgcolor="#81e0ff" | | ||
+ | |||
+ | |- | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | <center>x</center> | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | <center>x</center> | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | |||
+ | |- | ||
+ | | rowspan="2" bgcolor="#e8e8e8" | setcws | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" |<center>x</center> | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" |<center>x</center> | ||
+ | | bgcolor="#81e0ff" | | ||
+ | |||
+ | |- | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | <center>x</center> | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | <center>x</center> | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | |||
+ | |- | ||
+ | | rowspan="2" | ooO-Env.Set | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | |||
+ | |- | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | |||
+ | |- | ||
+ | | rowspan="2" bgcolor="#e8e8e8" | local env. | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | | bgcolor="#81e0ff" | | ||
+ | |||
+ | |- | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | | bgcolor="#c0ffc0" | | ||
+ | |||
+ | |} | ||
+ | |||
+ | === Userland-Scripts === | ||
+ | ;setsolar: $SHIPDRIVE/$INPATH/UserScripts/$PKGFORMAT/{OfficeVersion}/install | ||
+ | ;setcws: $SOLARVERSION/$INPATH/bin$$UPDMINOREXT/userscripts/install | ||
== [[EIS]] == | == [[EIS]] == | ||
Line 121: | Line 246: | ||
** implementation in Java is not useful. Too many dependencies. | ** implementation in Java is not useful. Too many dependencies. | ||
** best way is to call Perl scripts like [[cwsquery]] and pase the output | ** best way is to call Perl scripts like [[cwsquery]] and pase the output | ||
+ | *** Issue [http://qa.openoffice.org/issues/show_bug.cgi?id=85370 85370] | ||
* the EIS should display test runs | * the EIS should display test runs | ||
** currently some tests attach result files to the CWS, f.e. PerformanceTests | ** currently some tests attach result files to the CWS, f.e. PerformanceTests | ||
Line 133: | Line 259: | ||
The developer at Sun/StarOffice have some privileges that other don't have. This is f.e. an easier access to EIS. | The developer at Sun/StarOffice have some privileges that other don't have. This is f.e. an easier access to EIS. | ||
* SOAP-Interface needet | * SOAP-Interface needet | ||
+ | ** SOAP has too many dependencies. So cwsquery will be used. | ||
* runner: change default connection string from socket to named pipes if possible | * runner: change default connection string from socket to named pipes if possible | ||
* complex Test complex.unoapi.CheckModuleAPI must be stabilized | * complex Test complex.unoapi.CheckModuleAPI must be stabilized | ||
Line 145: | Line 272: | ||
[[Category:API]] | [[Category:API]] | ||
[[Category:UnoAPITest]] | [[Category:UnoAPITest]] | ||
− | [[Category: | + | [[Category:CWSQAStatus]] |
Latest revision as of 11:19, 26 October 2009
This project accompanied the way of UnoAPITests to be mandatory CWS tests.
Contents
Project description
The goal of the project is to set up a mechanism to find regression in the API on a cws. This should be done by run the UnoAPITests on added modules to a cws. In mostly all modules which support UnoAPI exist a qa/unoapi folder. This folder contains a make file which starts the UnoAPITests related to this module.
To make it easy a simple script, f.e cwscheckapi (in coherence to checkapi) should do this job.
Project Plan
develop concept and identify needing tools | done | M1 |
basic console script checkcws | done | M2 |
query EIS for added modules | done | M3 |
automatic Office installation | done | M4 |
deliver status to EIS | done | M5 |
Bugfixing | started | M6 |
Milestone | Due |
M1 | End of Dec. 07 |
M2 | Mid of Jan. 08 |
M3 | End of Jan. 08 |
M4 | Mid of Feb. 08 |
M5 | End of Feb. 08 |
M6 | End of Mar. 08 |
Concept of mandatory UnoAPITests for CWS
CWS is ready
If the CWS is ready the developer has to run cwscheckapi. checkcws calls an UnoAPITest for all modules which are added to the cws. If the result is FAILED the developer has
- to fix the failure inside the implementation
- to get an exception
To get the later one the developer must have good arguments why he/she is not able to fix the implementation.
If the UnoAPITest works fine with state state OK the CWS could be set readyForQA
Exception treatment
If an exception is approved, the developer has to do the following:
- write a new Issue
- update $MODULE.sce and knownissues.xcl
Exception handling
TODO
CwsCheckApi
cwscheckapi is an commandline script similar to checkapi. The runner query EIS for all added modules relayed to the CWS and execute the tests.
Issue 85368
Command line parameter
- -m Module[,Module]
- add the given modules to the test
- -log true
- enhance the logging output
- -debug true
- enabled the debugging mode: enhancing of output
- -tdoc Test-Document-Path
- path where testdocuments could be found
- -KeepDocument
- the tests normally close the documents which opened for testing purposes. If this flag is set to true all documents are available at the end of the test.
- -ini RunnerProps
- path to property file for CheckAPI. Default: $HOME/.runner.props
Notes
- only pro versions
- named pipes for connecting the office with pro+platform
- add solenv to a cws and commit checkws in solenv/bin
Automatic Office installation
Notes
- As the SmokeTest the cwscheckapi should be able to install the office automatically.
- the smoke test has no separate installation routine. It's much easier to install the office via UserlandScripts.
- $SOURCE_ROOT could be the setup root entry
- $SHIPDRIVE indicates the path to the seups of a master office
- $CWS_WORKSTAMP indicates a cws envirnment
- $SOL_TMP indicates a local environment
- support of local environments
Install-Sets
|
|
|
| |||||
|
|
|
| |||||
setsolar | |
|
||||||
|
|
|||||||
setcws | ||||||||
|
|
|||||||
ooO-Env.Set | ||||||||
local env. | ||||||||
Userland-Scripts
- setsolar
- $SHIPDRIVE/$INPATH/UserScripts/$PKGFORMAT/{OfficeVersion}/install
- setcws
- $SOLARVERSION/$INPATH/bin$$UPDMINOREXT/userscripts/install
EIS
- communication via SOAP
- the EIS should display test runs
- currently some tests attach result files to the CWS, f.e. PerformanceTests
- the EIS should have a extra data base for CWS tests
- the EIS should display successfully executed tests with a green bullet
- the EIS should support the possibility to set the "OK" status manual. See #runner
runner
first step
The developer at Sun/StarOffice have some privileges that other don't have. This is f.e. an easier access to EIS.
- SOAP-Interface needet
- SOAP has too many dependencies. So cwsquery will be used.
- runner: change default connection string from socket to named pipes if possible
- complex Test complex.unoapi.CheckModuleAPI must be stabilized
second step
The SOAP-Interface of EIS needs https connections.
- implement https for community developer
- implement user/password login into EIS