Difference between revisions of "Cwscheckapi"

Revision as of 13:52, 4 June 2008

D R A F T

To avoid regressions in the UnoAPI CwsCheckApi must be executed as a step before readyForQA.

Concept of mandatory UnoAPITests for CWS

Flowchart for mandatory UnoAPITests in CWS

CWS is ready

If the CWS is ready the developer has to run cwscheckapi. checkcws calls an UnoAPITest for all modules which are added to the cws. If the result is FAILED the developer has

to fix the failure inside the implementation
to get an exception

To get the later one the developer must have good arguments why he/she is not able to fix the implementation.

If the UnoAPITest works fine with state state OK the CWS could be set readyForQA

Exception treatment

If an exception is approved, the developer has to do the following:

write a new Issue
update $MODULE.sce and/or knownissues.xcl with the issue information.
run the cwscheckapi script again. The result must PASSED.OK

approve an Exception

If you think you need an exception you have to inform your QA representative and your manager. An exception could break in the UnoAPI! But it also possible that the test itself needs an update.

CwsCheckApi

cwscheckapi is an commandline script. You will find it here:

$SRC_ROOT/solenv/bin/cwscheckapi[.btm]

It is similar to checkapi but it runs UnoAPITests.

Office installation

At first we need an Office to test. Therefore an Office need to be installed. While Windows 2003 Terminal-Server makes trouble on installing an Office without user interaction, the cwscheckapi goes a different way. It pack (dmake PKGFORMAT=installed) a runnable office. You will think this not a correct installed office? You are right. The cwscheckapi should not test the installer, it should only test the UnoAPI. The Office will be installed here:

$TMP/$USERNAME/cwscheckapi/

install StarOffice or OpenOffice.org

The StarOffice developer mid to install StarOffice as default where the OpenOffice.org developer don't have a StarOffice. So the installation routine checks at first for an StarOffice version (instset_native <-> instetoo_native). If this is not available the OpenOffice.org version will be used. But with parameter -o it is possible to force the OpenOffice.org version

cwscheckapi -o

Test the modules

After a successful packaging the runner, which is the main application here, starts a query to EIS for all added modules to the CWS. If a module contains

$MODULE/qa/unoapi

all UnoAPITests will be executed.

Test is finished

At the end of the test run you will get a test result. This should look like this:

***** State for complex.unoapi.CheckModuleAPI ******
Whole unit: PASSED.OK
****************************************************
LOG> ********** end test *************
LOG> Finished module(auto)
***** State for complex.unoapi.CheckModuleAPI ******
Whole unit: PASSED.OK
****************************************************
Job -o complex.unoapi.CheckModuleAPI::module(auto) done 

A logfile could be found here: /tmp/$USERNAME/cwscheckapi/cwscheckapi.log
Dienstag,  6. Mai 2008 12:09 Uhr CEST

Test is OK

Your cws get an attachment in EIS which look like this:

/var/tmp/UnoApiCwsStatuscws_$CWS_unxsoli.PASSED.OK.txt

All is OK :-)

Test is FAILED

***** State for complex.unoapi.CheckModuleAPI ******
checkModule(toolkit) - module failed

Whole unit: PASSED.FAILED
****************************************************
LOG> ********** end test *************
LOG> CheckModuleAPI.module(auto) PASSED.FAILED
LOG> Finished module(auto)
***** State for complex.unoapi.CheckModuleAPI ******
module(auto) - CheckModuleAPI.module(auto) PASSED.FAILED

Whole unit: PASSED.FAILED
****************************************************
Job -o complex.unoapi.CheckModuleAPI::module(auto) failed

A logfile could be found here: /tmp/$USERNAME/cwscheckapi/cwscheckapi.log

What has failed?

As you can see the module toolkit has failed. Open the logfile in an editor you like (note that the logfile could be up to 10MB!) and search for the string:

checkModule(toolkit)

Now you have to ways to go:

follow the steps described in UnoAPITest
follow the following steps:

The string checkModule(toolkit) you will find at the beginning and at the end of the test for toolit. Go to the second match and you will see something like this:

LOG> out > Failures that appeared during scenario execution:
LOG> out > 	 toolkit.UnoSpinButtonControl
LOG> out > 1 of 55 tests failed
LOG> out > Job -sce toolkit.sce done
LOG> out > endlocal
LOG> out > quit
LOG> out > =======================================================================
LOG> out > In case you noticed a failures of toolkit.AccessibleToolBoxItem make sure that the object bar is configured as text and not as icons
LOG> out > =======================================================================	
LOG> [08.05.2008 - 22:41:15]PH:runCommand: waiting 76 seconds while command execution is ongoing... 9
LOG> [08.05.2008 - 22:42:31]PH:runCommand Could not detect changes in output stream!!!
LOG> [08.05.2008 - 22:42:31]PH:runCommand Process ist not finished but there are no changes in output stream.
LOG> [08.05.2008 - 22:42:36]PH:kill: process closed with exit code 0
LOG> verify output...
LOG> mached line: 1 of 55 tests failed
LOG> Module passed FAILED
LOG> ERROR: could not find '0 of [0-9]+? tests failed' in output
LOG> module failed
LOG> kill existing office...
[08.05.2008 - 22:42:37] try to get ProcessHandler
[08.05.2008 - 22:42:37] ProcessHandler != null
[08.05.2008 - 22:42:37] Trying to terminate the desktop
[08.05.2008 - 22:42:37] Desktop terminated
[08.05.2008 - 22:42:37] the Office has 15 seconds for closing...
[08.05.2008 - 22:42:52] OfficeProvider: User defined an application to destroy the started process.
    Trying to execute: Z:\so-cwsserv02\qadev32\DEV300\wntmsci11.pro\bin.m8\kill.exe -9 soffice.bin
[08.05.2008 - 22:43:05] copy 'e:\temp\user_backup' -> 'E:\temp\$USER\cwscheckapi\office\Sun\StarOffice 9\StarOffice9\user\'
[08.05.2008 - 22:43:06] copy 'e:\temp\user_backup' -> 'E:\temp\$USER\cwscheckapi\office\Sun\StarOffice 9\StarOffice9\user\' finished
[08.05.2008 - 22:43:06] try to get OfficeWatcher
[08.05.2008 - 22:43:06] OfficeWatcher finished
LOG> Finished checkModule(toolkit)

Here you can see that the UnoAPI-Test for toolkit.UnoSpinButtonControl has failed. Now find the lines which matches toolkit.UnoSpinButtonControl.

LOG> out > Creating: toolkit.UnoSpinButtonControl
LOG> out > LOG> Log started 08.04.2008 - 22:40:49
LOG> out > LOG> creating a textdocument
LOG> out > LOG> creating a new environment for UnoControlSpinButton object
LOG> out > ImplementationName: com.sun.star.comp.toolkit.UnoSpinButtonControl
LOG> out > Environment created
LOG> out > UnoSpinButtonControl recreated 
LOG> out > running: 'ifc.awt._XSpinValue'
LOG> out > LOG> Log started 08.04.2008 - 22:40:50
LOG> out > checking : _XSpinValue
LOG> out > LOG> Execute: addAdjustmentListener()
LOG> out > Method addAdjustmentListener() finished with state FAILED
LOG> out > LOG> addAdjustmentListener(): PASSED.FAILED
LOG> out > LOG> Execute: removeAdjustmentListener()
LOG> out > LOG> starting required method: addAdjustmentListener()
LOG> out > LOG> ! Required method addAdjustmentListener() failed
LOG> out > LOG> removeAdjustmentListener(): PASSED.FAILED
LOG> out > LOG> Execute: setValue()

Now you have to fix the issue in your code for XSpinValue. If this is currently not possible you have to follow the Exception treatment. Further details about the UnoAPI-Test status you can get at the UnoAPITests site.