TinderboxAndEISStatus

From Apache OpenOffice Wiki
Revision as of 11:55, 9 May 2007 by Bei (Talk | contribs)

Jump to: navigation, search


The technical aspect of tinderbox integration into EIS

The initial "tinderbox EIS integration" feature was to show the tinderbox status of a CWS on the CWS Overview page if it is available.


Additional features and issue fixes done recently:


1.) There´s a new menu in EIS: MasterWorkspaces / Tinderbox status which shows Tinderbox information of recent Milestone tinderbox builds for comparison. This allows to easily find out if a build break is already on the master.

2.) Links to Tinderbox Overview pages which might be useful for QA and development and to the Tinderbox Main page have been integrated into the EIS menu

3.) A Wiki page RedTinderboxStatusInEIS has been created explaining how to get more information out of tinderbox if the status is red and what might be done in this case. This page may contain anything that will be defined as a policy in the future.

4.) A link to that Wiki page is available with a help bubble right behind the tinderbox headline above the tinderbox status messages on the CWS Overview page making this information easily accessible.

5.) EIS now delivers the list of to be checked CWS/MWS with one single SOAP call fixing a performance problem between EIS and tinderbox.

6.) A potential problem that the tinderbox might try to start a build before modules have been added to the CWS and the status might get red in that case because there are no modules and that this happend even before the developer did do anything has been resolved. This fix was a side effect of 5.)

7.) Tinderbox now also checks OO*680 codelines, previously only SRC680 codelines had been checked, so that tinderbox information now becomes available for all public ChildWorkspaces. Also a side effect done with fix 5.)

8.) The "difficulty" that there is no MAC available inside Sun´s OpenOffice.org development department for testing potential bug fixes on Mac plattform is gone with recent events ;-)

9.) Only open issue IMHO is the feature wish to be able to tell tinderbox to restart a build in case of a failed build caused by things other than problems in the source code like network failures, anoncvs problems etc.

Workarounds for that are available:

a.) a cvs commit on the ChildWorkspace eg. submitting just a newline in some source code file would trigger a new tinderbox build.

b.) The to be defined policy can allow the Developer to start a buildbot build instead and if that doesn´t fail write a comment into the CWS that the red tinderbox status can be ignored because buildbot did build correctly.

c.) The to be defined policy can allow the Developer to just write a comment for the CWS in EIS that tinderbox red status can be ignored because his/her analysis of the tinderbox log file reveled <whatever_problem_there_was_unrelated_to_code_change> if this is the case.


10.) In case of there is a major problem with anoncvs which is used by tinderbox we need to announce that because it makes tinderbox results unusable.


==> IMHO technically there is nothing left to be clearified "for integration the result of the tinderbox into the QA approval process."


The Community aspect

Discussions took place on development and QA mailing lists from which possible policies and potential problems indiviuals might have with integrating or not integrating such a policy can be extracted. These discussions also did contain some explanations about how tinderbox actually works which might not have been known to stakeholders before and which might be useful to be moved into the existing tinderbox Wiki entry. At least one of the mails on that discussion informed that the Release Meeting will decide on a policy.

The Wiki entry CWS_Checklist_for_QA_responsibilities already contains the following sentence:

Check the status of the tinderbox in the Environment Information System. Is the status red the CWS has to go back to development.

Where tinderbox in that sentence is a link to that RedTinderboxStatusInEIS wiki page explained above.

So it looks like it is already partly being integrated into QA approval process whoever that did informally before the Release Meeting decided on and announced a policy ;-)


Possible QA Handling

If the tinderbox status is red first check if the corresponding milestone is also already red. Use EIS Menu Masterworkspace / Tinderbox status for that. Note this shows only last recent milestones for older milestones you can assume that the milestone was OK.

If the problem was not already on the milestone write a comment to please have a look at tinderbox into the CWS and set state back to new. If the problem was already on the milestone write a comment about that fact into the CWS and go on with approving or nominating that CWS.

Exception: if there is a global outage with anoncvs or similar infrastructure problem announced and you are aware of that tinderbox status can be ignored. Please provide a short comment in the CWS about the outage in this case before nomination or approval.

If you get the CWS back from development with a comment that it´s a false positive go on and approve or nominate the CWS. If you get the CWS back from development with a fix and a comment that that fix did build OK on buildbot go on and approve or nominate the CWS. If you get the CWS back from development without such comment tinderbox status should now no longer be red or else we have a fixed but failed issue.


Possible Development Handling

If the tinderbox status is red and you believe that your CWS should be in a buildable state have a look at the buildlog. Instructions for how to find and analyse the buildlog and handle the red status can be found via the help bubble in EIS beneath the tinderbox headline.

If it´s a real build breaker try to fix problem or try to find assistance for fixing problem. After you did commit your fix to CVS tinderbox will automatically requeue the CWS for a new test build. You can also start a build on buildbot immediately after fixing for proving that the build is now OK.

If you find out that it´s a false positive write a comment into the CWS and send it back to QA.

Reasons for false positives could be: infrastructure problems, anoncvs out-of-date (see the overview page - if the build was started just after the last commit, then anoncvs might not yet be up-to-date) only applies of course if the break is in a code-area affected by the last commit(s), other issues (disk full on buildslave or similar). Virtually all of these false positives are really easy to detect by having a look at the logs.

The only "false" positive (well, a real one actually, but not to blame to the cws) are parallel-build problems. But tinderbox buildslaves are set-up in a way that these kind of breaker is a rare exception.

Further suggestions

We need to define a responsible person who would announce relevant infrastructure outages on QA and DEV mailing lists.

If new tinderbox platforms appear in the future they will be automatically also shown in EIS. Maybe the release council should have to approve any additional platform to be "tinderbox red rule" ready ?

Personal tools