The technical aspect of tinderbox integration into EIS
Additional features and issue fixes done recently:
1.) There´s a new menu in EIS: MasterWorkspaces / Tinderbox status which shows Tinderbox information of recent Milestone tinderbox builds for comparison. This allows to easily find out if a build break is already on the master.
2.) Links to Tinderbox Overview pages which might be useful for QA and development and to the Tinderbox Main page have been integrated into the EIS menu
3.) A Wiki page RedTinderboxStatusInEIS has been created explaining how to get more information out of tinderbox if the status is red and what might be done in this case. This page may contain anything that will be defined as a policy in the future.
4.) A link to that Wiki page is available with a help bubble right behind the tinderbox headline above the tinderbox status messages on the CWS Overview page making this information easily accessible.
6.) A potential problem that the tinderbox might try to start a build before modules have been added to the CWS and the status might get red in that case because there are no modules and that this happend even before the developer did do anything has been resolved. This fix was a side effect of 5.)
7.) Tinderbox now also checks OO*680 codelines, previously only SRC680 codelines had been checked, so that tinderbox information now becomes available for all public ChildWorkspaces. Also a side effect done with fix 5.)
8.) The "difficulty" that there is no MAC available inside Sun´s OpenOffice.org development department for testing potential bug fixes on Mac plattform is gone with recent events ;-)
9.) Only open issue IMHO is the feature wish to be able to tell tinderbox to restart a build in case of a failed build caused by things other than problems in the source code like network failures, anoncvs problems etc.
Workarounds for that are available:
a.) a cvs commit on the ChildWorkspace eg. submitting just a newline in some source code file would trigger a new tinderbox build.
b.) The to be defined policy can allow the Developer to start a buildbot build instead and if that doesn´t fail write a comment into the CWS that the red tinderbox status can be ignored because buildbot did build correctly.
c.) The to be defined policy can allow the Developer to just write a comment for the CWS in EIS that tinderbox red status can be ignored because his/her analysis of the tinderbox log file reveled <whatever_problem_there_was_unrelated_to_code_change> if this is the case.
==> IMHO technically there is nothing left to be clearified "for integration the result of the tinderbox into the QA approval process."
The Community aspect
Discussions took place on development and QA mailing lists from which possible policies and potential problems indiviuals might have with integrating or not integrating such a policy can be extracted. These discussions also did contain some explanations about how tinderbox actually works which might not have been known to stakeholders before and which might be useful to be moved into the existing tinderbox Wiki entry. At least one of the mails on that discussion informed that the Release Meeting will decide on a policy.
The Wiki entry CWS_Checklist_for_QA_responsibilities already contains the following sentence:
Where tinderbox in that sentence is a link to that RedTinderboxStatusInEIS wiki page explained above.
So it looks like it is already partly being integrated into QA approval process whoever that did informally before the Release Meeting decided on and announced a policy ;-)
Possible QA Handling
If the tinderbox status is red write a comment to please have a look at tinderbox into the CWS and set state back to new. Exception: if there is a global outage with anoncvs announced or similar infrastructure problem tinderbox status can be ignored. Please provide a short comment in the CWS in this case before nomination or approval.
Possible Development Handling
If the tinderbox status is red have a look at the buildlog. Instructions can be found via the help bubble in EIS.
If it´s a real build breaker try to fix problem or try to find assistance for fixing problem. After you did commit your fix to CWS tinderbox will automatically requeue the CWS for a new test. You can also start a build on buildbot immidiatly after fixing for proving that the build is now OK.
If you find out that it´s a false positive write a comment into the CWS and send it back to QA.
Reasons for false positives could be: infrastructure problems, anoncvs out-of-date (see the overview page - if the build was started just after the last commit, then anoncvs might not yet be up-to-date) only applies of course if the break is in a code-area affected by the last commit(s), other issues (disk full on buildslave or similar). Virtually all of these false positives are really easy to detect by having a look at the logs.
The only "false" positive (well, a real one actually, but not to blame to the cws) are parallel-build problems. But tinderbox buildslaves are set-up in a way that these kind of breaker is a rare exception.