Handle fixed verified issues

From Apache OpenOffice Wiki
Revision as of 15:42, 22 February 2010 by Thorstenziehm (Talk | contribs)

Jump to: navigation, search


The process

Since 20th of July 2009 it isn't mandatory anymore to recheck a fixed/verified issue (status = VERIFIED, resolution = FIXED') in the master code line (Master build). The process is still to check a fixed issue in a Child Work Space (CWS) and set it to status VERIFIED when it is fixed. When the issue is integrated into the Master build the owner (or someone else) can recheck issue and set it to status CLOSED when it is also fixed in the Master build. But the last step isn't mandatory anymore, because the workload is very high and the quote of fault integrations is minimal.

After each release all 'fixed/verified' issues with targets older than 2 releases will be closed automatically. This will be announced in the Release Status Meeting and can be done by someone from the Release Status meeting or somebody else. This means for example : when OOo 3.2 is released and all issues equal or older than OOo 3.1 can be closed. This means that the issues are still open for more than half a year (quarterly release cycle).

If you want more details, please read the proposal.


The first closing session - 20th of July 2009

In this step all issues are closed which are longer than half a year in status fixed/verified. Additionally All issues in status 'VERIFIED' and resolution 'FIXED'

  • 12 issues without a target ('-')
  • 15 issues with target 'DevTools'
  • 11 issues with target 'milestone1', 'next build' or 'not determine'
  • 2 issues with target 'OOo 2.2'
  • 1 issue with target 'OOo 2.2.1'
  • 141 issues with target 'OOo 2.3'
  • 16 issues with target 'OOo 2.3.1'
  • 217 issues with target 'OOo 2.4'
  • 10 issues with target 'OOo 2.4.x'
  • 10 issues with target 'OOo 2.x'
  • 438 issues with target 'OOo 3.0'
  • 43 issues with target 'OOo 3.0.1'

By accident I closed 83 issues wrongly. I fixed this error with the result, that the owner of the issues get 3 additional mails. Now the issues should be again in status 'fixed/verified'. Sorry for that.


Closing session for issues with target OOo 3.1 - 22nd of February 2010

At 22nd of February 2010 it was approved by the Release Status Meeting to close all issues in state fixed/verified for target OOo 3.1. In sum 226 issues were closed.

  • 53 issues in component Database Access
  • 28 issues in component Spreadsheet
  • 18 issues in component L10N
  • 17 issues in component Framework
  • 15 issues in component Word Processing
  • 13 issues in component Porting
  • 12 issues in component API
  • 9 issues in component Tools
  • 8 issues in component Presentation
  • 7issues in component UDK
  • 7issues in component QA
  • 6 issues in component Utilities
  • 6 issues in component GSL
  • 5 issues in component Scripting
  • the rest of the issues were in components where less than 5 issues were open


Approval

The proposal was announced on 17th of July 2009. There wasn't any negative feedback in the past days, therefore the proposal was discussed in the next Release Status Meeting on 20th of July. The proposal was approved! http://wiki.services.openoffice.org/wiki/ReleaseStatus_Minutes#2009-07-20


Proposal : Recheck of 'fixed/verified'-issues in Master builds isn't mandatory anymore

Current situation

It is defined [1] in the life cycle of a fixed issue (defects, enhancements, features or tasks) that the last step is to check the fix in a Master Build. After verification of the fix the issue has to be set to status 'closed'. The work flow is documented in the Issue Handling section [2] on http://qa.openoffice.org in the Bug Life Cycle document [3] on page 3.

This work flow guaranteed that a bug is found in the Master Build, when it is integrated by the following faults.

  1. The code contributions aren't checked-in (committed) in the source tree and the tested build is done on a local copy of the source. The changes are in the tested build (CWS) but isn't in the Master Build.
  2. The code contribution in one CWS overwrites a code contribution in another CWS. This exists when more than one CWS handles code in the same code areas and when they are integrated into (the same) Master. Then integration-conflicts (merge-conflicts) exists and these have to be fixed by a Release Engineer or the responsible developer. When the conflicts aren't fixed correctly changes from one CWS will win and the code of another CWS isn't integrated.
  3. If dependencies from one CWS break changes in another CWS. For example in one CWS incompatible API changes were integrated and in another CWS the older API is used. The functionality in each CWS works fine. But after integration the functionality on the second CWS is broken.

In all these cases the product can crash, because missing or wrong code should be used by some functions.

All cases weren't very often in the past releases of OOo. When we had such a situation the bugs were identified very quickly, because general functionality was broken on master completely. It wasn't needed to check a special issue for verification to identify the problem. The general agreement between QA project [4] and the QA team inside Sun is that less than 1% of all fixed issues could be broken in Master.

Currently (July 2009) there are more than 1650 fixed/verified issues open which have to be checked in a Master build before closing.

  • 178 with targets older than OOo 2.4 (before Mar. 2008)
  • 250 with target OOo 2.4.x (released Mar./Oct. 2008)
  • 502 with target OOo 3.0.x (released Oct. 08/ Jan. 09)
  • 265 with target OOo 3.1 (released May 09)
  • 483 with current targets (OOo 3.1.1 and OOo 3.2) and which are already integrated in Master

In the past years more than 4250 issues were fixed per year. All these issues have to be checked in Master. To address this high number of unclosed issues the QA project started with Issue-Hunting-Days where they are working in chats on closing such issues (in these chats ~50 issues are closed without any fault found). But also these events doesn't help to reduce the number of such issues dramatically. The effort is too high for the QA project and the benefit is low.

Proposal

It isn't mandatory anymore to check each fixed/verified issue in a Master build. It is still needed to check all fixed issues (defects, enhancements, features or tasks) in CWS and set them to state 'fixed/verified'. If the owner, the submitter or someone else wants to check the issue in a Master build the issue will stay in this state for round about half a year. When the issues isn't closed in this time frame the issue will be closed automatically. This will be done by someone who has to be defined by the release status meeting. The issues should be closed automatically when 2 new released were done. This means when OOo 3.2 is released all 'fixed/verified' issues with target OOo 3.1 can be closed (quarterly releases).

The first step can be done by Thorsten Ziehm or Jörg Jahnke. In this step the opened 'fixed/verified' issues with targets older than or equal 2.4, 2.4.1, 3.0 and 3.0.1 will be closed. Issues with other targets which are in this state longer than March 2009 will be closed too. A text like 'This issue is closed automatically. When this issue isn't fixed in the current release, please reopen it.' should be added to each issue.

Benefits

The owner of an issue and the whole QA project will win time for other work.

Disadvantage

There is still a low risk to lower the quality of the product. General agreement with QA community and QA responsible inside Sun is that less than 1% of all fixed issues are broken in the Master. When something broke at integration of a CWS than the critical issues were identified without checking a special fixed/verified issue.

Footnotes/Links

[1] : http://qa.openoffice.org/ooQAReloaded/Docs/QA-Reloaded-IssueHandlingOverview.html
[2] : http://qa.openoffice.org/ooQAReloaded/ooQA-IssueRules.html
[3] : http://qa.openoffice.org/issue_handling/workflowcharts/defect_triaging.pdf
[4] : http://qa.openoffice.org/servlets/ReadMsg?list=dev&msgNo=12513

Personal tools