Difference between revisions of "Buildbot"

From Apache OpenOffice Wiki
Jump to: navigation, search
(Allocation: adding ppc mac and build type info)
m (Platform Coverage: fix spelling)
(44 intermediate revisions by 9 users not shown)
Line 1: Line 1:
A buildbot is now deployed for OpenOffice.org.
+
The buildbots are hosted by the Apache Software Foundation at
'''http://buildbot.go-oo.org/'''
+
'''http://ci.apache.org/projects/openoffice/'''
 +
The buildbot system used is [http://docs.buildbot.net/current/index.html extensively documented].
  
From the Buildbot README: http://buildbot.sourceforge.net/
+
The buildbot is a system to automate the compile/test cycle required by most software projects to validate code changes. With automatic and frequent builds, build problems can be pinpointed quickly, before other developers are inconvenienced by the failure. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not.
  
Please see how you can '''[[Buildbot Steps | add your own buildbot]]!'''
+
==Clean and Incremental Builds==
 +
There are two different types of builds: incremental and clean.  Incremental builds only compile files that have been changed since
 +
the last build and reuse object files and libraries of unmodified files where possible.  Clean builds start with removing old build artifacts from the last build.  Clean builds take more time to finish than incremental builds.  They are necessary after incompatible changes to the code.  A code change is considered incompatible when it affects files for which the build system does not have a dependency on the modified file.
  
Find out about the successor to the Buildbot, which is named [[Termite]].
+
==Nightly and Snapshot Builds==
 +
The buildbot performs a daily build, also called "nightly build", of the latest revision of trunk.  A snapshot build is done once per week for a fixed revision of trunk or one of the branches.
  
The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve.
+
===Nightly Builds===
 +
Nightly builds are built only for en-US.  Installation sets produced by the nightly builds can be found [http://ci.apache.org/projects/openoffice here].  Direct links for the platforms:
 +
*[http://ci.apache.org/projects/openoffice/#linux64 Linux 64bit]
 +
*[http://ci.apache.org/projects/openoffice/#win Windows (32bit)]
 +
* (currently out of order) [http://ci.apache.org/projects/openoffice/#linux32 Linux 32bit]
  
== OpenOffice.org Buildbot ==
+
Make sure to check if the build is up to date by looking at the [http://ci.apache.org/projects/openoffice overview].
The Buildbot will:  
+
* Build CWS/SVN branches on multiple platforms and make output available to developers
+
* Generate and make available install sets for multiple platforms
+
* Run tests (smoketest, performance tests, etc.) and make test reports available for tracking and regression.  
+
  
== Buildbot vs. Tinderbox ==  
+
===Snapshot Builds===
Currently, there is a [[Tinderbox]]. In fact, the Buildbot uses or adapts several Tinderbox scripts. The Buildbot provides several advantages:  
+
Snapshot builds are made for these languages:
* At-a-glance, customizable statusboard
+
ar cs da de en-GB en-US es fi fr gd gl hu it ja km ko nb nl pt-BR ru sk sl zh-CN zh-TW
* Flexible, modular, extensible framework
+
and platforms:
* Bots can be commanded and coordinated from the botmaster
+
Windows
  
Both Buildbots and Tinderboxes are easy to setup and can be behind the firewall. Thanks to [[User:Cloph|Cloph]] for reminding about the similarities.
+
Installation sets produced by the latest snapshot build can be found [https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds here].
  
== Deployment & Hardware ==
+
Changing the revision of the snapshot build is run is done by changing the SVN tag named SNAPSHOT:
Currently, the botmaster and several buildbots are hosted at OSU-OSL (http://osuosl.org). Additional machines will be added to the pool. Sun QA/RE plans to host a Solaris bot.  
+
*Remove the existing tag:
 +
: <code>svn rm https://svn.apache.org/repos/asf/openoffice/tags/SNAPSHOT</code>
 +
*Set the new tag to the desired revision:
 +
: <code>svn copy -r<revision> https://svn.apache.org/repos/asf/openoffice/<trunk-or-branch> https://svn.apache.org/repos/asf/openoffice/tags/SNAPSHOT</code>
  
=== Allocation ===  
+
==Platform Coverage==
Many buildbot based projects dedicate groups of bots to specific code branches. Thus, for example, checkins to HEAD will automatically trigger the associated pool of machines and the changes built, tested on multiple platforms. See for example: http://www.python.org/dev/buildbot/
+
The buildbot coverage differs from platform to platform:
 +
;Linux 64bit: The nightly build is always a clean build. There are no general known problems with the Linux 64bit builds.
 +
;Linux 32bit: Currently out of order (due to hardware problems?)
 +
;Windows: Clean builds are not reliable. The exact reasons are not yet known, symptoms are missing header files such as apr.h.
 +
Nightly builds are incremental builds by default.
 +
Once a week the nightly build is cleaned.
 +
Clean builds can be requested manually (how?).
 +
;Mac: At the moment there are no Mac builds because there is no suitable server for this platform.
  
However, this is impractical for OpenOffice.org due the numericity of branches (CWS'es). Instead, we focus on creating a reasonable sized pool of machines for each platform/variant, with flexible build policies (see below).
+
==Buildbot Configuration File==
 +
The [https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/openofficeorg.conf buildbot configuration file]
 +
is available and changable by every committer.
  
Currently, bots build for:  
+
[[Category:Build_System]]
* debian
+
* suse 10.0
+
* suse 10.1 (jre and gij)
+
* suse 7.3 (several)
+
* windows xp (3+)
+
* Mac OS X Intel (X11 for all CWS except aquavcl01)
+
* Mac OS X PPC (X11 for all CWS except aquavcl01)
+
* solaris 10 (sparc)
+
* solaris 10 (x86)
+
 
+
== Policies ==
+
The botmaster will schedule builds based on policies that take into account the pool size, machine status and build time. When a bot is busy doing a build, additional requests are automatically queued.
+
 
+
* At UTC 0200 + random() build HEAD
+
* At milestone release build milestone
+
* Build manually requested CWS or milestone
+
* Build automatically selected CWS
+
 
+
== Features ==
+
* When no CWS is provided --> checkout HEAD  - '''Done'''
+
* Summaries  - '''Done'''
+
** tail, warnings, errors, summary log like tinderbox
+
** error - show before/after buffer (15, 5) and skip ... between errors
+
* Deploy install-sets on bot-master - '''Done'''
+
* Increase length of status page.  - '''Done'''
+
* Email notification of build result (fail, success, install, etc. with link to buildlogs)
+
** Send mail to requestor@openoffice.org when finished. (add a hint on the request form) - '''Done'''
+
** For CWS - send to CWS owner (from EIS)
+
* Fix smoke-test on some hosts 
+
* CWS-Oriented View - '''Done'''
+
** Add header X axis (timeline) 
+
** Three different views - all, ready for QA, new
+
* Buildslave info on slave web page
+
* Scheduler to prioritize and distribute builds on various slaves
+
** Collect changes by one user (settling) in one build
+
** AllCVS mailing list listener to auto trigger build on commit notification
+
* Header Row - provide number/link to last build (related to status)
+
* Extra config switches field in force build form. Developers can specify some extra configure switches specific to there CWSs. - '''Done'''
+
* Add suse 7.3 buildslave
+
* Generate buildbot results as RSS feed for planetizing
+
* For windows slaves, provide link to html build status page
+
* UserId/Password verification for force build and stop buid
+
 
+
== People ==
+
* Prasad Madhav (pmadhav) deployed and customizes the buildbot
+
* mikeleib, kaib, vq, dkeskar support and provide ideas
+

Revision as of 14:50, 15 April 2013

The buildbots are hosted by the Apache Software Foundation at http://ci.apache.org/projects/openoffice/ The buildbot system used is extensively documented.

The buildbot is a system to automate the compile/test cycle required by most software projects to validate code changes. With automatic and frequent builds, build problems can be pinpointed quickly, before other developers are inconvenienced by the failure. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not.

Clean and Incremental Builds

There are two different types of builds: incremental and clean. Incremental builds only compile files that have been changed since the last build and reuse object files and libraries of unmodified files where possible. Clean builds start with removing old build artifacts from the last build. Clean builds take more time to finish than incremental builds. They are necessary after incompatible changes to the code. A code change is considered incompatible when it affects files for which the build system does not have a dependency on the modified file.

Nightly and Snapshot Builds

The buildbot performs a daily build, also called "nightly build", of the latest revision of trunk. A snapshot build is done once per week for a fixed revision of trunk or one of the branches.

Nightly Builds

Nightly builds are built only for en-US. Installation sets produced by the nightly builds can be found here. Direct links for the platforms:

Make sure to check if the build is up to date by looking at the overview.

Snapshot Builds

Snapshot builds are made for these languages:

ar cs da de en-GB en-US es fi fr gd gl hu it ja km ko nb nl pt-BR ru sk sl zh-CN zh-TW

and platforms:

Windows

Installation sets produced by the latest snapshot build can be found here.

Changing the revision of the snapshot build is run is done by changing the SVN tag named SNAPSHOT:

  • Remove the existing tag:
svn rm https://svn.apache.org/repos/asf/openoffice/tags/SNAPSHOT
  • Set the new tag to the desired revision:
svn copy -r<revision> https://svn.apache.org/repos/asf/openoffice/<trunk-or-branch> https://svn.apache.org/repos/asf/openoffice/tags/SNAPSHOT

Platform Coverage

The buildbot coverage differs from platform to platform:

Linux 64bit
The nightly build is always a clean build. There are no general known problems with the Linux 64bit builds.
Linux 32bit
Currently out of order (due to hardware problems?)
Windows
Clean builds are not reliable. The exact reasons are not yet known, symptoms are missing header files such as apr.h.

Nightly builds are incremental builds by default. Once a week the nightly build is cleaned. Clean builds can be requested manually (how?).

Mac
At the moment there are no Mac builds because there is no suitable server for this platform.

Buildbot Configuration File

The buildbot configuration file is available and changable by every committer.

Personal tools