Difference between revisions of "Buildbot"
(→Steps to add a new buildbot) |
|||
| Line 3: | Line 3: | ||
From the Buildbot README: http://buildbot.sourceforge.net/ | From the Buildbot README: http://buildbot.sourceforge.net/ | ||
| + | |||
| + | Please see how you can '''[[Buildbot Steps | add your own buildbot]]!''' | ||
The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve. | The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve. | ||
Revision as of 13:05, 11 August 2006
A buildbot is now deployed for OpenOffice.org. http://ooo-staging.osuosl.org:8010/
From the Buildbot README: http://buildbot.sourceforge.net/
Please see how you can add your own buildbot!
The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve.
Contents
OpenOffice.org Buildbot
The Buildbot will:
- Build CWS/SVN branches on multiple platforms and make output available to developers
- Generate and make available install sets for multiple platforms
- Run tests (smoketest, performance tests, etc.) and make test reports available for tracking and regression.
Buildbot vs. Tinderbox
Currently, there is a Tinderbox. In fact, the Buildbot uses or adapts several Tinderbox scripts. The Buildbot provides several advantages:
- At-a-glance, customizable statusboard
- Flexible, modular, extensible framework
- Bots can be commanded and coordinated from the botmaster
Both Buildbots and Tinderboxes are easy to setup and can be behind the firewall. Thanks to Cloph for reminding about the similarities.
Deployment & Hardware
Currently, the botmaster and several buildbots are hosted at OSU-OSL (http://osuosl.org). Additional machines will be added to the pool. Sun QA/RE plans to host a Solaris bot.
Allocation
Many buildbot based projects dedicate groups of bots to specific code branches. Thus, for example, checkins to HEAD will automatically trigger the associated pool of machines and the changes built, tested on multiple platforms. See for example: http://www.python.org/dev/buildbot/
However, this is impractical for OpenOffice.org due the numericity of branches (CWS'es). Instead, we focus on creating a reasonable sized pool of machines for each platform/variant, with flexible build policies (see below).
Currently, bots build for:
- debian
- suse 10.0
- suse 10.1 (jre and gij)
- suse 7.3 (several)
- windows xp (3+)
Policies
The botmaster will schedule builds based on policies that take into account the pool size, machine status and build time. When a bot is busy doing a build, additional requests are automatically queued.
- At UTC 0200 + random() build HEAD
- At milestone release build milestone
- Build manually requested CWS or milestone
- Build automatically selected CWS
Features
- When no CWS is provided --> checkout HEAD - Done
- Summaries
- tail, warnings, errors, summary log like tinderbox
- error - show before/after buffer (15, 5) and skip ... between errors
- Deploy install-sets on bot-master
- Increase length of status page.
- Email notification of build result (fail, success, install, etc. with link to buildlogs)
- Send mail to requestor@openoffice.org when finished. (add a hint on the request form)
- For CWS - send to CWS owner (from EIS)
- Show smoke-test results in status -- currently smoketest fails on Linux.
- CWS-Oriented View
- Add header X axis (timeline)
- Three different views - all, ready for QA, new
- Buildslave info on slave web page
- Scheduler to prioritize and distribute builds on various slaves
- Collect changes by one user (settling) in one build
- AllCVS mailing list listener to auto trigger build on commit notification
- Header Row - provide number/link to last build (related to status)
- Extra config switches field in force build form. Developers can specify some extra configure switches specific to there CWSs. - Done
- Add suse 7.3 buildslave
- Generate buildbot results as RSS feed for planetizing
- For windows slaves, provide link to html build status page
- UserId/Password verification for force build and stop buid
Steps to add a new buildbot
Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.
What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environment on this machine.
The Buildbot requires:
- Python 2.3 or later: http://www.python.org
- Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
- Note: For Twisted, you first need to install Zope.
- The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from Gnome CVS.
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login no password cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/buildbot/buildbot-0.7.3
- Installing the Buildbot:
- Note: Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
- After all requirements are met, install buildbot by running following commands in buildbot directory.
- The install may need to be to run as root (su).
python setup.py build python setup.py install
- Test installation is proper by running the command
"buildbot --version"
- Install is OK if you see the version information for Buildbot and Twisted.
- Errors such as 'no such command' or "ImportError" means something went wrong.
- Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code
Setting up the Buildbot:
- Make a buildbot directory on the machine at an appropriate place.
mkdir buildbot_ooo
- The full path name for this directory is hereinafter referred to as slavedir.
- Run the buildbot command to create a build slave:
buildbot slave ''slavedir'' ''MASTERHOST'':''PORT'' ''SLAVENAME'' ''PASSWORD''
where
- MASTERHOST = ooo-staging.osuosl.org and PORT = 9989
- Choose a SLAVENAME and PASSWORD and report them to botmaster administrator
- For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave
- For additional creation options see http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options
- Fill in the hostinfo files
- slavedir/info/admin should contain your name and email address. This is the buildslave admin address
- slavedir/info/host should be filled with a brief description of the host. For example: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running on the machine.
- Checkout some OO build specific scripts from http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login no password cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/buildbot/slave
- copy these files to slavedir
- buildbotget.pl - perl script for checking out OO source code from CVS
- buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
- smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
- Once all the above steps are done, You need to email buildmaster administrator with the following information.
- SLAVENAME & PASSWORD : This is the same that you used while creating buildslave.
- slavedir (full path) e.g. /home/bbot/buildslave_ooostaging
- The configure command you are using for configuring OO on this build platform. e.g. ./configure --with-system-freetype
- Once buildmaster administrator receives this above information, he will add your buildslave to buildfarm and then your slave will become OOBuildSlave and will be listed on buildmaster status page [1]
People
- Prasad Madhav (pmadhav) deployed and customizes the buildbot
- mikeleib, kaib, vq, dkeskar support and provide ideas
