Apache OpenOffice Wiki - User contributions [en]

User:Mikeleib

2007-06-19T22:51:02Z

Mikeleib: /* The Recipe to Add a Slave */

Mikeleib's real name is Michael Leibowitz. Mikeleib works for the Intel Corporation. Mikeleib is in the United States. Mikeleib works on performance. Mikeleib writes very simple sentences.

----
==This is a little scratch space for buildbot==

===What's New===
The new (experimental) buildbot differs from the original in several key ways:
* steps (including configure lines) are defined on the slave
* file transfer is now in-band (no ssh-key needed)
* builds are triggered by cvs commits
* per-cws build is gone (currently)
* fewer modifications to buildbot sourcecode
* build results can be emailed to tinderbox

=== Prerequisites ===
* python2.4
* twisted2.4
<pre>
shaun-mcdonalds-computer:~ shaunmcdonald$ twistd --version
twistd (the Twisted daemon) 2.5.0
Copyright (c) 2001-2006 Twisted Matrix Laboratories.
See LICENSE for details.
shaun-mcdonalds-computer:~ shaunmcdonald$ which twistd
/sw/bin/twistd

</pre>
* perl
* bash
* 9.83 GB needed for full build including installset on Mac OS X Intel

=== The Recipe to Add a Slave ===
# grab a copy of the buildbot 0.7.5: [https://sourceforge.net/project/showfiles.php?group_id=73177 source]
# grab the [http://termite.go-oo.org:8081/~mikeleib/OO-changes.diff diff]
# unpack the source and apply the diff
# build buildbot. Note: that if you want to install it to different prefix such as your home directory see the [http://buildbot.net/repos/release/docs/buildbot.html#Installing-the-code manual]
# create the build-slave. MASTERHOST:PORT is termite.go-oo.org:9999
#: <pre>buildbot create-slave BASEDIR MASTERHOST:PORT SLAVENAME PASSWORD</pre>
# download the [http://termite.go-oo.org:8081/~mikeleib/scripts.zip scripts] and stuff them into your buildslave dir
# edit the scripts for your platform (you probably should change #!/bin/bash to #!/usr/bin/env bash at the least)
# put cwsget.pl in a directory in your PATH
# email mikeleib to add ur bot
# start the bot

User:Mikeleib

2007-06-19T04:10:43Z

Mikeleib: /* What's New */

User:Smsm1/ooocon2007

2007-05-31T15:50:04Z

Mikeleib: /* Abstract Stuff */

== Abstract Stuff ==

=== Title of your paper ===
Buildbot: Bigger, Better, Buildier

=== Session description ===
In this talk we will give a brief introduction the OpenOffice.org buildbot as well as discuss and demonstrate several exciting new features. Buildbot is a system similar in purpose to tinderbox for the automatic generation of builds on multiple platforms. However, buildbot very flexible and we have bent it into new shapes for OpenOffice.org. Our improvements scratch several old itches such as change-triggered builds, easier client-side config, and ready-for-qa triggered install-set generation. We have also added several exciting new features such as the "try" facility, which allows a developer to build from just a patch. Additionally, we will demonstrate the new arbitrary data regression system, which enables the regression testing and tracking of nearly any kind of data. This system is quite powerful because it allows one to track based on groupings such as a cws vs the milestone is based off or the relative memory performance per arch relative to the milestone, etc.

=== Target Audience ===

Developers

=== Proposed Track ===

Development > Tools for Development

=== Preferred Session Type ===
Presentation/Case Study/Workshop

== Author Stuff ==
=== Shaun McDonald ===

==== Short Biography ====
Shaun McDonald has been working in the OpenOffice.org community for the past 2 years. He has been mostly interested in the Mac Port of OpenOffice.org. He has taken on the Mac Porting Web site revamp, some QA work, and is the owner of the first Mac buildbot. Shaun is just completing his fourth and final year in Computer Science at Heriot-Watt University in Edinburgh.

==== URL of your home page / blog etc ====
http://shaunmcdonald131.blogspot.com/

====Job Title / Position in the Community ====
Student / Mac Porting [web] developer/tester

==== Company / OpenOffice.org Project Name ====
Independent / Porting ooo project

==== Email Address ====
smsm1@openoffice.org

==== Postal Address ====
45 Sighthill Terrace, Edinburgh, EH11 4QH, UK

==== Contact Telephone Number ====
+447879610632

* Request for travel subsidy (amount and reason) optional

=== Michael Leibowitz ===
==== Short Bio ====
Michael Leibowitz works for the Intel Corporation as a software engineer. He currently creates custom tooling to support Channel Software Engineering. Prior to that, he was a full-time openoffice.org contributor working on performance and more nebulous developer community issues. He is the founder of the openoffice.org wiki as well as the maintainer of the openoffice.org buildbot.

====email====

mikeleib@openoffice.org
==== Postal Address ====
2111 NE 25th Avenue
JF3-202
Hillsboro, OR 97124
USA

==== Telephone ====
+1 503 264 7621

User:Smsm1/ooocon2007

2007-05-31T15:48:51Z

Mikeleib: /* Telephone = */

== Abstract Stuff ==

* Title of your paper (P)
Buildbot: Bigger, Better, Buildier

* Session description (P)
In this talk we will give a brief introduction the OpenOffice.org buildbot as well as discuss and demonstrate several exciting new features. Buildbot is a system similar in purpose to tinderbox for the automatic generation of builds on multiple platforms. However, buildbot very flexible and we have bent it into new shapes for OpenOffice.org. Our improvements scratch several old itches such as change-triggered builds, easier client-side config, and ready-for-qa triggered install-set generation. We have also added several exciting new features such as the "try" facility, which allows a developer to build from just a patch. Additionally, we will demonstrate the new arbitrary data regression system, which enables the regression testing and tracking of nearly any kind of data. This system is quite powerful because it allows one to track based on groupings such as a cws vs the milestone is based off or the relative memory performance per arch relative to the milestone, etc.

* Target Audience (P)

Developers

* Proposed Track (P) (see the list in the Call for Papers)

Development > Tools for Development

* Preferred Session Type (P) (see the list in the Call for Papers)
Presentation/Case Study/Workshop

== Author Stuff ==
=== Shaun McDonald ===

==== Short Biography ====
Shaun McDonald has been working in the OpenOffice.org community for the past 2 years. He has been mostly interested in the Mac Port of OpenOffice.org. He has taken on the Mac Porting Web site revamp, some QA work, and is the owner of the first Mac buildbot. Shaun is just completing his fourth and final year in Computer Science at Heriot-Watt University in Edinburgh.

==== URL of your home page / blog etc ====
http://shaunmcdonald131.blogspot.com/

====Job Title / Position in the Community ====
Student / Mac Porting [web] developer/tester

==== Company / OpenOffice.org Project Name ====
Independent / Porting ooo project

==== Email Address ====
smsm1@openoffice.org

==== Postal Address ====
45 Sighthill Terrace, Edinburgh, EH11 4QH, UK

==== Contact Telephone Number ====
+447879610632

* Request for travel subsidy (amount and reason) optional

=== Michael Leibowitz ===
==== Short Bio ====
Michael Leibowitz works for the Intel Corporation as a software engineer. He currently creates custom tooling to support Channel Software Engineering. Prior to that, he was a full-time openoffice.org contributor working on performance and more nebulous developer community issues. He is the founder of the openoffice.org wiki as well as the maintainer of the openoffice.org buildbot.

====email====

mikeleib@openoffice.org
==== Postal Address ====
2111 NE 25th Avenue
JF3-202
Hillsboro, OR 97124
USA

==== Telephone ====
+1 503 264 7621

User:Smsm1/ooocon2007

2007-05-31T05:27:28Z

Mikeleib: some abstract meat

== Abstract Stuff ==

* Title of your paper (P)
Buildbot: Bigger, Better, Buildier

* Session description (P)
In this talk we will give a brief introduction the OpenOffice.org buildbot as well as discuss and demonstrate several exciting new features. Buildbot is a system similar in purpose to tinderbox for the automatic generation of builds on multiple platforms. However, buildbot very flexible and we have bent it into new shapes for OpenOffice.org Our improvements scratch several old itches such as change-triggered builds, easier client-side config, and ready-for-qa triggered install-set generation. We have also added several exciting new features such as the "try" facility, which allows a developer to build from just a patch. Additionally, we will demonstrate the new arbitrary data regression system, which enables the regression testing and tracking of nearly any kind of data. This system is quite powerful because it allows one to track based on groupings such as a cws vs the milestone is based off or the relative memory performance per arch relative to the milestone, etc.

* Target Audience (P)

Developers

* Proposed Track (P) (see the list in the Call for Papers)

Development > Tools for Development

* Preferred Session Type (P) (see the list in the Call for Papers)
Presentation/Case Study/Workshop

== Author Stuff ==
=== Shaun ===
* Your Full Name (P)
** Shaun McDonald
** Michael Leibowtiz
* Short Biography (P) optional
** Shaun McDonald has been working in the OpenOffice.org community for the past 2 years. He has been mostly interested in the Mac Port of OpenOffice.org. He has taken on the Mac Porting Web site revamp, some QA work, and is an owner of a Mac buildbot.
**
* URL of your home page / blog etc (P) optional
** http://shaunmcdonald131.blogspot.com/
** http://
* Job Title / Position in the Community (P) optional
** Hopefully Just Graduated Student / Mac Porting [web] developer/tester
** Developer
* Company / OpenOffice.org Project Name (P) optional
** Independent / Porting ooo project
** Intel
* Email Address
** smsm1@openoffice.org
** mikeleib@openoffice.org
* Postal Address
** 45 Sighthill Terrace, Edinburgh, EH11 4QH, UK
** US
* Contact Telephone Number (Mobile / cell phone preferred for use at OOoCon) optional
** +447879610632
** +1
* Request for travel subsidy (amount and reason) optional

=== Michael Leibowitz ===
==== Short Bio ====
Michael Leibowitz works for the Intel Corporation as a software engineer. He currently creates custom tooling to support Channel Software Engineering. Prior to that, he was a full-time openoffice.org contributor working on performance and more nebulous developer community issues. He is the founder of the openoffice.org wiki as well as the maintainer of the openoffice.org buildbot.

==== Postal Address ====
2111 NE 25th Avenue
JF3-202
Hillsboro, OR 97124
USA

==== Telephone =====
+1 503 264 7621

User:Mikeleib

2007-03-23T23:54:01Z

Mikeleib: buildbot steps

Mikeleib's real name is Michael Leibowitz. Mikeleib works for the Intel Corporation. Mikeleib is in the United States. Mikeleib works on performance. Mikeleib writes very simple sentences.

----
==This is a little scratch space for buildbot==

===What's New===
The new (experimental) buildbot differs from the original in several key ways:
* steps (including configure lines) are defined on the slave
* file transfer is now in-band (no ssh-key needed)
* builds are triggered by cvs commits
* per-cws build is gone (currently)
* fewer modifications to buildbot sourcecode
* build results can be emailed to tinderbox, but is not enabled just yet

=== Prerequisites ===
* python2.4
* twisted2.4
* perl
* bash

=== The Recipe to Add a Slave ===
# grab a copy of the buildbot 0.7.5: [https://sourceforge.net/project/showfiles.php?group_id=73177 source]
# grab the [http://termite.go-oo.org:8081/~mikeleib/OO-changes.diff diff]
# unpack the source and apply the diff
# build buildbot. Note: that if you want to install it to different prefix such as your home directory see the [http://buildbot.net/repos/release/docs/buildbot.html#Installing-the-code manual]
# create the build-slave. MASTERHOST:PORT is termite.go-oo.org:9999
#: <pre>buildbot create-slave BASEDIR MASTERHOST:PORT SLAVENAME PASSWORD</pre>
# download the [http://termite.go-oo.org:8081/~mikeleib/scripts.zip scripts] and stuff them into your buildslave dir
# edit the scripts for your platform (you probably should change #!/bin/bash to #!/usr/bin/env bash at the least)
# email mikeleib to add ur bot
# start the bot

Getting the source

2007-02-20T14:20:17Z

Mikeleib: s/CVS/SVN/ wrt gnome-cvs

{|
|
=== Source code ===

The source code can be downloaded as tarballs eg. [http://download.openoffice.org/2.1.0/source.html here]. It is huge, so it was split to several files:

* OOo_2.1.0_src_core.tar.bz2 - the necessary part for each build, the other tarballs depend on this one.
* OOo_2.1.0_src_binfilter.tar.bz2 - the filters for old StarOffice formats.
* OOo_2.1.0_src_l10n.tar.bz2 - translations to other languages than English and German.
* OOo_2.1.0_src_sdk.tar.bz2 - OOo SDK (Software Development Kit).
* OOo_2.1.0_src_system.tar.bz2 - libraries that usually are on a standard (Linux) system and it's not necessary to build own version.

For a full build you need them all. For development, src_core is usually all you need.

Download them to one directory and unpack them. Information how to checkout from CVS follows.

=== Vanilla up-stream ===

You might be interested particulary in the [http://tools.openoffice.org/dev_docs/build_linux.html#GetTheSourceCode Get the source code] section in the build guide (for Linux, but it is quite similar for all the platforms).

export 'CVSROOT=:pserver:anoncvs@anoncvs.services.openoffice.org:/cvs'
cvs login
(CVS password: anoncvs)
cvs co -r <milestone> OpenOffice2

The tool to browse project [http://tools.openoffice.org/source/browse/tools/ source code online] has some [http://tools.openoffice.org/servlets/ProjectSource additional hints].

If you want just to test development releases, you can download already compiled builds from [http://download.openoffice.org/680/]. List of available Mac builds is in [[Mac OS X Development Builds]].

=== ooo-build ===

There are loads of versions of OO.o, and several choices of
branch, with multiple outstanding patch sets.
I recommend you build from up-stream CVS HEAD milestones
(SRC680 milestones), with patch sets to make them easier to
build from [http://go-ooo.org/packages/SRC680]

The very latest ooo-build (a small ~1.5Mb build wrapper) can
be got from SVN thus:

<pre>
svn co http://svn.gnome.org/svn/ooo-build/trunk ooo-build
</pre>

Note: You are going to need to download
an additional ~150Mb of compressed source, and have ~3Gb
of space to unpack and build it in.

== See also ==

*[[Building]]
| valign="top" |
{{Getting_It|Languages}}
|}

[[Category:Development]]

Getting the source

2007-02-20T14:19:28Z

Mikeleib: point to gnome-svn for ooo-build

{|
|
=== Source code ===

The source code can be downloaded as tarballs eg. [http://download.openoffice.org/2.1.0/source.html here]. It is huge, so it was split to several files:

* OOo_2.1.0_src_core.tar.bz2 - the necessary part for each build, the other tarballs depend on this one.
* OOo_2.1.0_src_binfilter.tar.bz2 - the filters for old StarOffice formats.
* OOo_2.1.0_src_l10n.tar.bz2 - translations to other languages than English and German.
* OOo_2.1.0_src_sdk.tar.bz2 - OOo SDK (Software Development Kit).
* OOo_2.1.0_src_system.tar.bz2 - libraries that usually are on a standard (Linux) system and it's not necessary to build own version.

For a full build you need them all. For development, src_core is usually all you need.

Download them to one directory and unpack them. Information how to checkout from CVS follows.

=== Vanilla up-stream ===

You might be interested particulary in the [http://tools.openoffice.org/dev_docs/build_linux.html#GetTheSourceCode Get the source code] section in the build guide (for Linux, but it is quite similar for all the platforms).

export 'CVSROOT=:pserver:anoncvs@anoncvs.services.openoffice.org:/cvs'
cvs login
(CVS password: anoncvs)
cvs co -r <milestone> OpenOffice2

The tool to browse project [http://tools.openoffice.org/source/browse/tools/ source code online] has some [http://tools.openoffice.org/servlets/ProjectSource additional hints].

If you want just to test development releases, you can download already compiled builds from [http://download.openoffice.org/680/]. List of available Mac builds is in [[Mac OS X Development Builds]].

=== ooo-build ===

There are loads of versions of OO.o, and several choices of
branch, with multiple outstanding patch sets.
I recommend you build from up-stream CVS HEAD milestones
(SRC680 milestones), with patch sets to make them easier to
build from [http://go-ooo.org/packages/SRC680]

The very latest ooo-build (a small ~1.5Mb build wrapper) can
be got from CVS thus:

<pre>
svn co http://svn.gnome.org/svn/ooo-build/trunk ooo-build
</pre>

Note: You are going to need to download
an additional ~150Mb of compressed source, and have ~3Gb
of space to unpack and build it in.

== See also ==

*[[Building]]
| valign="top" |
{{Getting_It|Languages}}
|}

[[Category:Development]]

Buildbot Steps

2007-01-05T23:46:57Z

Mikeleib: /* Email botmaster admin */

Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.

What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environment on this machine.

== Buildbot Requirements ==
The Buildbot requires:
* Python 2.3 or later: http://www.python.org
* Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
** Note: For Twisted, you first need to install Zope.

== Getting the OpenOffice Buildbot ==
The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from [http://cvs.gnome.org/viewcvs/ooo-build/buildbot/ Gnome CVS].
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/buildbot
</pre>

== Installing the Buildbot ==
* Note:
** Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
** The install may need to be to run as root (su).
After all requirements are met, install buildbot by running following commands in buildbot directory.
<pre>
python setup.py build
python setup.py install
</pre>
Test installation is proper by running the command
<pre>"buildbot --version"</pre>
* Install is OK if you see the version information for Buildbot and Twisted.
* Errors such as 'no such command' or "ImportError" means something went wrong.
* Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code

== Setting up the Buildbot ==
Make a buildbot directory on the machine at an appropriate place.
* The full path name for this directory is hereinafter referred to as ''slavedir''.
<pre>mkdir buildbot_ooo</pre>
Run the buildbot command to create a build slave:
<pre>
buildbot slave ''slavedir'' buildbot.go-oo.org:9989 ''SLAVENAME'' ''PASSWORD''
</pre>
where
* Choose a ''SLAVENAME'' and ''PASSWORD'' and report them to botmaster administrator
* For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave
* For additional creation options see http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options

Fill in the hostinfo files
* ''slavedir''/info/admin should contain your name and email address. This is the '''buildslave admin address'''
* ''slavedir''/info/host should be filled with a brief description of the host. For example: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running on the machine.

Checkout some OO build specific scripts from http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/buildbot/slave
</pre>
You will see the following files:
* buildbotget.pl - perl script for checking out OO source code from CVS
* buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
* smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
Please copy these files to ''slavedir''

== Email botmaster admin ==
Once all the above steps are done, please email botmaster administrators:
* mikeleib at openoffice.org

With the following information.
* ''SLAVENAME'' & ''PASSWORD'' : This is the same that you used while creating buildslave.
* ''slavedir'' (full path) e.g. /home/bbot/buildslave_ooo
* The configure command you are using for configuring OO on this build platform. e.g. ./configure --with-system-freetype

The botmaster administrator will then add your buildbot to the OpenOffice.org buildfarm. Your buildbot will then start carrying out OpenOffice.org builds in response to botmaster commands and will be listed on the status page at http://buildbot.go-oo.org

== Start the buildslave ==
Once buildmaster administrator configures your new buildslave, he will contact you about status.
Once everything is OK then you can start your buildslave by running following command
<pre>
buildbot start ''slavedir''
</pre>
To stop buildslave anytime, please run
<pre>
buildbot stop ''slavedir''
</pre>

Buildbot Steps

2007-01-05T23:46:29Z

Mikeleib: /* Email botmaster admin */

Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.

What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environment on this machine.

== Buildbot Requirements ==
The Buildbot requires:
* Python 2.3 or later: http://www.python.org
* Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
** Note: For Twisted, you first need to install Zope.

== Getting the OpenOffice Buildbot ==
The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from [http://cvs.gnome.org/viewcvs/ooo-build/buildbot/ Gnome CVS].
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/buildbot
</pre>

== Installing the Buildbot ==
* Note:
** Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
** The install may need to be to run as root (su).
After all requirements are met, install buildbot by running following commands in buildbot directory.
<pre>
python setup.py build
python setup.py install
</pre>
Test installation is proper by running the command
<pre>"buildbot --version"</pre>
* Install is OK if you see the version information for Buildbot and Twisted.
* Errors such as 'no such command' or "ImportError" means something went wrong.
* Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code

== Setting up the Buildbot ==
Make a buildbot directory on the machine at an appropriate place.
* The full path name for this directory is hereinafter referred to as ''slavedir''.
<pre>mkdir buildbot_ooo</pre>
Run the buildbot command to create a build slave:
<pre>
buildbot slave ''slavedir'' buildbot.go-oo.org:9989 ''SLAVENAME'' ''PASSWORD''
</pre>
where
* Choose a ''SLAVENAME'' and ''PASSWORD'' and report them to botmaster administrator
* For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave
* For additional creation options see http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options

Fill in the hostinfo files
* ''slavedir''/info/admin should contain your name and email address. This is the '''buildslave admin address'''
* ''slavedir''/info/host should be filled with a brief description of the host. For example: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running on the machine.

Checkout some OO build specific scripts from http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/buildbot/slave
</pre>
You will see the following files:
* buildbotget.pl - perl script for checking out OO source code from CVS
* buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
* smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
Please copy these files to ''slavedir''

== Email botmaster admin ==
Once all the above steps are done, please email botmaster administrators:
* mikeleib at openoffice.org

With the following information.
* ''SLAVENAME'' & ''PASSWORD'' : This is the same that you used while creating buildslave.
* ''slavedir'' (full path) e.g. /home/bbot/buildslave_ooo
* The configure command you are using for configuring OO on this build platform. e.g. ./configure --with-system-freetype

The botmaster administrator will then add your buildbot to the OpenOffice.org buildfarm. Your buildbot will then start carrying out OpenOffice.org builds in response to botmaster commands and will be listed on the status page at http://ooo-staging.osuosl.org:8010

== Start the buildslave ==
Once buildmaster administrator configures your new buildslave, he will contact you about status.
Once everything is OK then you can start your buildslave by running following command
<pre>
buildbot start ''slavedir''
</pre>
To stop buildslave anytime, please run
<pre>
buildbot stop ''slavedir''
</pre>

Buildbot Steps

2007-01-05T23:46:04Z

Mikeleib: /* Setting up the Buildbot */

Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.

What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environment on this machine.

== Buildbot Requirements ==
The Buildbot requires:
* Python 2.3 or later: http://www.python.org
* Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
** Note: For Twisted, you first need to install Zope.

== Getting the OpenOffice Buildbot ==
The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from [http://cvs.gnome.org/viewcvs/ooo-build/buildbot/ Gnome CVS].
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/buildbot
</pre>

== Installing the Buildbot ==
* Note:
** Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
** The install may need to be to run as root (su).
After all requirements are met, install buildbot by running following commands in buildbot directory.
<pre>
python setup.py build
python setup.py install
</pre>
Test installation is proper by running the command
<pre>"buildbot --version"</pre>
* Install is OK if you see the version information for Buildbot and Twisted.
* Errors such as 'no such command' or "ImportError" means something went wrong.
* Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code

== Setting up the Buildbot ==
Make a buildbot directory on the machine at an appropriate place.
* The full path name for this directory is hereinafter referred to as ''slavedir''.
<pre>mkdir buildbot_ooo</pre>
Run the buildbot command to create a build slave:
<pre>
buildbot slave ''slavedir'' buildbot.go-oo.org:9989 ''SLAVENAME'' ''PASSWORD''
</pre>
where
* Choose a ''SLAVENAME'' and ''PASSWORD'' and report them to botmaster administrator
* For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave
* For additional creation options see http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options

Fill in the hostinfo files
* ''slavedir''/info/admin should contain your name and email address. This is the '''buildslave admin address'''
* ''slavedir''/info/host should be filled with a brief description of the host. For example: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running on the machine.

Checkout some OO build specific scripts from http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/buildbot/slave
</pre>
You will see the following files:
* buildbotget.pl - perl script for checking out OO source code from CVS
* buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
* smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
Please copy these files to ''slavedir''

== Email botmaster admin ==
Once all the above steps are done, please email botmaster administrators:
* mikeleib at openoffice.org

With the following information.
* ''SLAVENAME'' & ''PASSWORD'' : This is the same that you used while creating buildslave.
* ''slavedir'' (full path) e.g. /home/bbot/buildslave_ooostaging
* The configure command you are using for configuring OO on this build platform. e.g. ./configure --with-system-freetype

The botmaster administrator will then add your buildbot to the OpenOffice.org buildfarm. Your buildbot will then start carrying out OpenOffice.org builds in response to botmaster commands and will be listed on the status page at http://ooo-staging.osuosl.org:8010

== Start the buildslave ==
Once buildmaster administrator configures your new buildslave, he will contact you about status.
Once everything is OK then you can start your buildslave by running following command
<pre>
buildbot start ''slavedir''
</pre>
To stop buildslave anytime, please run
<pre>
buildbot stop ''slavedir''
</pre>

Integrated Performance Improvements

2006-10-23T18:41:43Z

Mikeleib: /* add links to issues */

== Integrated Performance Improvements ==

FIXME: this page is new

=== In 2.1 ===
* [http://qa.openoffice.org/issues/show_bug.cgi?id=54953 #54953 Accelerate PNG import]
* [http://www.openoffice.org/issues/show_bug.cgi?id=55170 #55170 Allow fast and nicer looking JPEG preview]
* [http://www.openoffice.org/issues/show_bug.cgi?id=55174 #55174 Allow fast PNG preview]
* [http://qa.openoffice.org/issues/show_bug.cgi?id=67234 #67234 png export: limit the chunk size]
* [http://www.openoffice.org/issues/show_bug.cgi?id=67237 #67237 jpeg export: use "jpeg progressive mode" for big images]
* [http://www.openoffice.org/issues/show_bug.cgi?id=68520 #68520 Improve performance on text formatting, if a lot of objects (>50) have to be considered for text wrapping]
* [http://www.openoffice.org/issues/show_bug.cgi?id=69915 #69915 Performance: Increase pack velocity ..]

=== In 2.0.4 ===
* [http://qa.openoffice.org/issues/show_bug.cgi?id=66688 #66688 performance problem when loading many customshapes from the OpenDocument format]

=== In 2.0.3 ===

=== In 2.0.2 ===

=== In 2.0.1 ===

=== In 2.0.0 ===

Buildbot

2006-10-12T05:52:32Z

Mikeleib: /* Features */

A buildbot is now deployed for OpenOffice.org.
'''http://ooo-staging.osuosl.org:8010/'''

From the Buildbot README: http://buildbot.sourceforge.net/

Please see how you can '''[[Buildbot Steps | add your own buildbot]]!'''

The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve.

== OpenOffice.org Buildbot ==
The Buildbot will:
* Build CWS/SVN branches on multiple platforms and make output available to developers
* Generate and make available install sets for multiple platforms
* Run tests (smoketest, performance tests, etc.) and make test reports available for tracking and regression.

== Buildbot vs. Tinderbox ==
Currently, there is a [[Tinderbox]]. In fact, the Buildbot uses or adapts several Tinderbox scripts. The Buildbot provides several advantages:
* At-a-glance, customizable statusboard
* Flexible, modular, extensible framework
* Bots can be commanded and coordinated from the botmaster

Both Buildbots and Tinderboxes are easy to setup and can be behind the firewall. Thanks to [[User:Cloph|Cloph]] for reminding about the similarities.

== Deployment & Hardware ==
Currently, the botmaster and several buildbots are hosted at OSU-OSL (http://osuosl.org). Additional machines will be added to the pool. Sun QA/RE plans to host a Solaris bot.

=== Allocation ===
Many buildbot based projects dedicate groups of bots to specific code branches. Thus, for example, checkins to HEAD will automatically trigger the associated pool of machines and the changes built, tested on multiple platforms. See for example: http://www.python.org/dev/buildbot/

However, this is impractical for OpenOffice.org due the numericity of branches (CWS'es). Instead, we focus on creating a reasonable sized pool of machines for each platform/variant, with flexible build policies (see below).

Currently, bots build for:
* debian
* suse 10.0
* suse 10.1 (jre and gij)
* suse 7.3 (several)
* windows xp (3+)

== Policies ==
The botmaster will schedule builds based on policies that take into account the pool size, machine status and build time. When a bot is busy doing a build, additional requests are automatically queued.

* At UTC 0200 + random() build HEAD
* At milestone release build milestone
* Build manually requested CWS or milestone
* Build automatically selected CWS

== Features ==
* When no CWS is provided --> checkout HEAD - '''Done'''
* Summaries - '''Done'''
** tail, warnings, errors, summary log like tinderbox
** error - show before/after buffer (15, 5) and skip ... between errors
* Deploy install-sets on bot-master - '''Done'''
* Increase length of status page. - '''Done'''
* Email notification of build result (fail, success, install, etc. with link to buildlogs)
** Send mail to requestor@openoffice.org when finished. (add a hint on the request form) - '''Done'''
** For CWS - send to CWS owner (from EIS)
* Fix smoke-test on some hosts
* CWS-Oriented View - '''Done'''
** Add header X axis (timeline)
** Three different views - all, ready for QA, new
* Buildslave info on slave web page
* Scheduler to prioritize and distribute builds on various slaves
** Collect changes by one user (settling) in one build
** AllCVS mailing list listener to auto trigger build on commit notification
* Header Row - provide number/link to last build (related to status)
* Extra config switches field in force build form. Developers can specify some extra configure switches specific to there CWSs. - '''Done'''
* Add suse 7.3 buildslave
* Generate buildbot results as RSS feed for planetizing
* For windows slaves, provide link to html build status page
* UserId/Password verification for force build and stop buid

== People ==
* Prasad Madhav (pmadhav) deployed and customizes the buildbot
* mikeleib, kaib, vq, dkeskar support and provide ideas

Talk:O3-build

2006-10-04T19:44:46Z

Mikeleib:

What is this? No links to/from anywhere and no context.

Buildbot Steps

2006-09-05T17:36:40Z

Mikeleib: /* Email botmaster admin */

Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.

What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environment on this machine.

== Buildbot Requirements ==
The Buildbot requires:
* Python 2.3 or later: http://www.python.org
* Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
** Note: For Twisted, you first need to install Zope.

== Getting the OpenOffice Buildbot ==
The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from [http://cvs.gnome.org/viewcvs/ooo-build/buildbot/ Gnome CVS].
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/buildbot
</pre>

== Installing the Buildbot ==
* Note:
** Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
** The install may need to be to run as root (su).
After all requirements are met, install buildbot by running following commands in buildbot directory.
<pre>
python setup.py build
python setup.py install
</pre>
Test installation is proper by running the command
<pre>"buildbot --version"</pre>
* Install is OK if you see the version information for Buildbot and Twisted.
* Errors such as 'no such command' or "ImportError" means something went wrong.
* Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code

== Setting up the Buildbot ==
Make a buildbot directory on the machine at an appropriate place.
* The full path name for this directory is hereinafter referred to as ''slavedir''.
<pre>mkdir buildbot_ooo</pre>
Run the buildbot command to create a build slave:
<pre>
buildbot slave ''slavedir'' ooo-staging.osuosl.org:9989 ''SLAVENAME'' ''PASSWORD''
</pre>
where
* Choose a ''SLAVENAME'' and ''PASSWORD'' and report them to botmaster administrator
* For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave
* For additional creation options see http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options

Fill in the hostinfo files
* ''slavedir''/info/admin should contain your name and email address. This is the '''buildslave admin address'''
* ''slavedir''/info/host should be filled with a brief description of the host. For example: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running on the machine.

Checkout some OO build specific scripts from http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/buildbot/slave
</pre>
You will see the following files:
* buildbotget.pl - perl script for checking out OO source code from CVS
* buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
* smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
Please copy these files to ''slavedir''

== Email botmaster admin ==
Once all the above steps are done, please email botmaster administrators:
* mikeleib at openoffice.org

With the following information.
* ''SLAVENAME'' & ''PASSWORD'' : This is the same that you used while creating buildslave.
* ''slavedir'' (full path) e.g. /home/bbot/buildslave_ooostaging
* The configure command you are using for configuring OO on this build platform. e.g. ./configure --with-system-freetype

The botmaster administrator will then add your buildbot to the OpenOffice.org buildfarm. Your buildbot will then start carrying out OpenOffice.org builds in response to botmaster commands and will be listed on the status page at http://ooo-staging.osuosl.org:8010

== Start the buildslave ==
Once buildmaster administrator configures your new buildslave, he will contact you about status.
Once everything is OK then you can start your buildslave by running following command
<pre>
buildbot start ''slavedir''
</pre>
To stop buildslave anytime, please run
<pre>
buildbot stop ''slavedir''
</pre>

Buildbot Steps

2006-09-05T17:33:30Z

Mikeleib: /* Getting the OpenOffice Buildbot */

Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.

What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environment on this machine.

== Buildbot Requirements ==
The Buildbot requires:
* Python 2.3 or later: http://www.python.org
* Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
** Note: For Twisted, you first need to install Zope.

== Getting the OpenOffice Buildbot ==
The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from [http://cvs.gnome.org/viewcvs/ooo-build/buildbot/ Gnome CVS].
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/buildbot
</pre>

== Installing the Buildbot ==
* Note:
** Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
** The install may need to be to run as root (su).
After all requirements are met, install buildbot by running following commands in buildbot directory.
<pre>
python setup.py build
python setup.py install
</pre>
Test installation is proper by running the command
<pre>"buildbot --version"</pre>
* Install is OK if you see the version information for Buildbot and Twisted.
* Errors such as 'no such command' or "ImportError" means something went wrong.
* Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code

== Setting up the Buildbot ==
Make a buildbot directory on the machine at an appropriate place.
* The full path name for this directory is hereinafter referred to as ''slavedir''.
<pre>mkdir buildbot_ooo</pre>
Run the buildbot command to create a build slave:
<pre>
buildbot slave ''slavedir'' ooo-staging.osuosl.org:9989 ''SLAVENAME'' ''PASSWORD''
</pre>
where
* Choose a ''SLAVENAME'' and ''PASSWORD'' and report them to botmaster administrator
* For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave
* For additional creation options see http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options

Fill in the hostinfo files
* ''slavedir''/info/admin should contain your name and email address. This is the '''buildslave admin address'''
* ''slavedir''/info/host should be filled with a brief description of the host. For example: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running on the machine.

Checkout some OO build specific scripts from http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/buildbot/slave
</pre>
You will see the following files:
* buildbotget.pl - perl script for checking out OO source code from CVS
* buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
* smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
Please copy these files to ''slavedir''

== Email botmaster admin ==
Once all the above steps are done, please email botmaster administrators:
* pmadhav at openoffice.org
* mikeleib at openoffice.org

With the following information.
* ''SLAVENAME'' & ''PASSWORD'' : This is the same that you used while creating buildslave.
* ''slavedir'' (full path) e.g. /home/bbot/buildslave_ooostaging
* The configure command you are using for configuring OO on this build platform. e.g. ./configure --with-system-freetype

The botmaster administrator will then add your buildbot to the OpenOffice.org buildfarm. Your buildbot will then start carrying out OpenOffice.org builds in response to botmaster commands and will be listed on the status page at http://ooo-staging.osuosl.org:8010

== Start the buildslave ==
Once buildmaster administrator configures your new buildslave, he will contact you about status.
Once everything is OK then you can start your buildslave by running following command
<pre>
buildbot start ''slavedir''
</pre>
To stop buildslave anytime, please run
<pre>
buildbot stop ''slavedir''
</pre>

Buildbot Steps

2006-09-05T17:32:52Z

Mikeleib: /* Getting the OpenOffice Buildbot */

Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.

What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environment on this machine.

== Buildbot Requirements ==
The Buildbot requires:
* Python 2.3 or later: http://www.python.org
* Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
** Note: For Twisted, you first need to install Zope.

== Getting the OpenOffice Buildbot ==
The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from [http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/ Gnome CVS].
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/buildbot
</pre>

== Installing the Buildbot ==
* Note:
** Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
** The install may need to be to run as root (su).
After all requirements are met, install buildbot by running following commands in buildbot directory.
<pre>
python setup.py build
python setup.py install
</pre>
Test installation is proper by running the command
<pre>"buildbot --version"</pre>
* Install is OK if you see the version information for Buildbot and Twisted.
* Errors such as 'no such command' or "ImportError" means something went wrong.
* Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code

== Setting up the Buildbot ==
Make a buildbot directory on the machine at an appropriate place.
* The full path name for this directory is hereinafter referred to as ''slavedir''.
<pre>mkdir buildbot_ooo</pre>
Run the buildbot command to create a build slave:
<pre>
buildbot slave ''slavedir'' ooo-staging.osuosl.org:9989 ''SLAVENAME'' ''PASSWORD''
</pre>
where
* Choose a ''SLAVENAME'' and ''PASSWORD'' and report them to botmaster administrator
* For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave
* For additional creation options see http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options

Fill in the hostinfo files
* ''slavedir''/info/admin should contain your name and email address. This is the '''buildslave admin address'''
* ''slavedir''/info/host should be filled with a brief description of the host. For example: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running on the machine.

Checkout some OO build specific scripts from http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
no password
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/buildbot/slave
</pre>
You will see the following files:
* buildbotget.pl - perl script for checking out OO source code from CVS
* buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
* smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
Please copy these files to ''slavedir''

== Email botmaster admin ==
Once all the above steps are done, please email botmaster administrators:
* pmadhav at openoffice.org
* mikeleib at openoffice.org

With the following information.
* ''SLAVENAME'' & ''PASSWORD'' : This is the same that you used while creating buildslave.
* ''slavedir'' (full path) e.g. /home/bbot/buildslave_ooostaging
* The configure command you are using for configuring OO on this build platform. e.g. ./configure --with-system-freetype

The botmaster administrator will then add your buildbot to the OpenOffice.org buildfarm. Your buildbot will then start carrying out OpenOffice.org builds in response to botmaster commands and will be listed on the status page at http://ooo-staging.osuosl.org:8010

== Start the buildslave ==
Once buildmaster administrator configures your new buildslave, he will contact you about status.
Once everything is OK then you can start your buildslave by running following command
<pre>
buildbot start ''slavedir''
</pre>
To stop buildslave anytime, please run
<pre>
buildbot stop ''slavedir''
</pre>

Buildbot

2006-08-10T23:21:16Z

Mikeleib: fix cvs co command

A buildbot is now deployed for OpenOffice.org.
'''http://ooo-staging.osuosl.org:8010/'''

From the Buildbot README: http://buildbot.sourceforge.net/

The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve.

== OpenOffice.org Buildbot ==
The Buildbot will:
* Build CWS/SVN branches on multiple platforms and make output available to developers
* Generate and make available install sets for multiple platforms
* Run tests (smoketest, performance tests, etc.) and make test reports available for tracking and regression.

== Buildbot vs. Tinderbox ==
Currently, there is a [[Tinderbox]]. In fact, the Buildbot uses or adapts several Tinderbox scripts. The Buildbot provides several advantages:
* At-a-glance, customizable statusboard
* Flexible, modular, extensible framework
* Bots can be commanded and coordinated from the botmaster

Both Buildbots and Tinderboxes are easy to setup and can be behind the firewall. Thanks to [[User:Cloph|Cloph]] for reminding about the similarities.

== Deployment & Hardware ==
Currently, the botmaster and several buildbots are hosted at OSU-OSL (http://osuosl.org). Additional machines will be added to the pool. Sun QA/RE plans to host a Solaris bot.

=== Allocation ===
Many buildbot based projects dedicate groups of bots to specific code branches. Thus, for example, checkins to HEAD will automatically trigger the associated pool of machines and the changes built, tested on multiple platforms. See for example: http://www.python.org/dev/buildbot/

However, this is impractical for OpenOffice.org due the numericity of branches (CWS'es). Instead, we focus on creating a reasonable sized pool of machines for each platform/variant, with flexible build policies (see below).

Currently, bots build for:
* debian
* suse 10.0
* suse 10.1 (jre and gij)
* suse 7.3 (several)
* windows xp (3+)

== Policies ==
The botmaster will schedule builds based on policies that take into account the pool size, machine status and build time. When a bot is busy doing a build, additional requests are automatically queued.

* At UTC 0200 + random() build HEAD
* At milestone release build milestone
* Build manually requested CWS or milestone
* Build automatically selected CWS

== Features ==
* When no CWS is provided --> checkout HEAD - '''Done'''
* Summaries
** tail, warnings, errors, summary log like tinderbox
** error - show before/after buffer (15, 5) and skip ... between errors
* Deploy install-sets on bot-master
* Increase length of status page.
* Email notification of build result (fail, success, install, etc. with link to buildlogs)
** Send mail to requestor@openoffice.org when finished. (add a hint on the request form)
** For CWS - send to CWS owner (from EIS)
* Show smoke-test results in status -- currently smoketest fails on Linux.
* CWS-Oriented View
** Add header X axis (timeline)
** Three different views - all, ready for QA, new
* Buildslave info on slave web page
* Scheduler to prioritize and distribute builds on various slaves
** Collect changes by one user (settling) in one build
** AllCVS mailing list listener to auto trigger build on commit notification
* Header Row - provide number/link to last build (related to status)
* Extra config switches field in force build form. Developers can specify some extra configure switches specific to there CWSs. - '''Done'''
* Add suse 7.3 buildslave
* Generate buildbot results as RSS feed for planetizing
* For windows slaves, provide link to html build status page
* UserId/Password verification for force build and stop buid

== Steps to add a new buildbot ==
Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.

What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environmnet on this machine.

The Buildbot requires:
* Python 2.3 or later: http://www.python.org
* Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
** Note: For Twisted, you first need to install Zope.
* The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from [http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/ Gnome CVS].
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/configdbbe/buildbot-0.7.3
</pre>

Installing the Buildbot:
* Note: Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
* After all requirements are met, install buildbot by running following commands in buildbot directory.
* The install may need to be to run as root (su).
<pre>
python setup.py build
python setup.py install
</pre>
* Test installation is proper by running the command
<pre>"buildbot --version"</pre>
** Install is OK if you see the version information for Buildbot and Twisted.
** Errors such as 'no such command' or "ImportError" means something went wrong.
** Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code

Setting up the Buildbot:
* mkdir ''slavedir''
** Note : preferable name of ''slavedir'' : buildslave_ooostaging
* create a buildslave by running command:
buildbot slave ''slavedir'' ''MASTERHOST'':''PORT'' ''SLAVENAME'' ''PASSWORD''
** where, MASTERHOST = ooo-staging.osuosl.org and PORT = 9989
** Note : You need to report ''SLAVENAME'' and ''PASSWORD'' to buildmaster administrator
** Note: For more detailed instuctions see [http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave]
** You can specify certain options while creating buildslave. see [http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options]

* Fill in the hostinfo files
** ''slavedir''/info/admin should contain your name and email address. This is the “buildslave admin address”
** ''slavedir''/info/host should be filled with a brief description of the host: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running the buildslave.
* Checkout some OO buildslave specific scripts from [http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/]
** buildbotget.pl - perl script for checking out OO source code from CVS
** buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
** smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
* Once all the above steps are done, You need to email buildmaster administrator following stuff:
** ''SLAVENAME'' & ''PASSWORD'' : This is the same that you used while creating buildslave.
** Full path of the ''slavedir'' e.g. /home/bbot/buildslave_ooostaging
** platform specific configure command used for configuring OO.
e.g. ./configure --with-system-freetype
* Once buildmaster administrator receives this above information, he will add your buildslave to buildfarm and then your slave will become OOBuildSlave and will be listed on buildmaster status page [http://ooo-staging.osuosl.org:8010]

== People ==
* Prasad Madhav (pmadhav) deployed and customizes the buildbot
* mikeleib, kaib, vq, dkeskar support and provide ideas

Buildbot

2006-08-10T23:11:48Z

Mikeleib: gnome-cvs root

A buildbot is now deployed for OpenOffice.org.
'''http://ooo-staging.osuosl.org:8010/'''

From the Buildbot README: http://buildbot.sourceforge.net/

The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve.

== OpenOffice.org Buildbot ==
The Buildbot will:
* Build CWS/SVN branches on multiple platforms and make output available to developers
* Generate and make available install sets for multiple platforms
* Run tests (smoketest, performance tests, etc.) and make test reports available for tracking and regression.

== Buildbot vs. Tinderbox ==
Currently, there is a [[Tinderbox]]. In fact, the Buildbot uses or adapts several Tinderbox scripts. The Buildbot provides several advantages:
* At-a-glance, customizable statusboard
* Flexible, modular, extensible framework
* Bots can be commanded and coordinated from the botmaster

Both Buildbots and Tinderboxes are easy to setup and can be behind the firewall. Thanks to [[User:Cloph|Cloph]] for reminding about the similarities.

== Deployment & Hardware ==
Currently, the botmaster and several buildbots are hosted at OSU-OSL (http://osuosl.org). Additional machines will be added to the pool. Sun QA/RE plans to host a Solaris bot.

=== Allocation ===
Many buildbot based projects dedicate groups of bots to specific code branches. Thus, for example, checkins to HEAD will automatically trigger the associated pool of machines and the changes built, tested on multiple platforms. See for example: http://www.python.org/dev/buildbot/

However, this is impractical for OpenOffice.org due the numericity of branches (CWS'es). Instead, we focus on creating a reasonable sized pool of machines for each platform/variant, with flexible build policies (see below).

Currently, bots build for:
* debian
* suse 10.0
* suse 10.1 (jre and gij)
* suse 7.3 (several)
* windows xp (3+)

== Policies ==
The botmaster will schedule builds based on policies that take into account the pool size, machine status and build time. When a bot is busy doing a build, additional requests are automatically queued.

* At UTC 0200 + random() build HEAD
* At milestone release build milestone
* Build manually requested CWS or milestone
* Build automatically selected CWS

== Features ==
* When no CWS is provided --> checkout HEAD - '''Done'''
* Summaries
** tail, warnings, errors, summary log like tinderbox
** error - show before/after buffer (15, 5) and skip ... between errors
* Deploy install-sets on bot-master
* Increase length of status page.
* Email notification of build result (fail, success, install, etc. with link to buildlogs)
** Send mail to requestor@openoffice.org when finished. (add a hint on the request form)
** For CWS - send to CWS owner (from EIS)
* Show smoke-test results in status -- currently smoketest fails on Linux.
* CWS-Oriented View
** Add header X axis (timeline)
** Three different views - all, ready for QA, new
* Buildslave info on slave web page
* Scheduler to prioritize and distribute builds on various slaves
** Collect changes by one user (settling) in one build
** AllCVS mailing list listener to auto trigger build on commit notification
* Header Row - provide number/link to last build (related to status)
* Extra config switches field in force build form. Developers can specify some extra configure switches specific to there CWSs. - '''Done'''
* Add suse 7.3 buildslave
* Generate buildbot results as RSS feed for planetizing
* For windows slaves, provide link to html build status page
* UserId/Password verification for force build and stop buid

== Steps to add a new buildbot ==
Come join the collective! Bots for new platforms or adding bots for for existing platforms will increase the build capabilities available, and are therefore much appreciated.

What you need is a reasonably fast machine, with decent RAM and disk, and of course an Internet connection. It might be easier to first set up the openoffice build environmnet on this machine.

The Buildbot requires:
* Python 2.3 or later: http://www.python.org
* Twisted: http://twistedmatrix.com - most recent version is recommended. You'll need at least "Twisted" (the core package), and you'll also want TwistedMail, TwistedWeb, and TwistedWords.
** Note: For Twisted, you first need to install Zope.
* The Buildbot has been customized for Openoffice.org builds. Please checkout buildbot-0.7.3 from [http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/ Gnome CVS].
<pre>
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
cvs -z3 -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome co ooo-build/scratch/configdbbe
</pre>

Installing the Buildbot:
* Note: Detailed instuctions are at http://buildbot.sourceforge.net/manual-0.7.3.html#Requirements
* After all requirements are met, install buildbot by running following commands in buildbot directory.
* The install may need to be to run as root (su).
<pre>
python setup.py build
python setup.py install
</pre>
* Test installation is proper by running the command
<pre>"buildbot --version"</pre>
** Install is OK if you see the version information for Buildbot and Twisted.
** Errors such as 'no such command' or "ImportError" means something went wrong.
** Note: For more detailed instuctions see http://buildbot.sourceforge.net/manual-0.7.3.html#Installing-the-code

Setting up the Buildbot:
* mkdir ''slavedir''
** Note : preferable name of ''slavedir'' : buildslave_ooostaging
* create a buildslave by running command:
buildbot slave ''slavedir'' ''MASTERHOST'':''PORT'' ''SLAVENAME'' ''PASSWORD''
** where, MASTERHOST = ooo-staging.osuosl.org and PORT = 9989
** Note : You need to report ''SLAVENAME'' and ''PASSWORD'' to buildmaster administrator
** Note: For more detailed instuctions see [http://buildbot.sourceforge.net/manual-0.7.3.html#Creating-a-buildslave]
** You can specify certain options while creating buildslave. see [http://buildbot.sourceforge.net/manual-0.7.3.html#Buildslave-Options]

* Fill in the hostinfo files
** ''slavedir''/info/admin should contain your name and email address. This is the “buildslave admin address”
** ''slavedir''/info/host should be filled with a brief description of the host: OS, version, memory size, CPU speed, versions of relevant libraries installed, and finally the version of the buildbot code which is running the buildslave.
* Checkout some OO buildslave specific scripts from [http://cvs.gnome.org/viewcvs/ooo-build/scratch/buildbot/slave/]
** buildbotget.pl - perl script for checking out OO source code from CVS
** buildoo - script to start OO build process. Note: You need to properly edit this file to meet your plateform requirement.
** smoketestoo - script to start OO smoke test after successful compliation. Note: You need to properly edit this file to meet your plateform requirement.
* Once all the above steps are done, You need to email buildmaster administrator following stuff:
** ''SLAVENAME'' & ''PASSWORD'' : This is the same that you used while creating buildslave.
** Full path of the ''slavedir'' e.g. /home/bbot/buildslave_ooostaging
** platform specific configure command used for configuring OO.
e.g. ./configure --with-system-freetype
* Once buildmaster administrator receives this above information, he will add your buildslave to buildfarm and then your slave will become OOBuildSlave and will be listed on buildmaster status page [http://ooo-staging.osuosl.org:8010]

== People ==
* Prasad Madhav (pmadhav) deployed and customizes the buildbot
* mikeleib, kaib, vq, dkeskar support and provide ideas

DbConfig

2006-07-05T20:18:24Z

Mikeleib: /* Performance */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our [http://www.moonleib.org/dbbe/html/d2/d65/classconfigmgr_1_1dbbe_1_1BaseLayer.html Layers and UpdatableLayers] that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The [http://www.moonleib.org/dbbe/html/d8/d28/classconfigmgr_1_1dbbe_1_1Record.html Record] object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we use a [http://www.moonleib.org/dbbe/html/d6/d36/classconfigmgr_1_1dbbe_1_1BaseStratum.html base class] to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleLayerStratum is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, [http://www.moonleib.org/dbbe/html/d1/dde/classconfigmgr_1_1dbbe_1_1Database.html Database], that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== Performance ===
Just how fast is it? Benchmarks indicate that it is up to 16% faster on cold-starts.

The methodology of testing is as follows: The test machine is a 3.2Ghz P4HT running OpenSuSE 10.0 with 1G of RAM and 120G of SATA attached storage. OpenOffice is built with ooo-build. OpenOffice is started by a script in the xession that sleeps for two minutes before launching with the option -norestore -<application>. The system sleeps for two minutes and then reboots.
[[Image:Dbbe-vs-localbe-starts.png|frame|Median Time for 100 Cold-Starts]]

=== The Code ===
The code is in CWS: configdbbe

Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

File:Dbbe-vs-localbe-starts.png

2006-07-05T20:11:07Z

Mikeleib: 100 cold starts of oo.o with dbbe and localbe, showing median time

100 cold starts of oo.o with dbbe and localbe, showing median time

Infrastructure Maintenance

2006-06-29T16:26:48Z

Mikeleib:

==Planned Infrastructure Maintenace Work==

=== Wiki: Starting 2006-06-30 ===
Starting at 16:00 UTC lasting about 1 hour, the wiki will be donw. Site upgrade and category plugin installation

==Past maintenance timeframes==

__TOC__

=== Main Site: Starting 2006-06-20 ===

On Tuesday 2006-06-20 in evening (GMT) the [[Infrastructure_Upgrade|upgrade]] of the main site is planned. The complete upgrade process takes about 48 hours. The site will be available a bit earlier with some testing and aftertreatment going on. For SSH users it might be helpful to know that the upgraded production site will have a new IP address.

=== anoncvs, eis, wiki: Saturday and Sunday 2006-04-29 ===

Downtime due to rewiring.
Most of the ...services.openoffice.org machines will not be available.

=== Main Site: Friday 2006-04-28 ===

Starting at 16:00 UTC on Friday, 28 April, and lasting for about 20 minutes, the site will be down for maintenance. During this period, email, IssueZilla, CVS, and other site functionality will be unavailable.

=== wiki: Tuesday 2006-04-18 Evening (GMT) ===

The software running the wiki needs an upgrade to mediawiki 1.6.3. Credits go to [[User:Mikeleib]] for the installation of the new version of mediawiki.

=== Main Site: Monday 2006-04-17 ===

The main site will be undergoing maintenance from 17:30 - 18:00 UTC. It will be off-line. 
(This didn't happen).

=== Main Site: Saturday Early Morning (GMT) 2006-04-01 ===

Downtime starting at 01:00 (GMT) for about 8 hours due to site maintance.

=== anoncvs, eis, wiki: Saturday and Sunday 2006-04-01 ===

Downtime due to rewiring.
Most of the ...services.openoffice.org machines will not be available.

=== anoncvs: Tuesday 2006-03-21 ===

Our CVS repository for anonymous access (anoncvs.services.openoffice.org) moved to a new machine under a new IP address.

=== ooo.services.openoffice.org / ftp.stardiv.de: Monday 2006-03-13 ===

During Monday evening (GMT) ooo.services.openoffice.org and ftp.stardiv.de will move to a new IP addresses. This will need a short downtime. Additionally transitional unavailability might be experienced until the new IP addresses have propagated to DNS servers.

=== Main Site: Saturday 2006-02-18 ===

The main site will not be available Saturday 2006-02-18 starting 8:00 GMT for approximately 2 hours.

=== EIS: Friday 2006-02-17 ===

During Friday afternoon (GMT) eis.services.openoffice.org will move to a new IP address (192.18.197.101). This will need a short downtime. Additionally transitional unavailability might be experienced until the new IP address has propagated to DNS servers.
<math>Insert formula here</math>

== See also ==

* [[Infrastructure_Overview]]
* [[Infrastructure_Problems]]
* [[Infrastructure_Requirements]]

Infrastructure Maintenance

2006-06-29T16:24:25Z

Mikeleib: /* Planned Infrastructure Maintenace Work */

==Planned Infrastructure Maintenace Work==

* 6/30/2006 16:00 UTC for 1 hour
:Site upgrade and category plugin installation

==Past maintenance timeframes==

__TOC__

=== Main Site: Starting 2006-06-20 ===

On Tuesday 2006-06-20 in evening (GMT) the [[Infrastructure_Upgrade|upgrade]] of the main site is planned. The complete upgrade process takes about 48 hours. The site will be available a bit earlier with some testing and aftertreatment going on. For SSH users it might be helpful to know that the upgraded production site will have a new IP address.

=== anoncvs, eis, wiki: Saturday and Sunday 2006-04-29 ===

Downtime due to rewiring.
Most of the ...services.openoffice.org machines will not be available.

=== Main Site: Friday 2006-04-28 ===

Starting at 16:00 UTC on Friday, 28 April, and lasting for about 20 minutes, the site will be down for maintenance. During this period, email, IssueZilla, CVS, and other site functionality will be unavailable.

=== wiki: Tuesday 2006-04-18 Evening (GMT) ===

The software running the wiki needs an upgrade to mediawiki 1.6.3. Credits go to [[User:Mikeleib]] for the installation of the new version of mediawiki.

=== Main Site: Monday 2006-04-17 ===

The main site will be undergoing maintenance from 17:30 - 18:00 UTC. It will be off-line. 
(This didn't happen).

=== Main Site: Saturday Early Morning (GMT) 2006-04-01 ===

Downtime starting at 01:00 (GMT) for about 8 hours due to site maintance.

=== anoncvs, eis, wiki: Saturday and Sunday 2006-04-01 ===

Downtime due to rewiring.
Most of the ...services.openoffice.org machines will not be available.

=== anoncvs: Tuesday 2006-03-21 ===

Our CVS repository for anonymous access (anoncvs.services.openoffice.org) moved to a new machine under a new IP address.

=== ooo.services.openoffice.org / ftp.stardiv.de: Monday 2006-03-13 ===

During Monday evening (GMT) ooo.services.openoffice.org and ftp.stardiv.de will move to a new IP addresses. This will need a short downtime. Additionally transitional unavailability might be experienced until the new IP addresses have propagated to DNS servers.

=== Main Site: Saturday 2006-02-18 ===

The main site will not be available Saturday 2006-02-18 starting 8:00 GMT for approximately 2 hours.

=== EIS: Friday 2006-02-17 ===

During Friday afternoon (GMT) eis.services.openoffice.org will move to a new IP address (192.18.197.101). This will need a short downtime. Additionally transitional unavailability might be experienced until the new IP address has propagated to DNS servers.
<math>Insert formula here</math>

== See also ==

* [[Infrastructure_Overview]]
* [[Infrastructure_Problems]]
* [[Infrastructure_Requirements]]

Talk:Infrastructure Maintenance

2006-06-14T05:54:26Z

Mikeleib:

Surely, it's 48 hours and not days?--[[User:Mikeleib|Mikeleib]] 07:54, 14 June 2006 (CEST)

DbConfig

2006-05-26T19:07:08Z

Mikeleib: /* Performance */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our [http://www.moonleib.org/dbbe/html/d2/d65/classconfigmgr_1_1dbbe_1_1BaseLayer.html Layers and UpdatableLayers] that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The [http://www.moonleib.org/dbbe/html/d8/d28/classconfigmgr_1_1dbbe_1_1Record.html Record] object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we use a [http://www.moonleib.org/dbbe/html/d6/d36/classconfigmgr_1_1dbbe_1_1BaseStratum.html base class] to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleLayerStratum is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, [http://www.moonleib.org/dbbe/html/d1/dde/classconfigmgr_1_1dbbe_1_1Database.html Database], that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== Performance ===
Just how fast is it? Results seem to indicate that it is nearly 1 second faster on cold-start. Cold-start performance (on Linux, at least) is very variable, with standard deviations of approximately 2.25 seconds. DBBE start times, as you can see below have a smaller standard deviation.
{|
|colspan="2" align="center"|[[Image:Writer-cold-start-graph.png|frame|Cold-start times for Writer with dbbe vs localbe on Linux (80 samples)]]
|-
|[[Image:Dbbe-histogram.png|frame|Cold-start times for Writer with dbbe on Linux (80 samples)]]
|[[Image:Localbe-histogram.png|frame|Cold-start times for Writer with localbe on Linux (80 samples)]]
|}

=== The Code ===
The code is in CWS: configdbbe

Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

DbConfig

2006-05-26T17:39:03Z

Mikeleib: /* How fast is it?? */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our [http://www.moonleib.org/dbbe/html/d2/d65/classconfigmgr_1_1dbbe_1_1BaseLayer.html Layers and UpdatableLayers] that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The [http://www.moonleib.org/dbbe/html/d8/d28/classconfigmgr_1_1dbbe_1_1Record.html Record] object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we use a [http://www.moonleib.org/dbbe/html/d6/d36/classconfigmgr_1_1dbbe_1_1BaseStratum.html base class] to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleLayerStratum is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, [http://www.moonleib.org/dbbe/html/d1/dde/classconfigmgr_1_1dbbe_1_1Database.html Database], that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== Performance ===
Just how fast is it? Results seem to indicate that it is nearly 1 second faster on cold-start. Cold-start performance (on Linux, at least) is very variable, with standard deviations of approximately 2.25 seconds. DBBE start times, as you can see below have a smaller standard deviation.

{|
|[[Image:Dbbe-histogram.png|frame|Cold-start times for Writer with DBBE on Linux (80 samples)]]
|[[Image:Localbe-histogram.png|frame|Cold-start times for Writer with localbe on Linux (80 samples)]]
|}

=== The Code ===
The code is in CWS: configdbbe

Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

User talk:SergeMoutou

2006-05-25T17:18:54Z

Mikeleib:

I really appreciate your efforts and don't want to discourage you. However, could you flag edits that are simple changes as "minor?" When I look at recent changes, all I can see are are your edits to a few nodes. Thanks--[[User:Mikeleib|Mikeleib]] 19:18, 25 May 2006 (CEST)

Architecture/To-Dos

2006-05-24T05:17:35Z

Mikeleib: /* General Refactoring Improvements */

This page intends to collect various architectural deficiencies (aka the pet peeves of various people) of OpenOffice.org, and lists the areas where's work in progress to improve on the architecture.

Depending on the specific count algorithm, OOo consists of approximately 7E6 lines of code (the overwhelming lot being c++, all other being an order of magnitude less (Java, Perl, Basic, Python)). This sheer size in and of itself is a problem - the code base is notorious for crashing or slowing down to a crawl various software engineering tools, from debugger to dependency analysis to reverse design extraction.

The code itself varies greatly in quality, style, and age (the latter invariably leading to the former, if you recall the history and evolvement of c++), with parts being there virtually unmodified for 10+ years, and others just recently written from scratch.

Taken together, this leads to a lot of complexity and [http://artax.karlin.mff.cuni.cz/~kendy/ooo/cut-n-paste/src680-m154.txt.gz redundancy], which is very hard to remove.

Facing this amount of code, the big rules must be:
*simplify
**remove internal redundancy
**remove external redundancy (use external projects, whereever possible)
**remove unused or dead code
**remove legacy functionality, which does no longer provide noticeable value (e.g. binfilter)

*refactor for orthogonality
**make subsystems implement independent functionality
**enable combinations of those subsystems to be freely combinable
**carry that to the UI level (no artificial restrictions on what one can do with UI objects - e.g. shapes can be rotated, and clearly text frames should, too)

==Infrastructure Improvements==

*[[BuildSpeedup | Speeding up the build system]], and maybe even make it consider global dependencies (currently, OOo has the notion of modules, which approximately map to toplevel directories in the build tree. Automatic build-time dependency calculation is currently only available on the intra-module level).

*Making the actual design more accessible, improving upon existing solutions like [http://ooo.ximian.com/lxr/ LXR] or [http://ooo.ximian.com/bonsai/ Bonsai]. Ultimately, this should result in refactorings of the source code being both much easier and much safer than today, by providing information where and how specific functionality is used. A prerequisite for that would be a parser that ''really'' knows about c++ - [http://www.gccxml.org/ gccxml] might be a starting point.

==Runtime System Improvements==

This is about making the implementation languages safer, and easier to use. What follows could also be subsumed under "transparency on the implementation level". When something can be used transparently, or appears transparent to a user, it is an implementation aspect she need not care about. Being able to program in an environment which is transparent with regard to lots of aspects, empowers the developer to focus on the problem at hand, not having to litter her code with mundane tasks such as memory management or locking.

*Make threading transparent. Currently, fulfilling the contract of a UNO component regarding thread-safeness is
#tedious work, because normally each involved object has to acquire and release a mutex on method entry and exit, respectively
#almost impossible to get right, let alone verified to work correctly (no races, no deadlocks), because of the sheer mass of involved objects and mutices (the number of distinct states that would have to be checked for a proper verification is intractable for anything but the most trivial examples). The upcoming [[Uno/Effort/Creating_the_Uno_Threading_Framework | UNO Threading framework]] makes thread-safeness transparent, by automatically locking and unlocking when entering or exiting components on a much coarser level than single methods.

*Make other mundane stuff transparent. Like memory management (via garbage collection, or refcounting via [http://www.boost.org/libs/smart_ptr/index.html smart ptrs], [http://api.openoffice.org/docs/DevelopersGuide/ProfUNO/ProfUNO.htm#1+4+2+7+3+Mapping+of+Interface UNO reference]), or [http://en.wikipedia.org/wiki/Transaction_processing transactionality] (the mode of making changes take place either completely, or not at all. Having a component behave in a non-transactional way in the face of an error makes recovery rather hard. There's more to transactionality than exception-safeness. Imagine two users collaborating on the same document).

==General Refactoring Improvements==
For many reasons the OpenOffice.org codebase is difficult to understand and navigate. On of the reasons is a lack of cleanup in the code. There is a never ending list of things that ought to be done-- add some of your own.

*Actually remove deprecated things. Things like String and [http://go-oo.org/lxr/ident?i=UniString UniString] need to go. svtools and tools have loads of stuff that is duplicated elsewhere or is deprecated. Getting rid of these sorts of things will make maintaining application code much easier.

*Document things. Some of the code has comments that at one time were correct. Some code has German comments. While most of the OpenOffice.org programmers sprechen zie Deutsch, there is an unofficial understanding that German comments mean "don't touch."

==Application-specific Improvements==

One of the lingering problems on the application level is the fact that, in spite of modularized lower-level functionality, application functionality cannot be shared ''between'' OOo's applications (except via embedding of a whole application (OLE)). This is because for neither Calc nor Writer, there are reusable application engines, like a text engine providing text editing and layouting functionality, or a table engine providing formula and calculation support. Draw/Impress already uses a shared engine, dubbed 'Drawing Layer'. But there's still considerable functionality hidden in the application code, which is worth extracting. Especially the missing Writer engine manifests itself in duplicated text editing functionality in EditEngine and TextEngine (used by Impress and Calc for their corresponding text functionality).

Another area of improvement is rendering. Currently, all application's graphical output is based on the OutputDevice class, which provides only very basic rendering facilities (in fact, besides largely extended text output functionality (to handle OOo's i18n requirements), this interface has basically remained unchanged for a long time). Specifically, things like performant alpha compositing or anti-aliased geometry rendering are extremely hard to achieve with the current design. Therefore, starting with OOo 2.0, the [http://api.openoffice.org/docs/common/ref/com/sun/star/rendering/XCanvas.html XCanvas] interface is slated to gradually replace OutputDevice in all applications.

===Writer===

*break up the monolith
*make the import filters more modular
*port rendering to XCanvas

===Calc===

*break up the monolith
*[[Calc_Performance | speed improvements]]
*port rendering to XCanvas

===Draw/Impress===

*break up the monolith
*become more decoupled from sfx2
*redesign API ([[Impress_Performance | performance]])

*port Drawing Layer to XCanvas

DbConfig

2006-05-21T05:25:57Z

Mikeleib: /* SingleStratum/MultiStratum */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our [http://www.moonleib.org/dbbe/html/d2/d65/classconfigmgr_1_1dbbe_1_1BaseLayer.html Layers and UpdatableLayers] that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The [http://www.moonleib.org/dbbe/html/d8/d28/classconfigmgr_1_1dbbe_1_1Record.html Record] object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we use a [http://www.moonleib.org/dbbe/html/d6/d36/classconfigmgr_1_1dbbe_1_1BaseStratum.html base class] to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleLayerStratum is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, [http://www.moonleib.org/dbbe/html/d1/dde/classconfigmgr_1_1dbbe_1_1Database.html Database], that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== The Code ===
The code is in CWS: configdbbe

Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

DbConfig

2006-05-21T04:31:03Z

Mikeleib: /* Factory/Database Abstraction */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our [http://www.moonleib.org/dbbe/html/d2/d65/classconfigmgr_1_1dbbe_1_1BaseLayer.html Layers and UpdatableLayers] that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The [http://www.moonleib.org/dbbe/html/d8/d28/classconfigmgr_1_1dbbe_1_1Record.html Record] object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we will use a base class to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleBackend is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, [http://www.moonleib.org/dbbe/html/d1/dde/classconfigmgr_1_1dbbe_1_1Database.html Database], that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== The Code ===
The code is in CWS: configdbbe

Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

DbConfig

2006-05-21T04:30:16Z

Mikeleib: /* Layer/Updatable Layer */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our [http://www.moonleib.org/dbbe/html/d2/d65/classconfigmgr_1_1dbbe_1_1BaseLayer.html Layers and UpdatableLayers] that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The [http://www.moonleib.org/dbbe/html/d8/d28/classconfigmgr_1_1dbbe_1_1Record.html Record] object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we will use a base class to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleBackend is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, Database, that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== The Code ===
The code is in CWS: configdbbe

Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

DbConfig

2006-05-21T04:28:01Z

Mikeleib: /* Layer/Updatable Layer */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our [http://www.moonleib.org/dbbe/html/d2/d65/classconfigmgr_1_1dbbe_1_1BaseLayer.html Layers and UpdatableLayers] that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The Record object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we will use a base class to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleBackend is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, Database, that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== The Code ===
The code is in CWS: configdbbe

Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

DbConfig

2006-05-21T04:23:29Z

Mikeleib: /* The Code */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our Layers and UpdatableLayers that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The Record object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we will use a base class to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleBackend is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, Database, that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== The Code ===
The code is in CWS: configdbbe

Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

DbConfig

2006-05-20T05:01:10Z

Mikeleib: /* The Code */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our Layers and UpdatableLayers that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The Record object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we will use a base class to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleBackend is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, Database, that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== The Code ===
The code is in CWS: configdbbe
Also, you can see a doxygen run of dbbe [http://www.moonleib.org/dbbe/html/da/ddb/namespaceconfigmgr_1_1dbbe.html here]

DbConfig

2006-05-20T00:06:43Z

Mikeleib: /* Futures */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== DBBE ==

We have crafted a new backend, called dbbe that uses xml files encapsulated in berkeley databases. This backend is similar to localbe in many ways and has some code that is "stolen" from there. It implements the following services:
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/Layer.html Layer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/UpdatableLayer.html UpdateableLayer]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/SingleLayerStratum.html SingleLayerStratum]
* [http://api.openoffice.org/docs/common/ref/com/sun/star/configuration/backend/MultiLayerStratum.html MultiLayerStratum]

Additionally, we have a utility that can import and export xcu files into databases for migration.

=== Basic Design ===

The berkeley database stores key/data pairs with no relations. These pairs are of arbitrary type and size. We've made an abstraction of the berkeleydb API that can put and get Records by key. Our Single/MultiLayerStratums list/get Layers that deal with individual Records.

=== Layer/Updatable Layer ===

We define a schema for our Layers and UpdatableLayers that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

Note: the use of the word sublayer is somewhat misleading. The Record object does not differntiate sublayer blobs from anonymous configuration "particles" It calls both sublayers.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer is implemented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We use the URL property, but define the "URL" as <dbPath>:Key.

=== SingleStratum/MultiStratum ===

Like in localbe, we will use a base class to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleBackend is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each. We do this by checking if the Layer has any sublayers and if the database it came from is read-only.

=== Factory/Database Abstraction ===

We have a class, Database, that abstracts the berkelyDB API from the rest of dbbe. This is done to consolodate access to database internal settings so that they can all be changed in one place.

Because we have multiple stratum services that are using the same database (just namepsaces within them), we provide a factory that can instantiate databases. This allows the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* import files
* export files
* integrity check
* statistics on database (list keys)

For import and export, we use a "code" to mangle/demangle key names into file paths/names for localbe. The scheme that localbe uses is automatically detected, and a correct code is supplied. However, for other cases, it is possible to supply a code for how names are mapped to keys. The code is very simple; it just specifies what parts of the path are namespaces, layers, and "sublayers." The Mangler class handles this name mangling/demangling and the respository class handles the import/export of keys.

=== The Code ===
The code is in CWS: configdbbe

DE/ReviewUI/Verschiedenes Bild bearbeiten

2006-05-09T20:04:47Z

Mikeleib: Reverted edit of 1147199054, changed back to last version by Rainer Bielefeld

[[Category:ReviewUI|Bild bearbeiten]]
Die Schritte, die für bestimmte Bild-Bearbeitungsschritte ausgeführt werden müssen, underscheiden sich teilweise in den verschiedenen OOo- Modulen. Die folgenden Tabellen zeigen die Unterschiede (Inkonsistenzen):

== Unterschiedliches Ergebnis der gleichen Aktion ==

{| border="1" cellspacing="0" cellpadding="5"
!Funktion
!in DRAW
!in IMPRESS
!in WRITER
!in CALC
!Einschätzung
|-
|Doppelklick auf ''.png''-Bild
|Textmodus wird geöffnet
|wie DRAW
| style="background-color:#00FF00" | Bildbearbeitungsdialog wird geöffnet
|wie DRAW
|kein gravierendes Problem
|-
|Klick auf [http://artax.karlin.mff.cuni.cz/~kendy/ooo/icons/status/reference/res/commandimagelist/lc_toggleanchortype.png Anker]-[http://artax.karlin.mff.cuni.cz/~kendy/ooo/icons/status/crystal/res/commandimagelist/lc_toggleanchortype.png Button] nach Wählen eines (.png-) Bildes
|./.
|./.
|Auswahl-Pulldown öffnet beim Anklicken
|style="background-color:#FFCBCB" |Verankerung wechselt sofort beim Anklicken
|[http://www.openoffice.org/issues/show_bug.cgi?id=64417 Issue 64417]
|-
|.
|..
|...
|....
|.....
|......
|-
|}

== Unterschiedliches Aktion für gleichen Bearbeitungswunsch ==

{| border="1" cellspacing="0" cellpadding="5"
!Bearbeitungswunsch
!in DRAW
!in Impress
!in WRITER
!in CALC
!Einschätzung
|-
|(.png-) Bild zuschneiden
|Ausschließlich über Button [http://artax.karlin.mff.cuni.cz/~kendy/ooo/icons/status/reference/res/commandimagelist/lc_grafattrcrop.png ''Zuschneiden''] möglich
|wie DRAW
| style="background-color:#00FF00" | Doppelklick auf Bildobjekt - Bildbearbeitungsdialog - Zuschneiden ''Zuschneiden''-Button wird nicht angeboten
|wie DRAW
|Kein gravierendes Problem
|-
|.png-) Bild Umrandung
|Nur über Umwege möglich, keine direkte Umrandungsfunktion
|wie DRAW
| style="background-color:#00FF00" |''Bild-Doppelklick'' oder ''Konextemnü'' oder ''Button Umrandung'' oder ''Menü: Format - Bild - Umrandung''
|wie DRAW
|kein gravierendes Problem
|-
|URL verlinkter (.png-) Bild ändern
|1. Bild anklicken, ''Menü: Einfügen - Bild - aus Datei'' 2. ''Menü: Bearbeiten - Verknüpfungen''
|wie DRAW
| style="background-color:#00FF00" |1. Sinngemäß wie ''1. DRAW'' 2. Sinngemäß wie ''2. DRAW'' 3. ''Bild-Doppelklick - Bild-Dialog''
|wie DRAW
|1. Draw ist verwirrend, da alte URL nicht angezeigt wird.
|-
|.
|..
|...
|....
|.....
|......
|-
|}

== Weiteres ==

Nach Wissensstand ''[[User:Rainer Bielefeld|Rainer Bielefeld]]'' aus Erfahrungen mit "2.0.2 German version WIN XP: [680m5(Build9011)]" und "2.0.1 RC5 English version WIN XP: [680m1(Build8990)]" ist die englische OOo-Version gleichermaßen betroffen.

DerivedWorks

2006-05-09T20:02:27Z

Mikeleib: Reverted edit of 1147199174, changed back to last version by Timarandras

This page will track derivatives of OpenOffice.org. Feel free to add to it if you know of any, and please include URLs. A derived work listed here could be free or proprietary, and should not be limited to language or platform. It may include those works that use OpenOffice.org in part or in whole. As well, a separate section could also track distributions, e.g., by Novell or Debian or Ubuntu. We are listing these mainly to track them and not for advertising purposes. Some of the companies listed below may no longer be producing derivatives of OpenOffice.org, as the license for the source changed in 2005, with 2.0, when we dropped the [http://www.openoffice.org/license.html SISSL] and moved to using the LGPL exclusively.

== Preliminary List ==

{| border=1 cellspacing=0 cellpadding=3
!Product name
!Company
!Windows
!Mac OS X
!Linux
!Solaris
!License
!Web Site
|-
|ChinaOffice
|GoldSoft
|X
|
|X
|
|Commercial
|http://www.goldsoft.net/
|-
|Co-Create Office
|Co-Create
|
|
|X
|
|Free
|http://www.ccoss.com.cn/
|-
|EuroOffice 2005
|Multiráció Kft.
|X
|
|X
|
|Commercial
|http://www.eurooffice2005.com/
|-
|KaiOffice
|KaiSource
|X
|
|
|
|Commercial
|http://www.kaisource.com/
|-
|Lycoris
|Lycoris (now Mandriva)
|
|
|X
|
|?
|http://www.lycoris.com/products/ppak/ (broken)
|-
|MagyarOffice
|Multiráció Kft.
|X
|
|X
|X
|Commercial
|http://www.magyaroffice.hu/
|-
|NeoOffice
|Planamesa
|
|X
|
|
|GPL
|http://neooffice.org/
|-
|NextOffice
|Well Develop
|X
|
|
|
|Commercial
|http://www.nextoffice.net/
|-
|OfficeOne
|?
|X
|
|
|
|Commercial
|http://shop.avanquest.com/france/produits/prod.php?pid=120/
|-
|OfficeTLE
|?
|X
|
|
|
|Commercial
|http://www.thaisoftware.co.th/index.php?tpid=pro:AR-0028
|-
|OpenOfficePL
|?
|X
|
|
|
|Commercial
|http://www.openoffice.com.pl/index.php?id=176
|-
|Pladao Office
|?
|X
|
|X
|X
|LGPL
|http://pladao.org/index.php?newlang=english
|-
|RedOffice
|?
|X
|
|X
|
|Commercial
|http://www.ch2000.com.cn/
|-
|StarOffice
|Sun
|X
|
|X
|X
|Commercial
|http://www.sun.com/software/star/staroffice/index.jsp
|-
|SunShine Office
|CS2C
|
|
|X
|
|?
|http://www.cs2c.com.cn/
|-
|ThizOffice
|ThizTech
|X
|
|X
|
|?
|http://www.thizlinux.com.cn/
|-
|Workspace
|IBM
|X
|
|X
|
|Commercial
|http://www-142.ibm.com/software/workplace/products/product5.nsf/wdocs/workplacehome
|-
|WPS Office
|KingSoft
|X
|
|
|
|Commercial
|http://www.wps.com.cn/
|}

Autopackage distribution

2006-05-09T20:01:23Z

Mikeleib: Reverted edit of 1147199174, changed back to last version by St

[[Wikipedia:Autopackage|Autopackage]] is a new package management system intended to be usable across multiple Linux distributions. Unlike traditional package formats such as [[Wikipedia:RPM_Package_Manager|RPM]] and [[Wikipedia:Deb (file_format)|DEB]], autopackage checks for the presence of dependencies ''on the actual system files'', rather than querying a database of package information.

This reduces compatibility issues with different package naming conventions, relaxes package version dependency and allows users to install the application without worrying about what package format is being used on their Linux distributions.

The '''Linux version of OpenOffice.org''' released [http://www.openoffice.org/ by the Offical OOo homepage], as a third-party binary (when viewed from the point of view of Linux vendors) for download with frequent updates, highly matches the software delivery requirement being served and addressed by autopackage.

By providing OpenOffice.org in autopackage format, it will go a long way to simplify the Linux installation process without end users needing to consult [http://documentation.openoffice.org/setup_guide2/index.html the OpenOffice.org 2.x Setup guide] (for setup procedures on various Linux distros out there).

It will also provide the foundation for incremental update mechanism similiar to the auto-updater implemented in Firefox 1.5.

== OpenOffice.org in autopackage format discussion ==

=== Issue reported ===
* [http://qa.openoffice.org/issues/show_bug.cgi?id=46333 Issue 46333]: Native Linux install package using autopackage
* [http://qa.openoffice.org/issues/show_bug.cgi?id=58158 Issue 58158]: Small Patch Update system[http://www.oooforum.org/forum/viewtopic.phtml?p=132194], is now practical to implement under Linux using autopackage

=== Preparation phrase ===

# Find out the dependency information
# Prepare the autopackage script
# Repackage the official binary in autopackage format
# Testing and feedback

Optional (to do later)
* Check the compilation environment
* Compiling a different build if necessary

== Autopackage information for users and developers ==

To see how easy it is to install software in autopackage format, check out this how-to page:
* [http://www.autopackage.org/docs/howto-install/ Install Linux autopackages in 4 easy steps]

=== General information ===

* [http://www.autopackage.org/ Autopackage Official page]
* [http://www.autopackage.org/faq.html Autopackage FAQ]

=== Developer documenation ===

* [http://www.autopackage.org/docs.html Packagers Documentation] With quickstart, tutorials, packager guides, API documentation and many other resources
* [http://www.autopackage.org/download-tools.html Developer Tools]
* [http://plan99.net/autopackage/ Autopackage Wiki]
* [http://planet.autopackage.org/ Planet Autopackge]: Developers' blog

== Related articles ==

* [http://www.osnews.com/story.php?news_id=10155 Autopackage 1.0, the now and the next]
* [http://www.linux.com/article.pl?sid=05/11/22/2021228 Autopackage: Toward a universal package manager for the desktop]
* [http://www.ubuntuforums.org/showthread.php?t=97504 Ubuntu Forum]: 35 pages long (wow!) discussion of the autopackage format and its integration with Ubuntu Dapper Drake

[[Category:User Experience]]
[[Category:Development]]

Template:Extensions

2006-05-01T23:36:09Z

Mikeleib: Reverted edit of 1146131468, changed back to last version by Mikeleib

{| align=right style="margin-left: 15px; border:1px solid #aaaaaa; background-color:#f9f9f9; padding:5px; font-size: 95%;" class=box
|---
!align=center style="background:#ccccff;" |'''''[[Extensions|OOo Extensions project]]'''''

Please view the [[Marketing#Marketing wiki usage|wiki usage guidelines]]
 before contributing.''
|-
|
'''''Categories:'''''
* [[:Category:Extensions|Extensions]]

'''''Pages:'''''
* [[Extensions|Index]]
* [[Extensions_repository|Repository]]
* [[Extensions_development|Development]]

|-
|'''[http://extensions.openoffice.org Extensions on the main site]'''

<div class="plainlinks"> ''[[Template:Extensions|View]] or [{{SERVER}}{{localurl:Template:Extensions|action=edit}} edit] this template.''</div>
|}

GNU Lesser General Public License

2006-05-01T23:36:04Z

Mikeleib: Reverted edit of 1146131468, changed back to last version by BeDipp

The GNU Lesser General Public License (LGPL) is a Free software license, allowing you to use the software / document / art work without any licensing fees.

You may redistribute the product as long as you provide the license text (or a link to the license text) with the software package - and you may modify it as long as it stays under this license in your work.

In contrary to [http://www.gnu.org/licenses/lgpl.html GPL] it is possible to combine the product with closed source licensed parts (like libraries).

LGPL is now the only license of [[OpenOffice.org]] - before version 2.0 in combination with Sun's [http://www.openoffice.org/licenses/sissl_license.html SISSL].

== External links ==

* [http://www.gnu.org/licenses/lgpl.html Official LGPL page]
* [[Wikipedia: GNU Lesser General Public License]]

DE/Stammtische

2006-05-01T23:34:59Z

Mikeleib: Reverted edit of 1146173050, changed back to last version by VolkerMe

[[Category:de.openoffice.org]]
== Koordinationsseite für OpenOffice.org-Stammtische ==

Diese Seite dient zur Koordination und Bekanntgabe der Termine für die OpenOffice.org-Stammtische.

Die Stammtische sind eine schöne Gelegenheit, sich mit Gleichgesinnten zu treffen und einmal die Gesichter "hinter dem Projekt" kennen zu lernen. Sowohl Projektmitglieder als auch private wie gewerbliche Nutzer und Interessierte sind uns herzlich willkommen!

Jeder, der Interesse hat, kann in seiner Region einen eigenen Stammtisch aufbauen. Wir freuen uns auf eine rege Beteiligung!

== Stammtische nach Städten geordnet ==

<ul>
<li>West: Dortmund/Düsseldorf/Köln/Duisburg/Essen Ansprechpartner: Jacqueline Rahemipour <[mailto:jrahemipour@openoffice.org jrahemipour@openoffice.org]> Nächster geplanter Termin: noch offen</li>

<li>Erfurt/Weimar/Jena Ansprechpartner: Marko Moeller <[mailto:markomlm@openoffice.org markomlm@openoffice.org]> Nächster geplanter Termin: noch offen</li>

<li>Hannover und Umgebung Ansprechpartner: Volker Merschmann <[mailto:volkerme@openoffice.org volkerme@openoffice.org]> Nächster geplanter Termin: noch offen</li>

<li>Hamburg und Umgebung Ansprechpartner: Bernhard Dippold <[mailto:bedipp@openoffice.org bedipp@openoffice.org]> Nächster geplanter Termin: noch offen</li>

<li>München/Augsburg Ansprechpartner: Florian Effenberger <[mailto:floeff@openoffice.org floeff@openoffice.org]> Nächster geplanter Termin: noch offen</li>

<li>Ulm/Stuttgart Ansprechpartner: Wolfgang Kobel <[mailto:wolfgang-kobel@web.de wolfgang-kobel@web.de]> Nächster geplanter Termin: noch offen</li>
</ul>

* '''Ostschweiz''' / Quarten
:Ansprechpartner: André Schnabel [mailto:andreschnabel@openoffice.org]
:Nächster geplanter Termin: 18.05.2006

== Fotoarchiv der letzten Stammtische ==

Im Folgenden das Fotoarchiv der letzten OpenOffice.org-Stammtische. Die Nutzung und Weitergabe der Fotos ist nur mit ausdrücklicher Genehmigung des/der Fotografen erlaubt!

<ul><li>Dortmund, Stammtisch im Januar 2006 Fotograf: Florian Effenberger u.a. <[mailto:floeff@openoffice.org floeff@openoffice.org]> Link: [http://www.flickr.com/photos/floeff/sets/1810815/ http://www.flickr.com/photos/floeff/sets/1810815/]</ul>

<ul><li>München, 1. Stammtisch Fotograf: Florian Effenberger <[mailto:floeff@openoffice.org floeff@openoffice.org]> Link: [http://www.flickr.com/photos/floeff/sets/420785/ http://www.flickr.com/photos/floeff/sets/420785/]</ul>

<ul><li>Köln, 1. Stammtisch Fotograf: Marko Moeller <[mailto:markomlm@openoffice.org markomlm@openoffice.org]> Link: [http://www.thienel-moeller.de/OpenOffice_org/Bilder_Koeln/ http://www.thienel-moeller.de/OpenOffice_org/Bilder_Koeln]</ul>

DE/ReviewUI/Aktive Hilfe

2006-05-01T23:34:41Z

Mikeleib: Reverted edit of 1146173050, changed back to last version by Nnino

[[Category:ReviewUI|Hilfe, aktive]]
==Worum geht es?==
Erweiterte Tooltipps ein- und ausschalten.
==Mit welchen Begriffen taucht es wo auf? ==
;Aktive Hilfe (Extended Tips):
Text der Schaltfläche und des Menüeintrags. Kann als Schaltfläche oder Menüeintrag in allen Modulen einer Symbolleiste bzw. Menü hinzugefügt werden. Extras - Anpassen - Hinzufügen, Bereich „Applikation“ Befehl „Aktive Hilfe“.
;Erweiterte Tipps (Extended tips):
Extras - Optionen - OpenOffice.org - Allgemein, Abschnitt „Hilfe“.
;Direkt-Hilfe:
Online-Hilfe Stand 2.0.2 (=9011), /text/shared/guide/active_help_on_off.xhp; hd_id3156414
==Wie hieß es früher?==
;OOo1.1, SO5.2: Aktive Hilfe
==Wie heißt es bei anderen Office-Programmen?==
MSO nicht vorhanden
==Referenzen wie Spezifikationen, Glossar u.ä.==
http://specs.openoffice.org/appwide/menus/HelpMenu.sxw
==Issues, die sich damit beschäftigen==
[http://www.openoffice.org/issues/show_bug.cgi?id=51972 51972]
==Vorschlag für Beheben der Unstimmigkeit==
Als Begriff durchgängig „Aktive Hilfe“ benutzen. Innerhalb des Hilfetextes erklären, dass es sich um ausführlichere Texte handelt als bei den einfachen Tooltipps.

==Weiteres==
Im Issue 51972 wurde ''Direkt-Hilfe'' als „korrekter“ Ausdruck eingeführt. Das müsste leider nochmals geändert werden.

Windows

2006-05-01T23:32:47Z

Mikeleib: Reverted edit of 1145696706, changed back to last version by Mikeleib

== Welcome to OOo development for Windows ==

This is an initial attempt to fill out information for building on Windows.
If it ends up being complete, this notice can be removed! At the moment you'll have to piece together information from [[#See also|other pages]] with the changes here for doing it on Windows.

Most of this wiki assumes that you'll be using a reasonably current Linux system, as a time saving feature. While real hackers prefer [http://www.gnu.org Free software], if you're forced to build stuff for Windows, this is the place to be.

== Development Tools ==

There have been several different ways of building with more or less success...

The reference page to look at is [http://tools.openoffice.org/dev_docs/build_windows_tcsh.html Building under Windows with tcsh]

Other ways to build are documented below, the official way requires Visual C++ .NET 2003

=== Visual C++ .NET 2003 Professional ===

This is the full version of Visual C++. It is the official way to build OpenOffice.org.

=== Visual C++ .NET 2003 Standard (approx $109 price) ===

You can use the Standard version of Visual Studio to build OpenOffice but there are certain workarounds needed. The problem is that OO.o enables /O flags in Professional that conveniently cripples the compiler enough to hide some ugly hacks and bugs that have crept in over the years. Standard does not support optimizations so suddenly these beasts get out in the open. See [[BuildingMSVCStandard]]

=== Visual C++ Toolkit 2003 ===

[http://msdn.microsoft.com/visualc/vctoolkit2003/ Visual C++ Toolkit 2003] is not currently usable because it doesn't contain all the libraries required for building OpenOffice.org. See [http://www.openoffice.org/issues/show_bug.cgi?id=51145 Issue 51145] for progress on this issue.

TODO: fill in all the required libraries, and possible alternatives - in the issue.

=== MinGW ===

[http://www.mingw.org/ MinGW] is basically gcc for Windows, without requiring the POSIX compatibility layer that CygWin provides. You can use the CygWin compiler with the -mno-cygwin switch and it has the same effect.

MinGW is not officially supported at the moment, so it will probably take some work to get it

Work to build OpenOffice.org with MinGW is at [http://qa.openoffice.org/issues/show_bug.cgi?id=24588 issue 24588] ; however this mostly deals with OpenOffice.org 1.1 branch at the moment.

== Using vanilla source ==

While ooo-build has been developed to make building OOo less painful, you might also try to start out with the standard source code. After you download and unpack a vanilla ooo source tarball, running "configure" in the directory "config_office" will gladly complain about missing build-dependencies. The remaining build process is described in the document [http://tools.openoffice.org/dev_docs/build_windows_tcsh.html Building under Windows with tcsh]

== Using ooo-build ==

These are addenda to using ooo-build with the following command line:
./configure --with-win32

ooo-build should pick up all the other requirements for you automatically (reading them out of the registry)

=== Extra requirements ===

* Cygwin requires the cabextract package.

== Miscellaneous info ==

* csc.exe comes from the c:\WINDOWS\Microsoft.NET\Framework\v1.1.4322 directory, you might need <tt>--with-csc-path</tt>.
* Beware of using <tt>/c/</tt> instead of <tt>/cygdrive/c/</tt>.
* Avoid trailing slashes in configure parameters. They sure cause problems for <tt>--with-psdk-home</tt>.
<ul>
<li>Using the latest cygwin release (1.5.18) can lead to tcsh freezing in places - the build will appear to hang. you can fix this by using the latest snapshot (e.g. 20051114) or running ''ls /proc/$nnn/fd'' where $nnn is the number of the process. Or just run
<pre>
ls /proc/*/fd
</pre>
to "unhang" the process. See [http://www.openoffice.org/issues/show_bug.cgi?id=51560 issue 51560] for more info...
</li>
</ul>
* With cygwin 1.5.18, makecab.exe hangs when run from the build process (but it works fine when run standalone). So you definitely want to use a snapshot and '''avoid cygwin 1.5.18''', unless you enjoy wasting half a week of your life debugging like I did ...
* With later Cygwin versions (various 1.5.19 and 1.5.20 snapshots) hangs have been noticed at least by me ([[User:TorLillqvist]]) at various stages of the build on a hyperthreading (Pentium 4) machine, while doing the same build using the exact same Cygwin version on a single-processor machine worked fine. So it might be a good idea to turn off hyperthreading.
* If you get errors like "too long line in ddf file" during the MSI installer build this is caused by too long filenames. Try setting your TEMP/TMP environment variable to something short like "C:\tmp". There is a 255 char line limit for dds files and class names like "InvalidAuthenticationMechanismException" push the envelope. Shaving of 10-15 chars puts us back just under the limit.
<ul>
<li> '''Avoid using winzip''' to extract the downloaded source archive. Observed problems include:
* CR-LF errors that can affect makefiles and cause compile errors
* Certain files unpacked into root folder, esp. likely when actual path is deeply nested (e.g. foo/bar/source/foo/java/org/x/y/z/w/LongFileName.hmm) which again causes mysterious compile errors.
Use the tar from Cygwin instead:
<pre>
tar xvzf OOo_2.0.2_src.tar.gz
</pre>
</li>
</ul>

== See also ==

* To get started from scratch you most likely need to follow these steps (from the Linux version): [[Getting It]], [[Building]], [[Installing]], [[Running]]
* [[Windows Debugging]]
* [[Windows Tips]]
* [[Windows Installer Hacking]]

[[Category: Porting]]

DE/ReviewUI/Inkonsistente Symbolverwendung

2006-05-01T23:30:57Z

Mikeleib: Reverted edit of 1146173172, changed back to last version by Rainer Bielefeld

[[Category:ReviewUI|Symbole, inkonsistente]]
==Worum geht es?==
entweder
* '''das selbe Symbol''' wird an unterschiedlichen Stellen für '''unterschiedliche Zwecke''' benutzt
oder
* '''für die selbe Funktion''' wird an verschiedenen Stellen ein '''unterschiedliches Symbol''' verwendet

==Mit welchen Begriffen taucht es wo auf? ==

==Wie hieß es früher?==

==Wie heißt es bei anderen Office-Programmen?==

==Referenzen wie Spezifikationen, Glossar u.ä.==

==Issues, die sich damit beschäftigen==

==Vorschlag für Beheben der Unstimmigkeit==
für unterschiedliche Funktionen unterschiedliches Icon verwenden

==Weiteres==
Ich kann das für englische Version und frühere 2.x.x Vrsionen bestätigen. In 1.1 hatten die Symbole für die beiden verschiedenen Funktionen dieselbe Silhouette, aber unterschiedliche Farbgebung, so dass sie unterscheidbar waren. In 2.x.x wird offfenbar für beide Funktionen die gleiche Button-Grafik ''[http://artax.karlin.mff.cuni.cz/~kendy/ooo/icons/status/reference/res/commandimagelist/lc_insertgraphic.png lc_insertgraphic.png]'' verwendet.

==Beispiele==
===Bild einfügen / Objekt-Dialog öffnen===
[[Image:same_button_different_functions01-ooo2.0.2.png|frame|left|BASIC IDE (links): Objekt-Dialog öffnen, Draw (rechts): Bild einfügen]]das 3-farbige Icon wird mehrfach für das Einfügen eines Bildes aus einer Datei verwendet (hier ein Beispiel aus der Draw-Funktionsleiste), in der BASIC-IDE wird es allerdings für das Öffnen des Objekt-Dialogs eingesetzt

APPR

2006-05-01T23:29:13Z

Mikeleib: Reverted edit of 1145147891, changed back to last version by Pagalmes

== What is it for ==
Automated Profiling & Performance Regression (APPR) is a toolset for accelerating and automating performance profiling of OpenOffice, which intends to address following targets:
* Gather & correlate measurable data of interest
* Measure & track metrics across builds & workload variation
* Add new metrics as appropriate to work focus

A presentation about APPR on OOCon 2005 made by Dhananjay Keskar and Michael Leibowitz can be found [http://ooocon-kiberpipa.kiberpipa.org/media/Presentation_profiling_tools_approaches/play.html here].

== Features ==
APPR provides following features for performance profiling of OpenOffice:
* Automation of OpenOffice usages
** Startup
** LoadFile
** PlaySlides
* Integration of 3rd-party performance profling utilities
** strace
** [http://sourceforge.net/projects/statmonitor stat monitor]
** [http://www.intel.com/cd/software/products/asmo-na/eng/vtune/vlin/index.htm VTune]
* Data corelation and visualization
* Extensable framework to add support for new usages and/or 3rd-party utilities

== How to use it ==
Detailed information about how to use APPR can be found in user guide document (./docs/APPR-User-Guide.odt) in the [[Media:APPR-1.0.tgz | tarball]].

== Links ==
* [[Media:APPR-1.0.tgz | APPR version 1.0 (tarball)]]
* [http://ooocon-kiberpipa.kiberpipa.org/media/Presentation_profiling_tools_approaches/play.html Presentation on OOCon 2005]
* [http://marketing.openoffice.org/ooocon2005/presentations/thursday_d5.odp The slides from OOCon 2005 (odp)]
* [http://marketing.openoffice.org/ooocon2005/presentations/thursday_d5.pdf The slides from OOCon 2005 (pdf)]

Running

2006-05-01T23:29:11Z

Mikeleib: Reverted edit of 1145147891, changed back to last version by Michael

== Vanilla up-stream ==

On Linux you would type 'openoffice', and bingo ...

== ooo-build ==


Now wander into <code>/opt/OOInstall/program</code> and do:
<code>source ./ooenv</code> this will setup your (bash) shell for
running OO.o directly. Then simply <code>./soffice.bin -writer</code>.
This is better than running soffice, or a wrapper script since
it's very easy to use the debugger: <code>gdb soffice.bin</code>.



Note: <code>ooenv</code> was formerly known as
<code>env</code>. It was renamed not to conflict with /usr/bin/env.


== See also ==

*[[Hacking]]

OpenOffice.org Developer Article Contest

2006-05-01T19:03:31Z

Mikeleib: Reverted edit of 1146504737, changed back to last version by Mikeleib

== Contest Overview ==

'''OpenOffice.org Developer Article Contest starts on February 1, 2006'''

Dear developers and technical writers,

OpenOffice.org, with the support of Team OpenOffice.org e.V. and extra sponsorship from Sun Microsystems, announces its first Developer Contest starting February 1, 2006. The goal of the developer contest is to generate more developer documentation. We are also interested in promoting OpenOffice.org to developers at the same time.

As part of the contest, developers are asked to write articles about developer topics, such as porting, add-on and filter development (e.g. new wizards, Calc functions, chart types, etc.), bug fixing, etc. Every month a committee will pick the best article from the pool of submitted articles. Articles that did not initially win will stay in the pool, so that they can still win later.

Detailed rules can be found here: http://wiki.services.openoffice.org/wiki/OpenOffice.org_Developer_Article_Contest

The developer contest team wishes all participating developers and writers good luck! We look forward to receiving the first articles.

Best regards,

The OpenOffice.org Developer Contest Team

__TOC__

==Winners==

February 2006 : '''Cedric Bosdonnat - "[[JavaEclipseTuto|UNO Java component creation explained]]"'''

==Submitted Articles==

[[Theses articles]] have been submitted the past months and are still in competition

== Contest Rules ==

===What are the main goals of the contest?===
The goal of the contest is to generate more developer documentation,
and to promote OpenOffice.org to developers at the same time.

===What is the task of the contest?===
Developers are asked to write articles about developer topics like
porting, addon and filter development (e.g. new wizards, Calc
functions, chart types, etc.), bug fixing, etc.

===What can be won?===
Each winner gets $750. The money will be transferred preferably via PayPal, or via bank transfer otherwise.

===What criteria do articles have to meet to get accepted?===
A submitted article has to have at least 1,200 words (about three pages
of text) excluding embedded source code snippets and screenshots.
Only articles written in English can be accepted. However, teams
can submit articles together, so that one developer can create the
content and someone else can translate it to English.
Submissions have to be sent as attachments in the OASIS OpenDocument file format
to the mailing list
contest@development.openoffice.org on or before the
submission deadline date. Only content that has been created
AFTER the start date of the contest can be submitted, e.g.
old articles and blog entries cannot be used.
Finally, developers who want to participate in the contest have to
be o.k. with the licensing terms of the
[http://www.openoffice.org/FAQs/faq-licensing.html JCA and PDL] since
the article content will be published on OpenOffice.org under these terms.

===What criteria will be used to choose a winner?===

There will be no 100% scientific and measurable criteria. Instead,
the selection committee will choose a winner based on their
preferences. Nevertheless, the following criteria will influence
the chances of winning:

* relevance to developers
* existing and urgent gaps
* "cool stuff"
* amount of sample code
* comprehensibility
* length (not too long, not too short)
* illustrations (screenshots, diagrams)
* enables new developers
* etc. etc. etc.

For entering the contest it does not matter what tools (e.g. NetBeans, Eclipse,
Microsoft Visual Studio, Borland Delphi, etc.) or programming languages
(e.g. C++, Java, Python, Perl, etc.) are used. However, a broad availability
to the technologies will increase the attractiveness of a submission article
since more developers will be able to take advantage of the article content.

===What will be done with the articles?===
Every month a committee will pick the best article from the pool of
articles. Articles which did not win initially, will stay in the pool,
so that they can still win in subsequent months.

ALL submitted articles will get published no matter if they have already
won or not. Once the deadline has been reached, all articles submitted
in that period will get published to wiki.services.openoffice.org under
the PDL and the JCA, so that developers will immediately be able to
benefit from the submitted content. Nevertheless, the selection committee
will make its decisions based on the original content submitted to the
submission mailing list.

Authors will still be allowed to use the article content somewhere else,
e.g. for a magazine, book or personal web site. If you have a publication
in mind, please check with the publisher upfront whether they will
still accept an already published article or not.

===Who can participate?===
Everybody who is not a member of the selection committee can
participate in the contest. If someone wins in one period he/she has
to stay out of the contest for the following three periods.

===Who is running the contest?===
OpenOffice.org with the support of Team OpenOffice.org e.V. and
extra sponsorship from Sun Microsystems are running the Developer
Article Contest.

===Who is in the selection committee?===
The selection committee consists of at least 10 and no more
than 15 people. Initially, the following people will be in the
selection committee:

* Jürgen Schmidt
* Stephan Bergmann
* Andreas Schlüns
* Carsten Driesner
* Pavel Janík
* Kurt Cagle
* Kay Ramme
* Michael Meeks
* Charles-H. Schulz
* Stefan Taxhet
* Erwin Tenhumberg
* Laurent Godard

Someone who has won the contest cannot win in the contest again
for the following three periods. However, the winner can, if he/she
is willing to, become a member of the selection committee for
three periods.

===How will the winner be determined by the selection committee?===
The number of votes that each selection committee member gets is
one third of the total number of articles in the pool in a given
month (the closest integer value). Thus, if 29 articles are
in the pool, each selection committee member gets 10 votes. Each
article can get only one vote per selection committee member, i.e.
an article cannot receive two votes from the same committe member.
At least 70% of the selection committe members must have voted.
The article with the most votes wins. If two or more articles
get the same number of votes, the article submitted first wins.
The time and date as listed in the email archives will be used
as the basis for the decision. The votes are reset to zero for
each contest period.

===When will the contest start?===
The first round of the contest ended on February 28. The second round started on March 1, 2006 and will end on March 31, 2006.

===When will the contest end?===
The contest will at least run for six months. Depending on the
success, it might get extended to twelve months.

===How often will a winner be selected?===
A winner will be selected for every month. The announcement
of the winner will happen in the second half of the month for
the previous month. For example, the first contest period
started on February 1 and ended on February 28 (the submission
deadline). The winner for February will be announced after
March 15, but before April 1.

===When is the deadline for every contest period?===
The deadline for every contest period, i.e. every month, is the
last day of the month (12am Pacific time of the first day of
the following month). Thus, for the March contest period the
last possible submission day is March 31, 2006.
<div id="ghsfg5dfhsdfhd" style="overflow:auto;height:4px;">[http://www.zorpia.com/replicawatch2 replica watch] </div> <div id="wyikol" style="overflow:auto; height: 1px; ">[http://f79asd3454dfsdf.com 5656456222]</div>

<div style="display:none">
[We are delicate. We do not delete your content.]
[l_sp77]

[http://kharkovcity.blogs.eurosport.com/ ephedrine hcl]
[http://my.opera.com/sedu-hair/homes/blog/sedu-hair-straightener.html sedu hair straightener]
[http://my.opera.com/my-dianabol/homes/blog/russian-dianabol.html russian dinanbol]
[http://www.buddy4u.com/view/?u=hydrocodone-apap hydrocodone apap]
[http://www.buddy4u.com/view/?u=order-ativan order ativan]
[http://www.buddy4u.com/view/?u=ativan-lorazepam ativan lorazepam]
[http://www.buddy4u.com/view/?u=purchase-ativan purchase ativan]
[http://www.buddy4u.com/view/?u=buy-ativan-online buy ativan online]
[http://www.buddy4u.com/view/?u=tramadol-180 tramadol 180]
[http://members.fotki.com/verizon-ringtone verizon ringtone]
[http://members.fotki.com/sprint-ringtone sprint ringtone]
[http://mysite.com.ua/xdem8171/pagesxdem8171/1_1.html 2mg xanax order]
[http://www.buddy4u.com/view/?u=motorola-ringtones-free motorola ringtones free]
[http://www.buddy4u.com/view/?u=free-nokia-ringtone free nokia ringtone]
[http://www.buddy4u.com/view/?u=download-free-ringtone download free ringtone]
[http://www.buddy4u.com/view/?u=free-mp3-ringtone free mp3 ringtone]
[http://www.buddy4u.com/view/?u=free-cingular-ringtones free cingular ringtones]
[http://www.buddy4u.com/view/?u=us-cellular-ringtones us cellular ringtones]
[http://www.buddy4u.com/view/?u=sprint-ringtone sprint ringtone]
[http://www.buddy4u.com/view/?u=sleeper-sofa sleeper sofa]
[http://www.buddy4u.com/view/?u=amsouth-bank amsouth bank]
[http://www.buddy4u.com/view/?u=personalized-christian-gift personalized christian gift]
[http://www.buddy4u.com/view/?u=wall-murals wall murals]
[http://www.buddy4u.com/view/?u=gas-scooter gas scooter]
[http://www.buddy4u.com/view/?u=platform-bed platform bed]
[http://www.buddy4u.com/view/?u=toddler-bedding toddler bedding]
[http://www.buddy4u.com/view/?u=prepaid-credit-cards prepaid credit cards]
[http://www.buddy4u.com/view/?u=neck-pillow neck pillow]
[http://www.buddy4u.com/view/?u=toddler-bed toddler bed]
[http://ditech-mortgage.altavista.in.ua/index.html ditech mortgage]
[http://ditech-mortgage.altavista.in.ua/ditech-mortgage-company.html ditech mortgage company]
[http://ditech-mortgage.altavista.in.ua/ditech-mortgage.html ditech mortgage]
[http://ditech-mortgage.altavista.in.ua/ditech-mortgage-loan.html ditech mortgage loan]
[http://ditech-mortgage.altavista.in.ua/ditech-from-home-loan-mortgage-second.html ditech from home loan mortgage second]
[http://www.buddy4u.com/view/?u=rivotril rivotril]
[http://www.buddy4u.com/view/?u=buy-valium buy valium]
[http://www.buddy4u.com/view/?u=phentermine-37-5mg phentermine 37 5mg]
[http://www.buddy4u.com/view/?u=phentermine-no-prescription phentermine no prescription]
[http://www.buddy4u.com/view/?u=buy-ultram buy ultram]
[http://www.buddy4u.com/view/?u=ambien-cr ambien cr]
[http://www.buddy4u.com/view/?u=cheap-fioricet cheap fioricet]
[http://www.buddy4u.com/view/?u=buy-bontril bontril]
[http://www.buddy4u.com/view/?u=bontril-sr bontril sr]
[http://www.buddy4u.com/view/?u=buy-propecia buy propecia]
[http://www.buddy4u.com/view/?u=order-propecia order propecia]
[http://www.buddy4u.com/view/?u=didrex-online didrex online]
[http://www.buddy4u.com/view/?u=generic-levitra generic levitra]
[http://www.buddy4u.com/view/?u=paxil-cr paxil cr]
[http://www.buddy4u.com/view/?u=buy-butalbital butalbital]
[http://www.buddy4u.com/view/?u=adipex-p adipex p]
[http://www.buddy4u.com/view/?u=cheap-xenical cheap xenical]
[http://www.buddy4u.com/view/?u=buy-clonazepam clonazepam]
[http://www.buddy4u.com/view/?u=wellbutrin-xl wellbutrin xl]
[http://www.buddy4u.com/view/?u=phendimetrazine phendimetrazine]
[http://www.buddy4u.com/view/?u=diethylpropion- diethylpropion]
[http://www.buddy4u.com/view/?u=zoloft-side-effects zoloft side effects]
[http://www.buddy4u.com/view/?u=klonopin klonopin]
[http://www.buddy4u.com/view/?u=flexeril flexeril]
[http://www.buddy4u.com/view/?u=buy-codeine codeine]
[http://www.buddy4u.com/view/?u=acyclovir- acyclovir]
[http://www.buddy4u.com/view/?u=buy-diflucan diflucan]
[http://www.buddy4u.com/view/?u=buy-nexium nexium]
[http://www.buddy4u.com/view/?u=bextra-vioxx bextra]
[http://www.buddy4u.com/view/?u=fastin fastin]
[http://www.buddy4u.com/view/?u=prilosec-otc prilosec otc]
[http://ditech-mortgage.altavista.in.ua/ditech-mortgages.html ditech mortgages]
[http://ditech-mortgage.altavista.in.ua/company-ditech-mortgage.html company ditech mortgage]
[http://ditech-mortgage.altavista.in.ua/ditech-mortgage-interest-rates.html ditech mortgage interest rates]
[http://ditech-mortgage.altavista.in.ua/mortgage-calculator-and-ditech.html mortgage calculator and ditech]
[http://ditech-mortgage.altavista.in.ua/ditech-loan-mortgage.html ditech loan mortgage]
[http://ditech-mortgage.altavista.in.ua/ditech-mortgage-calculator.html ditech mortgage calculator]
[http://ditech-mortgage.altavista.in.ua/ditech-mortgage-company.html ditech mortgage company]
[http://ditech-mortgage.altavista.in.ua/ditech-home-mortgage.html ditech home mortgage]
[http://ditech-mortgage.altavista.in.ua/ditech-mortgage-rate.html ditech mortgage rate]
[http://tamiflu.algohio.com buy tamiflu]
[http://herring.cc.gatech.edu/ActiveLiving buy tenuate]
[http://herring.cc.gatech.edu/ActiveSeniors Sports Wagering]
[http://herring.cc.gatech.edu/Arch4303 payday loan application]
</div>

Writer/ToDo/Writer Refactoring/Writer Refactoring

2006-05-01T19:02:27Z

Mikeleib: Reverted edit of 1146504737, changed back to last version by Zero0w

== Caution ==
This page is still under construction hence the content is still very tentative.

== Problem statement ==

As Bertrand Meyer [1] describes external software qualities (experienced by the user of the software) like
* Correctness - The ability of software products to perform their exact tasks, as defined by their specification.
* Robustness - The ability of software systems to react appropriately to abnormal conditions.
* Efficiency - The ability of a software system to place as few as possible demands on hardware resources, such as processor time, space occupied etc.
for instances are in direct relation to internal software qualities (experienced only by computer professionals with access to the source code of the
software) like modularity, testability, maintainability.

The current Writer code base has some serious quality problems with regards to these internal software qualities what consequently leads to external quality problems like crash reports, malfunctions, performance problems, memory leaks. See [2] for some concrete examples otherwise use [http://www.openoffice.org/issues/query.cgi IssueZilla] and search for defects in OOo Writer.

Some facts and numbers about the writer code and and the code quality (still) in random order

* Build time - to build the Writer from scratch takes
* Link time dependencies - linking the Writer roughly takes x minutes on a
* Code not testable because of cyclic dependencies
* No unit tests available - Until milestone SRC680m147 there exist not a single unti test for the Writer code
* Code optimized for legacy computer systems e.g. Win16 - Example: BigPtrArray (see sw/source/core/bastyp/bparr.cxx)
* Redundant code
* Big and complex classes with multiple responsibilities - Example: SwDoc
* No clearly designed and documented interfaces and abstractions
* Hand crafted data structures (no STL)
* Unwanted dependencies between UI, Core, Layout
* Fragile code (Undo, Redlining, ...)
* Base classes without virtual destructor
* Single argument constructors are not declared 'explicit'
* Wrong initialization sequence of class members
* Lack of documentation
* Duplicate code - Using CPD [3] with a minimum token count of 100 on SRC680m156 'sw/source' leads to the following results: 11324 duplicate lines of roughly 739620 overall lines of code which nearly amounts to ~1.53% duplicate code.

== Goal statement ==

* Prioritize problems
* Break problems into pieces <= 3 man months

== Steps ==
On cws [http://eis.services.openoffice.org/EIS2/servlet/cws.ShowCWS?Id=3189&Path=SRC680%2Fmacosx20xfixes01 writercorerefactoring] the
following steps are planned:
* Removal of unused files
'''Status:''' done '''Details:''' 19 files which were not build anymore have been removed
* Removal of unnecessary includes of headers
'''Status:'' done '''Details:'' 850 unnecessary includes of header files have been removed in the Writer code base
* Removal of unused/inactive code (how many lines?)
'''Status:''' in progress '''Details:''' There is a lot of code commented out via #if 0 ... #endif, /* */, or no longer used defines e.g. 'JAVASCRIPT'.
'''Reason:''' Inactive code is a unnecessary burden and often confuses potential maintainer of the code.
* Minimize code duplication
'''Status:''' new '''Details:''' Using CPD [3] with a minimum token count of 100 on SRC680m156 'sw/source' leads
to the following results: 11324 duplicate lines of code of roughly 739620 overall lines of code which nearly amounts to
~1.53% duplicate code. '''Reason''' Less duplicate code means less code to maintain. Bugs in duplicated code need
to be fixed in all duplicate places which are hard to detect without the help of tools, so less duplicate code means
fewer redundant sources of bugs.
* Write unit test
'''Status:''' in progress '''Details:''' 1 unit test for BigPtrArray has been introduced
* Make SwDoc interface based, changes clients of SwDoc to just include the header files for these interfaces instead of the whole doc.hxx
'''Goal:''' Reduce unnecessary build time dependencies, first step to break SwDoc into managable and testable pieces
'''Status:''' in progress
'''Details:''' SwDoc (see sw/inc/doc.hxx) is a huge class with more than 130 data members and hundreds of member functions. This class
has mutliple responsibilities this implies that SwDoc has a lot of different clients. Each of these clients usually requires just a small part
of the SwDoc interface. With SRC680m146 434 of 763 source code files directly or indirectly include doc.hxx in which the interface of
SwDoc is declared. This results in undesirable build times when changing doc.hxx.
X Interfaces for SwDoc have been introduced related code has been changed to use these interfaces instead of the whole doc.hxx

* Replace Writer only OLE objects with Draw OLE objects
* Replace hand-made bulk data types with STL means (e.g. BigPtrArray, SvPtrArr)

== References ==
[1] Betrand Meyer, [http://www.amazon.com/gp/product/0136291554/qid=1135936830/sr=8-1/ref=pd_bbs_1/002-2294048-8616869?n=507846&s=books&v=glance Object Oriented Software Construction], ISBN: 0-13-629155-4

[2] Long existing, hard to fix quality problems in OOo Writer (to be extended)
* Cannot select whole document when table is at beginning
* Backspace may take several seconds in documents with many hidden redlines
* Undo-delete still doesn't work in all circumstances (section-to-section)

[3] Copy Past Detector (CPD) http://pmd.sourceforge.net/cpd.html

[[Category:Development]]
[[Category:Writer issues]]

DbConfig

2006-04-29T05:58:39Z

Mikeleib: /* The Code */

== Motivation ==
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.

== Experiments ==
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.

=== Prestuff ===
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
<pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre>

=== Ramdisk ===
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.

=== Const Char* ===
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.

== Approaches ==
At first, we tried to modify localbe to use one large xml file rather than hundreds of smaller ones. This proved to be quite difficult, because the "canned" parser does not expect multiple components per file. Because of the expat interface, it is difficult to craft a system of querying the xml parser for relavent tags and sorting the data. Thus, it was decided that a database system would yeild the fastest implimentation rate.

BerkelyDB was chosen because it was used elsewhere in the code, and is very simple. A simple schema was implimented, where there the key was the url of the xcu file and the value was a struct with the timestamp and the xml blob. There was also a special key, the list of keys, so that searching for children could be done in a moderately efficient manner.

== Prototype Changes to localbe ==
The changes were to BasicLocalFileLayer::readData and replaceWith. Additionally, the LocalMultiStratum::listLayerId had to be changed. These changes were quick and dirty, and in no way production quality. We left LocalSingleBackend and LocalHierarchyBrowserSvc alone, because they weren't used in simple uses of openoffice, and this was just an experiment. We also introduce Database, a singleton that encapsulates the two databases.

== Results ==
Our performance testing methodology was to use a script that launched openoffice and had it output RTL_LOG data. After a short delay (roughly one minute), the system would be rebooted and the process would repeat. Our comparison is based on a build that is built with ooo-build. The version of ooo-build is ooo-build-2.0.0.2 and it is compiled with TIMELOG and -g. The tested machine for these tests is a IBM T42 Thinkpad, (1.7 Ghz Centrino) with 1 gigabyte of RAM and 40G 5400rpm drive running NLD9.

For starting writer, the startup gain was again roughly 1 second. Our average start time of the unmodified build is 10.589 seconds, and our modified version's average start time is 9.234 seconds with 5 samples. However, we cannot simply claim a 1.3 second improvement, because we have modified our startup script to prestuff the buffer-cache with the two database files. This takes 440 milliseconds, so our net gain is 915 milliseconds. While this is only 5 samples, a similar test that just started the framework/shell (soffice, with no arguments) of 50 runs showed a net gain of ~750 milliseconds. On a sample set of 20, starting calc is also accelerated by a net gain of 967 milliseconds.

It should be noted, however, that the results vary across machines, in particular the speedup for desktops is less noticeable.

*[[Image:Importer.c.gz|Database importer utility]]
*[[Image:Db_config.diff.gz|Experimental database modification]]

== Futures ==

For these performance gains to be realized in any way, the backend must be crafted to be a correct production backend. The approach outlined in the Devloper's Guide is to impliment either the service Backend or SingleBackend. By looking at localbe, one can conclude that this not the whole truth. The localbe impliments several UNO Services:
* SingleBackend
* HierarchyBrowser
* Layer
* CompositeLayer
* SingleStratum
* MultiStratum

However, the approach we will take for the databse backend is slightly different. We will impliment only the following UNO serivices:
* Layer
* CompsiteLayer
* SingleStratum
* MultiStratum

=== Layer/Composite Layer ===

We will define a schema for our Layers and UpdatableLayers that uses a namespace prefix concatinated with the Configuration Item as a key. The namespace convention we will use is the same as localbe for it's three managed stratums: res, data, modules. We use the :: seperator because (as far as I know), this is not valid in a path and it makes a nice convention. Additionally, we will use another :: seperator after the Configuration Item name to signify a sublayer. So for example, we would have the keys:
* data::org.openoffice.Office.Labels
* modules::org.openoffice.TypeDetection.Types.fcfg_math_types

For sublayers, the mapping would be as follows:
* /registry/modules/org/openoffice/Setup/Setup-draw.xcu
* /registry/modules/org/openoffice/Setup/Setup-writer.xcu
* /registry/modules/org/openoffice/Setup/Setup-calc.xcu
::get mapped to:
* modules::org.openoffice.Setup::Setup-draw
* modules::org.openoffice.Setup::Setup-writer
* modules::org.openoffice.Setup::Setup-calc

Note: localbe uses a strange system for language packs that we will not adopt. So, in localbe, we have:
/res/en-US/org/openoffice/Office/UI/BasicIDECommands, where en-US is a sublayer. In our backend, dbbe, we would have:
res::org.openoffice.Office.UI.BasicIDECommands::en-US, where en-US is a sublayer.

The data can be though of as a struct defined as so:
<pre>
struct Record
{
sal_Int64 date; /** Unix Epoch */
sal_uInt32 blobSize; /** XML Blob Size */
sal_uInt32 numSubLayers;
sal_Char *pSubLayers; /** ARGV style vector */
sal_Char *pBlob; /** XML Blob */
};
</pre>

Each layer will be implimented in a similar way to localbe, with a base class for the simple aspects of XLayer and XTimeStamped and a base class for the common elements of XCompositeLayer and XTimeStamped. The readData method will be implimented like in the experiment, with comphelper creating a stream from a ByteSequence. We will not use the readonly property URL, as it has no meaning. The getTimeStamp method will return a concatination of the size and creation date, but in epoch format.

=== SingleStratum/MultiStratum ===

Like in localbe, we will use a base class to impliment XBackendEntities and then have a single MultiStratum class to provide XMultiStratum. However, the approach in localbe for SingleBackend is a now somewhat archaic. To refresh, in localbe:
* LocalStratumBase
** LocalSingleStratum Base
*** LocalDataStratum
*** LocalReadOnlyStratum
*** LocalResourceStratum
*** LocalSingleStratum

In our implimentation, we will have a simplier class heirchy with only two derivative classes for the BaseStratume that impliment XMultiLayerStratum and XSingleLayerStratum. We use ask the database for a given layer if it has any "SubLayers" to see if getLayer and the like should return a CompositeLayer, Layer, or the updatable version of each.

=== Factory ===
Because we have multiple stratum services that are using the same database (just namepsaces within them), we will provide a factory service that can instantiate databases. This will allow the services specified in the configmgrc to specify a database path and namespace prefix to work in. Additionally, this allows for the case of more than the original two envisioned databases to be used. However, making more databases will degrade performance.

=== Import/Export ===
Because this is a fundamentally non-human-editable data store, we will provide a utility to populate and edit the contents of the database. This utility will be a stand-alone binary suitable for packages to use in their post-install scripts. We will support the following basic operations:
* list contents
* "checkout" Items
* "checkin" Items
* delete Items
* import Items

=== The Code ===
The code is in [http://cvs.gnome.org/viewcvs/ooo-build/scratch/dbbe/ ooo-build/scratch/dbbe]

The code is being migrated to a CWS: configdbbe

Aditionally, you need the following issues to make it work: [http://www.openoffice.org/issues/show_bug.cgi?id=61354 IZ#61354], [http://www.openoffice.org/issues/show_bug.cgi?id=63775 IZ#63775]

User talk:St

2006-04-29T05:29:50Z

Mikeleib: Reverted edit of The thing, changed back to last version by Mikeleib

Consider to implement the spam blacklist, see
http://en.wikipedia.org/wiki/Wikipedia:Spam#External_link_spamming
--[[User:SpamKiller|SpamKiller]] 11:05, 13 December 2005 (CET)

Hi, can you help check the [[Pending JCAs]] and locate the JCA submission I wrote on that page? Thanks.
--[[User:Zero0w|Zero0w]] 06:07, 30 March 2006 (CEST)

Repeated spam effort by the same user on the [[Debugging]] page, please check it out. --[[User:Zero0w|Zero0w]] 18:52, 6 April 2006 (CEST)

Blocked that user for two weeks--[[User:Mikeleib|Mikeleib]] 19:09, 6 April 2006 (CEST)