Apache OpenOffice Wiki - User contributions [en]

User:Tora

2011-05-10T03:51:19Z

Tora: Added a picture

= Self-Introduction =
<table><tr><td>
[[Image:tora-2011-october.jpeg]]
</td><td>
*Name: Takamichi Akiyama
*Nickname: Tora
*E-mail: tora@openoffice.org
*Occupation: Freelance software engineer
</td></tr></table>

== PROFILE ==
Confident, competent, customer-focused, positive thinking professional. Global perspective based upon team work with diverse colleagues including American, German, French, Irish, Korean, Chinese, Indian, Thai, Japanese, and so on. Knowing both American and Japanese cultures by living in Berkeley, CA for several years. Possessing the fundamental knowledge of computer covering from software to electronic circuits with more than 25 year extensive experience in computer technology since the dawn of 8 bit processors.

== AREAS OF EXPERTISE ==
Internationalization and Localization of Software Development, UNIX Based Systems, Office Suites, System Administration, Network Administration, ISDN, Telecommunication

== PROFESSIONAL EXPERIENCE ==
=== Freelance Software Engineer (2007 - Current) ===
*Server-side computing with OpenOffice.org on Linux, C++
*Web-based server-side computing with OpenOffice.org, PHP with Ajax
*Enhancing OpenOffice.org, C++, http://wiki.services.openoffice.org/wiki/Jsdp2007

=== Good-day, Inc. - Osaka, Japan (2005 - 2006) ===
Led a government founded project aiming at enhancing functionality and usability of OpenOffice.org for Japanese users. The project, this fiscal year, covers conducting a research of user's requirements, prioritizing them, discussing specifications, and visiting engineers in Hamburg, Germany - a main location of OpenOffice.org development - to share the requirements and specifications, and successfully ended at the end of February, 2007.

Contributed to the OpenOffice.org Japanese community conducting quality assurance tests, finding bugs and filing them to an issue tracking system, answering to user s queries, giving presentations in software related events, developing web based OpenOffice.org server sided solution, writing articles for magazines, developing customized OpenOffice.org, and so on.

=== Sun Microsystems K.K. - Tokyo, Japan (1999 - 2003) ===
Subsidiary of Sun Microsystems Inc., California, USA 
Technical Advisory

Led a localization engineering team consisting of members in Menlo Park, Hamburg, Dublin, Beijing, and Tokyo and successfully delivered StarSuite 6.0 into the market in 2001, including proposing Asian specific features, writing source codes, conducting quality assurance tests, and reviewing translations. StarSuite, an Asian version of StarOffice, is an office suite application software which can be considered an alternative of Microsoft Office. StarOffice is a product version of OpenOffice.org, open source software. Moreover, supported marketing and sales for Japanese market and incorporated feedback of customers to bug fixed versions and the next product version.

Administrated development and test environment composed of several file servers, workstations, Windows PCs, and Linux machines.

Accomplished localization and conducted quality assurance tests of SunScreen, a firewall software, with French coworkers, and Java related software applications with US colleagues.

=== Nippon Telegraph and Telephone Corporation - Tokyo, Japan (1991 - 1999) ===
Research Engineer

Researched and developed an innovative software debugging system for Integrated Services Digital Network (ISDN) exchange system based on my own idea, which dramatically helped software development process and debugging efforts for running systems that served ISDN services over Japan.

Maintained ISDN switching system development environment made up of hundreds of UNIX workstations, file servers, email servers, routers, and firewalls that spread at a number of remote locations.

Launched a Personal Handy Phone (PHS) service, cost effective mobile phone service, being in charge of development, deployment, and support for operators.

Researched and invented new telephone related services for ISDN switching system such as an automatic call transferring service, telephone voting service, and calling with voice recognition.

== EDUCATION AND TRAINING ==
Project Management, Business Negotiation, and English Language Program, University of California, Berkeley, Extension (2003-2004)

Master of Science in Electronic and Information Engineering (1991) 
Graduate School of Technology, Tokyo University of Agriculture and Technology, Tokyo Japan

Bachelor of Science in Electronic Engineering (1989) 
Tokyo University of Agriculture and Technology, Tokyo Japan

== TECHNICAL QUALIFICATIONS ==
*Operating Systems: Solaris, SunOS, Linux, Windows 98/ME/2000/NT/XP, MacOS, MS-DOS
*Programming Languages: C, C++, Perl, yacc, lex, OpenOffice.org Basic, VBA, Java, JavaScript, PHP, shell scripts, assembler
*Applications: StarOffice / StarSuite / OpenOffice.org, Microsoft Office, Apache, Samba, sendmail, emacs, debuggers, CVS, SubVersion, etc.
*Certifications: Class I Information Technology Engineer, Information-technology Promotion Agency, Japan

== AFFILIATIONS ==
*Co-Lead of Mirror subproject in the [http://distribution.openoffice.org OpenOffice.org Distribution project]

== SPEAKER ==
*"Migrations in Japan", OpenOffice.org Conference (OOoCon 2007), Barcelona, Spain, 2007
*"OpenSource Software - OpenOffice.org 2.0 -", JavaOne Tokyo 2005, Japan, 2005
*Annual Symposium of Nihon Sun Users Group, Tokyo, Japan, 2002
*BSD/Linux Day, Yokohama, Japan, 2002

== WRITINGS ==
*Articles for Monthly Magazines: "OpenOffice.org BASIC" [http://itpro.nikkeibp.co.jp/linux/index.html Nikkei Linux]
*Article "OpenDocument with Perl" http://www.impressjapan.jp/books/2289
*Article "OpenOffice.org working with Java" http://gihyo.jp/magazine/SD/archive/2006/200605

== LANGUAGES ==
Fluent in Japanese; Enough to communicate in English

File:Tora-2011-october.jpeg

2011-05-10T03:47:03Z

Tora:

Mirrors Project

2011-05-06T05:26:58Z

Tora: temporarily added notifications

Co-Lead: [mailto:tora@openoffice.org Tora - Takamichi Akiyama]

= Notifications =

<blockquote>

This wiki page is under revision. The current contents in this page is slightly outdated.
 
Please refer to [[New OOo Mirror Structure]] for the latest information.

</blockquote>

= Mission Statement =

This project coordinates the listing of those sites that host OpenOffice.org software. A "mirror" is a website that has agreed to donate its server space and bandwidth for OpenOffice.org downloads. This page serves as a reference page for the download pages. For regular users, please go to the [http://download.openoffice.org/ OpenOffice.org Download webpage].

<blockquote>
Disclaimer: To be listed here, anything under the OpenOffice.org directory must come from any of the Category 1/Tier-1 servers mentioned in the [[Master Mirrors]]. We make no promises about this software, and anyone who downloads any software listed on OpenOffice.org, whether from the master servers or from any listed mirror or from any other source, including CD-ROMs or peer-to-peer networks, is doing so at his or her own risk. Downloaders should always exercise caution whenever downloading any executable software from the Internet. Please check your md5sums prior to installing downloaded software.
</blockquote>

= Setting Up a Mirror =

Setting up a mirror site is easy. You may select which [http://distribution.openoffice.org/files.html#sets sets] (main or extended) you wish to carry, but you must select one. We do not need mirrors for the OpenOffice.org Web content.

If you would like to set up a mirror, please read the information below and consult our
[http://www.openoffice.org/FAQs/mirrors_faq.html mirrors FAQs].

Quick start:

<ol>
<li>Choose one set for your mirror (look at [http://distribution.openoffice.org/files.html#sets sets]). 
We accept mirrors carrying the main set (category 1). But we are especially in need for mirrors capable to host the extended dataset.</li>
<li>Send an email to [mailto:mirrors@distribution.openoffice.org?subject=New%20Mirror project's discussion list], including: 

<ul>
<li>Name of mirror site:</li>
<li>Technical contact's name:</li>
<li>Technical contact's email address:</li>
<li>URL of the mirror (http):</li>
<li>URL of the mirror (ftp):</li>
<li>URL of the mirror (rsync):</li>
<li>Mirroring from:</li>
<li>Set You've chosen:</li>
<li>Your Internet connection's speed in k/M/Gbits per second:</li>
<li>Country:</li>
</ul>
</li>
<li>Start mirroring from a suitable host in the [[Master Mirrors]].
Note: GWDG is heavily loaded; you can also mirror from every Category 1 / Tier-1 mirror and
we ask you to do so.
The software shows up with a little delay there, but it is not more than 12 hours. You can start copying from them without ANY permissions at
any time (rsync or ftp).</li>
<li>Read further on this page.</li>
</ol>

= Minimum Requirements =
(This set of requirements is under discussion, not the final one.)

* Your server is connected to the Internet with static IP address(es), not dynamically assigned one.
* Your Internet connection is capable of providing adequate bandwidth for the demands of your region.
* Security measures is properly implemented.

= Responsibilities =

<ul>
<li>All maintainers of mirrors listed here must be subscribed to the [mailto:mirrors@distribution.openoffice.org mirrors discussion mailing list]. To subscribe it, send an empty email to mirrors-subscribe@distribution.openoffice.org and follow instructions that will be automatically sent to your email address. To view archives, please go to [http://distribution.openoffice.org/servlets/SummarizeList?listName=mirrors mailing list archive]</li>
<li>All mirrors are responsible for keeping their files up-to-date and for notifying this project of any changes in their set up (such as a change hostname, URL or a discontinuation of service). We understand that maintaining files can be difficult, and so encourage you to ask for assistance from the community</li>
</ul>

= Files: What To Distribute =

Please refer to the [http://distribution.openoffice.org/files.html Files Page] for the latest information on which files are grouped in which sets or categories to distribute. In brief, there are two [http://distribution.openoffice.org/files.html#sets sets] for mirrors:
main or extended. If you choose Category 4 for CD-ROM distribution, you must specify the details of what you will be offering when you send in your link
(for a CD-ROM) or when you set up a Mirror.

When submitting to the CD-ROM listing, please do not simply say "Category 4". This tells us absolutely nothing.

No mirror is to be discounted but every mirror should understand that the download pages can't be compiled by hand. Category 4 mirrors will certainly be listed. They may also offer local services, such as ISO images for CD-ROMs, or files for "p2p" exchanges.

= Master and Tier-1 Servers =

Category 1 / Tier-1 servers have all OpenOffice.org related main set data at one entry point for you. Please choose the closest
Tier-1 server listed in the [[Master Mirrors]]. Even better would be if you go the extra step and join the small group of extended mirrors.

It is understood that a mirror site may experience a slight lag in obtaining the code from the primary servers. The
[http://download.services.openoffice.org/mirmon/ mirror monitoring page] gives an overview of the mirror states and delay.
Additionally we use [http://www.mirrorbrain.org Mirrorbrain] to check the availability of files and distribute the load. But each mirror site is responsible in the end for hosting at the least the latest build, and in the same directory structure, as the master server.

= Rsync Instructions =

<ul>
<li>We require rsync to copy over files listed on the master server. Please point to: 
<tt>rsync rsync://tier-1-server/path/</tt> (with the appropriate options) 
and start mirroring. If you have problems, please post them to the
[mailto:mirrors@distribution.openoffice.org mirrors discussion mailing list].</li>
<li>Please indicate which data set you wish to occupy. We encourage all to follow the category structure.</li>
<li>Speed of download: We have indicated by each mirror the quality and speed of its connection to the Internet. There are three indicators:
***, **, *. More details are supplied on each mirror site's homepage. Question m<arks
(??) indicate no data, not bad connections.</li>
</ul>

= Directory Structure =

All mirrors, regardless of category, will have to follow the format below for OpenOffice.org files. This will be easy, as rsync will allow you to copy
over the directory structure:

<blockquote>
; stable, tested builds : /stable/[release version]/[filenames] Example: /stable/3.2.1/OOo_3.2.1_Win_x86_install-wJRE_en-US.exe
; tested localized builds : /localized/[language]/[release version]/[filenames] Example: /localized/ja/3.2.1/OOo_3.2.1_Win_x86_install-wJRE_ja.exe
; contributed files : /contrib/[directoryname]/[filenames] Example: /contrib/dictionaries/README.txt
; large datasets like Betas, RCs, POT and SDF files : /extended/[directoryname]/[filenames] Example: /extended/3.2.1rc2/OOo_3.2.1rc2_20100521_Win_x86_install-wJRE_en-US.exe
; developer snapshots (aka nightly builds or milestone builds) : /extended/developer/[developer build number]/[filenames] Example: /extended/developer/DEV300m83/OOo-Dev_DEV300m83_Win_x86_install_en-US.exe
</blockquote>

The tools project hosts a description of the file [http://development.openoffice.org/releases/filenames.html naming scheme].

= Archive Servers =

We also have archive servers for mirroring legacy and rarely-used content. See [[Archive Mirrors]] for futher information.

= All Mirror Sites =

Many thanks go not only to extended mirror sites but to all the mirrors supporting OpenOffice.org listed below.

* [http://download.services.openoffice.org/mirrors/all.html OpenOffice.org Download Mirrors - Overview]
* [http://distribution.openoffice.org/mirrors/#mirrors All Mirror Sites] (slightly obsolete)

<nowiki>*</nowiki> Please use a FTP program if your browser does not support FTP downloads.

[[Category:Website]]

Mirrors Project

2011-03-13T07:54:29Z

Tora: Added Minimum Requirements

Co-Lead: [mailto:tora@openoffice.org Tora - Takamichi Akiyama]

= Mission Statement =

This project coordinates the listing of those sites that host OpenOffice.org software. A "mirror" is a website that has agreed to donate its server space and bandwidth for OpenOffice.org downloads. This page serves as a reference page for the download pages. For regular users, please go to the [http://download.openoffice.org/ OpenOffice.org Download webpage].

<blockquote>
Disclaimer: To be listed here, anything under the OpenOffice.org directory must come from any of the Category 1/Tier-1 servers mentioned in the [[Master Mirrors]]. We make no promises about this software, and anyone who downloads any software listed on OpenOffice.org, whether from the master servers or from any listed mirror or from any other source, including CD-ROMs or peer-to-peer networks, is doing so at his or her own risk. Downloaders should always exercise caution whenever downloading any executable software from the Internet. Please check your md5sums prior to installing downloaded software.
</blockquote>

= Setting Up a Mirror =

Setting up a mirror site is easy. You may select which [http://distribution.openoffice.org/files.html#sets sets] (main or extended) you wish to carry, but you must select one. We do not need mirrors for the OpenOffice.org Web content.

If you would like to set up a mirror, please read the information below and consult our
[http://www.openoffice.org/FAQs/mirrors_faq.html mirrors FAQs].

Quick start:

<ol>
<li>Choose one set for your mirror (look at [http://distribution.openoffice.org/files.html#sets sets]). 
We accept mirrors carrying the main set (category 1). But we are especially in need for mirrors capable to host the extended dataset.</li>
<li>Send an email to [mailto:mirrors@distribution.openoffice.org?subject=New%20Mirror project's discussion list], including: 

<ul>
<li>Name of mirror site:</li>
<li>Technical contact's name:</li>
<li>Technical contact's email address:</li>
<li>URL of the mirror (http):</li>
<li>URL of the mirror (ftp):</li>
<li>URL of the mirror (rsync):</li>
<li>Mirroring from:</li>
<li>Set You've chosen:</li>
<li>Your Internet connection's speed in k/M/Gbits per second:</li>
<li>Country:</li>
</ul>
</li>
<li>Start mirroring from a suitable host in the [[Master Mirrors]].
Note: GWDG is heavily loaded; you can also mirror from every Category 1 / Tier-1 mirror and
we ask you to do so.
The software shows up with a little delay there, but it is not more than 12 hours. You can start copying from them without ANY permissions at
any time (rsync or ftp).</li>
<li>Read further on this page.</li>
</ol>

= Minimum Requirements =
(This set of requirements is under discussion, not the final one.)

* Your server is connected to the Internet with static IP address(es), not dynamically assigned one.
* Your Internet connection is capable of providing adequate bandwidth for the demands of your region.
* Security measures is properly implemented.

= Responsibilities =

<ul>
<li>All maintainers of mirrors listed here must be subscribed to the [mailto:mirrors@distribution.openoffice.org mirrors discussion mailing list]. To subscribe it, send an empty email to mirrors-subscribe@distribution.openoffice.org and follow instructions that will be automatically sent to your email address. To view archives, please go to [http://distribution.openoffice.org/servlets/SummarizeList?listName=mirrors mailing list archive]</li>
<li>All mirrors are responsible for keeping their files up-to-date and for notifying this project of any changes in their set up (such as a change hostname, URL or a discontinuation of service). We understand that maintaining files can be difficult, and so encourage you to ask for assistance from the community</li>
</ul>

= Files: What To Distribute =

Please refer to the [http://distribution.openoffice.org/files.html Files Page] for the latest information on which files are grouped in which sets or categories to distribute. In brief, there are two [http://distribution.openoffice.org/files.html#sets sets] for mirrors:
main or extended. If you choose Category 4 for CD-ROM distribution, you must specify the details of what you will be offering when you send in your link
(for a CD-ROM) or when you set up a Mirror.

When submitting to the CD-ROM listing, please do not simply say "Category 4". This tells us absolutely nothing.

No mirror is to be discounted but every mirror should understand that the download pages can't be compiled by hand. Category 4 mirrors will certainly be listed. They may also offer local services, such as ISO images for CD-ROMs, or files for "p2p" exchanges.

= Master and Tier-1 Servers =

Category 1 / Tier-1 servers have all OpenOffice.org related main set data at one entry point for you. Please choose the closest
Tier-1 server listed in the [[Master Mirrors]]. Even better would be if you go the extra step and join the small group of extended mirrors.

It is understood that a mirror site may experience a slight lag in obtaining the code from the primary servers. The
[http://download.services.openoffice.org/mirmon/ mirror monitoring page] gives an overview of the mirror states and delay.
Additionally we use [http://www.mirrorbrain.org Mirrorbrain] to check the availability of files and distribute the load. But each mirror site is responsible in the end for hosting at the least the latest build, and in the same directory structure, as the master server.

= Rsync Instructions =

<ul>
<li>We require rsync to copy over files listed on the master server. Please point to: 
<tt>rsync rsync://tier-1-server/path/</tt> (with the appropriate options) 
and start mirroring. If you have problems, please post them to the
[mailto:mirrors@distribution.openoffice.org mirrors discussion mailing list].</li>
<li>Please indicate which data set you wish to occupy. We encourage all to follow the category structure.</li>
<li>Speed of download: We have indicated by each mirror the quality and speed of its connection to the Internet. There are three indicators:
***, **, *. More details are supplied on each mirror site's homepage. Question m<arks
(??) indicate no data, not bad connections.</li>
</ul>

= Directory Structure =

All mirrors, regardless of category, will have to follow the format below for OpenOffice.org files. This will be easy, as rsync will allow you to copy
over the directory structure:

<blockquote>
; stable, tested builds : /stable/[release version]/[filenames] Example: /stable/3.2.1/OOo_3.2.1_Win_x86_install-wJRE_en-US.exe
; tested localized builds : /localized/[language]/[release version]/[filenames] Example: /localized/ja/3.2.1/OOo_3.2.1_Win_x86_install-wJRE_ja.exe
; contributed files : /contrib/[directoryname]/[filenames] Example: /contrib/dictionaries/README.txt
; large datasets like Betas, RCs, POT and SDF files : /extended/[directoryname]/[filenames] Example: /extended/3.2.1rc2/OOo_3.2.1rc2_20100521_Win_x86_install-wJRE_en-US.exe
; developer snapshots (aka nightly builds or milestone builds) : /extended/developer/[developer build number]/[filenames] Example: /extended/developer/DEV300m83/OOo-Dev_DEV300m83_Win_x86_install_en-US.exe
</blockquote>

The tools project hosts a description of the file [http://development.openoffice.org/releases/filenames.html naming scheme].

= Archive Servers =

We also have archive servers for mirroring legacy and rarely-used content. See [[Archive Mirrors]] for futher information.

= All Mirror Sites =

Many thanks go not only to extended mirror sites but to all the mirrors supporting OpenOffice.org listed below.

* [http://download.services.openoffice.org/mirrors/all.html OpenOffice.org Download Mirrors - Overview]
* [http://distribution.openoffice.org/mirrors/#mirrors All Mirror Sites] (slightly obsolete)

<nowiki>*</nowiki> Please use a FTP program if your browser does not support FTP downloads.

[[Category:Website]]

Jsdp2007

2011-03-07T17:28:42Z

Mirrors Project

2011-01-20T09:11:35Z

Tora: Inserted "Co-Lead: Tora"

Co-Lead: [mailto:tora@openoffice.org Tora - Takamichi Akiyama]

= Mission Statement =

This project coordinates the listing of those sites that host OpenOffice.org software. A "mirror" is a website that has agreed to donate its server space and bandwidth for OpenOffice.org downloads. This page serves as a reference page for the download pages. For regular users, please go to the [http://download.openoffice.org/ OpenOffice.org Download webpage].

<blockquote>
Disclaimer: To be listed here, anything under the OpenOffice.org directory must come from any of the Category 1/Tier-1 servers mentioned in the [[Master Mirrors]]. We make no promises about this software, and anyone who downloads any software listed on OpenOffice.org, whether from the master servers or from any listed mirror or from any other source, including CD-ROMs or peer-to-peer networks, is doing so at his or her own risk. Downloaders should always exercise caution whenever downloading any executable software from the Internet. Please check your md5sums prior to installing downloaded software.
</blockquote>

= Setting Up a Mirror =

Setting up a mirror site is easy. You may select which [http://distribution.openoffice.org/files.html#sets sets] (main or extended) you wish to carry, but you must select one. We do not need mirrors for the OpenOffice.org Web content.

If you would like to set up a mirror, please read the information below and consult our
[http://www.openoffice.org/FAQs/mirrors_faq.html mirrors FAQs].

Quick start:

<ol>
<li>Choose one set for your mirror (look at [http://distribution.openoffice.org/files.html#sets sets]). 
We accept mirrors carrying the main set (category 1). But we are especially in need for mirrors capable to host the extended dataset.</li>
<li>Send an email to [mailto:mirrors@distribution.openoffice.org?subject=New%20Mirror project's discussion list], including: 

<ul>
<li>Technical contact's name:</li>
<li>Technical contact's email address:</li>
<li>the URL of the mirror (http, ftp, rsync):</li>
<li>Mirroring from:</li>
<li>Set You've chosen:</li>
<li>Your Internet connection's speed in k/M/Gbits per second:</li>
<li>Country:</li>
</ul>
</li>
<li>Start mirroring from a suitable host in the [[Master Mirrors]].
Note: GWDG is heavily loaded; you can also mirror from every Category 1 / Tier-1 mirror and
we ask you to do so.
The software shows up with a little delay there, but it is not more than 12 hours. You can start copying from them without ANY permissions at
any time (rsync or ftp).</li>
<li>Read further on this page.</li>
</ol>

= Responsibilities =

<ul>
<li>All maintainers of mirrors listed here must be subscribed to the [mailto:mirrors@distribution.openoffice.org mirrors discussion mailing list]. To subscribe it, send an empty email to mirrors-subscribe@distribution.openoffice.org and follow instructions that will be automatically sent to your email address. To view archives, please go to [http://distribution.openoffice.org/servlets/SummarizeList?listName=mirrors mailing list archive]</li>
<li>All mirrors are responsible for keeping their files up-to-date and for notifying this project of any changes in their set up (such as a change hostname, URL or a discontinuation of service). We understand that maintaining files can be difficult, and so encourage you to ask for assistance from the community</li>
</ul>

= Files: What To Distribute =

Please refer to the [http://distribution.openoffice.org/files.html Files Page] for the latest information on which files are grouped in which sets or categories to distribute. In brief, there are two [http://distribution.openoffice.org/files.html#sets sets] for mirrors:
main or extended. If you choose Category 4 for CD-ROM distribution, you must specify the details of what you will be offering when you send in your link
(for a CD-ROM) or when you set up a Mirror.

When submitting to the CD-ROM listing, please do not simply say "Category 4". This tells us absolutely nothing.

No mirror is to be discounted but every mirror should understand that the download pages can't be compiled by hand. Category 4 mirrors will certainly be listed. They may also offer local services, such as ISO images for CD-ROMs, or files for "p2p" exchanges.

= Master and Tier-1 Servers =

Category 1 / Tier-1 servers have all OpenOffice.org related main set data at one entry point for you. Please choose the closest
Tier-1 server listed in the [[Master Mirrors]]. Even better would be if you go the extra step and join the small group of extended mirrors.

It is understood that a mirror site may experience a slight lag in obtaining the code from the primary servers. The
[http://download.services.openoffice.org/mirmon/ mirror monitoring page] gives an overview of the mirror states and delay.
Additionally we use [http://www.mirrorbrain.org Mirrorbrain] to check the availability of files and distribute the load. But each mirror site is responsible in the end for hosting at the least the latest build, and in the same directory structure, as the master server.

= Rsync Instructions =

<ul>
<li>We require rsync to copy over files listed on the master server. Please point to: 
<tt>rsync rsync://tier-1-server/path/</tt> (with the appropriate options) 
and start mirroring. If you have problems, please post them to the
[mailto:mirrors@distribution.openoffice.org mirrors discussion mailing list].</li>
<li>Please indicate which data set you wish to occupy. We encourage all to follow the category structure.</li>
<li>Speed of download: We have indicated by each mirror the quality and speed of its connection to the Internet. There are three indicators:
***, **, *. More details are supplied on each mirror site's homepage. Question m<arks
(??) indicate no data, not bad connections.</li>
</ul>

= Directory Structure =

All mirrors, regardless of category, will have to follow the format below for OpenOffice.org files. This will be easy, as rsync will allow you to copy
over the directory structure:

<blockquote>
; stable, tested builds : /stable/[release version]/[filenames] Example: /stable/3.2.1/OOo_3.2.1_Win_x86_install-wJRE_en-US.exe
; tested localized builds : /localized/[language]/[release version]/[filenames] Example: /localized/ja/3.2.1/OOo_3.2.1_Win_x86_install-wJRE_ja.exe
; contributed files : /contrib/[directoryname]/[filenames] Example: /contrib/dictionaries/README.txt
; large datasets like Betas, RCs, POT and SDF files : /extended/[directoryname]/[filenames] Example: /extended/3.2.1rc2/OOo_3.2.1rc2_20100521_Win_x86_install-wJRE_en-US.exe
; developer snapshots (aka nightly builds or milestone builds) : /extended/developer/[developer build number]/[filenames] Example: /extended/developer/DEV300m83/OOo-Dev_DEV300m83_Win_x86_install_en-US.exe
</blockquote>

The tools project hosts a description of the file [http://development.openoffice.org/releases/filenames.html naming scheme].

= Archive Servers =

We also have archive servers for mirroring legacy and rarely-used content. See [[Archive Mirrors]] for futher information.

= All Mirror Sites =

Many thanks go not only to extended mirror sites but to all the mirrors supporting OpenOffice.org listed below.

* [http://download.services.openoffice.org/mirrors/all.html OpenOffice.org Download Mirrors - Overview]
* [http://distribution.openoffice.org/mirrors/#mirrors All Mirror Sites] (slightly obsolete)

<nowiki>*</nowiki> Please use a FTP program if your browser does not support FTP downloads.

[[Category:Website]]

Archive Mirrors

2010-11-01T21:19:03Z

Tora: Initially copied from http://distribution.openoffice.org/mirrors/archive.html

= Introduction =

OpenOffice.org is a fairly large project, available for many combinations of languages and platforms. Thus, we currently offer a broad range of files on our mirroring network that are quite large in size. Some of them are requested quite often, whereas some of them are old, legacy and only downloaded once in a while. Therefore, we need to establish a policy on how to mirror the files in an economical way.

= Current situation =

Currently, we distribute all of them in two mirror modules: the 'main' module and the 'extended' module, whereas the 'main' module is divided into several categories. The main module is mirrored by quite a few mirrors, whereas the 'extended' module - due to its size - is being mirrored only by a dozen one. This hierarchy shouldn't be changed. However, in order to shrink the size of all modules, we want to establish a new module called 'archive', where we can store the mentioned old, legacy and rarely downloaded files and move them out of 'main' or 'extended'. The 'archive' module is only mirrored on two or three mirrors to avoid too much traffic consumption. We currently DO NOT need new mirrors mirroring the archive set.

= Location of the archive set =

The archive set is available at
* [http://archive.services.openoffice.org/pub/openoffice-archive/ http://archive.services.openoffice.org/pub/openoffice-archive/]
* [ftp://archive.services.openoffice.org/pub/openoffice-archive/ ftp://archive.services.openoffice.org/pub/openoffice-archive/]

= Detailed structure of the modules =

Roughly spoken, the modules should be functioning as follows:

The 'main' set should only contain the last released version for each platform and language, including source code. For a transition time after each release, 'main' should contain two builds until all download pages and Bouncer entries have been adjusted. Additionally the 'main' set includes en-US binaries of release candidates (../contrib/rc) and developer builds (../developer).

The 'extended' set should contain things like experimental Mac OS X builds or experimental builds for other platforms before they get final, as these files are requested quite often. 'extended' is also the right choice for URE, SDK, release candidates, CWSs, ISOs and additional dictionaries. Files that are only requested rarely should be put into 'archive'.

The 'archive' module should contain archived old versions. However, as storing each and any version is quite senseless, and explodes disk space rapidly, only the last version of each code tree for the combination platform and language should be stored. Examples: 1.0.3.1, 1.1.5, eventually 2.4.1; but also things like 1.1.2 for LinuxPPC, which is the last version for this platform. Apart from its name, 'archive' is also dedicated to builds that would normally fit into 'extended', but are rarely downloaded.

And, last but not least: Experimental stuff, very old builds et al. shouldn't be in the mirror at all. No one needs an unsupported OpenOffice.org version for a dead architecture on an unknown platform mirrored anywhere...

Master Mirrors

2010-11-01T21:13:35Z

Tora: Initially copied from http://distribution.openoffice.org/mirrors/master.html

= Introductions =

The following servers are available for downstream mirrors as a master mirror server.

As an end user please use our [http://download.openoffice.org/index.html Download Central].

If you are providing OpenOffice.org downloads from your server, please read more about our [[Mirrors Project]] and let us know by sending a message to [mailto:mirrors@distribution.openoffice.org mirrors@distribution.openoffice.org].

= Tier-1 / Category 1 / main set =

The Tier-1 / Category 1 servers mentioned below carry the main data set and are accessible without prior registration.
Access to the extended data set is described at the bottom of this page.

<blockquote>

=== Goodie Domain Service - Vienna Univ. of Tech. (Europe) ===
* rsync://gd.tuwien.ac.at/openoffice/
* [ftp://gd.tuwien.ac.at/office/openoffice/ ftp://gd.tuwien.ac.at/office/openoffice/]
* [http://gd.tuwien.ac.at/office/openoffice/ http://gd.tuwien.ac.at/office/openoffice/]

=== SNT (Europe) ===
* rsync://ftp.snt.utwente.nl/openoffice/
* [ftp://ftp.snt.utwente.nl/pub/software/openoffice/ ftp://ftp.snt.utwente.nl/pub/software/openoffice/]
* [http://ftp.snt.utwente.nl/pub/software/openoffice/ http://ftp.snt.utwente.nl/pub/software/openoffice/]

=== RWTH Aachen (Europe) ===
* rsync://rsync.informatik.rwth-aachen.de/openoffice/
* [ftp://sunsite.informatik.rwth-aachen.de/pub/mirror/OpenOffice/ ftp://sunsite.informatik.rwth-aachen.de/pub/mirror/OpenOffice/]
* [http://sunsite.informatik.rwth-aachen.de/ftp/pub/mirror/OpenOffice/ http://sunsite.informatik.rwth-aachen.de/ftp/pub/mirror/OpenOffice/]

=== Universität Münster (Europe) ===
* rsync://ftp.uni-muenster.de/OpenOffice/
* [ftp://ftp.uni-muenster.de/pub/mirrors/ftp.openoffice.org/ ftp://ftp.uni-muenster.de/pub/mirrors/ftp.openoffice.org/]

=== Planet Mirror (Australia) ===
* rsync://rsync.planetmirror.com/openoffice/
* [http://planetmirror.com/pub/openoffice/ http://planetmirror.com/pub/openoffice/]
* [ftp://ftp.planetmirror.com/pub/openoffice/ ftp://ftp.planetmirror.com/pub/openoffice/]

=== Indiana University (Americas) ===
* rsync://ftp.ussg.iu.edu/openoffice/
* [ftp://ftp.ussg.iu.edu/pub/openoffice/ ftp://ftp.ussg.iu.edu/pub/openoffice/]

=== Universidade Federal do Paraná (South America) ===
* rsync://openoffice.c3sl.ufpr.br/openoffice/
* [ftp://openoffice.c3sl.ufpr.br/openoffice/ ftp://openoffice.c3sl.ufpr.br/openoffice/]
* [http://openoffice.c3sl.ufpr.br http://openoffice.c3sl.ufpr.br]

</blockquote>

= Master servers for Tier-1 / Category 1 mirrors =

The following servers should be used by Tier-1 / Category 1 mirrors only.

<blockquote>

=== GWDG (sync'ed every 4 hours from StarDiv) ===
* rsync://ftp5.gwdg.de/OpenOffice/
* [ftp://ftp5.gwdg.de/pub/openoffice/ ftp://ftp5.gwdg.de/pub/openoffice/]
* [http://ftp5.gwdg.de/pub/openoffice/ http://ftp5.gwdg.de/pub/openoffice/]

</blockquote>

= Servers for extended dataset =

The following server can be used to mirror the extended dataset.

<blockquote>

=== rsync.services.openoffice.org ===
* rsync://rsync.services.openoffice.org/openoffice-extended/
* [http://rsync.services.openoffice.org http://rsync.services.openoffice.org] (HTTP access is not guaranteed to work. If it sucks too much bandwith we will shut it down in favor of downstream servers rsyncing the data.)

</blockquote>

= Archive Servers =

We also have archive servers for mirroring legacy and rarely-used content. See [[Archive Mirrors]] for futher information.

Mirrors Project

2010-11-01T20:48:08Z

Tora: Initially copied from http://distribution.openoffice.org/mirrors/

= Mission Statement =

This project coordinates the listing of those sites that host OpenOffice.org software. A "mirror" is a website that has agreed to donate its server space and bandwidth for OpenOffice.org downloads. This page serves as a reference page for the download pages. For regular users, please go to the [http://download.openoffice.org/ OpenOffice.org Download webpage].

<blockquote>
Disclaimer: To be listed here, anything under the OpenOffice.org directory must come from any of the Category 1/Tier-1 servers mentioned in the [[Master Mirrors]]. We make no promises about this software, and anyone who downloads any software listed on OpenOffice.org, whether from the master servers or from any listed mirror or from any other source, including CD-ROMs or peer-to-peer networks, is doing so at his or her own risk. Downloaders should always exercise caution whenever downloading any executable software from the Internet. Please check your md5sums prior to installing downloaded software.
</blockquote>

= Setting Up a Mirror =

Setting up a mirror site is easy. You may select which [http://distribution.openoffice.org/files.html#sets sets] (main or extended) you wish to carry, but you must select one. We do not need mirrors for the OpenOffice.org Web content.

If you would like to set up a mirror, please read the information below and consult our
[http://www.openoffice.org/FAQs/mirrors_faq.html mirrors FAQs].

Quick start:

<ol>
<li>Choose one set for your mirror (look at [http://distribution.openoffice.org/files.html#sets sets]). 
We accept mirrors carrying the main set (category 1). But we are especially in need for mirrors capable to host the extended dataset.</li>
<li>Send an email to [mailto:mirrors@distribution.openoffice.org?subject=New%20Mirror project's discussion list], including: 

<ul>
<li>Technical contact's name:</li>
<li>Technical contact's email address:</li>
<li>the URL of the mirror (http, ftp, rsync):</li>
<li>Mirroring from:</li>
<li>Set You've chosen:</li>
<li>Your Internet connection's speed in k/M/Gbits per second:</li>
<li>Country:</li>
</ul>
</li>
<li>Start mirroring from a suitable host in the [[Master Mirrors]].
Note: GWDG is heavily loaded; you can also mirror from every Category 1 / Tier-1 mirror and
we ask you to do so.
The software shows up with a little delay there, but it is not more than 12 hours. You can start copying from them without ANY permissions at
any time (rsync or ftp).</li>
<li>Read further on this page.</li>
</ol>

= Responsibilities =

<ul>
<li>All maintainers of mirrors listed here must be subscribed to the [mailto:mirrors@distribution.openoffice.org mirrors discussion mailing list]. To subscribe it, send an empty email to mirrors-subscribe@distribution.openoffice.org and follow instructions that will be automatically sent to your email address. To view archives, please go to [http://distribution.openoffice.org/servlets/SummarizeList?listName=mirrors mailing list archive]</li>
<li>All mirrors are responsible for keeping their files up-to-date and for notifying this project of any changes in their set up (such as a change hostname, URL or a discontinuation of service). We understand that maintaining files can be difficult, and so encourage you to ask for assistance from the community</li>
</ul>

= Files: What To Distribute =

Please refer to the [http://distribution.openoffice.org/files.html Files Page] for the latest information on which files are grouped in which sets or categories to distribute. In brief, there are two [http://distribution.openoffice.org/files.html#sets sets] for mirrors:
main or extended. If you choose Category 4 for CD-ROM distribution, you must specify the details of what you will be offering when you send in your link
(for a CD-ROM) or when you set up a Mirror.

When submitting to the CD-ROM listing, please do not simply say "Category 4". This tells us absolutely nothing.

No mirror is to be discounted but every mirror should understand that the download pages can't be compiled by hand. Category 4 mirrors will certainly be listed. They may also offer local services, such as ISO images for CD-ROMs, or files for "p2p" exchanges.

= Master and Tier-1 Servers =

Category 1 / Tier-1 servers have all OpenOffice.org related main set data at one entry point for you. Please choose the closest
Tier-1 server listed in the [[Master Mirrors]]. Even better would be if you go the extra step and join the small group of extended mirrors.

It is understood that a mirror site may experience a slight lag in obtaining the code from the primary servers. The
[http://download.services.openoffice.org/mirmon/ mirror monitoring page] gives an overview of the mirror states and delay.
Additionally we use [http://www.mirrorbrain.org Mirrorbrain] to check the availability of files and distribute the load. But each mirror site is responsible in the end for hosting at the least the latest build, and in the same directory structure, as the master server.

= Rsync Instructions =

<ul>
<li>We require rsync to copy over files listed on the master server. Please point to: 
<tt>rsync rsync://tier-1-server/path/</tt> (with the appropriate options) 
and start mirroring. If you have problems, please post them to the
[mailto:mirrors@distribution.openoffice.org mirrors discussion mailing list].</li>
<li>Please indicate which data set you wish to occupy. We encourage all to follow the category structure.</li>
<li>Speed of download: We have indicated by each mirror the quality and speed of its connection to the Internet. There are three indicators:
***, **, *. More details are supplied on each mirror site's homepage. Question m<arks
(??) indicate no data, not bad connections.</li>
</ul>

= Directory Structure =

All mirrors, regardless of category, will have to follow the format below for OpenOffice.org files. This will be easy, as rsync will allow you to copy
over the directory structure:

<blockquote>
; stable, tested builds : /stable/[release version]/[filenames] Example: /stable/3.2.1/OOo_3.2.1_Win_x86_install-wJRE_en-US.exe
; tested localized builds : /localized/[language]/[release version]/[filenames] Example: /localized/ja/3.2.1/OOo_3.2.1_Win_x86_install-wJRE_ja.exe
; contributed files : /contrib/[directoryname]/[filenames] Example: /contrib/dictionaries/README.txt
; large datasets like Betas, RCs, POT and SDF files : /extended/[directoryname]/[filenames] Example: /extended/3.2.1rc2/OOo_3.2.1rc2_20100521_Win_x86_install-wJRE_en-US.exe
; developer snapshots (aka nightly builds or milestone builds) : /extended/developer/[developer build number]/[filenames] Example: /extended/developer/DEV300m83/OOo-Dev_DEV300m83_Win_x86_install_en-US.exe
</blockquote>

The tools project hosts a description of the file [http://development.openoffice.org/releases/filenames.html naming scheme].

= Archive Servers =

We also have archive servers for mirroring legacy and rarely-used content. See [[Archive Mirrors]] for futher information.

= All Mirror Sites =

Many thanks go not only to extended mirror sites but to all the mirrors supporting OpenOffice.org listed below.

* [http://download.services.openoffice.org/mirrors/all.html OpenOffice.org Download Mirrors - Overview]
* [http://distribution.openoffice.org/mirrors/#mirrors All Mirror Sites] (slightly obsolete)

<nowiki>*</nowiki> Please use a FTP program if your browser does not support FTP downloads.

Rough ideas for the OpenOffice.org Open Mirror Network System version 3.0

2009-12-29T06:59:04Z

Tora: Make some subsection titles plural

== Status of this document ==
* Rough ideas for brainstorming

== Concepts ==
{| border=1 cellpadding=3
! Concept
! Descriptions
|-
| Distributed computer network
|
* One subsystem per one server basis deployment
* Each subsystem concentrates on solely its own duty
* Deploying servers as a virtual and/or real machine
|-
| Robustness
|
* Hard to lose data such as configurations, histories, and/or logs
* Data is kept in several subsystems as a replica, not kept in a single centralized server machine
* No need to keep irrelevant, uninteresting data in any subsystem
|-
| Simple, open API
|
* Platform, programming language, and application neutral API
* HTTP GET to retrieve remote data as a plain text, XML, and/or ZIP-archived file
* Anyone can freely develop their own subsystems using data files via the API
|-
| Maintainability
|
* Easy to upgrade each subsystem separately
* Easy to develop and test with experimental and staging servers while production servers serve users
|-
| Scalability & High Availability
|
* Easy to add servers like a small device and to replace them if broken
* DNS load balancing and automatic fail over
|-
| Surveillance & Alert
|
* Surveying malicious files and monitoring the status of servers
* Emergency stop in case of incident
|}

== Network Diagram ==
[[Image:open-mirror-network-system-draft-2009-12-17-2100.png]]

== Subsystems ==
The OpenOffice.org Open Mirror Network System consists of several subsystems. Each subsystem is loosely connected from one subsystem to another subsystem using the platform, programming language, and application independent API.

{| border=1 cellpadding=3
!Subsystem
!Descriptions
|-
| Mirror Server
|
* A mirror server is a server run by several types of entities such as universities, organizations, companies or individuals, which delivers files to users via HTTP, FTP, and/or RSYNC.
* Mirror servers are spread over the world and can be classified into three types: Master server, Tier-1 server, and Regular server.
* The master server is the origination from which files come. A Tier-1 server retrieves files from the Master server; A Regular server retrieve files from the Tire-1 server.
* The release engineering team uploads files to the master server.
|-
| OpenOffice.org Web
|
# A user visits the web site, download.openoffice.org.
# A JavaScript in the web page accesses to the Download Concierge Servers to get several types of small lists of deliverable and shows them to the user using AJAX.
# The user chooses a platform, product, version, language, and other items from the lists. The JavaScript helps the user find required files saying e.g. "You will need to install one of these language installers first and then install this language pack for your language." The user clicks on the shown links to start downloading file(s).
# One of the load-balanced Redirector Servers takes a user's download request to a closer Mirror Server.
# The Mirror Server sends back the requested file to the user.
|-
| Download Concierge Server
|
* A Download Concierge Server is a web server specially tuned for AJAX.
* This server fetches scan results from the Scanner Server, internally prepares several lists of items by categories such as platform, product, version, language, and other items, and responds with a small XML file upon a request from JavaScript embedded in a web page.
|-
| Redirector Server
|
* A Redirector Server fetches scan results from the Scanner Server and hits back a user's request to a closer server in which the requested file is available and puts log files on its web server.
|-
| Scanner Server
|
* A Scanner Server fetches repository data such as a list of URLs from the Repository, scans all mirror servers based on the URLs, and puts scan results and log files on its web server.
|-
| Repository
|
* The Repository is not a server but a set of data files such as a list of mirror sites, which are stored in distribution.openoffice.org and maintained by the staffs.
* GUI tools might be prepared for staffs to easily maintain the repository data.
|-
| Logging Server
|
* A Logging Server collects log files from several servers, concatenate some log files - simultaneously produced by load-balanced servers - into a single log file, and puts those log files on its web server.
|-
| Statistics Calculation Server
|
* A Statistics Calculation Server fetches log files from a Logging Server, accumulates logs by category such as location of user, choice of language, version of OpenOffice.org, calculates statistics, and provides users with the results through its web server.
* Any type of existing freely available software products could be used for this purpose.
* Or even, anyone can develop own cool program running on his server or PC.
|-
| Surveillance Servers & User's PC
|
* A Surveillance Server fetches repository data from the Repository and scan results from the Scanner Server and confirm the integrity of files kept in each mirror server by periodically and randomly scanning all mirror servers.
* This server also continuously checks the status of several other servers such as Redirector Server and Scanner Server.
* Likewise, User's PC also does the similar tasks.
* A Surveillance Server will inform Redirector Servers and alert Staffs and Administrators in case of incident.
* A PC user would inform suspicious phenomenon on the mailing list.
* Staffs can directly stop the Redirector Servers.
|}


== API ==
Each subsystem communicates by obtaining remote files via HTTP GET. There would be basically three types of file:
{| border=1 cellpadding=3
! Type
! Example
|-
| A plain text file (.txt)
| Scan results (URL, path, type of entry (d: directory, f: file, l: symbolic link)
<pre>
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/ d
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f
...
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/ d
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f
...
</pre>
|-
| XML file (.xml)
| A list of mirror server
<pre>
<site location="Australia" code="au" type="regular" name="AussieHQ">
<uri set="extended" scheme="ftp">ftp://openoffice.mirror.aussiehq.net.au/pub/openoffice/</uri>
<uri set="extended" scheme="http">http://openoffice.mirror.aussiehq.net.au/</uri>
<uri set="extended" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice-extended/</uri>
<uri set="main" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice/</uri>
<contact name="Foo Bar">
<email>foo.bar@xxx.yyy</email>
</contact>
</site>
</pre>
|-
| ZIP-archived file (.zip)
|
* A set of a list of mirror server (.xml) and scan results (.txt)
* A set of Apache access log file (.txt) and error log file (.txt)
|}


== Strength ==
{| border=1 cellpadding=3
!Strengh
!Descriptions
|-
| Maintainability
|
* Since each subsystem is loosely connected, we can stop, reboot, upgrade, and replace one server without interrupting other server's service.
* Since each subsystem is encapsulated, we can easily upgrade any subsystem without depending on other subsystem's implementation.
* We can develop a new version of software on an experimental server and test it on a staging server without interfering with the production server which is serving real users.
|-
| Scalability and high availability
|
* We can instantly deploy many instances of subsystem, such as a redirector server, in several different locations.
* Each instance of one subsystem can be given an individual IP address and these IP addresses can be accessed with a single FQDN - DNS load balancing.
* Since the API is based on HTTP, as its nature, a fail over to another IP address could be automatically done in a condition of multiply assigned IP address for a single FQDN.
* Each server can be a virtual machine in an existing server machine or can be a real machine depending on a resource strategy.
|-
| Openness
|
* We can easily deploy existing software products such as server monitoring tools and download statistics tools because of application neutral API
* Anyone can freely get and reuse any data provided by the repository and each subsystem.
* Any subsystem can be developed by any developers who are interested and experienced in the field. Web developers could develop a front-end for users. Programmers who are good at mathematics could develop a statistics calculation server. Operators who know network administration could develop a surveillance server.
|}


== Weakness ==
{| border=1 cellpadding=3
! Weakness
! Descriptions
|-
| Complexity
|
* Each subsystem is simple, but the entire system is complex.
* No one can know deeply the entire system from end to end because each subsystem is encapsulated and its detailed implementation is left to its developer or development team.
|}


== Generations ==

{| border=1 cellpadding=3
!Generation
!System
!Descriptions
|-
| 1
| Bouncer
|
* Apache, PHP, and MySQL based application
* "Bouncer is a database driven mirror management app..."
* http://wiki.osuosl.org/display/Bouncer/Home
|-
| 2
| MirrorBrain
|
* Apache, Python, Perl and PostgreSQL based application
* "MirrorBrain is a Download Redirector and Metalink Generator"
* http://mirrorbrain.org/
|- style="background-color: #8080ff;"
| 3
| Open Mirror Network System version 3
|
* Distributed computer network system
|-
|4
| Open Mirror Network System version 4
|
* In addition to version 3, (just some ideas)
** providing communication features between redirectors and each mirror server
** logging for preciser download statistics calculation
** taking timezone into account
|}


== The Next Generation ==
In the version 4, redirectors and mirror servers would communicate each other over SOAP or other protocol to make situation better.
* The availability of content
** Pooling
*** Redirector: "What files do you currently have?"
*** Mirror server: "I have this, this, ... and this file."
** Reporting
*** Mirror server: "I have gotten this file and deleted that file."
*** Redirector: "Thanks."

* The load of server
** Pooling
*** Redirector: "How is it going?"
*** Mirror server: "Well, I have been busy, could you throttle the amount of download requests towards me?"
** Reporting
*** Mirror server: "I have currently room for more download."
*** Redirector: "OK, I am increasing the amount of download requests for you."

To calculate preciser download statistics, installing a small program in every or most mirror servers and the program prepare download logs. Logging Servers gather download logs from each mirror server and calculate download statistics.

In addition to the communication, timezone would be also taken into account. E.g during a day time in Europa, some traffic from the inside of Europa could be routed to America where it is early morning.

== Developer Assignment ==
{| border=1 cellpadding=3
!Subsystem
!Developer
|-
| Mirror Server
| - (no need to be developed)
|-
| OpenOffice.org Web
| (website project, release engineer)
|-
| Download Concierge Server
| (website project, release engineer, mirror project)
|-
| Redirector Server
| (Persons who know redirector)
|-
| Scanner Server
| (Probably Tora)
|-
| Repository
| - (no need to be developed)
|-
| Logging Server
| (Probably Tora)
|-
| Statistics Calculation Server
| (Persons who know statistics, marketing project, or simply use existing software)
|-
| Surveillance Servers & User's PC
| (Persons who are interested in this area)
|}


== Schedule ==
(To be discussed.)

[[Category:Mirror Network]]

Rough ideas for the OpenOffice.org Open Mirror Network System version 3.0

2009-12-29T04:47:19Z

Tora: Concepts, Strength, and some expressions

Rough ideas for the OpenOffice.org Open Mirror Network System version 3.0

2009-12-29T04:45:49Z

Tora: Undo revision 154027 by Tora (Talk)

Rough ideas for the OpenOffice.org Open Mirror Network System version 3.0

2009-12-29T04:44:39Z

Tora: Revised Concepts and Strength and some expressions

== Status of this document ==
* Rough ideas for brainstorming

== Concept ==
{| border=1 cellpadding=3
! Concept
! Descriptions
|-
| Distributed computer network
|
* One subsystem per one server basis deployment
* Each subsystem concentrates on solely its own duty
* Deploying servers as a virtual and/or real machine
|-
| Robustness
|
* Hard to lose data such as configurations, histories, and/or logs
* Data is kept in several subsystems as a replica, not kept in a single centralized server machine
* No need to keep irrelevant, uninteresting data in any subsystem
|-
| Simple, open API
|
* Platform, programming language, and application neutral API
* HTTP GET to retrieve remote data as a plain text, XML, and/or ZIP-archived file
* Anyone can freely develop their own subsystems using data files via the API
|-
| Maintainability
|
* Easy to upgrade each subsystem separately
* Easy to develop and test with experimental and staging servers while production servers serve users
|-
| Scalability & High Availability
|
* Easy to add servers like a small device and to replace them if broken
* DNS load balancing and automatic fail over
|-
| Surveillance & Alert
|
* Surveying malicious files and monitoring the status of servers
* Emergency stop in case of incident
|}

== Network Diagram ==
[[Image:open-mirror-network-system-draft-2009-12-17-2100.png]]

== Subsystem ==
The OpenOffice.org Open Mirror Network System consists of several subsystems. Each subsystem is loosely connected from one subsystem to another subsystem using the platform, programming language, and application independent API.

{| border=1 cellpadding=3
!Subsystem
!Descriptions
|-
| Mirror Server
|
* A mirror server is a server run by several types of entities such as universities, organizations, companies or individuals, which delivers files to users via HTTP, FTP, and/or RSYNC.
* Mirror servers are spread over the world and can be classified into three types: Master server, Tier-1 server, and Regular server.
* The master server is the origination from which files come. A Tier-1 server retrieves files from the Master server; A Regular server retrieve files from the Tire-1 server.
* The release engineering team uploads files to the master server.
|-
| OpenOffice.org Web
|
# A user visits the web site, download.openoffice.org.
# A JavaScript in the web page accesses to the Download Concierge Servers to get several types of small lists of deliverable and shows them to the user using AJAX.
# The user chooses a platform, product, version, language, and other items from the lists. The JavaScript helps the user find required files saying e.g. "You will need to install one of these language installers first and then install this language pack for your language." The user clicks on the shown links to start downloading file(s).
# One of the load-balanced Redirector Servers takes a user's download request to a closer Mirror Server.
# The Mirror Server sends back the requested file to the user.
|-
| Download Concierge Server
|
* A Download Concierge Server is a web server specially tuned for AJAX.
* This server fetches scan results from the Scanner Server, internally prepares several lists of items by categories such as platform, product, version, language, and other items, and responds with a small XML file upon a request from JavaScript embedded in a web page.
|-
| Redirector Server
|
* A Redirector Server fetches scan results from the Scanner Server and hits back a user's request to a closer server in which the requested file is available and puts log files on its web server.
|-
| Scanner Server
|
* A Scanner Server fetches repository data such as a list of URLs from the Repository, scans all mirror servers based on the URLs, and puts scan results and log files on its web server.
|-
| Repository
|
* The Repository is not a server but a set of data files such as a list of mirror sites, which are stored in distribution.openoffice.org and maintained by the staffs.
* GUI tools might be prepared for staffs to easily maintain the repository data.
|-
| Logging Server
|
* A Logging Server collects log files from several servers, concatenate some log files - simultaneously produced by load-balanced servers - into a single log file, and puts those log files on its web server.
|-
| Statistics Calculation Server
|
* A Statistics Calculation Server fetches log files from a Logging Server, accumulates logs by category such as location of user, choice of language, version of OpenOffice.org, calculates statistics, and provides users with the results through its web server.
* Any type of existing freely available software products could be used for this purpose.
* Or even, anyone can develop own cool program running on his server or PC.
|-
| Surveillance Servers & User's PC
|
* A Surveillance Server fetches repository data from the Repository and scan results from the Scanner Server and confirm the integrity of files kept in each mirror server by periodically and randomly scanning all mirror servers.
* This server also continuously checks the status of several other servers such as Redirector Server and Scanner Server.
* Likewise, User's PC also does the similar tasks.
* A Surveillance Server will inform Redirector Servers and alert Staffs and Administrators in case of incident.
* A PC user would inform suspicious phenomenon on the mailing list.
* Staffs can directly stop the Redirector Servers.
|}


== API ==
Each subsystem communicates by obtaining remote files via HTTP GET. There would be basically three types of file:
{| border=1 cellpadding=3
! Type
! Example
|-
| A plain text file (.txt)
| Scan results (URL, path, type of entry (d: directory, f: file, l: symbolic link)
<pre>
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/ d
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f
...
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/ d
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f
...
</pre>
|-
| XML file (.xml)
| A list of mirror server
<pre>
<site location="Australia" code="au" type="regular" name="AussieHQ">
<uri set="extended" scheme="ftp">ftp://openoffice.mirror.aussiehq.net.au/pub/openoffice/</uri>
<uri set="extended" scheme="http">http://openoffice.mirror.aussiehq.net.au/</uri>
<uri set="extended" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice-extended/</uri>
<uri set="main" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice/</uri>
<contact name="Foo Bar">
<email>foo.bar@xxx.yyy</email>
</contact>
</site>
</pre>
|-
| ZIP-archived file (.zip)
|
* A set of a list of mirror server (.xml) and scan results (.txt)
* A set of Apache access log file (.txt) and error log file (.txt)
|}


== Strength ==
{| border=1 cellpadding=3
!Strengh
!Descriptions
|-
| Maintainability
|
* Since each subsystem is loosely connected, we can stop, reboot, upgrade, and replace one server without interrupting other server's service.
* Since each subsystem is encapsulated, we can easily upgrade any subsystem without depending on other subsystem's implementation.
* We can develop a new version of software on an experimental server and test it on a staging server without interfering with the production server which is serving real users.
|-
| Scalability and high availability
|
* We can instantly deploy many instances of subsystem, such as a redirector server, in several different locations.
* Each instance of one subsystem can be given an individual IP address and these IP addresses can be accessed with a single FQDN - DNS load balancing.
* Since the API is based on HTTP, as its nature, a fail over to another IP address could be automatically done in a condition of multiply assigned IP address for a single FQDN.
* Each server can be a virtual machine in an existing server machine or can be a real machine depending on a resource strategy.
|-
| Openness
|
* We can easily deploy existing software products such as server monitoring tools and download statistics tools because of application neutral API
* Anyone can freely get and reuse any data provided by the repository and each subsystem.
* Any subsystem can be developed by any developers who are interested and experienced in the field. Web developers could develop a front-end for users. Programmers who are good at mathematics could develop a statistics calculation server. Operators who know network administration could develop a surveillance server.
|}


== Weakness ==
{| border=1 cellpadding=3
! Weakness
! Descriptions
|-
| Complexity
|
* Each subsystem is simple, but the entire system is complex.
* No one can know deeply the entire system from end to end because each subsystem is encapsulated and its detailed implementation is left to its developer or development team.
|}


== Generation ==

{| border=1 cellpadding=3
!Generation
!System
!Descriptions
|-
| 1
| Bouncer
|
* Apache, PHP, and MySQL based application
* "Bouncer is a database driven mirror management app..."
* http://wiki.osuosl.org/display/Bouncer/Home
|-
| 2
| MirrorBrain
|
* Apache, Python, Perl and PostgreSQL based application
* "MirrorBrain is a Download Redirector and Metalink Generator"
* http://mirrorbrain.org/
|- style="background-color: #8080ff;"
| 3
| Open Mirror Network System version 3
|
* Distributed computer network system
|-
|4
| Open Mirror Network System version 4
|
* In addition to version 3, (just some ideas)
** providing communication features between redirectors and each mirror server
** logging for preciser download statistics calculation
** taking timezone into account
|}


== The Next Generation ==
In the version 4, redirectors and mirror servers would communicate each other over SOAP or other protocol to make situation better.
* The availability of content
** Pooling
*** Redirector: "What files do you currently have?"
*** Mirror server: "I have this, this, ... and this file."
** Reporting
*** Mirror server: "I have gotten this file and deleted that file."
*** Redirector: "Thanks."

* The load of server
** Pooling
*** Redirector: "How is it going?"
*** Mirror server: "Well, I have been busy, could you throttle the amount of download requests towards me?"
** Reporting
*** Mirror server: "I have currently room for more download."
*** Redirector: "OK, I am increasing the amount of download requests for you."

To calculate preciser download statistics, installing a small program in every or most mirror servers and the program prepare download logs. Logging Servers gather download logs from each mirror server and calculate download statistics.

In addition to the communication, timezone would be also taken into account. E.g during a day time in Europa, some traffic from the inside of Europa could be routed to America where it is early morning.

== Developer Assignment ==
{| border=1 cellpadding=3
!Subsystem
!Developer
|-
| Mirror Server
| - (no need to be developed)
|-
| OpenOffice.org Web
| (website project, release engineer)
|-
| Download Concierge Server
| (website project, release engineer, mirror project)
|-
| Redirector Server
| (Persons who know redirector)
|-
| Scanner Server
| (Probably Tora)
|-
| Repository
| - (no need to be developed)
|-
| Logging Server
| (Probably Tora)
|-
| Statistics Calculation Server
| (Persons who know statistics, marketing project, or simply use existing software)
|-
| Surveillance Servers & User's PC
| (Persons who are interested in this area)
|}


== Schedule ==
(To be discussed.)

[[Category:Mirror Network]]
== Status of this document ==
* Rough ideas for brainstorming

== Concept ==
{| border=1 cellpadding=3
! Concept
! Descriptions
|-
| Distributed computer network
|
* One subsystem per one server basis deployment
* Each subsystem simply acts for own duty
* Deploying servers as a virtual and/or real machine
|-
| Simple, open API
|
* Platform, programming language, and application independent API
* Using HTTP GET to get a plain text file, XML file, or ZIP-archive file
* Anyone can freely develop own cool subsystems using data files
|-
| Maintainability
|
* Easy to upgrade each subsystem individually
* Easy to develop and test with experimental, staging, and production servers
|-
| Scalability & High Availability
|
* Easy to add servers
* DNS load balancing and automatic fail over
|-
| Surveillance & Alert
|
* Surveying malicious files and monitoring the status of servers
* Emergency stop in case of incident
|}

== Network Diagram ==
[[Image:open-mirror-network-system-draft-2009-12-17-2100.png]]

== Subsystem ==
The OpenOffice.org Open Mirror Network System consists of several subsystems. Each subsystem is loosely connected from one subsystem to another subsystem using the platform, programming language, and application independent API.

{| border=1 cellpadding=3
!Subsystem
!Descriptions
|-
| Mirror Server
|
* A mirror server is a server run by several types of entities such as universities, organizations, companies or individuals, which delivers files to users via HTTP, FTP, and/or RSYNC.
* Mirror servers are spread over the world and can be classified into three types: Master server, Tier-1 server, and Regular server.
* The master server is the origination from which files come. A Tier-1 server retrieves files from the Master server; A Regular server retrieve files from the Tire-1 server.
* The release engineering team uploads files to the master server.
|-
| OpenOffice.org Web
|
# A user visits the web site, download.openoffice.org.
# A JavaScript in the web page accesses to the Download Concierge Servers to get several types of small lists of deliverable and shows them to the user using AJAX.
# The user chooses a platform, product, version, language, and other items from the lists. The JavaScript helps the user find required files saying e.g. "You will need to install one of these language installers first and then install this language pack for your language." The user clicks on the shown links to start downloading file(s).
# One of the load-balanced Redirector Servers takes a user's download request to a closer Mirror Server.
# The Mirror Server sends back the requested file to the user.
|-
| Download Concierge Server
|
* A Download Concierge Server is a web server specially tuned for AJAX.
* This server fetches scan results from the Scanner Server, internally prepares several lists of items by categories such as platform, product, version, language, and other items, and responds with a small XML file upon a request from JavaScript embedded in a web page.
|-
| Redirector Server
|
* A Redirector Server fetches scan results from the Scanner Server and hits back a user's request to a closer server in which the requested file is available and puts log files on its web server.
|-
| Scanner Server
|
* A Scanner Server fetches repository data such as a list of URLs from the Repository, scans all mirror servers based on the URLs, and puts scan results and log files on its web server.
|-
| Repository
|
* The Repository is not a server but a set of data files such as a list of mirror sites, which are stored in distribution.openoffice.org and maintained by the staffs.
* GUI tools might be prepared for staffs to easily maintain the repository data.
|-
| Logging Server
|
* A Logging Server collects log files from several servers, concatenate some log files - simultaneously produced by load-balanced servers - into a single log file, and puts those log files on its web server.
|-
| Statistics Calculation Server
|
* A Statistics Calculation Server fetches log files from a Logging Server, accumulates logs by category such as location of user, choice of language, version of OpenOffice.org, calculates statistics, and provides users with the results through its web server.
* Any type of existing freely available software products could be used for this purpose.
* Or even, anyone can develop own cool program running on his server or PC.
|-
| Surveillance Servers & User's PC
|
* A Surveillance Server fetches repository data from the Repository and scan results from the Scanner Server and confirm the integrity of files kept in each mirror server by periodically and randomly scanning all mirror servers.
* This server also continuously checks the status of several other servers such as Redirector Server and Scanner Server.
* Likewise, User's PC also does the similar tasks.
* A Surveillance Server will inform Redirector Servers and alert Staffs and Administrators in case of incident.
* A PC user would inform suspicious phenomenon on the mailing list.
* Staffs can directly stop the Redirector Servers.
|}


== API ==
Each subsystem communicates by obtaining remote files via HTTP GET. There are basically three types of file:
{| border=1 cellpadding=3
! Type
! Example
|-
| A plain text file (.txt)
| Scan results (URL, path, type of entry (d: directory, f: file, l: symbolic link)
<pre>
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/ d
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f
...
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/ d
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f
...
</pre>
|-
| XML file (.xml)
| A list of mirror server
<pre>
<site location="Australia" code="au" type="regular" name="AussieHQ">
<uri set="extended" scheme="ftp">ftp://openoffice.mirror.aussiehq.net.au/pub/openoffice/</uri>
<uri set="extended" scheme="http">http://openoffice.mirror.aussiehq.net.au/</uri>
<uri set="extended" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice-extended/</uri>
<uri set="main" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice/</uri>
<contact name="Foo Bar">
<email>foo.bar@xxx.yyy</email>
</contact>
</site>
</pre>
|-
| ZIP-archive file (.zip)
|
* A set of a list of mirror server (.xml) and scan results (.txt)
* A set of Apache access log file (.txt) and error log file (.txt)
|}


== Strength ==
{| border=1 cellpadding=3
!Strengh
!Descriptions
|-
| Maintainability
|
* Since each subsystem is loosely connected, we can stop, reboot, upgrade, and replace one server without interrupting other server's service.
* Since each subsystem is encapsulated, we can easily upgrade any subsystem without depending on other subsystem's implementation.
* We can develop a new version of software on an experimental server and test it on a staging server without interfering with the production server which is serving real users.
|-
| Scalability and high availability
|
* We can instantly deploy many instances of subsystem, such as a redirector server, in several different locations.
* Each instance of one subsystem can be given an individual IP address and these IP addresses can be accessed with a single FGDN - DNS load balancing.
* Since the API is based on HTTP, as its nature, a fail over to another IP address could be automatically done in a condition of multiply assigned IP address for a single FQDN.
* Each server can be a virtual machine in an existing server machine or can be a real machine depending on a resource strategy.
|-
| Openness
|
* Anyone can freely get and reuse any data provided by the repository and each subsystem.
* Any subsystem can be developed by any developers who are interested and experienced in the field. Web developers could develop a front-end for users. Programmers who are good at mathematics could develop a statistics calculation server. Operators who know network administration could develop a surveillance server.
|}


== Weakness ==
{| border=1 cellpadding=3
! Weakness
! Descriptions
|-
| Complexity
|
* Each subsystem is simple, but the entire system is complex.
* No one can know deeply the entire system from end to end because each subsystem is encapsulated and its detailed implementation is left to its developer or development team.
|}


== Generation ==

{| border=1 cellpadding=3
!Generation
!System
!Descriptions
|-
| 1
| Bouncer
|
* Apache, PHP, and MySQL based application
* "Bouncer is a database driven mirror management app..."
* http://wiki.osuosl.org/display/Bouncer/Home
|-
| 2
| MirrorBrain
|
* Apache, Python, Perl and PostgreSQL based application
* "MirrorBrain is a Download Redirector and Metalink Generator"
* http://mirrorbrain.org/
|- style="background-color: #8080ff;"
| 3
| Open Mirror Network System version 3
|
* Distributed computer network system
|-
|4
| Open Mirror Network System version 4
|
* In addition to version 3, (just some ideas)
** providing communication features between redirectors and each mirror server
** logging for preciser download statistics calculation
** taking timezone into account
|}


== The Next Generation ==
In the version 4, redirectors and mirror servers would communicate each other over SOAP or other protocol to make situation better.
* The availability of content
** Pooling
*** Redirector: "What files do you currently have?"
*** Mirror server: "I have this, this, ... and this file."
** Reporting
*** Mirror server: "I have gotten this file and deleted that file."
*** Redirector: "Thanks."

* The load of server
** Pooling
*** Redirector: "How is it going?"
*** Mirror server: "Well, I have been busy, could you throttle the amount of download requests towards me?"
** Reporting
*** Mirror server: "I have currently room for more download."
*** Redirector: "OK, I am increasing the amount of download requests for you."

To calculate preciser download statistics, installing a small program in every or most mirror servers and the program prepare download logs. Logging Servers gather download logs from each mirror server and calculate download statistics.

In addition to the communication, timezone would be also taken into account. E.g during a day time in Europa, some traffic from the inside of Europa could be routed to America where it is early morning.

== Developer Assignment ==
{| border=1 cellpadding=3
!Subsystem
!Developer
|-
| Mirror Server
| - (no need to be developed)
|-
| OpenOffice.org Web
| (website project, release engineer)
|-
| Download Concierge Server
| (website project, release engineer, mirror project)
|-
| Redirector Server
| (Persons who know redirector)
|-
| Scanner Server
| (Probably Tora)
|-
| Repository
| - (no need to be developed)
|-
| Logging Server
| (Probably Tora)
|-
| Statistics Calculation Server
| (Persons who know statistics, marketing project, or simply use existing software)
|-
| Surveillance Servers & User's PC
| (Persons who are interested in this area)
|}


== Schedule ==
(To be discussed.)

[[Category:Mirror Network]]
== Status of this document ==
* Rough ideas for brainstorming

== Concept ==
{| border=1 cellpadding=3
! Concept
! Descriptions
|-
| Distributed computer network
|
* One subsystem per one server basis deployment
* Each subsystem simply acts for own duty
* Deploying servers as a virtual and/or real machine
|-
| Simple, open API
|
* Platform, programming language, and application independent API
* Using HTTP GET to get a plain text file, XML file, or ZIP-archive file
* Anyone can freely develop own cool subsystems using data files
|-
| Maintainability
|
* Easy to upgrade each subsystem individually
* Easy to develop and test with experimental, staging, and production servers
|-
| Scalability & High Availability
|
* Easy to add servers
* DNS load balancing and automatic fail over
|-
| Surveillance & Alert
|
* Surveying malicious files and monitoring the status of servers
* Emergency stop in case of incident
|}

== Network Diagram ==
[[Image:open-mirror-network-system-draft-2009-12-17-2100.png]]

== Subsystem ==
The OpenOffice.org Open Mirror Network System consists of several subsystems. Each subsystem is loosely connected from one subsystem to another subsystem using the platform, programming language, and application independent API.

{| border=1 cellpadding=3
!Subsystem
!Descriptions
|-
| Mirror Server
|
* A mirror server is a server run by several types of entities such as universities, organizations, companies or individuals, which delivers files to users via HTTP, FTP, and/or RSYNC.
* Mirror servers are spread over the world and can be classified into three types: Master server, Tier-1 server, and Regular server.
* The master server is the origination from which files come. A Tier-1 server retrieves files from the Master server; A Regular server retrieve files from the Tire-1 server.
* The release engineering team uploads files to the master server.
|-
| OpenOffice.org Web
|
# A user visits the web site, download.openoffice.org.
# A JavaScript in the web page accesses to the Download Concierge Servers to get several types of small lists of deliverable and shows them to the user using AJAX.
# The user chooses a platform, product, version, language, and other items from the lists. The JavaScript helps the user find required files saying e.g. "You will need to install one of these language installers first and then install this language pack for your language." The user clicks on the shown links to start downloading file(s).
# One of the load-balanced Redirector Servers takes a user's download request to a closer Mirror Server.
# The Mirror Server sends back the requested file to the user.
|-
| Download Concierge Server
|
* A Download Concierge Server is a web server specially tuned for AJAX.
* This server fetches scan results from the Scanner Server, internally prepares several lists of items by categories such as platform, product, version, language, and other items, and responds with a small XML file upon a request from JavaScript embedded in a web page.
|-
| Redirector Server
|
* A Redirector Server fetches scan results from the Scanner Server and hits back a user's request to a closer server in which the requested file is available and puts log files on its web server.
|-
| Scanner Server
|
* A Scanner Server fetches repository data such as a list of URLs from the Repository, scans all mirror servers based on the URLs, and puts scan results and log files on its web server.
|-
| Repository
|
* The Repository is not a server but a set of data files such as a list of mirror sites, which are stored in distribution.openoffice.org and maintained by the staffs.
* GUI tools might be prepared for staffs to easily maintain the repository data.
|-
| Logging Server
|
* A Logging Server collects log files from several servers, concatenate some log files - simultaneously produced by load-balanced servers - into a single log file, and puts those log files on its web server.
|-
| Statistics Calculation Server
|
* A Statistics Calculation Server fetches log files from a Logging Server, accumulates logs by category such as location of user, choice of language, version of OpenOffice.org, calculates statistics, and provides users with the results through its web server.
* Any type of existing freely available software products could be used for this purpose.
* Or even, anyone can develop own cool program running on his server or PC.
|-
| Surveillance Servers & User's PC
|
* A Surveillance Server fetches repository data from the Repository and scan results from the Scanner Server and confirm the integrity of files kept in each mirror server by periodically and randomly scanning all mirror servers.
* This server also continuously checks the status of several other servers such as Redirector Server and Scanner Server.
* Likewise, User's PC also does the similar tasks.
* A Surveillance Server will inform Redirector Servers and alert Staffs and Administrators in case of incident.
* A PC user would inform suspicious phenomenon on the mailing list.
* Staffs can directly stop the Redirector Servers.
|}


== API ==
Each subsystem communicates by obtaining remote files via HTTP GET. There are basically three types of file:
{| border=1 cellpadding=3
! Type
! Example
|-
| A plain text file (.txt)
| Scan results (URL, path, type of entry (d: directory, f: file, l: symbolic link)
<pre>
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/ d
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f
...
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/ d
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f
http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f
...
</pre>
|-
| XML file (.xml)
| A list of mirror server
<pre>
<site location="Australia" code="au" type="regular" name="AussieHQ">
<uri set="extended" scheme="ftp">ftp://openoffice.mirror.aussiehq.net.au/pub/openoffice/</uri>
<uri set="extended" scheme="http">http://openoffice.mirror.aussiehq.net.au/</uri>
<uri set="extended" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice-extended/</uri>
<uri set="main" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice/</uri>
<contact name="Foo Bar">
<email>foo.bar@xxx.yyy</email>
</contact>
</site>
</pre>
|-
| ZIP-archive file (.zip)
|
* A set of a list of mirror server (.xml) and scan results (.txt)
* A set of Apache access log file (.txt) and error log file (.txt)
|}


== Strength ==
{| border=1 cellpadding=3
!Strengh
!Descriptions
|-
| Maintainability
|
* Since each subsystem is loosely connected, we can stop, reboot, upgrade, and replace one server without interrupting other server's service.
* Since each subsystem is encapsulated, we can easily upgrade any subsystem without depending on other subsystem's implementation.
* We can develop a new version of software on an experimental server and test it on a staging server without interfering with the production server which is serving real users.
|-
| Scalability and high availability
|
* We can instantly deploy many instances of subsystem, such as a redirector server, in several different locations.
* Each instance of one subsystem can be given an individual IP address and these IP addresses can be accessed with a single FGDN - DNS load balancing.
* Since the API is based on HTTP, as its nature, a fail over to another IP address could be automatically done in a condition of multiply assigned IP address for a single FQDN.
* Each server can be a virtual machine in an existing server machine or can be a real machine depending on a resource strategy.
|-
| Openness
|
* Anyone can freely get and reuse any data provided by the repository and each subsystem.
* Any subsystem can be developed by any developers who are interested and experienced in the field. Web developers could develop a front-end for users. Programmers who are good at mathematics could develop a statistics calculation server. Operators who know network administration could develop a surveillance server.
|}


== Weakness ==
{| border=1 cellpadding=3
! Weakness
! Descriptions
|-
| Complexity
|
* Each subsystem is simple, but the entire system is complex.
* No one can know deeply the entire system from end to end because each subsystem is encapsulated and its detailed implementation is left to its developer or development team.
|}


== Generation ==

{| border=1 cellpadding=3
!Generation
!System
!Descriptions
|-
| 1
| Bouncer
|
* Apache, PHP, and MySQL based application
* "Bouncer is a database driven mirror management app..."
* http://wiki.osuosl.org/display/Bouncer/Home
|-
| 2
| MirrorBrain
|
* Apache, Python, Perl and PostgreSQL based application
* "MirrorBrain is a Download Redirector and Metalink Generator"
* http://mirrorbrain.org/
|- style="background-color: #8080ff;"
| 3
| Open Mirror Network System version 3
|
* Distributed computer network system
|-
|4
| Open Mirror Network System version 4
|
* In addition to version 3, (just some ideas)
** providing communication features between redirectors and each mirror server
** logging for preciser download statistics calculation
** taking timezone into account
|}


== The Next Generation ==
In the version 4, redirectors and mirror servers would communicate each other over SOAP or other protocol to make situation better.
* The availability of content
** Pooling
*** Redirector: "What files do you currently have?"
*** Mirror server: "I have this, this, ... and this file."
** Reporting
*** Mirror server: "I have gotten this file and deleted that file."
*** Redirector: "Thanks."

* The load of server
** Pooling
*** Redirector: "How is it going?"
*** Mirror server: "Well, I have been busy, could you throttle the amount of download requests towards me?"
** Reporting
*** Mirror server: "I have currently room for more download."
*** Redirector: "OK, I am increasing the amount of download requests for you."

To calculate preciser download statistics, installing a small program in every or most mirror servers and the program prepare download logs. Logging Servers gather download logs from each mirror server and calculate download statistics.

In addition to the communication, timezone would be also taken into account. E.g during a day time in Europa, some traffic from the inside of Europa could be routed to America where it is early morning.

== Developer Assignment ==
{| border=1 cellpadding=3
!Subsystem
!Developer
|-
| Mirror Server
| - (no need to be developed)
|-
| OpenOffice.org Web
| (website project, release engineer)
|-
| Download Concierge Server
| (website project, release engineer, mirror project)
|-
| Redirector Server
| (Persons who know redirector)
|-
| Scanner Server
| (Probably Tora)
|-
| Repository
| - (no need to be developed)
|-
| Logging Server
| (Probably Tora)
|-
| Statistics Calculation Server
| (Persons who know statistics, marketing project, or simply use existing software)
|-
| Surveillance Servers & User's PC
| (Persons who are interested in this area)
|}


== Schedule ==
(To be discussed.)

[[Category:Mirror Network]]

Rough ideas for the OpenOffice.org Open Mirror Network System version 3.0

2009-12-29T04:00:58Z

Tora: Corrected some typos

Rough ideas for the OpenOffice.org Open Mirror Network System version 3.0

2009-12-17T15:56:34Z

Tora: Added items to the next generation

File:Open-mirror-network-system-draft-2009-12-17-2100.png

2009-12-17T15:05:47Z

Tora:

2009-09-01T02:15:59Z

Tora: Initial check-in

Mirror/Stat/OOo3.1.1rc2

2009-08-22T13:33:35Z

Tora:

= Statistics on OpenOffice.org 3.1.1rc2 =

A set of 3.1.1rc2 files are available in the directory extended/3.1.1rc2/ of the extended [http://distribution.openoffice.org/mirrors/#mirrors mirror servers], which is marked with "E".

== Proposals ==
{| border=1 cellpadding=10 style="background-color: #eeffee;"
|<div>If many users want to have Windows versions of OpenOffice.org release candidate first, the following idea might be effective.

For instance,
# Upload Windows versions and source tar balls to the master server one day in the afternoon.
# Upload the rest of files in the next morning.
# Announce the availability in the afternoon.
</div>
|}

The reason why Windows versions of release candidate take much time to become available in the mirror servers comes from the order of file transfer with 'rsync' command widely used among mirror server administrators to obtain remote files from other servers.

A set of release candidate files are, normally, 30 times of a normal set of release files in total size.

<div style="margin-left: 30px;">
{| border=1 style="text-align: center;"
!Release
!Directory
!Number of files
!Total file size
|-
|3.1.1rc2
|extended/3.1.1rc2
|1020
|65 GB
|-
|DEV300_m55
|extended/developer/DEV300_m55
|51
|2 GB
|-
|3.1.0
|stable/3.1.0
|27
|2 GB
|}
</div>

'rsync' command first get a list of files from a remote server and compares it with locally existing files. If some of or all of files are missing in a local server, then start to get the new files.

'rsync' command normally gets files in an alphabetical order of file names. In a case of 3.1.1rc2 listed below, the order could be 3.1.1rc2_md5sums.txt first, then BrOOo_3.1.1rc2_, ... , eventually Win32Intel and src. Assume that the rate of file transfer was 1 Giga bytes per hour, it might take 65 hours to get all of files and Windows version would become available in the last 10 hours.

{| border=1 cellpadding=10 style="background-color: #eeffee;"
|<div>The idea above would give the mirror network a chance to circulate most desired files first and then to do the rest of files later.
</div>
|}

The order could be more appropriately arranged. For example, First English, German, French, and some language versions for active language projects, second Windows versions, finally the rests.

There might be no need to take account of this fashion for neither regular releases nor developer builds since their total file size is relatively small.
 

== Figures ==

=== 3.1.1rc2 ===
<pre>
Catetory AvgSize Count TtlSize
==========================================================================
3.1.1rc2_md5sums.txt 0 1 0
BrOOo_3.1.1rc2_20090820 100 18 1804
OOo-SDK_3.1.1rc2_20090820 9 8 77
OOo_3.1.1rc2_20090820_LinuxIntel_install 158 84 13308
OOo_3.1.1rc2_20090820_LinuxIntel_langpack 19 194 3735
OOo_3.1.1rc2_20090820_LinuxX86-64_install 177 28 4960
OOo_3.1.1rc2_20090820_LinuxX86-64_langpack 19 194 3737
OOo_3.1.1rc2_20090820_MacOSXIntel_install 167 28 4678
OOo_3.1.1rc2_20090820_SolarisSparc_install 172 56 9665
OOo_3.1.1rc2_20090820_SolarisSparc_langpack 18 97 1761
OOo_3.1.1rc2_20090820_Solarisx86_install 163 56 9176
OOo_3.1.1rc2_20090820_Solarisx86_langpack 17 97 1742
OOo_3.1.1rc2_20090820_Win32Intel_install 139 56 7828
OOo_3.1.1rc2_20090820_Win32Intel_langpack 29 97 2879
OOo_3.1.1rc2_20090820_src 64 6 387
==========================================================================
Total 64 1020 65742
</pre>
Where
* AvgSize: average file size (Mega bytes)
* Count: the number of files
* TtlSize: subtotal of the sizes (Mega bytes)

The following UNIX command would get a list of files in the above directory.
<blockquote><code>
rsync rsync://xxxxx/extended/3.1.1rc2/ > [[Media:mirror_a_list_of_extended_3.1.1rc2.txt|mirror_a_list_of_extended_3.1.1rc2.txt]]
</code></blockquote>
Then the following UNIX command would produce the above figures from the list of files.
<blockquote><code>
perl [[Media:get_statistics_of_uploaded_files_in_the_mirror-2009-08-22_pl.txt|get_statistics_of_yyyyy_pl.txt]] mirror_a_list_of_extended_3.1.1rc2.txt
</code></blockquote>

=== DEV300_m55 ===
<pre>
Catetory AvgSize Count TtlSize
==========================================================================
DEV300_m55_md5sums.txt 0 1 0
OOo-Dev-SDK_DEV300_m55 9 8 72
OOo-Dev_DEV300_m55_LinuxIntel_install 151 2 303
OOo-Dev_DEV300_m55_LinuxIntel_langpack 20 8 163
OOo-Dev_DEV300_m55_LinuxX86-64_install 166 2 332
OOo-Dev_DEV300_m55_LinuxX86-64_langpack 20 8 163
OOo-Dev_DEV300_m55_MacOSXIntel_install 165 1 165
OOo-Dev_DEV300_m55_SolarisSparc_install 193 1 193
OOo-Dev_DEV300_m55_SolarisSparc_langpack 20 4 80
OOo-Dev_DEV300_m55_Solarisx86_install 182 1 182
OOo-Dev_DEV300_m55_Solarisx86_langpack 19 4 79
OOo-Dev_DEV300_m55_Win32Intel_install 144 1 144
OOo-Dev_DEV300_m55_Win32Intel_langpack 21 4 84
OOo-Dev_DEV300_m55_src 66 6 401
==========================================================================
Total 46 51 2368
</pre>
The following UNIX command would get a list of files in the above directory.
<blockquote><code>
rsync rsync://xxxxx/extended/developer/DEV300_m55/ > [[Media:mirror_a_list_of_extended_developer_DEV300_m55.txt|mirror_a_list_of_extended_developer_DEV300_m55.txt]]
</code></blockquote>

=== 3.1.0 ===
<pre>
Catetory AvgSize Count TtlSize
==========================================================================
OOo-SDK_3.1.0 9 8 77
OOo_3.1.0_LinuxIA64_install 189 1 189
OOo_3.1.0_LinuxIntel_install 159 3 478
OOo_3.1.0_LinuxX86-64_install 158 2 316
OOo_3.1.0_MacOSXIntel_install 166 1 166
OOo_3.1.0_SolarisSparc_install 173 2 347
OOo_3.1.0_Solarisx86_install 164 2 329
OOo_3.1.0_Win32Intel_install 141 2 282
OOo_3.1.0_src 64 6 387
==========================================================================
Total 95 27 2575
</pre>

The following UNIX command would get a list of files in the above directory.
<blockquote><code>
rsync rsync://xxxxx/stable/3.1.0/ > [[Media:mirror_a_list_of_stable_3.1.0.txt|mirror_a_list_of_stable_3.1.0.txt]]
</code></blockquote>

File:Get statistics of uploaded files in the mirror-2009-08-22 pl.txt

2009-08-22T13:31:40Z

Tora:

File:Mirror a list of stable 3.1.0.txt

2009-08-22T13:11:39Z

Tora:

File:Mirror a list of extended developer DEV300 m55.txt

2009-08-22T13:10:38Z

Tora:

2009-08-22T05:40:33Z

Tora: Created page with '= Statistics on OpenOffice.org 3.1.1rc2 = A set of 3.1.1rc2 files are available in the directory extended/3.1.1rc2/ of extended [http://distribution.openoffice.org/mirrors/#mirr…'

= Statistics on OpenOffice.org 3.1.1rc2 =

A set of 3.1.1rc2 files are available in the directory extended/3.1.1rc2/ of extended [http://distribution.openoffice.org/mirrors/#mirrors mirror servers], which is marked with "E".

== Proposals ==
{| border=1 cellpadding=10 style="background-color: #eeffee;"
|<div>If many users want to have Windows versions of OpenOffice.org release candidate first, the following idea might be effective.

For instance,
# Upload Windows versions and source tar balls to the master server one day in the afternoon.
# Upload the rest of files in the next morning.
# Announce the availability in the afternoon.
</div>
|}

The reason why Windows versions of release candidate take much time to become available in the mirror servers comes from the order of file transfer with 'rsync' command, which is widely used among mirror server administrators to obtain remote files from other servers.

A set of release candidate files are, normally, 30 times of a normal set of release files in total size.

<div style="margin-left: 30px;">
{| border=1 style="text-align: center;"
!Release
!Directory
!Number of files
!Total file size
|-
|3.1.1rc2
|extended/3.1.1rc2
|1020
|65 GB
|-
|DEV300_m55
|extended/developer/DEV300_m55
|51
|2 GB
|-
|3.1.0
|stable/3.1.0
|27
|2 GB
|}
</div>

'rsync' command first get a list of files from a remote server and compares it with locally existing files. If some of or all of files are missing in a local server, then start to get the new files.

'rsync' command normally gets files in an alphabetical order of file names. In a case of 3.1.1rc2 listed below, the order could be 3.1.1rc2_md5sums.txt first, then BrOOo_3.1.1rc2_, ... , eventually Win32Intel and src. Assume that the rate of file transfer was 1 Giga bytes per hour, it might take 65 hours to get all of files and Windows version would become available in the last 3 hours.

{| border=1 cellpadding=10 style="background-color: #eeffee;"
|<div>The idea above would give the mirror network a chance to circulate most desired files first and then to do the rest of files later.
</div>
|}

 

== Figures ==

=== 3.1.1rc2 ===
<pre>
Catetory AvgSize Count TtlSize
==========================================================================
3.1.1rc2_md5sums.txt 0 1 0
BrOOo_3.1.1rc2_20090820 100 18 1804
OOo-SDK_3.1.1rc2_20090820 9 8 77
OOo_3.1.1rc2_20090820_LinuxIntel_install 158 84 13308
OOo_3.1.1rc2_20090820_LinuxIntel_langpack 19 194 3735
OOo_3.1.1rc2_20090820_LinuxX86-64_install 177 28 4960
OOo_3.1.1rc2_20090820_LinuxX86-64_langpack 19 194 3737
OOo_3.1.1rc2_20090820_MacOSXIntel_install 167 28 4678
OOo_3.1.1rc2_20090820_SolarisSparc_install 172 56 9665
OOo_3.1.1rc2_20090820_SolarisSparc_langpack 18 97 1761
OOo_3.1.1rc2_20090820_Solarisx86_install 163 56 9176
OOo_3.1.1rc2_20090820_Solarisx86_langpack 17 97 1742
OOo_3.1.1rc2_20090820_Win32Intel_install 139 56 7828
OOo_3.1.1rc2_20090820_Win32Intel_langpack 29 97 2879
OOo_3.1.1rc2_20090820_src 64 6 387
==========================================================================
Total 64 1020 65742
</pre>
Where
* AvgSize: average file size (Mega bytes)
* Count: the number of files
* TtlSize: subtotal of the sizes (Mega bytes)

The following UNIX command would get a list of files in the above directory.
<blockquote><code>
rsync rsync://xxxxx/extended/3.1.1rc2/ > [[Media:mirror_a_list_of_extended_3.1.1rc2.txt|mirror_a_list_of_extended_3.1.1rc2.txt]]
</code></blockquote>
Then the following UNIX command would produce the above figures from the list of files.
<blockquote><code>
perl [[Media:get_statistics_of_uploaded_files_in_the_mirror-2009-08-22.pl.txt|get_statistics_of_yyyyy.pl.txt]] mirror_a_list_of_extended_3.1.1rc2.txt
</code></blockquote>

=== DEV300_m55 ===
<pre>
Catetory AvgSize Count TtlSize
==========================================================================
DEV300_m55_md5sums.txt 0 1 0
OOo-Dev-SDK_DEV300_m55 9 8 72
OOo-Dev_DEV300_m55_LinuxIntel_install 151 2 303
OOo-Dev_DEV300_m55_LinuxIntel_langpack 20 8 163
OOo-Dev_DEV300_m55_LinuxX86-64_install 166 2 332
OOo-Dev_DEV300_m55_LinuxX86-64_langpack 20 8 163
OOo-Dev_DEV300_m55_MacOSXIntel_install 165 1 165
OOo-Dev_DEV300_m55_SolarisSparc_install 193 1 193
OOo-Dev_DEV300_m55_SolarisSparc_langpack 20 4 80
OOo-Dev_DEV300_m55_Solarisx86_install 182 1 182
OOo-Dev_DEV300_m55_Solarisx86_langpack 19 4 79
OOo-Dev_DEV300_m55_Win32Intel_install 144 1 144
OOo-Dev_DEV300_m55_Win32Intel_langpack 21 4 84
OOo-Dev_DEV300_m55_src 66 6 401
==========================================================================
Total 46 51 2368
</pre>
The following UNIX command would get a list of files in the above directory.
<blockquote><code>
rsync rsync://xxxxx/extended/developer/DEV300_m55/ > [[Media:mirror_a_list_of_extended_developer_DEV300_m55.txt|mirror_a_list_of_extended_developer_DEV300_m55.txt]]
</code></blockquote>

=== 3.1.0 ===
<pre>
Catetory AvgSize Count TtlSize
==========================================================================
OOo-SDK_3.1.0 9 8 77
OOo_3.1.0_LinuxIA64_install 189 1 189
OOo_3.1.0_LinuxIntel_install 159 3 478
OOo_3.1.0_LinuxX86-64_install 158 2 316
OOo_3.1.0_MacOSXIntel_install 166 1 166
OOo_3.1.0_SolarisSparc_install 173 2 347
OOo_3.1.0_Solarisx86_install 164 2 329
OOo_3.1.0_Win32Intel_install 141 2 282
OOo_3.1.0_src 64 6 387
==========================================================================
Total 95 27 2575
</pre>

The following UNIX command would get a list of files in the above directory.
<blockquote><code>
rsync rsync://xxxxx/stable/3.1.0/ > [[Media:mirror_a_list_of_stable_3.1.0.txt|mirror_a_list_of_stable_3.1.0.txt]]
</code></blockquote>