Difference between revisions of "Rough ideas for the OpenOffice.org Open Mirror Network System version 3.0"
(Added items to the next generation) |
(Make some subsection titles plural) |
||
(4 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
* Rough ideas for brainstorming | * Rough ideas for brainstorming | ||
− | == | + | == Concepts == |
{| border=1 cellpadding=3 | {| border=1 cellpadding=3 | ||
! Concept | ! Concept | ||
Line 10: | Line 10: | ||
| | | | ||
* One subsystem per one server basis deployment | * One subsystem per one server basis deployment | ||
− | * Each subsystem | + | * Each subsystem concentrates on solely its own duty |
* Deploying servers as a virtual and/or real machine | * Deploying servers as a virtual and/or real machine | ||
+ | |- | ||
+ | | Robustness | ||
+ | | | ||
+ | * Hard to lose data such as configurations, histories, and/or logs | ||
+ | * Data is kept in several subsystems as a replica, not kept in a single centralized server machine | ||
+ | * No need to keep irrelevant, uninteresting data in any subsystem | ||
|- | |- | ||
| Simple, open API | | Simple, open API | ||
| | | | ||
− | * Platform, programming language, and application | + | * Platform, programming language, and application neutral API |
− | * | + | * HTTP GET to retrieve remote data as a plain text, XML, and/or ZIP-archived file |
− | * Anyone can freely develop own | + | * Anyone can freely develop their own subsystems using data files via the API |
|- | |- | ||
| Maintainability | | Maintainability | ||
| | | | ||
− | * Easy to upgrade each subsystem | + | * Easy to upgrade each subsystem separately |
− | * Easy to develop and test with experimental | + | * Easy to develop and test with experimental and staging servers while production servers serve users |
|- | |- | ||
| Scalability & High Availability | | Scalability & High Availability | ||
| | | | ||
− | * Easy to add servers | + | * Easy to add servers like a small device and to replace them if broken |
* DNS load balancing and automatic fail over | * DNS load balancing and automatic fail over | ||
|- | |- | ||
Line 38: | Line 44: | ||
[[Image:open-mirror-network-system-draft-2009-12-17-2100.png]] | [[Image:open-mirror-network-system-draft-2009-12-17-2100.png]] | ||
− | == | + | == Subsystems == |
The OpenOffice.org Open Mirror Network System consists of several subsystems. Each subsystem is loosely connected from one subsystem to another subsystem using the platform, programming language, and application independent API. | The OpenOffice.org Open Mirror Network System consists of several subsystems. Each subsystem is loosely connected from one subsystem to another subsystem using the platform, programming language, and application independent API. | ||
Line 62: | Line 68: | ||
| Download Concierge Server | | Download Concierge Server | ||
| | | | ||
− | * A Download Concierge Server is a web server | + | * A Download Concierge Server is a web server specially tuned for AJAX. |
* This server fetches scan results from the Scanner Server, internally prepares several lists of items by categories such as platform, product, version, language, and other items, and responds with a small XML file upon a request from JavaScript embedded in a web page. | * This server fetches scan results from the Scanner Server, internally prepares several lists of items by categories such as platform, product, version, language, and other items, and responds with a small XML file upon a request from JavaScript embedded in a web page. | ||
|- | |- | ||
Line 100: | Line 106: | ||
== API == | == API == | ||
− | Each subsystem communicates by obtaining remote files via HTTP GET. There | + | Each subsystem communicates by obtaining remote files via HTTP GET. There would be basically three types of file: |
{| border=1 cellpadding=3 | {| border=1 cellpadding=3 | ||
! Type | ! Type | ||
Line 132: | Line 138: | ||
</pre> | </pre> | ||
|- | |- | ||
− | | ZIP- | + | | ZIP-archived file (.zip) |
| | | | ||
* A set of a list of mirror server (.xml) and scan results (.txt) | * A set of a list of mirror server (.xml) and scan results (.txt) | ||
Line 153: | Line 159: | ||
| | | | ||
* We can instantly deploy many instances of subsystem, such as a redirector server, in several different locations. | * We can instantly deploy many instances of subsystem, such as a redirector server, in several different locations. | ||
− | * Each instance of one subsystem can be given an individual IP address and these IP addresses can be accessed with a single | + | * Each instance of one subsystem can be given an individual IP address and these IP addresses can be accessed with a single FQDN - DNS load balancing. |
* Since the API is based on HTTP, as its nature, a fail over to another IP address could be automatically done in a condition of multiply assigned IP address for a single FQDN. | * Since the API is based on HTTP, as its nature, a fail over to another IP address could be automatically done in a condition of multiply assigned IP address for a single FQDN. | ||
* Each server can be a virtual machine in an existing server machine or can be a real machine depending on a resource strategy. | * Each server can be a virtual machine in an existing server machine or can be a real machine depending on a resource strategy. | ||
Line 159: | Line 165: | ||
| Openness | | Openness | ||
| | | | ||
+ | * We can easily deploy existing software products such as server monitoring tools and download statistics tools because of application neutral API | ||
* Anyone can freely get and reuse any data provided by the repository and each subsystem. | * Anyone can freely get and reuse any data provided by the repository and each subsystem. | ||
* Any subsystem can be developed by any developers who are interested and experienced in the field. Web developers could develop a front-end for users. Programmers who are good at mathematics could develop a statistics calculation server. Operators who know network administration could develop a surveillance server. | * Any subsystem can be developed by any developers who are interested and experienced in the field. Web developers could develop a front-end for users. Programmers who are good at mathematics could develop a statistics calculation server. Operators who know network administration could develop a surveillance server. | ||
Line 172: | Line 179: | ||
| | | | ||
* Each subsystem is simple, but the entire system is complex. | * Each subsystem is simple, but the entire system is complex. | ||
− | * No one can know deeply the entire system from | + | * No one can know deeply the entire system from end to end because each subsystem is encapsulated and its detailed implementation is left to its developer or development team. |
|} | |} | ||
<p></p> | <p></p> | ||
− | == | + | == Generations == |
{| border=1 cellpadding=3 | {| border=1 cellpadding=3 |
Latest revision as of 06:59, 29 December 2009
Contents
Status of this document
- Rough ideas for brainstorming
Concepts
Concept | Descriptions |
---|---|
Distributed computer network |
|
Robustness |
|
Simple, open API |
|
Maintainability |
|
Scalability & High Availability |
|
Surveillance & Alert |
|
Network Diagram
Subsystems
The OpenOffice.org Open Mirror Network System consists of several subsystems. Each subsystem is loosely connected from one subsystem to another subsystem using the platform, programming language, and application independent API.
Subsystem | Descriptions |
---|---|
Mirror Server |
|
OpenOffice.org Web |
|
Download Concierge Server |
|
Redirector Server |
|
Scanner Server |
|
Repository |
|
Logging Server |
|
Statistics Calculation Server |
|
Surveillance Servers & User's PC |
|
API
Each subsystem communicates by obtaining remote files via HTTP GET. There would be basically three types of file:
Type | Example |
---|---|
A plain text file (.txt) | Scan results (URL, path, type of entry (d: directory, f: file, l: symbolic link)
http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/ d http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f http://mirror.aarnet.edu.au/pub/openoffice /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f ... http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/ d http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe f http://openoffice.mirror.aussiehq.net.au /stable/3.1.1/OOo_3.1.1_Win32Intel_install_wJRE_en-US.exe f ... |
XML file (.xml) | A list of mirror server
<site location="Australia" code="au" type="regular" name="AussieHQ"> <uri set="extended" scheme="ftp">ftp://openoffice.mirror.aussiehq.net.au/pub/openoffice/</uri> <uri set="extended" scheme="http">http://openoffice.mirror.aussiehq.net.au/</uri> <uri set="extended" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice-extended/</uri> <uri set="main" scheme="rsync">rsync://openoffice.mirror.aussiehq.net.au/openoffice/</uri> <contact name="Foo Bar"> <email>foo.bar@xxx.yyy</email> </contact> </site> |
ZIP-archived file (.zip) |
|
Strength
Strengh | Descriptions |
---|---|
Maintainability |
|
Scalability and high availability |
|
Openness |
|
Weakness
Weakness | Descriptions |
---|---|
Complexity |
|
Generations
Generation | System | Descriptions |
---|---|---|
1 | Bouncer |
|
2 | MirrorBrain |
|
3 | Open Mirror Network System version 3 |
|
4 | Open Mirror Network System version 4 |
|
The Next Generation
In the version 4, redirectors and mirror servers would communicate each other over SOAP or other protocol to make situation better.
- The availability of content
- Pooling
- Redirector: "What files do you currently have?"
- Mirror server: "I have this, this, ... and this file."
- Reporting
- Mirror server: "I have gotten this file and deleted that file."
- Redirector: "Thanks."
- Pooling
- The load of server
- Pooling
- Redirector: "How is it going?"
- Mirror server: "Well, I have been busy, could you throttle the amount of download requests towards me?"
- Reporting
- Mirror server: "I have currently room for more download."
- Redirector: "OK, I am increasing the amount of download requests for you."
- Pooling
To calculate preciser download statistics, installing a small program in every or most mirror servers and the program prepare download logs. Logging Servers gather download logs from each mirror server and calculate download statistics.
In addition to the communication, timezone would be also taken into account. E.g during a day time in Europa, some traffic from the inside of Europa could be routed to America where it is early morning.
Developer Assignment
Subsystem | Developer |
---|---|
Mirror Server | - (no need to be developed) |
OpenOffice.org Web | (website project, release engineer) |
Download Concierge Server | (website project, release engineer, mirror project) |
Redirector Server | (Persons who know redirector) |
Scanner Server | (Probably Tora) |
Repository | - (no need to be developed) |
Logging Server | (Probably Tora) |
Statistics Calculation Server | (Persons who know statistics, marketing project, or simply use existing software) |
Surveillance Servers & User's PC | (Persons who are interested in this area) |
Schedule
(To be discussed.)