Difference between revisions of "Infrastructure Requirements"

From Apache OpenOffice Wiki
Jump to: navigation, search
m
 
(14 intermediate revisions by 7 users not shown)
Line 5: Line 5:
 
== Prioritized Item List ==
 
== Prioritized Item List ==
  
# Framework
+
# Framework / Core
## lightweight role / user administration / SSH key management<br>[incl. authorization / authentication interfaces for cmdline / web based apps]<br>(e.g. LDAP directory, could be placed on the SVN/CVS server)
+
#* Scalability
## structuring/projects/categories/namespaces<br>for tools like bugzilla, wiki, ...
+
#* lightweight role / user administration / SSH key management<br>[incl. authorization / authentication interfaces for cmdline / web based apps / persistent login]<br>(e.g. LDAP directory, could be placed on the SVN/CVS server)
# Version Control System for source code<br>[for web publishing and documentation see CMS]
+
#* SubProjects and SubProject Categories:
#* CVS
+
#** The OpenOffice.org project is being organized into different SubProjects having their own DNS entries for WebSpace and Mailinglists eg. xml.openoffice.org; '''must'''; see also DNS
#* [[SVNMigration|Migration to SVN]] planned<br> (NB. auth for SVN usually done using HTTPS or SSH)
+
#** Subprojects are classified into the 3 subproject catagories Accepted Project, Incubator Project, Native Language Project; '''must'''
#* [svk/distributed later?]
+
#** Access rights configurable for individual Subprojects; '''must'''
#* admin access for cvsroot manipulation, deletion, move
+
#** A <b>Domain Developer</b> role gives access rights to all subprojects of the openoffice.org project; '''must'''
#* cvsup, anoncvs, viewcvs, notification mails, ...
+
#** documents and files feature per subproject where documents can be uploaded; edited etc.; '''must'''
#* simple permission system (s. 1)
+
#** search feature - search in all OpenOffice.org subprojects or inside current OpenOffice.org subproject; '''must'''
#* authenticated access via SSH tunnel (and/or direct?)
+
#** project announcements; '''should'''
#* Preferrably sudo root access for selected admins
+
#* Sign-on
#* [bonsai, lxr, fisheye / CIA, ...]
+
#** Single sign-on across the different applications; '''must'''
# Bug Tracking System (Bugzilla)
+
#** API to enable third-party applications to use the single sign-on; '''must'''
#* central configuration
+
#* Site Search
#* export [/ import] interface
+
#** covering web pages, issues, mailing lists; '''must'''
 +
#** no of pages: ~450k
 +
# DNS
 +
#* Required Domain Name Service DNS Entries:
 +
#** DNS Records for SubProjects of openoffice.org eg. gsl.openoffice.org, api.openoffice.org, tools.openoffice.org, de.openoffice.org. This includes Entries for Native Language Communities for example de.openoffice.org. All entries must have IP Address under which the sub-projects web content is being served and also MX records for the projects mailing lists.; '''must'''
 +
#** DNS Entry for SubDomain services.openoffice.org in control of DNS Server operated by StarOffice department, for tools like the Environment Information System (EIS); '''must'''
 +
#** DNS Entry for download.openoffice.org serving own web content; '''must'''
 +
#** DNS Entry for support.openoffice.org serving own web content; '''must'''
 +
#** DNS Entry for contributing.openoffice.org serving own web content; '''must'''
 +
#** DNS Entry for about.openoffice.org serving own web content; '''must'''
 +
# Web pages
 +
#* subprojects need own DNS entries, see DNS; '''must'''; external sites contain links to web content of subprojects which we would otherwise break
 +
#* special purpose web pages on own website (download, support, contributing, about), see DNS; '''must'''; <i>If not otherwise possible these may be configured as individual subprojects but it´s preferable to have the www.openoffice.org main site plus these special sites belong to just one subproject named website where content developers can join</i>.
 +
#* Navigation Bar on top available to all sub-projects and special purpose websites; '''must'''
 +
#* Navigation Bar left, project specific; '''must'''
 +
#* global info boxes on right side of MyPage, with eg. Community Council Announcements, Download section
 +
#* project specific announcements on MyPage; '''must'''
 +
#** Default Content of left navigation bar per subproject but overrideable per subdirectory; '''must'''
 +
#** Migration for files specifying content of left navigation bar in subdirectories; '''nice to have'''
 +
#** Content modification:
 +
#* all webpages in the repository are automatically surrounded by standard header (including login possibility, logo and top navigation bar), procject/directory specific left navigation bar and standard footer (including eg. copyright information); '''must'''
 +
#* possibility to get HTML pages without the surrounding navigation, header and footer content via /nonav/ in URL; '''must'''
 +
# Version Control System for source code<br>[For web publishing and documentation see CMS.]<br>[Some time ago we created a requirements list for a new OpenOffice.org SCM system. This can be found here: http://wiki.services.openoffice.org/wiki/SCM_Requirements.]<br>The most important ones (distilled from the mentioned list):
 +
#* Clients:
 +
#** Clients must be available for all our supported platforms; '''must'''; <font color='green'>OK for Subversion</font>
 +
#* Branches:
 +
#** SCM system needs to easily support a multitude of branches; '''must'''; <font color='green'>OK for Subversion</font>
 +
#** Merge tracking should be either available or can be implemented by hand; '''must'''; <font color='green'>OK for Subversion</font>;
 +
#* Authentication:
 +
#** Passwords must be securely transmitted to prevent malicious commits; '''must'''
 +
#* Import:
 +
#** Import from CVS needs to be history preserving; '''must'''
 +
#** Import from CVS should be customizable (for example leave out old branches etc); '''should'''
 +
#** Import from CVS needs to be doable in days during final migration phase; '''must'''; <font color='green'>OK for Subversion with fsfs backend</font>; <font color='red'>not OK for Subversion with bdb backend</font>; <font color='red'>not OK for Mercurial</font>;
 +
#* Performance:
 +
#** The performance of common used operations (checkout, commit, log, merge) should not be significantly slower than CVS as presented by CN; '''must'''
 +
#* Replication:
 +
#** There needs to be a way to replicate the repository to save bandwidth; '''must'''
 +
#* Binary files, LF/CR conversions
 +
#** Proper handling of binary files; '''must'''; <font color='green'>OK for Subversion</font>;
 +
#** LF/CR conversions; '''should''';  <font color='green'>OK for Subversion</font>;
 +
#* admin access for deletion, move
 +
#* functionality similar to cvsup, anoncvs, viewcvs, notification mails; '''must'''
 +
#* simple permission system; '''should'''
 +
#* Sudo root access for selected admins; '''should'''
 +
#* Functionality similar to bonsai, lxr, fisheye / CIA etc.; '''should'''
 +
#Bugtracking
 +
#* Number of issues inside OOo:
 +
#** total 82600 (closed 57693)
 +
#** number of distinct submitters: 18500
 +
#** number of distinct assignees: 1600
 +
#** number of attached files: 49000
 +
#* States:
 +
#** ASSIGNED, CLOSED, NEW, REOPEN, RESOLVED, STARTED, UNCONFIRMED, VERIFIED; '''must'''
 +
#* Resolution:
 +
#** DUPLICATE, FIXED, INVALID, LATER, REMIND, WONTFIX, WORKSFORME; '''must''' (Further details about the workflow and the relationship between State and Resolution http://www.openoffice.org/scdocs/issue_lifecycle.html)
 +
#* Types:
 +
#** DEFECT, ENHANCEMENT, FEATURE, PATCH, TASK; '''must''' (may be PATCH can be dropped later -> Was a request for CollabNet)
 +
#* Additional fields and values:
 +
#* Import- and export of data:
 +
#** XML; '''must'''
 +
#** remove export as 'Word'/'Excel'; '''must'''
 +
#* Attachments:
 +
#** Possibility to attach any type of file to the issue, eg. OpenOffice.org documents, pictures, source code, source code patches; '''must'''
 +
#* Keywords:
 +
#** List of Keywords or tags to assign to issues as a method to organize QA work; '''must'''
 +
#* central configuration; '''must'''
 +
#* Email-notification of modified issues; '''must'''
 +
#* Email (offline) access to bugtracking (comment, status change, ...); '''should'''
 +
#* One issue-tracking instance for more than one project; '''must'''
 
# Wiki (CMS)
 
# Wiki (CMS)
#* for webcontent
+
#* MediaWiki, version ? ; '''should'''
#* allow branding
+
#* Required extensions/features:
#* [localized UI]
+
#** DisplayTitle feature ; '''must''' for Docs
#* versioned
+
#** DynamicPageList; '''should'''
#* namespaces
+
#** Syntax Highlighter (GeSHi) ; '''must''' for Docs
#* reuse of content (templates, server-side-include)
+
#** Breadcrumbs ; '''must''' for Docs
 +
#** Hierarchical pages ; '''must''' for Docs;
 +
#** Category Tree; '''should'''
 +
#** wfGFlashExtension; '''should'''
 +
#** wfLabeledSectionTransclusion ; '''must''' for Docs
 +
#** Parser extensions (own implementation) ; '''must''' for Docs
 +
#* Possibility of installing own extensions ; '''must'''
 +
#* Number of categories: 200
 +
#* Number of pages: 7723 within 2000 content pages (Docs will soon increase this number by 900 (DevGuide) and possibly another 2,000 (Help)
 +
#* Number of uploaded files: 2200
 +
#* Number of edit/day: 7
 +
#* Number of users: 10181<br>Further details http://wiki.services.openoffice.org/wiki/Special:Statistics
 +
#* allow branding; '''must'''
 +
#* localized UI; '''should'''
 +
#* versioned; '''must'''
 +
#* namespaces; '''must'''
 +
#* reuse of content (templates, server-side-include); '''must'''
 +
#* publication routes for wiki-based documentation (ODF/PDF); '''should'''
 +
#* localization of wiki-based content; '''should'''
 
# Mailing Lists
 
# Mailing Lists
#* different types (moderated, private)
+
#* Types:
#* non-site-member subscription
+
#**unmoderated;open to anyone(mostly unused at collab); '''must'''
#* Archive
+
#**discuss;subcribed users Post/Read allowed,archive Read allowed others moderated (most common); '''must'''
#* SPAM / Virus protected
+
#**moderated;Post always moderated; Read archive; '''must'''
#* [RSS]
+
#**allowed poster:
#* [webbased forums]
+
#**domain wide allowed poster:
#* NNTP
+
#**sublist;subcribed to another List; gathering global Information eg; issues@<project>.openoffice.org to ML issues@openoffice; '''must'''
#* attachment stripping
+
#* non-site-member subscription (external archives); '''must'''
#* customizable headers, footers, robot replies
+
#* Archiving; '''must'''
 +
#* Toplevel mailinglists: allcvs@openoffice.org; '''must'''
 +
#* migrating current Archives; '''must'''
 +
#* round about 850 Mailinglists with different types to migrate; '''must'''
 +
#* SPAM / Virus protected (Important especially for agenda@council.openoffice.org. See "Comm b/w CC and CMs" below); '''must'''
 +
#* [RSS] for easy syndication of new from various parts of the site, including the home page and mac port pages.; '''should'''
 +
#* webbased forums; '''should'''
 +
#* NNTP; '''must'''
 +
#* attachment stripping; '''must'''
 +
#* customizable headers, footers, robot replies; '''must'''
 
# Audit
 
# Audit
 +
# Communication between Community Council and Community Members
 +
#* Protect [http://council.openoffice.org/servlets/BrowseList?listName=agenda&by=date&from=2007-10-01&to=2007-10-31&first=1&count=22 agenda@council.openoffice.org] against spams and make it easy-to-post and easy-to-find community-proposals which should not be buried in spams.
 +
#* Additional infrastructure where CC can listen to contributors, developers and other Community members and talk with them daily.
 +
# L10n and T9n for about 100 languages
 +
#* Storage for:
 +
#** Pootle
 +
#** translation memories
 +
#** glossaries
 +
#** KeyID builds
 +
#** extensions translation process
 +
#** documentation translation process
 +
#* Management tools
  
== Unprioritized List ==
+
== Uncategorized List ==
  
* localization of UI parts (bugtracking, webpages, ...) and process for new locales
+
* localization of UI parts (bugtracking, webpages, ...) and process for new locales '''must'''
* reporting / stats / logfiles
+
* reporting / stats / logfiles, '''must'''
* Search via Google, Yahoo, ...
+
* Search via Google, Yahoo, ..., '''must'''
* mail forwarding (xxx@openoffice.org)
+
* mail forwarding (xxx@openoffice.org), '''must'''
* IP / domain blocking
+
* IP / domain blocking, '''must'''
* blog / planet
+
* blog / planet, '''should'''
* download, documents & files structured in folders
+
* download, documents & files structured in folders '''must'''
* surveys / votes / polls
+
* surveys / votes / polls '''must'''
* general open-ness for integration of scripting (PHP, Perl, ...)
+
* general open-ness for integration of scripting (PHP, Perl, ...) '''must'''
 +
* databases (*SQL) '''should'''
 +
* provide SSL logon for security reasons '''must'''
 +
* raise the limit for mail forwarding from 2 MB to 10 MB '''should'''
 +
 
 +
== Migration process ==
 +
* controlled public access to the staging server '''must'''
  
 
== Key Numbers ==
 
== Key Numbers ==

Latest revision as of 15:56, 27 October 2009

The following list contains the major functionalities expected from the OpenOffice.org infrastructure. The bullet points have been collected with focus on software developerment but are also applicable for the user oriented needs.

Prioritized Item List

  1. Framework / Core
    • Scalability
    • lightweight role / user administration / SSH key management
      [incl. authorization / authentication interfaces for cmdline / web based apps / persistent login]
      (e.g. LDAP directory, could be placed on the SVN/CVS server)
    • SubProjects and SubProject Categories:
      • The OpenOffice.org project is being organized into different SubProjects having their own DNS entries for WebSpace and Mailinglists eg. xml.openoffice.org; must; see also DNS
      • Subprojects are classified into the 3 subproject catagories Accepted Project, Incubator Project, Native Language Project; must
      • Access rights configurable for individual Subprojects; must
      • A Domain Developer role gives access rights to all subprojects of the openoffice.org project; must
      • documents and files feature per subproject where documents can be uploaded; edited etc.; must
      • search feature - search in all OpenOffice.org subprojects or inside current OpenOffice.org subproject; must
      • project announcements; should
    • Sign-on
      • Single sign-on across the different applications; must
      • API to enable third-party applications to use the single sign-on; must
    • Site Search
      • covering web pages, issues, mailing lists; must
      • no of pages: ~450k
  2. DNS
    • Required Domain Name Service DNS Entries:
      • DNS Records for SubProjects of openoffice.org eg. gsl.openoffice.org, api.openoffice.org, tools.openoffice.org, de.openoffice.org. This includes Entries for Native Language Communities for example de.openoffice.org. All entries must have IP Address under which the sub-projects web content is being served and also MX records for the projects mailing lists.; must
      • DNS Entry for SubDomain services.openoffice.org in control of DNS Server operated by StarOffice department, for tools like the Environment Information System (EIS); must
      • DNS Entry for download.openoffice.org serving own web content; must
      • DNS Entry for support.openoffice.org serving own web content; must
      • DNS Entry for contributing.openoffice.org serving own web content; must
      • DNS Entry for about.openoffice.org serving own web content; must
  3. Web pages
    • subprojects need own DNS entries, see DNS; must; external sites contain links to web content of subprojects which we would otherwise break
    • special purpose web pages on own website (download, support, contributing, about), see DNS; must; If not otherwise possible these may be configured as individual subprojects but it´s preferable to have the www.openoffice.org main site plus these special sites belong to just one subproject named website where content developers can join.
    • Navigation Bar on top available to all sub-projects and special purpose websites; must
    • Navigation Bar left, project specific; must
    • global info boxes on right side of MyPage, with eg. Community Council Announcements, Download section
    • project specific announcements on MyPage; must
      • Default Content of left navigation bar per subproject but overrideable per subdirectory; must
      • Migration for files specifying content of left navigation bar in subdirectories; nice to have
      • Content modification:
    • all webpages in the repository are automatically surrounded by standard header (including login possibility, logo and top navigation bar), procject/directory specific left navigation bar and standard footer (including eg. copyright information); must
    • possibility to get HTML pages without the surrounding navigation, header and footer content via /nonav/ in URL; must
  4. Version Control System for source code
    [For web publishing and documentation see CMS.]
    [Some time ago we created a requirements list for a new OpenOffice.org SCM system. This can be found here: http://wiki.services.openoffice.org/wiki/SCM_Requirements.]
    The most important ones (distilled from the mentioned list):
    • Clients:
      • Clients must be available for all our supported platforms; must; OK for Subversion
    • Branches:
      • SCM system needs to easily support a multitude of branches; must; OK for Subversion
      • Merge tracking should be either available or can be implemented by hand; must; OK for Subversion;
    • Authentication:
      • Passwords must be securely transmitted to prevent malicious commits; must
    • Import:
      • Import from CVS needs to be history preserving; must
      • Import from CVS should be customizable (for example leave out old branches etc); should
      • Import from CVS needs to be doable in days during final migration phase; must; OK for Subversion with fsfs backend; not OK for Subversion with bdb backend; not OK for Mercurial;
    • Performance:
      • The performance of common used operations (checkout, commit, log, merge) should not be significantly slower than CVS as presented by CN; must
    • Replication:
      • There needs to be a way to replicate the repository to save bandwidth; must
    • Binary files, LF/CR conversions
      • Proper handling of binary files; must; OK for Subversion;
      • LF/CR conversions; should; OK for Subversion;
    • admin access for deletion, move
    • functionality similar to cvsup, anoncvs, viewcvs, notification mails; must
    • simple permission system; should
    • Sudo root access for selected admins; should
    • Functionality similar to bonsai, lxr, fisheye / CIA etc.; should
  5. Bugtracking
    • Number of issues inside OOo:
      • total 82600 (closed 57693)
      • number of distinct submitters: 18500
      • number of distinct assignees: 1600
      • number of attached files: 49000
    • States:
      • ASSIGNED, CLOSED, NEW, REOPEN, RESOLVED, STARTED, UNCONFIRMED, VERIFIED; must
    • Resolution:
    • Types:
      • DEFECT, ENHANCEMENT, FEATURE, PATCH, TASK; must (may be PATCH can be dropped later -> Was a request for CollabNet)
    • Additional fields and values:
    • Import- and export of data:
      • XML; must
      • remove export as 'Word'/'Excel'; must
    • Attachments:
      • Possibility to attach any type of file to the issue, eg. OpenOffice.org documents, pictures, source code, source code patches; must
    • Keywords:
      • List of Keywords or tags to assign to issues as a method to organize QA work; must
    • central configuration; must
    • Email-notification of modified issues; must
    • Email (offline) access to bugtracking (comment, status change, ...); should
    • One issue-tracking instance for more than one project; must
  6. Wiki (CMS)
    • MediaWiki, version ? ; should
    • Required extensions/features:
      • DisplayTitle feature ; must for Docs
      • DynamicPageList; should
      • Syntax Highlighter (GeSHi) ; must for Docs
      • Breadcrumbs ; must for Docs
      • Hierarchical pages ; must for Docs;
      • Category Tree; should
      • wfGFlashExtension; should
      • wfLabeledSectionTransclusion ; must for Docs
      • Parser extensions (own implementation) ; must for Docs
    • Possibility of installing own extensions ; must
    • Number of categories: 200
    • Number of pages: 7723 within 2000 content pages (Docs will soon increase this number by 900 (DevGuide) and possibly another 2,000 (Help)
    • Number of uploaded files: 2200
    • Number of edit/day: 7
    • Number of users: 10181
      Further details http://wiki.services.openoffice.org/wiki/Special:Statistics
    • allow branding; must
    • localized UI; should
    • versioned; must
    • namespaces; must
    • reuse of content (templates, server-side-include); must
    • publication routes for wiki-based documentation (ODF/PDF); should
    • localization of wiki-based content; should
  7. Mailing Lists
    • Types:
      • unmoderated;open to anyone(mostly unused at collab); must
      • discuss;subcribed users Post/Read allowed,archive Read allowed others moderated (most common); must
      • moderated;Post always moderated; Read archive; must
      • allowed poster:
      • domain wide allowed poster:
      • sublist;subcribed to another List; gathering global Information eg; issues@<project>.openoffice.org to ML issues@openoffice; must
    • non-site-member subscription (external archives); must
    • Archiving; must
    • Toplevel mailinglists: allcvs@openoffice.org; must
    • migrating current Archives; must
    • round about 850 Mailinglists with different types to migrate; must
    • SPAM / Virus protected (Important especially for agenda@council.openoffice.org. See "Comm b/w CC and CMs" below); must
    • [RSS] for easy syndication of new from various parts of the site, including the home page and mac port pages.; should
    • webbased forums; should
    • NNTP; must
    • attachment stripping; must
    • customizable headers, footers, robot replies; must
  8. Audit
  9. Communication between Community Council and Community Members
    • Protect agenda@council.openoffice.org against spams and make it easy-to-post and easy-to-find community-proposals which should not be buried in spams.
    • Additional infrastructure where CC can listen to contributors, developers and other Community members and talk with them daily.
  10. L10n and T9n for about 100 languages
    • Storage for:
      • Pootle
      • translation memories
      • glossaries
      • KeyID builds
      • extensions translation process
      • documentation translation process
    • Management tools

Uncategorized List

  • localization of UI parts (bugtracking, webpages, ...) and process for new locales must
  • reporting / stats / logfiles, must
  • Search via Google, Yahoo, ..., must
  • mail forwarding (xxx@openoffice.org), must
  • IP / domain blocking, must
  • blog / planet, should
  • download, documents & files structured in folders must
  • surveys / votes / polls must
  • general open-ness for integration of scripting (PHP, Perl, ...) must
  • databases (*SQL) should
  • provide SSL logon for security reasons must
  • raise the limit for mail forwarding from 2 MB to 10 MB should

Migration process

  • controlled public access to the staging server must

Key Numbers

Per day the main site (actually the accelerators) receives 4-6M hits / 100,000 - 150,000 visits and sends 40-60GB. In peaks like the OpenOffice.org 2.0 launch there were > 16M hits per day with more than 400,000 visits.

The mail server has to be prepared to handle 50 msg/sec and 300 parallel sessions when the rising tide flows in.

See also

Personal tools