Difference between revisions of "Efforts/Package Restructuring/Modelling"

From Apache OpenOffice Wiki
Jump to: navigation, search
m (More Complex: Two Features in two Languages)
(Composition)
 
(44 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
To dos:
 +
* Add images to examples.
 +
* Define algorithms.
 +
 +
==The [[Terms/Product|Product]] Pipeline==
 +
[[Image:Product Pipeline.jpg|center]]
 +
 +
The product pipeline should leverage "make" at the various stages to "build" intermediate deliverables, to ensure that dependencies are reflected correctly and to allow the re-usage of intermediate deliverables for different targets products.
 +
 
==De-Composition==
 
==De-Composition==
Looking at an installed OOo and its files, registry entries etc., we can see, that all these entities belong to one or multiple of the following categories, such that they provide or depend on it
+
Looking at an installed OOo and its files, registry entries etc., we can see, that all these entities belong to one or multiple of the following domains.
# brand
+
;Brand: This is everything somehow related to a specific [[Terms/Product|product]], such as StarOffice or OpenOffice.org. E.g. the [[Splash_Screens|splash screen]] typically belongs the a brand.
# Operating System
+
;Operating System: This is everything somehow related to a specific operating system as Solaris, MS Windows or Linux. Shell or batch scripts, such as #!/bin/sh or .bat are Operating System specific, but typically not machine dependent.
# Machine Architecture / interpreter
+
;Machine Architecture: This is everything related to a native (machine) interpreter, such as x86 or SPARC, as well as its ABI.
# localization
+
;Localization: This is everything related to a human language, such as English or German.
 +
;Version: This is everything related to a particular version. Actually every entity has a version, such as SRC680_m245.
  
The consequences of this observation are, that entities unrelated to one or multiple categories, but shared by products differing in these categories, are identical and thus may be re-used for creating, changing or updating products respectively installed products.
+
{{Uno/Note}} Ideally the [[Terms/Part|parts]] (packages) of two different [[Terms/Deliverable|deliverables]] are reflecting the redundancy of the according installations, e.g. if OOo and BrOOo have 90% in common, than their [[Terms/Deliverable|deliverables]] need to share 90% of their [[Terms/Part|parts]] (packages).
 +
 
 +
{{Uno/Note}} If we assume that the part format (e.g. rpm, deb, pkg, msi) implies the operating system, we may leave out Operating System as an own domain, as no part may be shared between any two OS.  The differentiation into Operating System and Machine Architecture becomes the moment interesting, we are going to support multiple machine architectures of the same Operating System, e.g. Linux x86 and x64.
 +
 
 +
The consequences of this observation are, that entities unrelated to one or multiple categories, but shared by [[Terms/Product|products]] differing in these categories, are identical and thus may be re-used for creating, changing or updating [[Terms/Product|products]] respectively installed [[Terms/Product|products]].
  
 
Additionally every entity exactly belongs to one
 
Additionally every entity exactly belongs to one
* feature
+
* [[Terms/Feature|feature]]
as otherwise it would not be needed for any feature, thus it would not be needed at all. Certainly features may be related such that they require one another, even if this is not noticeable in the [[Terms/Product View|Product View]].
+
as otherwise it would not be needed for any [[Terms/Feature|feature]], thus it would not be needed at all. Certainly [[Terms/Feature|features]] may be related such that they require one another, even if this is not noticeable in the [[Terms/Product View|Product View]].
 +
 
 +
Every [[Terms/Product|product]] we define can be assembled out of [[Terms/Part|parts]] (e.g. RPM packages), which are created along the above categories.
 +
 
 +
===Examples===
 +
====One [[Terms/Feature|Feature]] in Two Languages====
 +
In the simplest case, we have exactly one [[Terms/Part|part]] for every [[Terms/Product|product]], e.g. for the Writer:
 +
* The '''writer''' [[Terms/Part|part]].
 +
This [[Terms/Part|part]] may include all entities for the Writer for Linux x86, in English, utilizing the OpenOffice.org brand.
 +
 
 +
Later on, we would like to provide the Writer not only in English, but may be also in German, which leads to the following [[Terms/Product|products]]:
 +
* '''writer_en''' - The Writer in English.
 +
* '''writer_de''' - The Writer in German.
 +
Obviously we now need to take care of any redundancy in these [[Terms/Product|products]]. We can do so by
 +
* naming / placing entities (e.g. files) differently, or
 +
* by sharing them.
 +
 
 +
For obvious reasons, that sharing approach is better, leading to the following [[Terms/Part|parts]]:
 +
* '''writer''' - This is everything but the locale.
 +
* '''writer_en''' - This is the English locale only.
 +
* '''writer_en''' - This is the German locale only.
 +
But wait, now we have another problem. What if the user only installs one of the [[Terms/Part|parts]]? This would be an incomplete / inconsistent and useless installation.
 +
 
 +
On UNIX and alike platforms, [[Terms/Part|parts]] are typically called packages. Package managers typically use dependencies and virtual packages (or provisions) to address the consistency problem. So, lets try to set the above [[Terms/Part|parts]] into relationship using dependencies and provisions.
 +
 
 +
The '''writer''' ("all but the locale") [[Terms/Part|part]] needs at least one writer locale [[Terms/Part|part]] to be installed, to function correctly, but must not depend on any concrete locale (otherwise one would need to install all locales ;-). So we may want to model the dependencies and provisions as follows:
 +
* '''writer''': depends on '''writer_lcl''', provides '''writer'''
 +
* '''writer_en''': depends on '''writer''', provides '''writer_en''', '''writer_lcl'''
 +
* '''writer_de''': depends on '''writer''', provides '''writer_de''', '''writer_lcl'''
 +
By now, we can only install any of the above [[Terms/Part|parts]], if we install a completing [[Terms/Part|part]] as well, though we certainly can install all three [[Terms/Part|parts]].
 +
 
 +
====Two [[Terms/Feature|Features]]/Two Layers====
 +
In the previous example, our only [[Terms/Feature|feature]] was the Writer. Lets now have two [[Terms/Feature|features]], e.g.
 +
* Writer, and
 +
* Calc.
 +
 
 +
As you may know, OOo has a [[Architecture|layered Architecture]], therefore thus providing these two [[Terms/Feature|features]] as two [[Terms/Product|products]]:
 +
* '''writer''' - The Writer [[Terms/Product|product]].
 +
* '''calc''' - The Calc [[Terms/Product|product]].
 +
leads to many same entities to be installed twice. So, obviously we need again to take care of this redundancy. And again, we can do so by
 +
* naming / placing these entities (e.g. files) differently, or
 +
* by sharing them.
 +
 
 +
For obvious reasons, that sharing approach is better, leading to the following [[Terms/Part|parts]]:
 +
* '''writer''' - Everything needed for Writer, not being needed for Calc.
 +
* '''calc''' - Everything needed for Calc, not being needed for Writer.
 +
* '''basis''' - Everything needed for both, Calc and Writer.
 +
Again, we now need to take care of completeness and consistency.
 +
 
 +
So, lets define some dependencies:
 +
* '''writer''': depends on basis, provides '''writer'''
 +
* '''calc''': depends on basis, provides '''calc'''
 +
* '''basis''': depends on nothing, provides '''basis'''
 +
Now, the '''writer''' [[Terms/Part|part]] respectively the '''calc''' [[Terms/Part|part]] can only be installed if the '''basis''' [[Terms/Part|part]] is going to be installed as well.
 +
 
 +
By now, we can only install a [[Terms/Feature|feature]] (Writer or Calc) in a complete way.
 +
 
 +
====Two [[Terms/Feature|Features]]/Two Layers in Two Languages====
 +
So, lets see what happens, if we have two [[Terms/Feature|features]], e.g. Writer and Calc, and two localizations, e.g. English and German.
 +
 
 +
Applying the above separation, we get the following [[Terms/Part|parts]]:
 +
* '''writer'''
 +
* '''calc'''
 +
* '''basis'''
 +
* '''writer_en'''
 +
* '''writer_de'''
 +
* '''calc_en'''
 +
* '''calc_de'''
 +
* '''basis_en'''
 +
* '''basis_de'''
 +
 
 +
So, what do the dependencies should now look like? By combining the previous dependencies orthogonally we get
 +
* '''writer''': depends on '''writer_lcl''', '''basis''' provides '''writer'''
 +
* '''calc''': depends on '''calc_lcl''', '''basis''' provides '''calc'''
 +
* '''basis''': depends on '''basis_lcl''' provides '''basis''',
 +
* '''writer_en''': depends on '''writer''', '''basis_en''' provides '''writer_lcl''', '''writer_en''',
 +
* '''writer_de''': depends on '''writer''', '''basis_de''' provides '''writer_lcl''', '''writer_de''',
 +
* '''calc_en''': depends on '''calc''', '''basis_en''' provides '''calc_lcl''', '''calc_en''',
 +
* '''calc_de''': depends on '''calc''', '''basis_de''' provides '''calc_lcl''', '''calc_de''',
 +
* '''basis_en''': depends on '''basis''' provides '''basis_lcl''', '''basis_en''',
 +
* '''basis_de''': depends on '''basis''' provides '''basis_lcl''', '''basis_de'''.
 +
To ensure consistence among layers, we need to make locale packages (([Terms/Part|parts]])  (e.g. '''calc_en''') dependent on any lower layer locale packages ([[Terms/Part|parts]]) (e.g. '''basis_en''').
 +
 
 +
Looking at it, we the get the following minimal (and consistent) feature installations:
 +
# '''writer''', '''writer_en''', '''basis''', '''basis_en''', == Writer English
 +
# '''writer''', '''writer_de''', '''basis''', '''basis_de''', == Writer German
 +
# '''calc''', '''calc_en''', '''basis''', '''basis_en''', == Calc English
 +
# '''calc''', '''calc_de''', '''basis''', '''basis_de''' == Calc German
 +
 
 +
Which we can combine to:
 +
# 1+2: '''writer''', '''writer_en''', '''writer_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer English/German,
 +
# 1+3: '''writer''', '''writer_en''', '''calc''', '''calc_en''', '''basis''', '''basis_en''' == Writer/Calc English
 +
# 1+4: '''writer''', '''writer_en''', '''calc''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer English / Calc German
 +
# 2+3: '''writer''', '''writer_de''', '''calc''', '''calc_en''', '''basis''', '''basis_de''', '''basis_em''' == Writer German / Calc English
 +
# 2+4: '''writer''', '''writer_de''', '''calc''', '''calc_de''', '''basis''', '''basis_de''' == Writer/Calc German
 +
# 3+4: '''calc''', '''calc_en''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Calc English/German
 +
# 1+2+3: '''writer''', '''writer_en''', '''writer_de''', '''calc''', '''calc_en''', '''basis''', '''basis_en''', '''basis_de''' == Writer English/German, Calc English
 +
# 1+2+4: '''writer''', '''writer_en''', '''writer_de''', '''calc''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer English/German, Calc German
 +
# 1+3+4: '''writer''', '''writer_en''', '''calc''', '''calc_en''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer English, Calc English/German
 +
# 1+2+3+4: '''writer''', '''writer_en''', '''writer_de''', '''calc''', '''calc_en''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer/Calc English/German
 +
# 2+3+4: '''writer''', '''writer_de''', '''calc''', '''calc_en''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer German, Calc English/German
 +
 
 +
{{Uno/Note}} Obviously we need to be able to deal in our implementations with the fact, that different [[Terms/Feature|features]] may only be available in different locales. This is unavoidable, if we want to separate [[Terms/Feature|features]]. Or to phrase it differently, all complete (depends, provides) sets of [[Terms/Part|parts]] need to be valid.
 +
 
 +
====Complex Example====
 +
Splitting the Writer along some the of above identified categories (brand, OS/Architecture, locale), we get the following [[Terms/Part|parts]]:
 +
* '''writer''' - Everything of the Writer, which does not belong to any of the categories.
 +
* '''writer___en''' - All English localization content not belonging to any other category.
 +
* '''writer_OOo__en''' - All English localization content for the OOo brand.
 +
* '''writer___de''' - All German localization content not belonging to any other category.
 +
* '''writer_OOo__de''' - All German localization content for the OOo brand.
 +
* '''writer__linux''' - All Linux specific stuff.
 +
 
 +
To ensure consistency, we may model the dependencies and provisions as follows:
 +
* '''writer''': depends on '''writer___OS/Arch''', '''writer_brnd__lcl''' provides '''writer'''
 +
* '''writer___en''': depends on '''writer''' provides '''writer___en'''
 +
* '''writer_OOo__en''': depends on '''writer___en''' provides '''writer_brnd__lcl'''
 +
* '''writer___de''': depends on '''writer''' provides '''writer___de'''
 +
* '''writer_OOo__de''': depends on '''writer___de''' provides '''writer_brnd__lcl'''
 +
* '''writer__linux''': depends on '''writer''' provides '''writer__OS/Arch'''
 +
 
 +
This is one way to model the dependencies, but is it optimal? It at least seems to be minimal. But what happens if we can get rid of any brand specific localization, thus the '''writer_OOo__de''' respectively the '''writer_OOo__en''' [[Terms/Part|parts]]? Even we don't change anything in '''writer''' it seems that we need to update it, because of changing dependencies, '''writer''' would now directly depend on '''writer___lcl'''.
 +
 
 +
===Rules of Thumb===
 +
* A [[Terms/Part|part]] may depend on lower layer [[Terms/Part|parts]].
 +
* A [[Terms/Part|part]] may depend on more general [[Terms/Part|parts]] (e.g. the '''writer_OOo__en''' [[Terms/Part|part]] depends on the '''writer_OOo''' and the '''writer___en''' [[Terms/Part|parts]])!
 +
* A more specific [[Terms/Part|part]] implies any more general [[Terms/Part|parts]] (e.g. the '''writer_OOo__en''' [[Terms/Part|part]] implies the '''writer___en''' [[Terms/Part|part]])!
 +
* A virtual [[Terms/Part|part]] is more general than a concrete [[Terms/Part|part]].
 +
* Every [[Terms/Part|part]] not completing any other [[Terms/Part|part]] is a feature.
 +
 
 +
===Algorithms===
 +
* Minimal dependencies.
 +
* Minimal provisions.
 +
* Minimal updates because of changing dependencies / provisions.
 +
* Optimal dependencies / provisions.
 +
* Installation variants of one set of [[Terms/Part|parts]].
  
 
==Composition==
 
==Composition==
Model products by setting them into  
+
* [[Terms/Deliverable|deliverable]]
 +
* [[Terms/Product|product]]
 +
* [[ProductPacker]]
 +
* [[Terms/Product Configuration|configuration]]
 +
* [[Terms/Feature|feature]]
 +
* [[Terms/Part|part]]
 +
* Domain
 +
* RootDomain
 +
* NullDomain (AKA "NOARCH")
 +
 
 +
Model [[Terms/Product|products]] by setting them into  
 
* inheritance, respectively
 
* inheritance, respectively
 
* instantiation (template)
 
* instantiation (template)
 
relationship.
 
relationship.
 +
 +
product -features/languages-> product configuration -container/format-> deliverable
 +
 +
* product: OOo, BrOOo
 +
* platform: unxlngi6.pro
 +
* version: build (e.g. OOH680_m7)
 +
* configuration: language(s) (e.g. en-US, de)
 +
* format: container/packager (e.g. directory/rpm or tar.gz/deb)
 +
 +
===Requirements===
 +
* Packaging by Dependency (make driven)
 +
* Producting by Dependency (make driven)
 +
* Support for Distributed Packaging.
 +
* Concurrent Packaging
 +
* Concurrent Producting
 +
* Parts must be creatable independently (e.g. a new localization or brand).
 +
* Parts must be self describing.
 +
* Support for pre-build parts
 +
 +
===Packaging===
 +
Taking a look at how packages (like RPM) are typically organized, we can see, that the intermediate [[Terms/Deliverable|deliverables]] should express their needs and offers in terms of
 +
* dependencies against a (virtual) package, as well as by listing the
 +
* provisions.
  
 
===Inheritance===
 
===Inheritance===
Line 24: Line 206:
 
===Example===
 
===Example===
 
<pre>
 
<pre>
 +
 +
"Categories.scp":
 +
  Category os_arch {
 +
  }
 +
 +
  Category locale {
 +
  }
 +
 +
"Framework.scp":
 +
  #include "categories.scp"
 +
 +
  Part {
 +
    Name: "Framework"
 +
    ...
 +
  }
 +
 +
"Writer.scp"
 +
  #include "categories.scp"
 +
 +
  Part {
 +
    Name: "Writer"
 +
    Description: "A Wordprocessor"
 +
    Depends: Framework, locale
 +
    Producer: "OOo"
 +
    Category: os_arch
 +
 +
    Files: ...
 +
    Registry: ...
 +
  }
 +
 +
"Writer-en-US.scp"
 +
  #include "categories.scp"
 +
 +
  Part {
 +
    Name: "en-US"
 +
    Description: "English localization for Writer"
 +
    Completes: "file://Writer.scp"
 +
    Producer: "OOo"
 +
    Category: locale
 +
 +
    Files: ...
 +
    Registry: ...
 +
  }
 +
 +
 +
"Writer-de.scp"
 +
  #include "categories.scp"
 +
 +
  Part {
 +
    Name: "de"
 +
    Description: "German localization for Writer"
 +
    Completes: "file://Writer.scp"
 +
    Producer: "OOo"
 +
    Category: locale
 +
 +
    Files: ...
 +
    Registries: ...
 +
  }
 +
 +
 +
"Standard.scp"
 +
  Product {
 +
    Name: standard
 +
    Features: "Writer", "Calc"
 +
  }
 +
 +
"StarOffice.scp"
 +
  Configuration {
 +
    Name: "StarOffice"
 +
 +
    Product: standard
 +
 +
  # The categories from above.
 +
    Brands: StarOffice
 +
    Locales: de, en-US
 +
    Platforms: Linux/x86, Linux/x64, Windows/x86
 +
    Versions: SRC680_m248
 +
  }
 +
 +
"Sun_Debian.scp"
 +
  Deliverable {
 +
    Configurations: "StarOffice.scp", "StarSuite.scp"
 +
    Container: directory
 +
    Format: .deb
 +
  }
 +
 +
 +
 +
 +
==========================================
 +
 
Template Product OOo-Standard {
 
Template Product OOo-Standard {
 
   Features: writer, calc, impress, draw
 
   Features: writer, calc, impress, draw
Line 83: Line 356:
 
* Creation of Installation Sets
 
* Creation of Installation Sets
 
* Visualization
 
* Visualization
 
+
* Deliverable Table
==Product Pipeline==
+
* Module Table
[[Image:Product Pipeline.jpg|center]]
+
 
+
The modelling by inheritance and instantiation needs to lead to deliverables which are re-usable during productization and after installation, ideally leading to zero redundancy in case of the installation of many variants (OOo and derivatives).
+
 
+
Taking a look at how packages (like RPM) are typically organized, we can see, that the intermediate deliverables should express their needs and offers in terms of
+
* dependencies against a (virtual) package, as well as by listing the
+
* provisions.
+
  
 
===Naming Schema===
 
===Naming Schema===
Line 100: Line 366:
  
 
or generalized
 
or generalized
   <feature>[_<dimension>]*
+
   <feature>[_<category>]*
 
+
Packages independent of a particular dimension just leave this position empty.
+
 
+
===Simple Example: One Feature in two Languages===
+
In the simplest example, we have exactly one package e.g. for the Writer:
+
* '''writer.rpm'''
+
This package may include all entities for the Writer for Linux x86, in English, using the OpenOffice.org brand.
+
 
+
Later on, we would like to provide the Writer not only in English, but may be also in German:
+
* '''writer_en.rpm'''
+
* '''writer_de.rpm'''
+
Obviously we now need to take care of any redundancy in these packages. We can do so by
+
* naming / placing this files differently, or
+
* by sharing them.
+
 
+
For obvious reasons, that sharing approach is better, leading to the following packages:
+
* '''writer.rpm''' - this is everything bug the locale,
+
* '''writer_en.rpm''' - this is the English locale only,
+
* '''writer_en.rpm''' - this is the German locale only.
+
But wait, no we have another problem. What if the user only installs one of the packages? This would be an incomplete / inconsistent and useless installation.
+
 
+
Package managers typically use dependencies and virtual packages (or provisions) to address this problem.
+
 
+
The Writer package needs at least one Writer locale package to be installed, to function correctly. So may want to model the dependencies as follows:
+
* '''writer''' depends on '''writer_lcl''', provides '''writer'''
+
* '''writer_en''' depends on '''writer''', provides '''writer_en''', '''writer_lcl'''
+
* '''writer_de''' depends on '''writer''', provides '''writer_de''', '''writer_lcl'''
+
By now, we can only install one of the above packages, if we install a completing package, though we certainly can install all three packages.
+
 
+
===More Complex: Two Features in one Language===
+
In the previous example, our only feature was the Writer. Lets now have two features, e.g.
+
* Writer, and
+
* Calc.
+
 
+
As you may know, OOo has a [[Architecture|layered Architecture]], therefore thus providing these two features as
+
* '''writer.rpm''',
+
* '''calc.rpm'''
+
leads to many same files to be installed twice. So, obviously we need again to take care of this redundancy. And again, we can do so by
+
* naming / placing this files differently, or
+
* by sharing them.
+
 
+
For obvious reasons, that sharing approach is better, leading to the following packages
+
* '''writer.rpm''',
+
* '''calc.rpm''',
+
* '''basis.rpm'''
+
Again, we now need to take care of completeness / consistency etc.
+
 
+
So, lets define some dependencies:
+
* '''writer''' - depends on basis, provides feature, writer
+
* '''calc''' - depends on basis, provides feature, calc
+
* '''basis''' - optionally depends on feature, provides basis
+
Depending on the usefulness, we may want the basis to require at least one feature to be installed.
+
 
+
By now, we can only install a feature (writer or calc) if the basis is installed.
+
 
+
 
+
===Still More Complex: Two Features in two Languages===
+
So, lets see what happens, if we have two features, e.g. Writer and Calc, and two localizations, e.g. English and German.
+
 
+
Applying the above separation, we get
+
* '''writer.rpm'''
+
* '''calc.rpm'''
+
* '''basis.rpm'''
+
* '''writer_en.rpm'''
+
* '''writer_de.rpm'''
+
* '''calc_en.rpm'''
+
* '''calc_de.rpm'''
+
* '''basis_en.rpm'''
+
* '''basis_de.rpm'''
+
 
+
So, what do the dependencies should now look like? By combining the above orthogonally we get
+
* '''writer''' depends on '''writer_lcl''', '''basis''', provides '''writer''', '''feature'''
+
* '''calc''' depends on '''calc_lcl''', '''basis''', provides '''calc''', '''feature'''
+
* '''basis''' depends on '''basis_lcl''', '''provides basis''',
+
* '''writer_en''' depends on '''writer''', '''basis_en''', provides '''writer_lcl''', '''writer_en''',
+
* '''writer_de''' depends on '''writer''', '''basis_de''', provides '''writer_lcl''', '''writer_de''',
+
* '''calc_en''' depends on '''calc''', '''basis_en''', provides '''calc_lcl''', '''calc_en''',
+
* '''calc_de''' depends on '''calc''', '''basis_de''', provides '''calc_lcl''', '''calc_de''',
+
* '''basis_en''' depends on '''basis''', optionally on '''feature_en''', provides '''basis_lcl''', '''basis_en''',
+
* '''basis_de''' depends on '''basis''', optionally on '''feature_de''', provides '''basis_lcl''', '''basis_de'''.
+
To ensure consistence among layers, we need to make feature locales (e.g. calc_en) dependent on the lower layer locale (e.g. basis_en).
+
 
+
Looking at it, we the get the following minimal and consistent installations:
+
# '''writer''', '''writer_en''', '''basis''', '''basis_en''', == Writer English
+
# '''writer''', '''writer_de''', '''basis''', '''basis_de''', == Writer German
+
# '''calc''', '''calc_en''', '''basis''', '''basis_en''', == Calc English
+
# '''calc''', '''calc_de''', '''basis''', '''basis_de''' == Calc German
+
 
+
Which we can combine to:
+
# 1+2: '''writer''', '''writer_en''', '''writer_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer English/German,
+
# 1+3: '''writer''', '''writer_en''', '''calc''', '''calc_en''', '''basis''', '''basis_en''' == Writer/Calc English
+
# 1+4: '''writer''', '''writer_en''', '''calc''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer English / Calc German
+
# 2+3: '''writer''', '''writer_de''', '''calc''', '''calc_en''', '''basis''', '''basis_de''', '''basis_em''' == Writer German / Calc English
+
# 2+4: '''writer''', '''writer_de''', '''calc''', '''calc_de''', '''basis''', '''basis_de''' == Writer/Calc German
+
# 3+4: '''calc''', '''calc_en''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Calc English/German
+
# 1+2+3: '''writer''', '''writer_en''', '''writer_de''', '''calc''', '''calc_en''', '''basis''', '''basis_en''', '''basis_de''' == Writer English/German, Calc English
+
# 1+2+4: '''writer''', '''writer_en''', '''writer_de''', '''calc''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer English/German, Calc German
+
# 1+3+4: '''writer''', '''writer_en''', '''calc''', '''calc_en''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer English, Calc English/German
+
# 1+2+3+4: '''writer''', '''writer_en''', '''writer_de''', '''calc''', '''calc_en''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer/Calc English/German
+
# 2+3+4: '''writer''', '''writer_de''', '''calc''', '''calc_en''', '''calc_de''', '''basis''', '''basis_en''', '''basis_de''' == Writer German, Calc English/German
+
Note: Obviously we need to be able, to deal in our implementations with the fact, that different features may only be available in different locales. This is unavoidable, if we want to separate features.
+
 
+
===Complex Example===
+
For example, splitting the writer along the above "dimensions" (brand, OS, Architecture, locale, rest), we get
+
* writer.rpm - this is everything of the writer, which does not provide anything a long the dimensions,
+
* writer___en.rpm - containing all English localization content not depending on anything else,
+
* writer_OOo__en.rpm - containing all English localization content of the OOo brand,
+
* writer___de.rpm - containing all German localization content not depending on anything else,
+
* writer_OOo__de.rpm - containing all German localization content of the OOo brand,
+
* writer__linux_.rpm - containing all Linux specific stuff,
+
 
+
All product entities (files, registry entries, short cuts etc.) get packaged according their dimensions.
+
 
+
We now need to see, how we can express dependencies. Looking at the above example, we see, that the writer package (writer.rpm) certainly needs some of the other packages, to become usable. It obviously depends on the following
+
* writer localization,
+
* writer brand,
+
* writer platform specific files,
+
or expressed more general writer.rpm depends on
+
* writer_brnd__lcl
+
* writer___lcl
+
* writer__linux_
+
While the specific packages do provide
+
* writer - writer
+
* writer___en - writer___en, writer___lcl
+
* writer_OOo__en - writer_brnd__lcl, writer_OOo__lcl, writer_brnd__en, writer_OOo__en,
+
* writer___de - writer___de, writer___lcl,
+
* wrtier_OOo__de - writer_brnd__lcl, writer_OOo__lcl, writer_brnd__de, writer_OOo__de
+
 
+
Packages with less dimensions are '''more general''' than packages with more dimensions, which are '''more specific'''.
+
 
+
Two rules of thumb help to ease modelling the dependencies:
+
* A package may only have a dependency to a more general package (the writer_OOo__en package may depend on the writer_OOo__ package, but not the opposit)!
+
* A more specific package implies the more general (the writer_OOo__en package implies an writer___en package)!
+
  
 +
Packages independent of a particular dimension just leave this position empty.
  
  
 
[[Category:Packaging]]
 
[[Category:Packaging]]

Latest revision as of 14:32, 16 April 2009

To dos:

  • Add images to examples.
  • Define algorithms.

The Product Pipeline

Product Pipeline.jpg

The product pipeline should leverage "make" at the various stages to "build" intermediate deliverables, to ensure that dependencies are reflected correctly and to allow the re-usage of intermediate deliverables for different targets products.

De-Composition

Looking at an installed OOo and its files, registry entries etc., we can see, that all these entities belong to one or multiple of the following domains.

Brand
This is everything somehow related to a specific product, such as StarOffice or OpenOffice.org. E.g. the splash screen typically belongs the a brand.
Operating System
This is everything somehow related to a specific operating system as Solaris, MS Windows or Linux. Shell or batch scripts, such as #!/bin/sh or .bat are Operating System specific, but typically not machine dependent.
Machine Architecture
This is everything related to a native (machine) interpreter, such as x86 or SPARC, as well as its ABI.
Localization
This is everything related to a human language, such as English or German.
Version
This is everything related to a particular version. Actually every entity has a version, such as SRC680_m245.

Note: Ideally the parts (packages) of two different deliverables are reflecting the redundancy of the according installations, e.g. if OOo and BrOOo have 90% in common, than their deliverables need to share 90% of their parts (packages).

Note: If we assume that the part format (e.g. rpm, deb, pkg, msi) implies the operating system, we may leave out Operating System as an own domain, as no part may be shared between any two OS. The differentiation into Operating System and Machine Architecture becomes the moment interesting, we are going to support multiple machine architectures of the same Operating System, e.g. Linux x86 and x64.

The consequences of this observation are, that entities unrelated to one or multiple categories, but shared by products differing in these categories, are identical and thus may be re-used for creating, changing or updating products respectively installed products.

Additionally every entity exactly belongs to one

as otherwise it would not be needed for any feature, thus it would not be needed at all. Certainly features may be related such that they require one another, even if this is not noticeable in the Product View.

Every product we define can be assembled out of parts (e.g. RPM packages), which are created along the above categories.

Examples

One Feature in Two Languages

In the simplest case, we have exactly one part for every product, e.g. for the Writer:

This part may include all entities for the Writer for Linux x86, in English, utilizing the OpenOffice.org brand.

Later on, we would like to provide the Writer not only in English, but may be also in German, which leads to the following products:

  • writer_en - The Writer in English.
  • writer_de - The Writer in German.

Obviously we now need to take care of any redundancy in these products. We can do so by

  • naming / placing entities (e.g. files) differently, or
  • by sharing them.

For obvious reasons, that sharing approach is better, leading to the following parts:

  • writer - This is everything but the locale.
  • writer_en - This is the English locale only.
  • writer_en - This is the German locale only.

But wait, now we have another problem. What if the user only installs one of the parts? This would be an incomplete / inconsistent and useless installation.

On UNIX and alike platforms, parts are typically called packages. Package managers typically use dependencies and virtual packages (or provisions) to address the consistency problem. So, lets try to set the above parts into relationship using dependencies and provisions.

The writer ("all but the locale") part needs at least one writer locale part to be installed, to function correctly, but must not depend on any concrete locale (otherwise one would need to install all locales ;-). So we may want to model the dependencies and provisions as follows:

  • writer: depends on writer_lcl, provides writer
  • writer_en: depends on writer, provides writer_en, writer_lcl
  • writer_de: depends on writer, provides writer_de, writer_lcl

By now, we can only install any of the above parts, if we install a completing part as well, though we certainly can install all three parts.

Two Features/Two Layers

In the previous example, our only feature was the Writer. Lets now have two features, e.g.

  • Writer, and
  • Calc.

As you may know, OOo has a layered Architecture, therefore thus providing these two features as two products:

leads to many same entities to be installed twice. So, obviously we need again to take care of this redundancy. And again, we can do so by

  • naming / placing these entities (e.g. files) differently, or
  • by sharing them.

For obvious reasons, that sharing approach is better, leading to the following parts:

  • writer - Everything needed for Writer, not being needed for Calc.
  • calc - Everything needed for Calc, not being needed for Writer.
  • basis - Everything needed for both, Calc and Writer.

Again, we now need to take care of completeness and consistency.

So, lets define some dependencies:

  • writer: depends on basis, provides writer
  • calc: depends on basis, provides calc
  • basis: depends on nothing, provides basis

Now, the writer part respectively the calc part can only be installed if the basis part is going to be installed as well.

By now, we can only install a feature (Writer or Calc) in a complete way.

Two Features/Two Layers in Two Languages

So, lets see what happens, if we have two features, e.g. Writer and Calc, and two localizations, e.g. English and German.

Applying the above separation, we get the following parts:

  • writer
  • calc
  • basis
  • writer_en
  • writer_de
  • calc_en
  • calc_de
  • basis_en
  • basis_de

So, what do the dependencies should now look like? By combining the previous dependencies orthogonally we get

  • writer: depends on writer_lcl, basis provides writer
  • calc: depends on calc_lcl, basis provides calc
  • basis: depends on basis_lcl provides basis,
  • writer_en: depends on writer, basis_en provides writer_lcl, writer_en,
  • writer_de: depends on writer, basis_de provides writer_lcl, writer_de,
  • calc_en: depends on calc, basis_en provides calc_lcl, calc_en,
  • calc_de: depends on calc, basis_de provides calc_lcl, calc_de,
  • basis_en: depends on basis provides basis_lcl, basis_en,
  • basis_de: depends on basis provides basis_lcl, basis_de.

To ensure consistence among layers, we need to make locale packages (([Terms/Part|parts]]) (e.g. calc_en) dependent on any lower layer locale packages (parts) (e.g. basis_en).

Looking at it, we the get the following minimal (and consistent) feature installations:

  1. writer, writer_en, basis, basis_en, == Writer English
  2. writer, writer_de, basis, basis_de, == Writer German
  3. calc, calc_en, basis, basis_en, == Calc English
  4. calc, calc_de, basis, basis_de == Calc German

Which we can combine to:

  1. 1+2: writer, writer_en, writer_de, basis, basis_en, basis_de == Writer English/German,
  2. 1+3: writer, writer_en, calc, calc_en, basis, basis_en == Writer/Calc English
  3. 1+4: writer, writer_en, calc, calc_de, basis, basis_en, basis_de == Writer English / Calc German
  4. 2+3: writer, writer_de, calc, calc_en, basis, basis_de, basis_em == Writer German / Calc English
  5. 2+4: writer, writer_de, calc, calc_de, basis, basis_de == Writer/Calc German
  6. 3+4: calc, calc_en, calc_de, basis, basis_en, basis_de == Calc English/German
  7. 1+2+3: writer, writer_en, writer_de, calc, calc_en, basis, basis_en, basis_de == Writer English/German, Calc English
  8. 1+2+4: writer, writer_en, writer_de, calc, calc_de, basis, basis_en, basis_de == Writer English/German, Calc German
  9. 1+3+4: writer, writer_en, calc, calc_en, calc_de, basis, basis_en, basis_de == Writer English, Calc English/German
  10. 1+2+3+4: writer, writer_en, writer_de, calc, calc_en, calc_de, basis, basis_en, basis_de == Writer/Calc English/German
  11. 2+3+4: writer, writer_de, calc, calc_en, calc_de, basis, basis_en, basis_de == Writer German, Calc English/German

Note: Obviously we need to be able to deal in our implementations with the fact, that different features may only be available in different locales. This is unavoidable, if we want to separate features. Or to phrase it differently, all complete (depends, provides) sets of parts need to be valid.

Complex Example

Splitting the Writer along some the of above identified categories (brand, OS/Architecture, locale), we get the following parts:

  • writer - Everything of the Writer, which does not belong to any of the categories.
  • writer___en - All English localization content not belonging to any other category.
  • writer_OOo__en - All English localization content for the OOo brand.
  • writer___de - All German localization content not belonging to any other category.
  • writer_OOo__de - All German localization content for the OOo brand.
  • writer__linux - All Linux specific stuff.

To ensure consistency, we may model the dependencies and provisions as follows:

  • writer: depends on writer___OS/Arch, writer_brnd__lcl provides writer
  • writer___en: depends on writer provides writer___en
  • writer_OOo__en: depends on writer___en provides writer_brnd__lcl
  • writer___de: depends on writer provides writer___de
  • writer_OOo__de: depends on writer___de provides writer_brnd__lcl
  • writer__linux: depends on writer provides writer__OS/Arch

This is one way to model the dependencies, but is it optimal? It at least seems to be minimal. But what happens if we can get rid of any brand specific localization, thus the writer_OOo__de respectively the writer_OOo__en parts? Even we don't change anything in writer it seems that we need to update it, because of changing dependencies, writer would now directly depend on writer___lcl.

Rules of Thumb

  • A part may depend on lower layer parts.
  • A part may depend on more general parts (e.g. the writer_OOo__en part depends on the writer_OOo and the writer___en parts)!
  • A more specific part implies any more general parts (e.g. the writer_OOo__en part implies the writer___en part)!
  • A virtual part is more general than a concrete part.
  • Every part not completing any other part is a feature.

Algorithms

  • Minimal dependencies.
  • Minimal provisions.
  • Minimal updates because of changing dependencies / provisions.
  • Optimal dependencies / provisions.
  • Installation variants of one set of parts.

Composition

Model products by setting them into

  • inheritance, respectively
  • instantiation (template)

relationship.

product -features/languages-> product configuration -container/format-> deliverable

  • product: OOo, BrOOo
  • platform: unxlngi6.pro
  • version: build (e.g. OOH680_m7)
  • configuration: language(s) (e.g. en-US, de)
  • format: container/packager (e.g. directory/rpm or tar.gz/deb)

Requirements

  • Packaging by Dependency (make driven)
  • Producting by Dependency (make driven)
  • Support for Distributed Packaging.
  • Concurrent Packaging
  • Concurrent Producting
  • Parts must be creatable independently (e.g. a new localization or brand).
  • Parts must be self describing.
  • Support for pre-build parts

Packaging

Taking a look at how packages (like RPM) are typically organized, we can see, that the intermediate deliverables should express their needs and offers in terms of

  • dependencies against a (virtual) package, as well as by listing the
  • provisions.

Inheritance

Inheritance models a "is a" relationship. In practice that would mean, that a StarOffice 8 update 7 is an OOo 2.2.1 (respectively its basis) adding something.

Instantiation

Example


"Categories.scp":
  Category os_arch {
  }

  Category locale {
  }

"Framework.scp":
  #include "categories.scp"

  Part {
    Name: "Framework"
    ...
  }

"Writer.scp"
  #include "categories.scp"

  Part {
    Name: "Writer"
    Description: "A Wordprocessor"
    Depends: Framework, locale
    Producer: "OOo"
    Category: os_arch

    Files: ...
    Registry: ...
  }

"Writer-en-US.scp"
  #include "categories.scp"

  Part {
    Name: "en-US"
    Description: "English localization for Writer"
    Completes: "file://Writer.scp"
    Producer: "OOo"
    Category: locale

    Files: ...
    Registry: ...
  }


"Writer-de.scp"
  #include "categories.scp"

  Part {
    Name: "de"
    Description: "German localization for Writer"
    Completes: "file://Writer.scp"
    Producer: "OOo"
    Category: locale

    Files: ...
    Registries: ...
  }


"Standard.scp"
  Product {
    Name: standard
    Features: "Writer", "Calc"
  }

"StarOffice.scp"
  Configuration {
    Name: "StarOffice"

    Product: standard

  # The categories from above.
    Brands: StarOffice
    Locales: de, en-US
    Platforms: Linux/x86, Linux/x64, Windows/x86
    Versions: SRC680_m248
  }

"Sun_Debian.scp"
  Deliverable {
    Configurations: "StarOffice.scp", "StarSuite.scp"
    Container: directory
    Format: .deb
  }




==========================================

Template Product OOo-Standard {
  Features: writer, calc, impress, draw
}

Abstract Product OOo2.4 {
  Name: OOo 2.4
  Code-Base: SRC680m236
  Implements: OOo-Standard

}

Product OOo2.4-ISO : OOo2.4 {
  Name: OpenOffice 2.4
  Format: ISO-750
  Platform: Linux-x86, Windows-x86, Mac OS X x86
}

Product OOo2.4-download-linux-x86 : OOo2.4 {
  Name: OpenOffice 2.4
  Format: donwload
  Platform: Linux-x86
}

Product OOo2.4-download-windows-x86 : OOo2.4 {
  Name: OpenOffice 2.4
  Format: donwload
  Platform: windows-x86
}

Abstract Product FooOffice3u4 : OOo2.4 {
  Name: FooOffice 3 update 4
  Features: foo-templates, foo-fonts, foo-brand
  Updates: < FooOffice 3 u 4
}

Product FooOOo3u4-ISO : FooOffice3u4 {
  Name: FooOffice 3 update 4 ISO
  Format: ISO-750
  Platform: Linux-x86, Solaris-x86, Solaris-Sparc, Windows-x86, Mac OS X x86
}

Product FooOffice3u4-donwload : FooOffice3u4{
  Name: FooOffice 3 update 4 ISO
  Format: download
  Platform: Linux-x86
}

Abstract Product : OOo_2.4 {
  Name: BarOffice 5 update 6
  Features: bar-templates, bar-fonts, bar-brand
  Updates: < BarOffice 5 u 6
}

Tooling

  • Comparison of Installation Sets
  • Check for conflicts
  • Creation of Installation Sets
  • Visualization
  • Deliverable Table
  • Module Table

Naming Schema

A name schema reflecting this approach may look like this:

<feature>_<brnd>_<pltfrm>_<lcl>
writer_foo_linux_en

or generalized

 <feature>[_<category>]*

Packages independent of a particular dimension just leave this position empty.

Personal tools