Efforts/Package Restructuring/Modelling

From Apache OpenOffice Wiki
Jump to: navigation, search

To dos:

  • Add images to examples.
  • Define algorithms.

The Product Pipeline

Product Pipeline.jpg

The product pipeline should leverage "make" at the various stages to "build" intermediate deliverables, to ensure that dependencies are reflected correctly and to allow the re-usage of intermediate deliverables for different targets products.

De-Composition

Looking at an installed OOo and its files, registry entries etc., we can see, that all these entities belong to one or multiple of the following domains.

Brand
This is everything somehow related to a specific product, such as StarOffice or OpenOffice.org. E.g. the splash screen typically belongs the a brand.
Operating System
This is everything somehow related to a specific operating system as Solaris, MS Windows or Linux. Shell or batch scripts, such as #!/bin/sh or .bat are Operating System specific, but typically not machine dependent.
Machine Architecture
This is everything related to a native (machine) interpreter, such as x86 or SPARC, as well as its ABI.
Localization
This is everything related to a human language, such as English or German.
Version
This is everything related to a particular version. Actually every entity has a version, such as SRC680_m245.

Note: Ideally the parts (packages) of two different deliverables are reflecting the redundancy of the according installations, e.g. if OOo and BrOOo have 90% in common, than their deliverables need to share 90% of their parts (packages).

Note: If we assume that the part format (e.g. rpm, deb, pkg, msi) implies the operating system, we may leave out Operating System as an own domain, as no part may be shared between any two OS. The differentiation into Operating System and Machine Architecture becomes the moment interesting, we are going to support multiple machine architectures of the same Operating System, e.g. Linux x86 and x64.

The consequences of this observation are, that entities unrelated to one or multiple categories, but shared by products differing in these categories, are identical and thus may be re-used for creating, changing or updating products respectively installed products.

Additionally every entity exactly belongs to one

as otherwise it would not be needed for any feature, thus it would not be needed at all. Certainly features may be related such that they require one another, even if this is not noticeable in the Product View.

Every product we define can be assembled out of parts (e.g. RPM packages), which are created along the above categories.

Examples

One Feature in Two Languages

In the simplest case, we have exactly one part for every product, e.g. for the Writer:

This part may include all entities for the Writer for Linux x86, in English, utilizing the OpenOffice.org brand.

Later on, we would like to provide the Writer not only in English, but may be also in German, which leads to the following products:

  • writer_en - The Writer in English.
  • writer_de - The Writer in German.

Obviously we now need to take care of any redundancy in these products. We can do so by

  • naming / placing entities (e.g. files) differently, or
  • by sharing them.

For obvious reasons, that sharing approach is better, leading to the following parts:

  • writer - This is everything but the locale.
  • writer_en - This is the English locale only.
  • writer_en - This is the German locale only.

But wait, now we have another problem. What if the user only installs one of the parts? This would be an incomplete / inconsistent and useless installation.

On UNIX and alike platforms, parts are typically called packages. Package managers typically use dependencies and virtual packages (or provisions) to address the consistency problem. So, lets try to set the above parts into relationship using dependencies and provisions.

The writer ("all but the locale") part needs at least one writer locale part to be installed, to function correctly, but must not depend on any concrete locale (otherwise one would need to install all locales ;-). So we may want to model the dependencies and provisions as follows:

  • writer: depends on writer_lcl, provides writer
  • writer_en: depends on writer, provides writer_en, writer_lcl
  • writer_de: depends on writer, provides writer_de, writer_lcl

By now, we can only install any of the above parts, if we install a completing part as well, though we certainly can install all three parts.

Two Features/Two Layers

In the previous example, our only feature was the Writer. Lets now have two features, e.g.

  • Writer, and
  • Calc.

As you may know, OOo has a layered Architecture, therefore thus providing these two features as two products:

leads to many same entities to be installed twice. So, obviously we need again to take care of this redundancy. And again, we can do so by

  • naming / placing these entities (e.g. files) differently, or
  • by sharing them.

For obvious reasons, that sharing approach is better, leading to the following parts:

  • writer - Everything needed for Writer, not being needed for Calc.
  • calc - Everything needed for Calc, not being needed for Writer.
  • basis - Everything needed for both, Calc and Writer.

Again, we now need to take care of completeness and consistency.

So, lets define some dependencies:

  • writer: depends on basis, provides writer
  • calc: depends on basis, provides calc
  • basis: depends on nothing, provides basis

Now, the writer part respectively the calc part can only be installed if the basis part is going to be installed as well.

By now, we can only install a feature (Writer or Calc) in a complete way.

Two Features/Two Layers in Two Languages

So, lets see what happens, if we have two features, e.g. Writer and Calc, and two localizations, e.g. English and German.

Applying the above separation, we get the following parts:

  • writer
  • calc
  • basis
  • writer_en
  • writer_de
  • calc_en
  • calc_de
  • basis_en
  • basis_de

So, what do the dependencies should now look like? By combining the previous dependencies orthogonally we get

  • writer: depends on writer_lcl, basis provides writer
  • calc: depends on calc_lcl, basis provides calc
  • basis: depends on basis_lcl provides basis,
  • writer_en: depends on writer, basis_en provides writer_lcl, writer_en,
  • writer_de: depends on writer, basis_de provides writer_lcl, writer_de,
  • calc_en: depends on calc, basis_en provides calc_lcl, calc_en,
  • calc_de: depends on calc, basis_de provides calc_lcl, calc_de,
  • basis_en: depends on basis provides basis_lcl, basis_en,
  • basis_de: depends on basis provides basis_lcl, basis_de.

To ensure consistence among layers, we need to make locale packages (([Terms/Part|parts]]) (e.g. calc_en) dependent on any lower layer locale packages (parts) (e.g. basis_en).

Looking at it, we the get the following minimal (and consistent) feature installations:

  1. writer, writer_en, basis, basis_en, == Writer English
  2. writer, writer_de, basis, basis_de, == Writer German
  3. calc, calc_en, basis, basis_en, == Calc English
  4. calc, calc_de, basis, basis_de == Calc German

Which we can combine to:

  1. 1+2: writer, writer_en, writer_de, basis, basis_en, basis_de == Writer English/German,
  2. 1+3: writer, writer_en, calc, calc_en, basis, basis_en == Writer/Calc English
  3. 1+4: writer, writer_en, calc, calc_de, basis, basis_en, basis_de == Writer English / Calc German
  4. 2+3: writer, writer_de, calc, calc_en, basis, basis_de, basis_em == Writer German / Calc English
  5. 2+4: writer, writer_de, calc, calc_de, basis, basis_de == Writer/Calc German
  6. 3+4: calc, calc_en, calc_de, basis, basis_en, basis_de == Calc English/German
  7. 1+2+3: writer, writer_en, writer_de, calc, calc_en, basis, basis_en, basis_de == Writer English/German, Calc English
  8. 1+2+4: writer, writer_en, writer_de, calc, calc_de, basis, basis_en, basis_de == Writer English/German, Calc German
  9. 1+3+4: writer, writer_en, calc, calc_en, calc_de, basis, basis_en, basis_de == Writer English, Calc English/German
  10. 1+2+3+4: writer, writer_en, writer_de, calc, calc_en, calc_de, basis, basis_en, basis_de == Writer/Calc English/German
  11. 2+3+4: writer, writer_de, calc, calc_en, calc_de, basis, basis_en, basis_de == Writer German, Calc English/German

Note: Obviously we need to be able to deal in our implementations with the fact, that different features may only be available in different locales. This is unavoidable, if we want to separate features. Or to phrase it differently, all complete (depends, provides) sets of parts need to be valid.

Complex Example

Splitting the Writer along some the of above identified categories (brand, OS/Architecture, locale), we get the following parts:

  • writer - Everything of the Writer, which does not belong to any of the categories.
  • writer___en - All English localization content not belonging to any other category.
  • writer_OOo__en - All English localization content for the OOo brand.
  • writer___de - All German localization content not belonging to any other category.
  • writer_OOo__de - All German localization content for the OOo brand.
  • writer__linux - All Linux specific stuff.

To ensure consistency, we may model the dependencies and provisions as follows:

  • writer: depends on writer___OS/Arch, writer_brnd__lcl provides writer
  • writer___en: depends on writer provides writer___en
  • writer_OOo__en: depends on writer___en provides writer_brnd__lcl
  • writer___de: depends on writer provides writer___de
  • writer_OOo__de: depends on writer___de provides writer_brnd__lcl
  • writer__linux: depends on writer provides writer__OS/Arch

This is one way to model the dependencies, but is it optimal? It at least seems to be minimal. But what happens if we can get rid of any brand specific localization, thus the writer_OOo__de respectively the writer_OOo__en parts? Even we don't change anything in writer it seems that we need to update it, because of changing dependencies, writer would now directly depend on writer___lcl.

Rules of Thumb

  • A part may depend on lower layer parts.
  • A part may depend on more general parts (e.g. the writer_OOo__en part depends on the writer_OOo and the writer___en parts)!
  • A more specific part implies any more general parts (e.g. the writer_OOo__en part implies the writer___en part)!
  • A virtual part is more general than a concrete part.
  • Every part not completing any other part is a feature.

Algorithms

  • Minimal dependencies.
  • Minimal provisions.
  • Minimal updates because of changing dependencies / provisions.
  • Optimal dependencies / provisions.
  • Installation variants of one set of parts.

Composition

Model products by setting them into

  • inheritance, respectively
  • instantiation (template)

relationship.

product -features/languages-> product configuration -container/format-> deliverable

  • product: OOo, BrOOo
  • platform: unxlngi6.pro
  • version: build (e.g. OOH680_m7)
  • configuration: language(s) (e.g. en-US, de)
  • format: container/packager (e.g. directory/rpm or tar.gz/deb)

Requirements

  • Packaging by Dependency (make driven)
  • Producting by Dependency (make driven)
  • Support for Distributed Packaging.
  • Concurrent Packaging
  • Concurrent Producting
  • Parts must be creatable independently (e.g. a new localization or brand).
  • Parts must be self describing.
  • Support for pre-build parts

Packaging

Taking a look at how packages (like RPM) are typically organized, we can see, that the intermediate deliverables should express their needs and offers in terms of

  • dependencies against a (virtual) package, as well as by listing the
  • provisions.

Inheritance

Inheritance models a "is a" relationship. In practice that would mean, that a StarOffice 8 update 7 is an OOo 2.2.1 (respectively its basis) adding something.

Instantiation

Example


"Categories.scp":
  Category os_arch {
  }

  Category locale {
  }

"Framework.scp":
  #include "categories.scp"

  Part {
    Name: "Framework"
    ...
  }

"Writer.scp"
  #include "categories.scp"

  Part {
    Name: "Writer"
    Description: "A Wordprocessor"
    Depends: Framework, locale
    Producer: "OOo"
    Category: os_arch

    Files: ...
    Registry: ...
  }

"Writer-en-US.scp"
  #include "categories.scp"

  Part {
    Name: "en-US"
    Description: "English localization for Writer"
    Completes: "file://Writer.scp"
    Producer: "OOo"
    Category: locale

    Files: ...
    Registry: ...
  }


"Writer-de.scp"
  #include "categories.scp"

  Part {
    Name: "de"
    Description: "German localization for Writer"
    Completes: "file://Writer.scp"
    Producer: "OOo"
    Category: locale

    Files: ...
    Registries: ...
  }


"Standard.scp"
  Product {
    Name: standard
    Features: "Writer", "Calc"
  }

"StarOffice.scp"
  Configuration {
    Name: "StarOffice"

    Product: standard

  # The categories from above.
    Brands: StarOffice
    Locales: de, en-US
    Platforms: Linux/x86, Linux/x64, Windows/x86
    Versions: SRC680_m248
  }

"Sun_Debian.scp"
  Deliverable {
    Configurations: "StarOffice.scp", "StarSuite.scp"
    Container: directory
    Format: .deb
  }




==========================================

Template Product OOo-Standard {
  Features: writer, calc, impress, draw
}

Abstract Product OOo2.4 {
  Name: OOo 2.4
  Code-Base: SRC680m236
  Implements: OOo-Standard

}

Product OOo2.4-ISO : OOo2.4 {
  Name: OpenOffice 2.4
  Format: ISO-750
  Platform: Linux-x86, Windows-x86, Mac OS X x86
}

Product OOo2.4-download-linux-x86 : OOo2.4 {
  Name: OpenOffice 2.4
  Format: donwload
  Platform: Linux-x86
}

Product OOo2.4-download-windows-x86 : OOo2.4 {
  Name: OpenOffice 2.4
  Format: donwload
  Platform: windows-x86
}

Abstract Product FooOffice3u4 : OOo2.4 {
  Name: FooOffice 3 update 4
  Features: foo-templates, foo-fonts, foo-brand
  Updates: < FooOffice 3 u 4
}

Product FooOOo3u4-ISO : FooOffice3u4 {
  Name: FooOffice 3 update 4 ISO
  Format: ISO-750
  Platform: Linux-x86, Solaris-x86, Solaris-Sparc, Windows-x86, Mac OS X x86
}

Product FooOffice3u4-donwload : FooOffice3u4{
  Name: FooOffice 3 update 4 ISO
  Format: download
  Platform: Linux-x86
}

Abstract Product : OOo_2.4 {
  Name: BarOffice 5 update 6
  Features: bar-templates, bar-fonts, bar-brand
  Updates: < BarOffice 5 u 6
}

Tooling

  • Comparison of Installation Sets
  • Check for conflicts
  • Creation of Installation Sets
  • Visualization
  • Deliverable Table
  • Module Table

Naming Schema

A name schema reflecting this approach may look like this:

<feature>_<brnd>_<pltfrm>_<lcl>
writer_foo_linux_en

or generalized

 <feature>[_<category>]*

Packages independent of a particular dimension just leave this position empty.

Personal tools