Difference between revisions of "Build System Analysis:build.pl versus makefile"

From Apache OpenOffice Wiki
Jump to: navigation, search
(Minor fixes)
 
(10 intermediate revisions by one other user not shown)
Line 4: Line 4:
  
 
In order to refactor our build system, the functionality of build.pl and how it can be translated into a central makefile have been analyzed.
 
In order to refactor our build system, the functionality of build.pl and how it can be translated into a central makefile have been analyzed.
 +
 +
The proposal contains:
 +
1 new Makefile in main, that handles all intermodule dependencies
 +
1 new Makefile in every <module>, that replaces build.lst and deliver.lst
 +
 +
in main the following calls are legal:
 +
{| class="wikitable" border="1"
 +
|-
 +
! call
 +
! description (call in main)'''Bold text'''
 +
! description (call in <module>)
 +
|-
 +
| make
 +
make all
 +
| generate and deliver all modules
 +
| generate and deliver <module>
 +
remark intermodule dependencies not checked.
 +
|-
 +
| make xyz
 +
| generate xyz and all depending modules as needed
 +
| Not Valid
 +
|-
 +
| make genPO
 +
| extract .pot files from all modules
 +
| extract .pot files from <module>
 +
|-
 +
| make genPO xyz
 +
| extract .pot files in xyz, use --always-make to extract in depending modules
 +
| Not Valid
 +
|-
 +
| make clean
 +
| clean temporary files in all modules
 +
| clean temporary files in <module>
 +
|-
 +
| make clean xyz
 +
| clean temporary files in xyz, use --always-make to clean in depending modules
 +
| Not Valid
 +
|-
 +
| make rebuild
 +
| rebuild all modules
 +
| rebuild <module>
 +
|-
 +
| make rebuild xyz
 +
| rebuild xyz, use --always-make to rebuild depending modules
 +
| Not Valid
 +
|}
 +
 +
Remark, there are no .done files, the <module> directories are elegantly used for that purpose.
 +
  
 
The following table contains all build.pl options, with description (build --help), and the make (in main) equivalent  
 
The following table contains all build.pl options, with description (build --help), and the make (in main) equivalent  
Line 15: Line 64:
 
| --all
 
| --all
 
| build all projects from very beginning till current one
 
| build all projects from very beginning till current one
| make <module>
+
| "make" builds all modules
 +
"make <module>" builds to and with <module> (same as --from)
 
|-
 
|-
| row 2, cell 1
+
| --from
| row 2, cell 2
+
| build all projects dependent from the specified (including it) till current one
| row 2, cell 3
+
| "make <module>" builds to and with <module>. Remark the Makefile represent real dependencies. <module>/Makefile touches a file "solver/.../<module>.done" in every run and the central makefile have rules like "<module>.done : <module_depend>.done ...", this means that modules needed to be generated are called.
 +
|-
 +
| --genPO
 +
| extract en-US PO file(s), no build/deliver (usable for '-all' and '-from' keys)
 +
| "make genPO" to extract from all modules
 +
"<module>/make genPO" to extract from a single module
 +
|-
 +
| --exclude_branch_fromd
 +
| exclude module(s) and its branch from the build
 +
| '''not supported'''
 +
|-
 +
| --mode OOo
 +
| build only projects needed for OpenOffice.org
 +
| '''not supported'''
 +
|-
 +
| --prepare
 +
| clear all projects for incompatible build from prj_name till current one [for platform] (cws version)
 +
| "make clear" to clear all modules
 +
"<module>/make clear" to clear a single module
 +
"<module>/make clear --depend" to clear all depending modules
 +
|-
 +
| --with_branches-
 +
| the same as "--from" but with build all projects in neighbour branches
 +
| '''not supported'''
 +
|-
 +
| --skip
 +
| do not build certain module(s)
 +
| '''not supported'''
 +
|-
 +
| --since
 +
| build all projects beginning from the specified till current one (the same as "--all:prj_name", but skipping prj_name)
 +
| "make <module>"
 +
|-
 +
| --checkmodules
 +
| check if all required parent projects are available
 +
| "make -q" dry-run for dependencies all modules
 +
"make <module> -q" dry-run for dependencies of <module>
 +
|-
 +
| --show
 +
| show what is going to be built
 +
| "make -q" dry-run for dependencies all modules
 +
"make <module> -q" dry-run for dependencies of <module>
 +
"<module>/make -q" dry-run for all files in <module>
 +
|-
 +
| --file
 +
| generate command file file_name
 +
| '''not supported'''
 +
|-
 +
| --deliver
 +
| only deliver, no build (usable for '-all' and '-from' keys)
 +
| "make deliver" delivers all modules
 +
"<module>/make deliver" delivers a single module
 +
|-
 +
| -P
 +
| start multiprocessing build, with number of processes passed
 +
| "make -j<x>" where <x> is number of parallel processes
 +
|-
 +
| --server
 +
| start build in server mode (clients required)
 +
| '''not supported'''
 +
|-
 +
| --setenvstring (--server)
 +
| string for configuration of the client environment
 +
| '''not supported'''
 +
|-
 +
| --port (--server)
 +
| set server port, default is 7890. You may pass several ports, the server will be started on the first available
 +
otherwise the server will be started on first available port from the default range 7890-7894
 +
| '''not supported'''
 +
|-
 +
| --client_timeout (--server)
 +
| time frame after which the client/job is considered to be lost. Default is 120 min
 +
| '''not supported'''
 +
|-
 +
| --dlv_switch
 +
| use deliver with the switch specified
 +
| '''not supported'''
 +
|-
 +
| --help
 +
| print help info
 +
| '''not supported'''
 +
|-
 +
| --ignore
 +
| force tool to ignore errors
 +
| "make -i"
 +
|-
 +
| --html
 +
| generate html page with build status file named unxlngx6.pro.build.html will be generated in /share/opensource/aoo/branches/l10n40/main
 +
| '''not supported''', make does not provide such a feature, but 2 scripts will be provided make_log, that calls make and redirect output to a log file, and make_html, that calls make and pipes output through a html formatter.
 +
|-
 +
| --html_path
 +
| set html page path
 +
| '''not supported'''
 +
|-
 +
| --dontgraboutput
 +
| do not grab console output when generating html page
 +
| '''not supported'''
 +
|-
 +
| --genconf
 +
| generate/modify workspace configuration file
 +
| '''not suported'''
 +
|-
 +
| --add
 +
| add active module(s) to configuration file
 +
| edit Makefile
 +
|-
 +
| --remove
 +
| removeactive  modules(s) from configuration file
 +
| edit Makefile
 +
|-
 +
| --removeall
 +
--clear
 +
| remove all active modules(s) from configuration file
 +
| edit Makefile
 +
|-
 +
| --stoponerror
 +
| stop build when error occurs (for mp builds)
 +
| default unless -i is used
 +
|-
 +
| --interactive
 +
| start interactive build process (process can be managed via html page)
 +
| '''not supported'''
 +
|-
 +
| --job=job_string
 +
| execute custom job in (each) module. job_string is a shell script/command to be executed instead of regular dmake jobs
 +
| "make --job=job_string"
 +
|-
 +
| --pre_job=pre_job_string
 +
| execute preliminary job in (each) module. pre_job_string is a shell script/command to be executed before regular job in the module
 +
| "make --pre_job=job_string"
 +
|-
 +
| --post_job=job_string
 +
| execute a postprocess job in (each) module. post_job_string is a shell script/command to be executed after regular job in the module
 +
| "make --post_job=job_string"
 
|}
 
|}
  
 
+
There will be 1 Makefile in main with all intermodule dependencies (first line in prj/build.lst) and
 
+
1 Makefile in each module with the dependencies in prj/build.lst, this makefile will also have target=deliver which contains all lines from prj/deliver.lst
 
+
--from      - build all projects dependent from the specified (including it) till current one
+
 
+
--genPO      - extract en-US PO file(s), no build/deliver (usable for '-all' and '-from' keys)
+
 
+
--exclude_branch_from    - exclude module(s) and its branch from the build
+
 
+
--mode OOo  - build only projects needed for OpenOffice.org
+
 
+
--prepare    - clear all projects for incompatible build from prj_name till current one [for platform] (cws version)
+
 
+
--with_branches- the same as "--from" but with build all projects in neighbour branches
+
 
+
--skip      - do not build certain module(s)
+
 
+
--since      - build all projects beginning from the specified till current one (the same as "--all:prj_name", but skipping prj_name)
+
 
+
--checkmodules      - check if all required parent projects are availlable
+
 
+
--show      - show what is going to be built
+
 
+
--file      - generate command file file_name
+
 
+
--deliver    - only deliver, no build (usable for '-all' and '-from' keys)
+
 
+
-P          - start multiprocessing build, with number of processes passed
+
 
+
--server    - start build in server mode (clients required)
+
 
+
  --setenvstring  - string for configuration of the client environment
+
 
+
  --port          - set server port, default is 7890. You may pass several ports, the server will be started on the first available
+
                            otherwise the server will be started on first available port from the default range 7890-7894
+
  --client_timeout  - time frame after which the client/job is considered to be lost. Default is 120 min
+
 
+
--dlv_switch - use deliver with the switch specified
+
 
+
--help      - print help info
+
 
+
--ignore    - force tool to ignore errors
+
 
+
--html      - generate html page with build status
+
                      file named unxlngx6.pro.build.html will be generated in /share/opensource/aoo/branches/l10n40/main
+
 
+
--html_path      - set html page path
+
 
+
--dontgraboutput - do not grab console output when generating html page
+
 
+
--genconf    - generate/modify workspace configuration file
+
 
+
--add            - add active module(s) to configuration file
+
 
+
--remove        - removeactive  modules(s) from configuration file
+
 
+
--removeall|--clear          - remove all active modules(s) from configuration file
+
 
+
--stoponerror      - stop build when error occurs (for mp builds)
+
 
+
--interactive      - start interactive build process (process can be managed via html page)
+
  Custom jobs:
+
--job=job_string        - execute custom job in (each) module. job_string is a shell script/command to be executed instead of regular dmake jobs
+
 
+
--pre_job=pre_job_string        - execute preliminary job in (each) module. pre_job_string is a shell script/command to be executed before regular job in the module
+
 
+
--post_job=job_string        - execute a postprocess job in (each) module. post_job_string is a shell script/command to be executed after regular job in the module
+
 
+
default: build current project
+

Latest revision as of 10:18, 2 November 2013

Edit.png

Build Environment Effort

Quick Navigation

About this template


build.pl versus central makefile

In order to refactor our build system, the functionality of build.pl and how it can be translated into a central makefile have been analyzed.

The proposal contains: 1 new Makefile in main, that handles all intermodule dependencies 1 new Makefile in every <module>, that replaces build.lst and deliver.lst

in main the following calls are legal:

call description (call in main)Bold text description (call in <module>)
make

make all

generate and deliver all modules generate and deliver <module>

remark intermodule dependencies not checked.

make xyz generate xyz and all depending modules as needed Not Valid
make genPO extract .pot files from all modules extract .pot files from <module>
make genPO xyz extract .pot files in xyz, use --always-make to extract in depending modules Not Valid
make clean clean temporary files in all modules clean temporary files in <module>
make clean xyz clean temporary files in xyz, use --always-make to clean in depending modules Not Valid
make rebuild rebuild all modules rebuild <module>
make rebuild xyz rebuild xyz, use --always-make to rebuild depending modules Not Valid

Remark, there are no .done files, the <module> directories are elegantly used for that purpose.


The following table contains all build.pl options, with description (build --help), and the make (in main) equivalent

Option Description Makefile
--all build all projects from very beginning till current one "make" builds all modules

"make <module>" builds to and with <module> (same as --from)

--from build all projects dependent from the specified (including it) till current one "make <module>" builds to and with <module>. Remark the Makefile represent real dependencies. <module>/Makefile touches a file "solver/.../<module>.done" in every run and the central makefile have rules like "<module>.done : <module_depend>.done ...", this means that modules needed to be generated are called.
--genPO extract en-US PO file(s), no build/deliver (usable for '-all' and '-from' keys) "make genPO" to extract from all modules

"<module>/make genPO" to extract from a single module

--exclude_branch_fromd exclude module(s) and its branch from the build not supported
--mode OOo build only projects needed for OpenOffice.org not supported
--prepare clear all projects for incompatible build from prj_name till current one [for platform] (cws version) "make clear" to clear all modules

"<module>/make clear" to clear a single module "<module>/make clear --depend" to clear all depending modules

--with_branches- the same as "--from" but with build all projects in neighbour branches not supported
--skip do not build certain module(s) not supported
--since build all projects beginning from the specified till current one (the same as "--all:prj_name", but skipping prj_name) "make <module>"
--checkmodules check if all required parent projects are available "make -q" dry-run for dependencies all modules

"make <module> -q" dry-run for dependencies of <module>

--show show what is going to be built "make -q" dry-run for dependencies all modules

"make <module> -q" dry-run for dependencies of <module> "<module>/make -q" dry-run for all files in <module>

--file generate command file file_name not supported
--deliver only deliver, no build (usable for '-all' and '-from' keys) "make deliver" delivers all modules

"<module>/make deliver" delivers a single module

-P start multiprocessing build, with number of processes passed "make -j<x>" where <x> is number of parallel processes
--server start build in server mode (clients required) not supported
--setenvstring (--server) string for configuration of the client environment not supported
--port (--server) set server port, default is 7890. You may pass several ports, the server will be started on the first available

otherwise the server will be started on first available port from the default range 7890-7894

not supported
--client_timeout (--server) time frame after which the client/job is considered to be lost. Default is 120 min not supported
--dlv_switch use deliver with the switch specified not supported
--help print help info not supported
--ignore force tool to ignore errors "make -i"
--html generate html page with build status file named unxlngx6.pro.build.html will be generated in /share/opensource/aoo/branches/l10n40/main not supported, make does not provide such a feature, but 2 scripts will be provided make_log, that calls make and redirect output to a log file, and make_html, that calls make and pipes output through a html formatter.
--html_path set html page path not supported
--dontgraboutput do not grab console output when generating html page not supported
--genconf generate/modify workspace configuration file not suported
--add add active module(s) to configuration file edit Makefile
--remove removeactive modules(s) from configuration file edit Makefile
--removeall

--clear

remove all active modules(s) from configuration file edit Makefile
--stoponerror stop build when error occurs (for mp builds) default unless -i is used
--interactive start interactive build process (process can be managed via html page) not supported
--job=job_string execute custom job in (each) module. job_string is a shell script/command to be executed instead of regular dmake jobs "make --job=job_string"
--pre_job=pre_job_string execute preliminary job in (each) module. pre_job_string is a shell script/command to be executed before regular job in the module "make --pre_job=job_string"
--post_job=job_string execute a postprocess job in (each) module. post_job_string is a shell script/command to be executed after regular job in the module "make --post_job=job_string"

There will be 1 Makefile in main with all intermodule dependencies (first line in prj/build.lst) and 1 Makefile in each module with the dependencies in prj/build.lst, this makefile will also have target=deliver which contains all lines from prj/deliver.lst

Personal tools