|
|
| (9 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| | ==Introduction== | | ==Introduction== |
| − | This document is based on and extents [[Localization_for_developers]]. The document is work in
| |
| − | progress showing the result of a detailed technical analysis of the current process (version 3.4.1) .
| |
| − | As such this document should be seen as a replacement of [[Localization_for_developers]].
| |
| | | | |
| − | The l10n process only concerns itself about localizing defined supported languages. Adding a new
| |
| − | language is a i18 process. This document is further restricted to the ongoing translation process and
| |
| − | closely related build process. In case of external happenings, like e.g. Germany changing rules of
| |
| − | spelling, it should be covered with i18 procedures.
| |
| | | | |
| − | The document will hopefully spark a discussion so it can be updated with other views from the [mailto:ooo-L10n@incubator.apache.org ooo-L10n@incubator.apache.org].
| + | math problem |
| | + | <math>E = m.c^2</math> |
| | | | |
| − | It is important to understand the current process before we start discussing detailed changes, so this
| |
| − | is the main purpose of this page. Once all the open issues at the end of document have been
| |
| − | discussed as solutions agreed upon, a new document will be made describing the process as it
| |
| − | should be in the near future.
| |
| | | | |
| − | Thanks to all those persons who contributed to [[Localization_for_developers]] that has been a great
| + | test |
| − | starting point for this document.
| |
| − | | |
| − | [[File:L10proc.pdf]]
| |
| − | | |
| − | | |
| − | ==Overview==
| |
| − | Localization, often abbreviated as l10n, defines the process to make a software package available in
| |
| − | local languages, different to the language of the developer.
| |
| − | | |
| − | Localization is from the perspective of the involved person a multi-step process that involves a
| |
| − | variety of tools and procedures. Most importantly the 4 main categories of involved persons have
| |
| − | quite different and to some extent conflicting views and requirement, therefore the process should
| |
| − | be a real “best of all worlds” approach.
| |
| − | | |
| − | The current process is more or less purely developer oriented, contains a lot of different tools and
| |
| − | depends a lot on the responsibility of the involved people. It seems to be a process that has grown
| |
| − | out of necessity more than a planned road.
| |
| − | | |
| − | Most of the tools used as well as the central data format (SDF) are specific to AOO and not used
| |
| − | anywhere else even though both source (c++, resource, UI files) and target (po files) are standard
| |
| − | file formats.
| |
| − | | |
| − | Only a part of the workflow are integrated in the build system. Much of it requires manual steps to
| |
| − | be taken. Some of the tools involved are not part of the OpenOffice SVN and, due to a hard disk
| |
| − | crash of the old [http://translate.sourceforge.net/wiki/pootle/index pootle server], are lost.
| |
| − | | |
| − | Translations are done with the help of a [http://translate.sourceforge.net/wiki/pootle/index pootle server]. The localization work flow can very short be
| |
| − | seen as:
| |
| − | * extraction messages from source files.
| |
| − | * uploading message to the [http://translate.sourceforge.net/wiki/pootle/index pootle server].
| |
| − | * translating messages on the [http://translate.sourceforge.net/wiki/pootle/index pootle server].
| |
| − | * downloading messages from the [http://translate.sourceforge.net/wiki/pootle/index pootle server].
| |
| − | * merging messages into source files.
| |
| − | | |
| − | If you are looking for information about how to contribute translations then [[Localization]] gives an overview.
| |
| − | | |
| − | The document has 5 parts:
| |
| − | * a relative non-technical overview of the process,
| |
| − | * a detailed technical overview of the process,
| |
| − | * a detailed technical data flow/storage view,
| |
| − | * a detailed technical view of the tools used with parameters etc,
| |
| − | * an open issues list,
| |
| − | ==Actors and Systems==
| |
| − | The l10n process can and should be viewed with respect to 4 different categories of people who
| |
| − | access the process through 2 different systems. The translator consider [http://translate.sourceforge.net/wiki/pootle/index pootle server] to be repository
| |
| − | whereas the others consider [http://www.apache.org/dev/version-control.html SVN] the main repository.
| |
| − | | |
| − | [[File:Localization_AOO_1.jpg|center]]
| |
| − | | |
| − | '''Note:''' this view only relates to the l10n procedure, the picture for the whole project is a lot more
| |
| − | complex.
| |
| − | | |
| − | The red lighting indicates that the [http://translate.sourceforge.net/wiki/pootle/index pootle server] only works indirectly on the [http://www.apache.org/dev/version-control.html SVN] server.
| |
| − | | |
| − | The red lightning indicates that data is being copied:
| |
| − | * to/from [http://translate.sourceforge.net/wiki/pootle/index pootle server], which requires manual intervention during the build process
| |
| − | * to tester which is quite normal, since a tester normally get an install-set.
| |
| − | | |
| − | | |
| − | ===Developers===
| |
| − | Developers construct the actual program, using dedicated development tools.
| |
| − | | |
| − | Developers will as part of the development process embed messages (errors, warnings …) in the
| |
| − | source code and/or build UI. The embedded texts are defined to be in English but the source code
| |
| − | are in different programming languages, making extraction a challenge.
| |
| − | | |
| − | Developers are fluent in their language (C++, java, python etc.) but for sure not in all the native
| |
| − | languages supported by AOO therefore localization is needed.
| |
| − | | |
| − | Developers uses solely
| |
| − | [http://www.apache.org/dev/version-control.html SVN] as their repository.
| |
| − | | |
| − | | |
| − | ===Translators===
| |
| − | Translators add texts in the local native language, relating (translating) to the original message. In a
| |
| − | release there is a 1-n relation between the original message and the supported languages, where n is
| |
| − | the number of supported languages.
| |
| − | | |
| − | Translators does in principle not need to have programming knowledge because in essence they are
| |
| − | presented with a list of texts extracted from the source and delivers the translated text back.
| |
| − | | |
| − | Translators work solely with the [http://translate.sourceforge.net/wiki/pootle/index pootle server] which today has no direct connection to [http://www.apache.org/dev/version-control.html SVN] but
| |
| − | work in parallel with [http://www.apache.org/dev/version-control.html SVN] and are updated manually with regular intervals.
| |
| − | | |
| − | | |
| − | ===Integrators===
| |
| − | Integrators initiate and control the build process.
| |
| − | | |
| − | Integrators does in principle not need to have programming or translation knowledge, because they
| |
| − | are basically doing administrative tasks.
| |
| − | | |
| − | | |
| − | ===Testers===
| |
| − | Testers check the total system and do a quality assurance of the behavior.
| |
| − | | |
| − | Testers need a deep knowledge of the behavior of the system, but deep technical knowledge is not
| |
| − | needed.
| |
| − | | |
| − | Today testing seems to be very limited and not formalized in respect of the l10n process.
| |
| − | | |
| − | | |
| − | ===System: [http://www.apache.org/dev/version-control.html SVN]===
| |
| − | The sub version server is the actual repository and ideally all systems should work directly on this
| |
| − | server.
| |
| − | | |
| − | All source files, documents etc. are stored in
| |
| − | [http://www.apache.org/dev/version-control.html SVN].
| |
| − | | |
| − | | |
| − | ===System: [http://translate.sourceforge.net/wiki/pootle/index pootle server]===
| |
| − | The [http://translate.sourceforge.net/wiki/pootle/index pootle server] provides an environment for translators to work in.
| |
| − | | |
| − | Today the [http://translate.sourceforge.net/wiki/pootle/index pootle server] contains all the translations and are updated from SVN and are as a
| |
| − | consequence not synchronized and without version control (during the translation process).
| |
| − | | |
| − | Furthermore many translators work offline without any control.
| |
| − | | |
| − | ==L10n workflow high altitude view==
| |
| − | The workflow seen from the outside is quite simple, but still some of the shortcommings should be
| |
| − | very obvious.
| |
| − | | |
| − | The workflow is designed as a waterfall, but one of the good norwegian ones where water is
| |
| − | pumped back up at night time. Idealy for each release each section is done only once (waterfall), but
| |
| − | in real life two things happen (norwegian night pumping):
| |
| − | * Some sections happens in parallel (e.g. Translators start working with early code)
| |
| − | * Some sections are repeated due to problems found in later sections
| |
| − | | |
| − | This is quite normal and normally not a real problem provided the process is automated and has a
| |
| − | number of quality gates.
| |
| − | | |
| − | However the current process there is only a single automated quality gate which are pure technical
| |
| − | (solving: “Can the product be built without errors?”) the rest is left to us humans.
| |
| − | | |
| − | The workflow only concentrates on the l10n process which is only a subset of the total lifecycle
| |
| − | process.
| |
| − | [[File:Localization_AOO_2.jpg|center]]
| |
| − | | |
| − | The model shows at least one problem, the parallelism of “Translation online” and “Translation
| |
| − | offline”. To put it a bit on edge, this works because there are no alternatives and because there are
| |
| − | few volunteers.
| |
| − | | |
| − | | |
| − | ===Content creation===
| |
| − | Developers construct/develop new functionality or correct bugs/issues using different tools and
| |
| − | programming languages. During the programming they may insert texts in the source files, this is
| |
| − | done very differently depending on programming language and type of application (UI or
| |
| − | error/information messages).
| |
| − | | |
| − | All text are written in English according to the programming guidelines, however there are no
| |
| − | review process to secure the quality of the text or consistency with the rest of the product.
| |
| − | | |
| − | '''Note:''' A developer can insert the text directly in the source file or in a resource file, for the program
| |
| − | both ways work, however only a limited number of file extension types are today scanned for texts,
| |
| − | so in worst case some texts are never translated.
| |
| − | | |
| − | | |
| − | ===Upload [http://translate.sourceforge.net/wiki/pootle/index pootle server]===
| |
| − | The source files are stored in
| |
| − | [http://www.apache.org/dev/version-control.html SVN]. In general the content of
| |
| − | [http://www.apache.org/dev/version-control.html SVN] is floating since it contains the
| |
| − | absolute last updates, with the consequence that a total build very often will fail. To circumvent this
| |
| − | problem a snapshot is made from time to time, guaranteeing a successful build but the package
| |
| − | might not function correctly.
| |
| − | | |
| − | The snapshots can be used for a manually started extraction to the [http://translate.sourceforge.net/wiki/pootle/index pootle server].
| |
| − | | |
| − | The extraction program loop over all files in
| |
| − | [http://www.apache.org/dev/version-control.html SVN]
| |
| − | * building one big sdf file.
| |
| − | * the sdf file are then split into multiple template files.
| |
| − | * the template files are merged with the existing po files in the [http://translate.sourceforge.net/wiki/pootle/index pootle server].
| |
| − | * [http://translate.sourceforge.net/wiki/pootle/index pootle server] database contain one set of po files for each language.
| |
| − | | |
| − | The purpose is to decouple the development process from the translation process. The purpose is
| |
| − | achieved, but the route is highly manual and error prone.
| |
| − | | |
| − | If life was ideal, translation would only take place when development is completed, but typically
| |
| − | translation takes place at several stages of the development process for several reasons:
| |
| − | * A release consist of changes to multiple function group (e.g. draw, write and calc), and these developments are finished at different point in times. Whenever a development of a group is finished this group can be translated and thus the decoupling will be repeated.
| |
| − | * Translation often takes place while testing is ongoing, any bug fixing must lead to a new decoupling, and since there are no version control of the translated parts it can only be controlled manually if there are changes.
| |
| − | * There are currently no short-cuts to fast translate a bug fix that involves a known text change
| |
| − | | |
| − | '''Note:''' This part of the process is highly manual and very error prone, since it involves coordinating
| |
| − | the effort of a high number of people
| |
| − | | |
| − | | |
| − | ===Translation===
| |
| − | Translation takes place on an offline copy consisting of multiple po files. These po files are
| |
| − | generated each time, so any additional information the translators would like to keep (e.g.
| |
| − | comments) are lost.
| |
| − | | |
| − | At the moment there are 276 different files to translate for each language. In order to split the work
| |
| − | UI and Help are separated, there are
| |
| − | * 20 help files (but they are big!)
| |
| − | * 256 UI/message files (typically an average of 20lines)
| |
| − | | |
| − | Having that many files to translate makes it more likely to get content inconsistency (same term is
| |
| − | translated differently).
| |
| − | | |
| − | Since the files are solely generated from the sources, there are no glossary file available, making it
| |
| − | very difficult for new volunteers to help. Furthermore there are no control of how accelerators are
| |
| − | used.
| |
| − | | |
| − | The online and offline translation process are handled quite differently.
| |
| − | | |
| − | '''Note:''' Today there are no version control and as such no computer controlled review and as a
| |
| − | consequence the content quality varies.
| |
| − | | |
| − | | |
| − | ====Translation online (“committer”)====
| |
| − | The po files are stored in [http://translate.sourceforge.net/wiki/pootle/index pootle server] database and thereby available to translators with through
| |
| − | the HTML interface.
| |
| − | | |
| − | Due to the lack of version control, team work must be controlled carefully.
| |
| − | | |
| − | Once a translation is complete, the translator(s) must manually inform the integrator that the set is
| |
| − | ready for merge.
| |
| − | | |
| − | | |
| − | ====Translation offline (non “committer”)====
| |
| − | The integrator will manually extract the po files from the [http://translate.sourceforge.net/wiki/pootle/index pootle server] and send the files to the
| |
| − | translators without “committer” status. The copy is not under version control or otherwise
| |
| − | controlled.
| |
| − | | |
| − | Once the translation is complete the the translator must send the files back to the integrator.
| |
| − | | |
| − | There are no computer control with which translations are outstanding, which are in manual review
| |
| − | and which are completed, this is currently controlled by the integrator.
| |
| − | | |
| − | '''Note:''' Neither bugzilla nor the mailing list allows these big attachments, so it must be sent to a private mail address or posted on a private web page.
| |
| − | | |
| − | | |
| − | ===Merge [http://www.apache.org/dev/version-control.html SVN]===
| |
| − | The integrator must manually decide that all offline translations are back and all online translators
| |
| − | have finished (translation review is left to the single translator team).
| |
| − | | |
| − | At a point in time decided by the integrator to start the merge, which consist of several manual
| |
| − | steps:
| |
| − | * synchronize po files with content of the [http://translate.sourceforge.net/wiki/pootle/index pootle server] database
| |
| − | * add the offline translated files
| |
| − | * convert po files to sdf file (one pr language)
| |
| − | * store sdf file in [http://www.apache.org/dev/version-control.html SVN].
| |
| − | | |
| − | This part of the process does not allow for glossary files, because the converters would have no
| |
| − | source parts to relate the glossary to.
| |
| − | | |
| − | | |
| − | ===Update [http://translate.sourceforge.net/wiki/pootle/index pootle server]===
| |
| − | Now it is time to synchronize the pootle server, to make sure then content is identical with SVN.
| |
| − | Based on the new sdf file (one pr language) the following actions are taken:
| |
| − | * Convert sdf til template file
| |
| − | * update templates in [http://translate.sourceforge.net/wiki/pootle/index pootle server]
| |
| − | | |
| − | | |
| − | ===Language build===
| |
| − | Finally a test release can be built, and the testers can control the final result.
| |
| − | It should be noted that there currently no formal testing of the native language versions.
| |
| − | | |
| − | | |
| − | ==Simplified data flow==
| |
| − | The current data flow is pretty complex, and it seems more like a “invented as needed” structure.
| |
| − | | |
| − | The first part shows the text flow from developer to translator:
| |
| − | | |
| − | [[File:Localization_AOO_3.jpg|center]]
| |
| − | | |
| − | The second part shows the text flow from translator to tester:
| |
| − | | |
| − | As seen from the diagrams there are many manual steps, and many different temporary files only
| |
| − | needed to come from a to b.
| |
| − | | |
| − | [[File:Localization_AOO_4.jpg|center]]
| |
| − | | |
| − | | |
| − | ==L10n workflow technical view==
| |
| − | This chapter is identical to L10n workflow high altitude view but seen from a technical view
| |
| − | showing actual commands, names of files and directories as well details of the tool behavior.
| |
| − | | |
| − | | |
| − | ===Content Creation===
| |
| − | Developers write text that needs to be localized. In principle the texts can be kept in files with any
| |
| − | extension since most compilers are quite large in that respect. However the programming guidelines
| |
| − | should secure that only defined extensions are used.
| |
| − | | |
| − | It is worth to note that the most common files (.cxx, .hxx, .cpp, .hpp, .py) are NOT scanned.
| |
| − | Note: If a developer for some good reason decides to use a file with a non-standard suffix, it will
| |
| − | NOT be searched for messages.
| |
| − | | |
| − | | |
| − | ===Upload [http://translate.sourceforge.net/wiki/pootle/index pootle server]===
| |
| − | The upload process is the very complicated and totally manual.
| |
| − | | |
| − | The outcome of the process in general it makes a snapshot copy of the texts in [http://www.apache.org/dev/version-control.html SVN] and makes it
| |
| − | available on the [http://translate.sourceforge.net/wiki/pootle/index pootle server] and as zip files to contributor translators.
| |
| − | | |
| − | After the texts is extracted and until they are merged back they are NOT in any source control, nor
| |
| − | is parallel development controlled.
| |
| − | | |
| − | | |
| − | ===Extraction from sources (generate new sdf file)===
| |
| − | Before starting this process, all sources needs to be checked out (read-only). In order to ensure that
| |
| − | the source is complete it is good practice to do a “build –all” first.
| |
| − | | |
| − | The process is started with:
| |
| | <dl> | | <dl> |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;" | + | {| style="margin-left: 2cm;border-spacing:0;border:0.002cm solid #000000;padding:0.2cm;" |
| | | | | | |
| − | <code>cd main | + | <code>cd main<br> |
| − | | |
| | localize -e -l en-US -f en-US.sdf</code> | | localize -e -l en-US -f en-US.sdf</code> |
| | |} | | |} |
| Line 331: |
Line 16: |
| | This is a perl script that will call | | This is a perl script that will call |
| | <dl> | | <dl> |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;" | + | {| style="margin-left: 2cm;border-spacing:0;border:0.002cm solid #000000;padding:0.2cm;" |
| | | | | | |
| | <code>solver/350/<platform>/bin/localize_sl<.exe></code> | | <code>solver/350/<platform>/bin/localize_sl<.exe></code> |
| | |} | | |} |
| | <dl> | | <dl> |
| − | which is the actual executable. Sources for this executable is found in l10tools/source.
| |
| | | | |
| − | localize_sl loop across the entire tree looking for files with a known extension. As seen in the table
| |
| − | below the number of relevant files are small compared to the total number of files.
| |
| | | | |
| − | <table border="2px" cellpadding="5">
| + | test table |
| − | <tr><th colspan=4><b>Extensions scanned for text</b></th></tr>
| + | total number of files. |
| − | <tr><th><b>Files</b></th><th><b>Extension</b></th><th><b>Tool</b></th><th><b>Description</b></th></tr>
| |
| − | <tr><td>814</td><td>.hrc</td><td>transex3</td><td>header for resource files</td></tr>
| |
| − | <tr><td>98</td><td>.properties</td><td>jpropex</td><td>java property files</td></tr>
| |
| − | <tr><td>1040</td><td>.src</td><td>transex3</td><td>source for resource files</td></tr>
| |
| − | <tr><td>15</td><td>.tree</td><td>xhtex</td><td>help files</td></tr>
| |
| − | <tr><td>0</td><td>.ulf</td><td>ulfex</td><td>?</td></tr>
| |
| − | <tr><td>53</td><td>.xcd</td><td>cfgex</td><td>xml files only in postprocess</td></tr>
| |
| − | <tr><td>314</td><td>.xcs</td><td>cfgex</td><td>xml file for java</td></tr>
| |
| − | <tr><td>1365</td><td>.xcu</td><td>cfex</td><td>xml files for UI</td></tr>
| |
| − | <tr><td>0</td><td>.xgf</td><td>xmlex</td><td>?</td></tr>
| |
| − | <tr><td>4543</td><td>.xhp</td><td>helpex</td><td>AOO help files</td></tr>
| |
| − | <tr><td>0</td><td>.xrb</td><td>xmlex</td><td>?</td></tr>
| |
| − | <tr><td>1</td><td>.xrm</td><td>xrmex</td><td>xml readme file</td></tr>
| |
| − | <tr><td>0</td><td>.xtx</td><td>xtxex</td><td>?</td></tr>
| |
| − | <tr><td>0</td><td>.xxl</td><td>xmlex</td><td>?</td></tr>
| |
| − | <tr><td><b>8243</b></td><td colspan=3>Files to be scanned, total number of files is 438189</td></tr>
| |
| − | </table>
| |
| | | | |
| − | The tools are all separate executables meaning that for each file to be scanned a separate process
| |
| − | with the corresponding tool is started, especially in MS-Windows this leads to prolonged duration.
| |
| | | | |
| − | The results of the single scans is contained in a single sdf file, which are then passed to the next
| + | {| style="border-spacing:0;" |
| − | phase.
| + | | colspan="4" style="border:0.002cm solid #000000;padding:0.097cm;"| <center>'''Extensions scanned for text'''</center> |
| | | | |
| − | The resulting sdf file is generated in directory containing main (normally trunk).
| + | |- |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>'''Files'''</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>'''Extension'''</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>'''Tool'''</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>'''Desription'''</center> |
| | | | |
| − | The resulting foo.sdf.main has at the moment:
| + | |- |
| − | * 12.994.113 bytes
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>814</center> |
| − | * 72.492 lines
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.hrc</center> |
| − | * 45.341 lines of the 72.492 originate from the helpcontent2 module
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>transex3</center> |
| − | * 27.151 lines of the 72.492 originate from UI and simple messages
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>header for resource files</center> |
| | | | |
| − | At the moment localize runs with errors on Windows: jpropex, a shell script that calls a java
| + | |- |
| − | program does not run. Linux is OK.
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>98</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.properties</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>jpropex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>java property files</center> |
| | | | |
| − | '''Note:''' On Linux or MacOS you have to use a full qualified path to the output file. Otherwise you
| + | |- |
| − | won't get an output file and also no error. The tooling seems to be very error-prone.
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>1040</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.src</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>transex3</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>source for resource files</center> |
| | | | |
| | + | |- |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>15</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.tree</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>xhtex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>help files</center> |
| | | | |
| − | ===Merge with pootle server database=== | + | |- |
| − | The sdf file created by localize is transformed/converted into template pot files using
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>0</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.ulf</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>ulfex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>?</center> |
| | | | |
| − | <dl> | + | |- |
| − | This is a perl script that will call
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>53</center> |
| − | <dl> | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xcd</center> |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>cfgex</center> |
| − | | | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>xml files only in postprocess</center> |
| − | <code>oo2po -P en-US.sdf templates</code> | |
| − | |}
| |
| − | <dl> | |
| | | | |
| − | This set of pot files in the directory templates should now be updated on the [http://translate.sourceforge.net/wiki/pootle/index pootle server]. Copy
| + | |- |
| − | the complete templates directory in the po directory of the [http://translate.sourceforge.net/wiki/pootle/index pootle server] in the related project
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>314</center> |
| − | directory.
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xcs</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>cfgex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>xml file for java</center> |
| | | | |
| − | Assuming our project id is aoo34 and the [http://translate.sourceforge.net/wiki/pootle/index pootle server] is under /var/www/Pootle:
| + | |- |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>1365</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xcu</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>cfex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>xml files for UI</center> |
| | | | |
| − | All help files are located in a single module so it easy to distinguish between UI and help. First
| + | |- |
| − | move the help files (in order not to copy them into the UI directory):
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>0</center> |
| − | <dl> | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xgf</center> |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>xmlex</center> |
| − | | | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>?</center> |
| − | <code>cp -r templates/helpcontent2 \\ | |
| | | | |
| − | /var/www/Pootle/po/aoo34help/templates/helpcontent2 | + | |- |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>4543</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xhp</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>helpex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>AOO help files</center> |
| | | | |
| − | rm -rf helpcontent2</code>
| + | |- |
| − | |} | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>0</center> |
| − | <dl>
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xrb</center> |
| − | Then copy the UI files:
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>xmlex</center> |
| − | <dl>
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>?</center> |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | | | |
| − | <code>cp -r templates /var/www/Pootle/po/aoo34/templates</code> | |
| − | |} | |
| − | <dl> | |
| − | Update all existing languages to be aligned with the new templates.
| |
| − | <dl> | |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | | | |
| − | <code>cd /var/www/Pootle | |
| − | | |
| − | ./manage.py update_from_templates –project=aoo34
| |
| − | | |
| − | ./manage.py update_from_templates –project=aoo34help</code>
| |
| − | |} | |
| − | <dl>
| |
| − | or
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | | | |
| − | <code>./manage.py update_from_templates --project=aoo34 –language=de | |
| | | | |
| − | ./manage.py update_from_templates --project=aoo34help –language=de</code> | + | |- |
| − | |} | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>1</center> |
| − | <dl> | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xrm</center> |
| − | to update a specific language.
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>xrmex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>xml readme file</center> |
| | | | |
| − | Probably it is also possible to specify both projects with --project=aoo34, aoo34help and a list of
| + | |- |
| − | languages with --language=de,fr,es,... (not tested it yet)
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>0</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xtx</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>xtxex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>?</center> |
| | | | |
| | + | |- |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>0</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>.xxl</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>xmlex</center> |
| | + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>?</center> |
| | | | |
| − | ===Translation=== | + | |- |
| − | Translation takes place, either directly via the [http://translate.sourceforge.net/wiki/pootle/index pootle server] html frontend or via an offline editor.
| + | | style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:none;padding:0.097cm;"| <center>'''8243'''</center> |
| | + | | colspan="3" style="border-top:none;border-bottom:0.002cm solid #000000;border-left:0.002cm solid #000000;border-right:0.002cm solid #000000;padding:0.097cm;"| <center>Files to be scanned, total number of files is 438189</center> |
| | | | |
| − |
| |
| − | ===Translation online (“committer”)===
| |
| − | Translators with status as “committer” can work directly on the pootle server.
| |
| − |
| |
| − | However they have no glossary available, so it is highly possible that the same term is translated
| |
| − | differently in different modules and it happens for sure over time as different people work on the
| |
| − | translation.
| |
| − |
| |
| − | The changes are done directly in the po files, there are NO version control, and NO review control.
| |
| − | The separation of help content from UI content has many advantages but one huge disadvantage,
| |
| − | there are no control that e.g. menu names are identical in help as in the UI.
| |
| − |
| |
| − |
| |
| − | ===Translation offline (non “committer”)===
| |
| − | Many translators do not have “committer” status and can therefore not use the online [http://translate.sourceforge.net/wiki/pootle/index pootle server].
| |
| − |
| |
| − | The normal procedure is that a “committer” generates a zip file with all the files, mails the location
| |
| − | to the translator.
| |
| − |
| |
| − | The translator uses an offline tool like [http://www.poedit.net poedit].
| |
| − |
| |
| − | However they have no glossary available, so it is highly possible that the same term is translated
| |
| − | differently in different modules and it happens for sure over time as different people work on the
| |
| − | translation.
| |
| − |
| |
| − | Once the translation is complete the translator send the files back to the “committer” that updates
| |
| − | the po files behind the back of the [http://translate.sourceforge.net/wiki/pootle/index pootle server].
| |
| − |
| |
| − | There are no special quality checks in place to secure that the content of the translation are
| |
| − | consistent with earlier translations.
| |
| − |
| |
| − | If you update po files for an existing language (translated external) you should update the stores
| |
| − | with (after having copied to po files)
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>./manage.py update_translation_projects -–project=aoo34,aoo34help
| |
| − |
| |
| − | ./manage.py update_stores --project=aoo34,aoo34help –language=de</code>
| |
| | |} | | |} |
| − | <dl>
| + | The tools |
| | | | |
| | + | The tools are all separate executables meaning that for each file to be scanned a separate process with the corresponding tool is |
| | | | |
| − | ===Merge SVN===
| |
| − | Once the integrator decides that all parts are translated and quality controlled it is time to get the
| |
| − | texts back into [http://www.apache.org/dev/version-control.html SVN].
| |
| | | | |
| − | First step is to resync the database into the po files because otherwise the made changes are only in
| |
| − | the database. For example sync the UI strings for de back into the po files.
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>./manage.py sync_stores --project=aoo34 –language=<lang>
| |
| | | | |
| − | ./manage.py sync_stores --project=aoo34help –language=<lang></code>
| + | [[File:L10proc.pdf]] |
| − | |}
| |
| − | <dl>
| |
| − | The next step is create a new sdf based on this updated po files.
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>po2oo -l de -t en-US.sdf --keeptimestamp --skipsource \
| |
| − | | |
| − | <lang> new_<lang>.sdf
| |
| − | | |
| − | cp new_<lang>.sdf extras/l18n/source/<lang>/localize.sdf</code>
| |
| − | |}
| |
| − | <dl>
| |
| − | This command used the template en-US.sdf and created a new sdf file containing the new de
| |
| − | translations. If you skip the parameter skipsource the en-US source translations are also included in
| |
| − | the sdf file. Can be useful for some verification.
| |
| − | | |
| − | There is a utility gsicheck to check the files syntactically, this is however currently not in use.
| |
| − | | |
| − | '''Note:''' this step has to be repeated for each language.
| |
| − | | |
| − | | |
| − | ===Update [http://translate.sourceforge.net/wiki/pootle/index pootle server]===
| |
| − | In order to update the [http://translate.sourceforge.net/wiki/pootle/index pootle server] with the newest templates, we repeat earlier steps:
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>oo2po -P en-US.sdf templates</code>
| |
| − | |}
| |
| − | <dl>
| |
| − | Let assume we are currently in some temp directory and have existing po files in
| |
| − | '''aoo34/es/...''' and
| |
| − | have new templates in
| |
| − | '''aoo34/templates/...''' then we can create a new set of po files with
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>pot2po -t aoo34/es aoo34/templates es_new</code>
| |
| − | |}
| |
| − | <dl>
| |
| − | This command will merge the existing translations found in
| |
| − | '''aoo34/es''' and merge them with the new
| |
| − | templates and stores the new po files in *es_new*. This new po files can be copied in the Pootle
| |
| − | project directory
| |
| − | '''<pootle_install_dir>/po/aoo34/es'''. The database have to be synchronized with the
| |
| − | new po files.
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>./manage.py update_translation_projects -–project=aoo34,aoo34help
| |
| − | | |
| − | ./manage.py update_stores --project=aoo34,aoo34help –language=de</code>
| |
| − | |}
| |
| − | <dl>
| |
| − | | |
| − | | |
| − | ===Language build===
| |
| − | Use the normal command:
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>build –with-lang="..."</code>
| |
| − | |}
| |
| − | <dl>
| |
| − | When the office is built with configure switch --with-lang="..." then extras/l10n is built and the
| |
| − | localize.sdf files are rearranged. In l10n they are grouped according to language. Now they are
| |
| − | grouped according to module (and directory.) The sdf files in extras/l10/<platform>/misc/sdf are
| |
| − | zipped into one archive per module and delivered into
| |
| − | main/solver/340/<platform>/sdf/<module>.zip and then forgotten (at least for the processing of src
| |
| − | files.)
| |
| − | | |
| − | Resource files (src files) are processed when the other modules are built. The original src files
| |
| − | contain strings only for en_US in lines that look like
| |
| − | transex3 adds the missing languages by adding lines like
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>Text [en_US] = "...";</code>
| |
| − | |}
| |
| − | <dl>
| |
| − | transex3 adds the missing languages by adding lines like
| |
| − | <dl>
| |
| − | {|border="1" cellpadding="5" style="margin-left: 20px;"
| |
| − | |
| |
| − | <code>Text [de] = "...";</code>
| |
| − | |}
| |
| − | <dl>
| |
| − | By default all (available) languages are added not just the ones given to configure's --with-lang
| |
| − | switch. The augmented src files are placed in <module>/<platform>/misc/... These are then
| |
| − | aggregated into some srs files in <module>/<platform>/srs/. In a (or several) following step(s) the
| |
| − | srs files are aggregated into res files, one for each language.
| |
| − | | |
| − | The resulting res files are delivered to main/solver and become part of the installation sets. Multilanguage
| |
| − | versions contain res files for more than one language.
| |
| − | | |
| − | At runtime the ResMgr class from the tools module is responsible to use the resource files of the
| |
| − | currently selected language whenever a string is requested (as is the case for e.g. all button texts and
| |
| − | in general for all text visible in the GUI.)
| |
| − | | |
| − | | |
| − | ==File Formats==
| |
| − | Quite a number of different file formats are involved in the localization process. The following list
| |
| − | is not complete and may be inaccurate:
| |
| − | | |
| − | <table border="2px" cellpadding="5">
| |
| − | <tr><th colspan=2><b>Used file formats</b></th></tr>
| |
| − | <tr><th><b>Extension</b></th><th><b>Desription</b></th></tr>
| |
| − | <tr><td>.hrc</td><td>header for resource files</td></tr>
| |
| − | <tr><td>.properties</td><td>java property files</td></tr>
| |
| − | <tr><td>.po</td><td>contains the translated strings from a .pot file. Used on the [http://translate.sourceforge.net/wiki/pootle/index pootle server].</td></tr>
| |
| − | <tr><td>.pot</td><td>created by gettext from source files. Contains strings that need translation. Not
| |
| − | used by OpenOffice except as part of the [http://translate.sourceforge.net/wiki/pootle/index pootle server] update.</td></tr>
| |
| − | <tr><td>.res</td><td>created by transex3 from .srs files</td></tr>
| |
| − | <tr><td>.sdf</td><td>used to store localized/localizable strings and their origins. Comparable to .po
| |
| − | files</td></tr>
| |
| − | <tr><td>.src</td><td>source for resource files Most strings used in the GUI are defined in .src files</td></tr>
| |
| − | <tr><td>.srs</td><td>Made by rsc (which calls rscpp and rsc2) from multiple src files with *all* language strings included</td></tr>
| |
| − | <tr><td>.tree</td><td>help files</td></tr>
| |
| − | <tr><td>.ulf</td><td>?</td></tr>
| |
| − | <tr><td>.xcd</td><td>xml files only in postprocess</td></tr>
| |
| − | <tr><td>.xcs</td><td>xml file for java</td></tr>
| |
| − | <tr><td>.xcu</td><td>xml files for UI</td></tr>
| |
| − | <tr><td>.xgf</td><td>?</td></tr>
| |
| − | <tr><td>.xhp</td><td>AOO help files</td></tr>
| |
| − | <tr><td>.xliff</td><td>a format with the same usage of .po, but it has more functionalities and is
| |
| − | standardized</td></tr>
| |
| − | <tr><td>.xrb</td><td>?</td></tr>
| |
| − | <tr><td>.xrm</td><td>xml readme file</td></tr>
| |
| − | <tr><td>.xtx</td><td>?</td></tr>
| |
| − | <tr><td>.xxl</td><td>?</td></tr>
| |
| − | </table>
| |
| − | | |
| − |
| |
| − | ==Tools==
| |
| − | A large number of tools, implemented in a variety of languages (C++, Java, Perl, Python, sh) are
| |
| − | involved in the localization process. They mostly extract strings from source files and merge the
| |
| − | translated strings back in, or transform between different data formats.
| |
| − | | |
| − | The following list is not (yet) complete and may (still) be inaccurate:
| |
| − | <table border="2px" cellpadding="5">
| |
| − | <tr><th colspan=2><b>Used tools</b></th></tr>
| |
| − | <tr><th><b>tool</b></th><th><b>Desription</b></th></tr>
| |
| − | <tr><td>build</td><td>Standard build tool</td></tr>
| |
| − | <tr><td>cfgex</td><td>Called from localize_sl to translate .xcd .xcs .xcu files</td></tr>
| |
| − | <tr><td>gsicheck</td><td>Tool to do a syntax check on sdf files</td></tr>
| |
| − | <tr><td>helpex</td><td>Called from localize_sl to translate .xhp files</td></tr>
| |
| − | <tr><td>localize</td><td>Perl script to control localize_sl</td></tr>
| |
| − | <tr><td>localize_sl</td><td>Program that scan all sources for text strings</td></tr>
| |
| − | <tr><td>manage.py</td><td>A Python script to manage the [http://translate.sourceforge.net/wiki/pootle/index pootle server] database</td></tr>
| |
| − | <tr><td>oo2po</td><td>Standard program used to convert sdf files to po files</td></tr>
| |
| − | <tr><td>po2oo</td><td>Standard program used to convert po files to sdf files</td></tr>
| |
| − | <tr><td>rsc</td><td>Resource compiler</td></tr>
| |
| − | <tr><td>rscpp</td><td>Resource compiler</td></tr>
| |
| − | <tr><td>Rsc2</td><td>Resource converter</td></tr>
| |
| − | <tr><td>jpropex</td><td>Called from localize_sl to translate .properties files</td></tr>
| |
| − | <tr><td>ulfex </td><td>Called from localize_sl to extract strings from .ulf files. NOT USED</td></tr>
| |
| − | <tr><td>xhtex</td><td>Called from localize_sl to translate .htex files</td></tr>
| |
| − | <tr><td>xmlex</td><td>Called from localize to extract strings from .xrb .xxl .xgf files. NOT USED</td></tr>
| |
| − | <tr><td>xrmex</td><td>Called from localize_sl to translate .xrm files</td></tr>
| |
| − | </table>
| |
| − | | |
| − | | |
| − | ==Open issues==
| |
| − | The current localization workflow as outlined above has several drawbacks and plenty of room for
| |
| − | improvement.
| |
| − | | |
| − | The drawbacks as well as other ideas to make the l10n process robust and stable have been
| |
| − | collected below. These issues should be discussed either through the wiki or through the mailing
| |
| − | list.
| |
| − | | |
| − | When there is a proposed solution to all issues, that the community in general agree to, this
| |
| − | document will be converted into the proposed structure with a list of to-dos.
| |
| − | The list of issues is not prioritized.
| |
| − | | |
| − | ===Workflow is not a designed approach===
| |
| − | The current workflow is probably created as needed and as a consequence it has big portions of
| |
| − | “left-over” from
| |
| − | * the original openOffice (not localized)
| |
| − | * the SUN era
| |
| − | * the ongoing integration of openOffice in the Apache environment
| |
| − | * the l10n process is merely a “must” and not as interesting to work on as other parts
| |
| − | * The localization workflow is convoluted and hard to understand
| |
| − | * Much tooling is involved outside the build process.
| |
| − | * Some of this tooling seems to be lost after a disk crash of the old OpenOffice [http://translate.sourceforge.net/wiki/pootle/index pootle server]
| |
| − | | |
| − | This results in a manual process that is undocumented and known only to a select few.
| |
| − | Proposal
| |
| − | Once we agree on all issues a design paper on a proposed structure will be make available and be
| |
| − | basis for discussion.
| |
| − | Tools are writing in multiple languages
| |
| − | The tools involved are written in a variety of languages: C++, Java, Perl, and Python. This is not
| |
| − | bad in itself. For example it makes sense to parse Java property files with Java code. But there is
| |
| − | also C++ code for iterating over the tree of source files that uses hard coded lists of other
| |
| − | executables and scripts for processing individual files. That leads to many processes to be created
| |
| − | and destroyed, something that is notoriously slow on Windows.
| |
| − | Some of the tools are not used anymore. For example there are no .xtx, .xrb, .xxl, .xgf, or .xcd files.
| |
| − | Therefore the xbtxex and xmlex tools can be dropped. (May have already happened for xmlex)
| |
| − | Others are used but do not run (like the jpropex tool). And then there is our own preprocessor for
| |
| − | Page 20
| |
| − | handling resource files, which might be replaceable by the standard C/C++ preprocessor (which
| |
| − | parses the included hrc files anyway since they are included in C++ code.)
| |
| − | On Linux or MacOS you have to use a full qualified path to the output file. Otherwise you won't get
| |
| − | an output file and also no error. The tooling seems to be very error-prone. A lot of space for
| |
| − | improvements.
| |
| − | At the moment localize runs with errors on Windows: jpropex, a shell script that calls a java
| |
| − | program does not run. Linux is OK.
| |
| − | Streamline the number and implementation of the tools used for extraction and merging of
| |
| − | localizable strings. Use the right language for each task.
| |
| − | Proposal
| |
| − | Rewrite localize_sl, include the conversion programs (more efficiently).
| |
| − | Use gcc preprocessor instead of our own.
| |
| − | Use of sdf file
| |
| − | AOO uses its own non-standard file format (SDF) for handling localized strings. In order to use a
| |
| − | pootle server for the actual translation, all .sdf files have to be transformed into .po files and, after
| |
| − | translation, back into .sdf files. It should be also taken into consideration a future migration to xliff
| |
| − | format for translation handout.
| |
| − | Proposal
| |
| − | The .sdf files are merely intermediary files between the source files and the po files, and should be
| |
| − | eliminated.
| |
| − | The choice of .po or .xliff is not so easy:
| |
| − | 1) source <-> .po and .pot files
| |
| − | The advantage of this approach is that all translators knows .po
| |
| − | The very big disadvantage is that the format has no standard way of storing extra
| |
| − | information. We need to store the relative path of the originating source file (as in .sdf) in
| |
| − | order to be able to split the information.
| |
| − | 2) Source ↔ .xliff
| |
| − | The advantage of this approach is that we can store extra information as needed,
| |
| − | furthermore there are xliff editors out there including pootle server. It would also eliminate
| |
| − | the need for template files.
| |
| − | The disadvantage is that it is a new format, and offline translators would need to change
| |
| − | editor.
| |
| − | Page 21
| |
| − | Personally I would prefer .xliff since it makes programming a lot easier, but I think we need to
| |
| − | listen carefully to the translators.
| |
| − | Separate projects for UI and help
| |
| − | We should create 2 separate projects: one for UI and one for Help. And we should keep it separated
| |
| − | between versions because there will be probably some overlap with potential conflicts. Maybe an
| |
| − | approach of keeping two versions in pootle to give translators the chance to work on translation
| |
| − | after a release. And to allow future development toward the next release in parallel.
| |
| − | For example something like:
| |
| − | Apache OpenOffice 3.4 UI (aoo34)
| |
| − | Apache OpenOffice 3.4 Help (aoo34help)
| |
| − | Apache OpenOffice 4.0 UI (aoo40)
| |
| − | Apache OpenOffice 4.0 UI (aoo40help)
| |
| − | note: there are already 2 projects (a0034 and a0034help)
| |
| − | At the moment there are 276 different files to translate. Having that many files to translate makes it
| |
| − | more likely that the same term is translated differently and currently there are no glossary list
| |
| − | available.
| |
| − | Proposal
| |
| − | The process makes 2 files (.xliff or .po) for each language:
| |
| − | • localize_ui.<xx>
| |
| − | • localize_help.<xx>
| |
| − | • glossary.<xx> this file is not generated but maintained by the translators
| |
| − | These 3 files are delivered to the pootle server, translated and sent back for storage in SVN.
| |
| − | These files are handled as other files in respect to versions and releases.
| |
| − | Build process is highly manual and error prone
| |
| − | Total workflow should be automated.
| |
| − | A developer can insert the text directly in the source file or in a resource file, for the program both
| |
| − | ways work, however only a limited number of file extension types are today scanned for texts, so in
| |
| − | worst case some texts are never translated.
| |
| − | Integrate the string extraction into the build process. Most of the files that can contain localizable
| |
| − | strings are already part of the build system, mostly for the merge process. For example there are
| |
| − | make-rules for transforming and merging rsc files into .srs and then into .res files. Add rules for the
| |
| − | string extraction. This would allow developers to count new strings and the buildbot could extract
| |
| − | the new strings and upload them to the pootle server.
| |
| − | Page 22
| |
| − | Proposal
| |
| − | Add a new target in the makefiles (l10n_gen). Developers can then assign which files belong to this
| |
| − | target.
| |
| − | Localize_sl should be rewritten so it can run in multiple makefiles (no directory scanning).
| |
| − | Localize_sl will generate a snippet file that will be stored in a staging area (l10n/stating) and as last
| |
| − | step in the “build –all” process, l10n will be “built”, that is the snippets will be used to update the
| |
| − | single language files. With this process the language files will always be “ready” for use in the build
| |
| − | process.
| |
| − | However the pootle server still need to be manually updated.
| |
| − | Automatic update of pootle server
| |
| − | Translators need versioning possibilities
| |
| − | Offline translation needs to be controlled (delivery etc).
| |
| − | At the moment there are no computerized control over when a translation is ready for merge, nor
| |
| − | can a translation be given a status like e.g. “ready for review”.
| |
| − | pootle server can use SVN directly, and thereby offer version control, however at the moment this is
| |
| − | not used.
| |
| − | Proposal
| |
| − | Make a new subproject in main called l10n, this project contains the language files (basically extras
| |
| − | today), but also .mk file for generation.
| |
| − | The pootle server works direct on SVN. With this philosophy translators are seen as just another
| |
| − | breed of developer (bot work with languages) and we have all the advantages of a version system
| |
| − | when working on larger translations.
| |
| − | Content control
| |
| − | PO->SDF There are currently no control of the content quality (it is possible to make a translation,
| |
| − | where all translated text are “not-translated” and it will pass.
| |
| − | PO->SDF There are no check, that changed text are changed in the translation.
| |
| − | Proposal
| |
| − | Write a new tool that controls the tranlated part (based on the idea from poConsistency) and
| |
| − | integrate in the “build –all” process.
| |
| − | | |
| − |
| |
| − |
| |
| − |
| |
| − | | |
| − | [[File:Localization_AOO_5.jpg|center]]
| |
| − | [http://www.apache.org/dev/version-control.html SVN]
| |
| − | [http://translate.sourceforge.net/wiki/pootle/index pootle server]
| |
