https://wiki.openoffice.org/w/api.php?action=feedcontributions&user=Os&feedformat=atomApache OpenOffice Wiki - User contributions [en]2024-03-19T14:05:27ZUser contributionsMediaWiki 1.23.13https://wiki.openoffice.org/wiki/Notes2Notes22009-11-03T14:53:28Z<p>Os: </p>
<hr />
<div>{| align="right"<br />
| __TOC__<br />
|}<br />
<br />
{| cellpadding="5" border="0"<br />
|- <br />
| style="background:#d3dee8" |'''Hi! You just want to see the highlights?'''<br />
|-<br />
| style="background:#e2eaf1" |<br />
* What will it finally look like? Go to [[#Mockups_and_Screenshots|Mockups and Screenshots]].<br />
* Want a test drive? Tough users try the [[#Development_Releases|Development Releases]]!<br />
* You love detailed specifications? Look at [[#Status_and_Proposals|Status and Proposals]].<br />
|}<br />
<br />
==Introduction==<br />
<br />
[[Image:team_notes.png|center]]<br />
<br />
Notes2 addresses the revision of the notes functionality in OpenOffice.org Writer to improve its usability and accessibility. Main goal is to resolve the main issues of the current implementation, which are reported by users. Furthermore, the notes implementation should at least provide the functionality of the competitors in the same class. In long term considerations, it is intended to consistently implement the notes functionality in the other OpenOffice.org modules. The main target audience is the Small-Business-User.<br />
<br />
Notes2 is a project which has been started as one accepted project in Google's [[Summer_of_Code_2007|Summer of Code 2007]] (GSOC).<br />
<br />
The team working on the feature (in alphabetical order): <br />
<br />
{| class="prettytable" border="1" <br />
<br />
|-<br />
<br />
! Name<br />
<br />
! OOo Nickname<br />
<br />
! Role<br />
<br />
|-<br />
<br />
| Mathias Bauer<br />
<br />
| mba<br />
<br />
| Development<br />
<br />
|-<br />
<br />
| Uwe Fischer<br />
<br />
| ufi<br />
<br />
| Documentation<br />
<br />
|-<br />
<br />
| Christian Jansen<br />
<br />
| cj<br />
<br />
| User Experience<br />
<br />
|-<br />
<br />
|-<br />
<br />
| Christoph Noack<br />
<br />
| christophnoack<br />
<br />
| User Experience<br />
<br />
|-<br />
<br />
| Maximilian Odendahl<br />
<br />
| mod<br />
<br />
| Development<br />
<br />
|-<br />
<br />
| Éric Savary<br />
<br />
| es<br />
<br />
| QA<br />
<br />
|}<br />
<br />
<br />
Please note that the information in this Wiki is work in progress and may change.<br />
<br />
==Requirements==<br />
<br />
===Scope of the Work===<br />
<br />
* Provide direct and easy editing of notes to get rid of the old notes editing window.<br />
* Establish good accessibility for disabled users by providing keyboard shortcuts or using appropriate colors.<br />
* Integrate the new functions in the existing functionality, e.g. the OpenOffice.org Navigator.<br />
* Respect the needs for advanced text editing, e.g. spell checking, rich-text formatting or sorting/filtering of notes.<br />
* Create both fresh and usable visual design of the notes.<br />
<br />
=== Issues and Requests for Enhancements ===<br />
The work will address the issues and request for enhancements: <br />
[http://www.openoffice.org/issues/show_bug.cgi?id=767 767], [http://www.openoffice.org/issues/show_bug.cgi?id=1981 1981], [http://www.openoffice.org/issues/show_bug.cgi?id=4964 4964], [http://www.openoffice.org/issues/show_bug.cgi?id=5487 5487], [http://www.openoffice.org/issues/show_bug.cgi?id=6193 6193], [http://www.openoffice.org/issues/show_bug.cgi?id=9776 9776], [http://www.openoffice.org/issues/show_bug.cgi?id=23465 23465], [http://www.openoffice.org/issues/show_bug.cgi?id=32232 32232], [http://www.openoffice.org/issues/show_bug.cgi?id=45788 45788], [http://www.openoffice.org/issues/show_bug.cgi?id=48150 48150], [http://www.openoffice.org/issues/show_bug.cgi?id=62625 62625], [http://www.openoffice.org/issues/show_bug.cgi?id=63759 63759], [http://www.openoffice.org/issues/show_bug.cgi?id=74157 74157], [http://www.openoffice.org/issues/show_bug.cgi?id=61644 61644], [http://www.openoffice.org/issues/show_bug.cgi?id=80135 80135], and [http://www.openoffice.org/issues/show_bug.cgi?id=94514 94514].<br />
<br />
You can use a [http://www.openoffice.org/issues/buglist.cgi?Submit+query=Submit+query&issue_type=DEFECT&issue_type=ENHANCEMENT&issue_type=FEATURE&issue_type=PATCH&issue_status=UNCONFIRMED&issue_status=NEW&issue_status=STARTED&issue_status=REOPENED&issue_status=RESOLVED&email1=&emailtype1=exact&emailassigned_to1=1&email2=&emailtype2=exact&emailreporter2=1&issueidtype=include&issue_id=767%2C+1981%2C+4964%2C+5487%2C+6193%2C+9776%2C+23465%2C+32232%2C+45788%2C+48150%2C+62625%2C+63759%2C+74157%2C+61644%2C+80135&changedin=&votes=&chfieldfrom=&chfieldto=&chfieldvalue=&short_desc=&short_desc_type=allwords&long_desc=&long_desc_type=allwords&issue_file_loc=&issue_file_loc_type=fulltext&status_whiteboard=&status_whiteboard_type=fulltext&keywords=&keywords_type=anytokens&field0-0-0=noop&type0-0-0=noop&value0-0-0=&cmdtype=doit&namedcmd=accessibility&newqueryname=&order=Reuse+same+sort+as+last+time Issue Tracker query] to see all issues.<br />
<br />
=== Competitive Analysis ===<br />
<br />
Competitor software is discussed on the separate page [[Notes2_CompetitiveAnalysis|Competitive Analysis for Notes]].<br />
<br />
=== Requirements Derived from Use Cases === <br />
<br />
More detailed requirements have been collected and are published at [[Notes2_RequirementsFromUseCases|Requirements from Use Cases]] (partly German).<br />
<br />
===Definition of Terms===<br />
<br />
The following list contains some definitions used in the subsequent text (in alphabetical order):<br />
<br />
{| class="prettytable" border="1"<br />
|-<br />
<br />
! Term<br />
! Description<br />
! Shown in Graphic Below<br />
<br />
|-<br />
|Author<br />
|The user who edits or edited the Document.<br />
|no<br />
<br />
|-<br />
|Document<br />
|The Document is the file which contains the user's data. The type of the Document is dependent on the OpenOffice.org module used for editing, e.g. Writer text document, Calc spreadsheet document, Impress presentation document, Draw drawing document.<br />
|no<br />
<br />
|-<br />
|Document Content<br />
|The document content is dependent on the type of the Document. It includes e.g. <br />
*characters, strings, words, sentences, paragraphs <br />
*Sections (Writer) <br />
*Cells (Calc) <br />
*Slides (Impress) <br />
*Graphic Objects (Impress, Draw) <br />
* Input Fields (Base)<br />
|no<br />
<br />
|-<br />
|Document, Current<br />
|The Current Document is the Document that the user uses at the moment.<br />
|no<br />
<br />
|-<br />
|Note<br />
|The Note refers to all elements and data which belong to the Notes Functionality (e.g. Note Anchor, Note Window, Note Property Data, Note Connector Line). Simplified, the Note contains all user visible elements.<br />
|no<br />
<br />
|-<br />
|Note Anchor<br />
|General name for the reference position of the Note Data which is located in the Document Content. For improved clearness, the Notes Data is not displayed inside the Document. Consequently, an Anchor is necessary to indicate the reference position.<br />
|no<br />
<br />
|-<br />
|Note Anchor Area<br />
|A special version of the Anchor which refers to a group of Document Content elements. The Anchor Area It is defined by a starting point and a end point according to the reading direction of the Document.<br />
|yes<br />
<br />
|-<br />
|Note Anchor Point<br />
|A special version of the Anchor which refers to a position inside the Document. The Anchor Point may be located between characters and therefore treated as character.<br />
|yes<br />
<br />
|-<br />
|Note Connector Line<br />
|The notes connector line is the graphical representation of the relationship between Notes Anchor and Notes Window.<br />
|yes<br />
<br />
|-<br />
|Note Data<br />
|General name for the data which is stored for the Notes, e.g. Note User Data or Note Property Data.<br />
|no<br />
<br />
|-<br />
|Note Functionality<br />
|The generic term which refers to the revised implementation of Notes handling in OpenOffice.org<br />
|no<br />
<br />
|-<br />
|Note Property Data<br />
|The Note Property Data refers to all the data of a note which does not belong to the Note user data. It may contain: <br />
* Status of the Note Window (e.g. shown/hidden, rolled in/out, position) <br />
* Rating of the User (e.g. priority, task completion, filtered/unfiltered) <br />
* History (author who created the note, author of the last change, date of creation, date of last change) <br />
* Relationship to other Notes (e.g. reply status)<br />
|yes<br />
<br />
|-<br />
|Note Side Pane<br />
|The screen area next to the Document which is reserved for displaying the Notes Windows.<br />
|yes<br />
<br />
|-<br />
|Note User Data<br />
|The user defined content of the Note, e.g. text, graphics.<br />
|yes<br />
<br />
|-<br />
|Note Window<br />
|The screen area which contain the Note Data. This window is usually used to view or edit the Note User Data.<br />
|yes<br />
<br />
|}<br />
<br />
The following picture shows some examples for the terms defined above:<br />
[[Image:Notes2 DefinitionOfTerms.png|414px|thumb|center|Definition of Terms]]<br />
<br />
== Status and Proposals ==<br />
The following table lists the topics currently worked on and refers to the more '''detailed design proposals'''.<br />
<br />
{| class="prettytable" border="1"<br />
! Topic<br />
! Description<br />
! Status<br />
! Detailed Design<br />
<br />
|-<br />
| General Interaction<br />
| Interaction with the Notes Anchors and Note Windows inside the Writer document window.<br />
| Implementation started.<br />
| [[Notes_Design_WorkingWithNotes|Working with Notes]]<br />
[[Notes_Design_NotePreview|Notes Preview]]<br />
<br />
|-<br />
| Design of the Notes Side Pane<br />
| The notes side pane will extend the document pages to contain the notes windows. <br />
| Implementation finished.<br />
| [[Notes2_Design_NotesSidePane|Note Side Pane]]<br />
<br />
|-<br />
| Design of the Notes Windows<br />
| The notes window will contain the notes text and further information like the author's name and time and date information.<br />
| Implementation started. If necessary, the window will be extended by a scrollbar for the text.<br />
| [[Notes2_Design_NoteWindow|Note Window]]<br />
[[Notes2_NotePropertyData|Note Property Data]]<br />
<br />
|-<br />
| Design of the Notes Anchor<br />
| The Note Anchor is the markup inside the document text to show that there is some notes information present.<br />
| Proposal in discussion.<br />
| [[Notes2_Design_NoteAnchor|Note Anchor]]<br />
<br />
|-<br />
| Design of the Notes Connector Line<br />
| The Notes Connector Line is the visual connection between the notes anchor and the corresponding notes window.<br />
| Proposal in discussion.<br />
| [[Notes2_Design_NoteConnectorLine|Note Connector Line]]<br />
<br />
|-<br />
| Colors of the Notes Windows<br />
| Colors for the Notes Windows. These colors are shared with the Writer change tracking feature.<br />
| Implementation for Notes started. Implementation for the Writer change tracking feature finished.<br />
| [[Notes2_Design_MainColors|Main Colors]]<br />
<br />
|-<br />
| Distribution of the Notes Windows in the Side Pane<br />
| Placement of the Notes Windows inside the Notes Side Pane (e.g. size, position).<br />
| Proposal in discussion. Implementation started.<br />
| [[Notes2_Design_NoteWindowDistribution|Note Window Distribution]]<br />
<br />
|-<br />
| Menus and Keybindings<br />
| Covers menu changes and keybindings (e.g. shortcut for insert notes) for the interaction with the Notes functionality.<br />
| Proposal in discussion. Implementation started.<br />
| [[Notes2_Design_MenusAndKeybindings | Menus and Keybindings]]<br />
<br />
|-<br />
| Rich-Text-Formatting<br />
| Use of attributes to markup the notes text (e.g. bold text).<br />
| Implementation started.<br />
| n/a<br />
<br />
|-<br />
| Focus Visualization<br />
| The Note Windows of a currently viewed or edited Note is highlighted. <br />
| Proposals in discussion. Preliminary implementation started.<br />
| [[Notes2_Design_Visualization_of_Focus|Visualization of Focus]]<br />
<br />
|-<br />
| Notes Printing<br />
| Notes can be part of a document, therefore it is desired to print the Notes information. <br />
| Proposals are being created.<br />
| [[Notes2_Design_Printing|Notes Printing]]<br />
<br />
|-<br />
| Reply to Note<br />
| An author can reply to an existing Note of another author. For such a case, Notes interaction and visualization is optimized.<br />
| Proposal in discussion. Implementation started.<br />
| [[Notes2_Reply|Reply to Note]]<br />
<br />
|-<br />
| Search in Notes<br />
| The contents of the notes are sometimes as important as the original text. Therefore the user should be able to search text inside notes also.<br />
| Proposal in discussion. Implementation started.<br />
| [[Notes2_Search|Search in Notes]]<br />
|}<br />
<br />
== Roadmap ==<br />
<br />
=== Development Releases ===<br />
<br />
The functionality will be implemented step-by-step.<br />
<br />
'''Development releases are available for testing at [[Notes2 Development Releases]]!''' Since developer milestone m248, the new Notes functionality has been continuously integrated into the main branch. Please test this functionality and provide feedback, as this helps us to improve the Notes functionality until it will be released officially. Please refer to the page [[Notes2 Development Releases]] for more information and download locations.<br />
<br />
Information about the '''further development''' (including CWS status, issues and links) on what is planned can be found at [[Notes2_CWS | Notes2 CWS]].<br />
<br />
=== Official Releases ===<br />
<br />
Functionality available in the first release (OpenOffice.org 3.0):<br />
* Viewing and editing notes next to the document page<br />
* Accessibility support<br />
* The functionality which is already available in current version of OpenOffice.org<br />
* solid foundation for further improvements<br />
* Improved integration with the Writer change tracking feature (e.g. use the same colors for the authors of the document)<br />
* Time information for the notes (besides the already available date information)<br />
* Rich-text editing (e.g. bold text, numerations)<br />
* Online help<br />
* Support for the OpenOffice.org [http://qa.openoffice.org/ooQAReloaded/AutomationTeamsite/ooQA-TeamAutomation.html Testtool]<br />
<br />
<br />
For functionality expected in later releases (OpenOffice.org 3.1 and beyond), please have a look here:<br />
[http://wiki.services.openoffice.org/wiki/Notes2_CWS#planned_for_upcoming_CWS planned for upcoming versions]<br />
<br />
===Other Ideas===<br />
Other ideas are discussed on the separate page [[Notes2_OtherIdeas|Other Ideas for Notes]].<br />
<br />
==Mockups and Screenshots==<br />
<br />
This section contains mockups and screenshots:<br />
* ''Mockups'' are working models used for demonstration purposes. There is no real functionality at all, because the following pictures have been composed with graphic programs just to get a first impression how it will look like.<br />
* ''Screenshots'' show the behavior of the real development in OpenOffice.org Writer. Those pictures are updated more or less regularly.<br />
<br />
{| class="prettytable" border="0"<br />
| [[Image:Notes2_2007-09-08_GeneralMockups_View.png|512px|thumb|center|Mockup for viewing the notes]]<br />
| Mockup shows the mode for "viewing" the notes:<br />
* different kinds of notes anchors in the document text:<br />
** anchor point (represents a character)<br />
** anchor selection<br />
* a side pane next to the document page for displaying and arranging the notes windows (including some information in the ruler above)<br />
* display of author and date inside the notes windows<br />
* different colors for different authors<br />
* (idea) a placeholder to just start typing the notes text (anchor would be placed at current document text position)<br />
|-<br />
| [[Image:Notes2_2007-09-08_GeneralMockups_Edit.png|512px|thumb|center|Mockup for editing the notes]]<br />
| Mockup shows the mode for "editing" the notes:<br />
<br />
* visualization of currently edited note<br />
** continuous anchor connector line<br />
** highlighted notes anchor selection (slightly darker border lines)<br />
** shadow for the notes window (whereas the shadow for the document page is removed)<br />
* spell checking inside the notes windows<br />
|-<br />
| [[Image:Notes2_2007-09-08_GeneralMockups_HiddenNotesPane.png|512px|thumb|center|Mockup for a hidden notes pane]]<br />
| Mockup shows the mode for a "hidden notes" pane (just an early idea):<br />
<br />
* notes pane is completely hidden (including notes windows)<br />
* the ruler above provides some information on the notes<br />
** information that there are (at least) some notes present in the document<br />
** the notes pane can be shown by the control element in the ruler bar<br />
|-<br />
| [[Image:Notes2_Screenshot_CurrentVersion.png|512px|thumb|center|Notes in the current OpenOffice.org 3.0]]<br />
| Example for how the Notes2 functionality looks like on Ubuntu Linux 8.10 with "Clearlooks" visual theme.<br />
|}<br />
<br />
==Press Coverage and Comments==<br />
<br />
This section lists some of the noteworthy blog posts, web news or press coverage for the new Notes2 feature.<br />
<br />
* The '''German Linux magazine''' presents the new notes feature (along with the zoom slider and the multi-page view) in a short video (German): [http://www.linux-magazin.de/news/video_neues_in_open_office_writer_3_0 Linux-Magazin Online (Video: Neues in Open Office Writer 3.0)] (2008-09-26)<br />
* The German OpenOffice.org community contributed to the special magazine '''OpenOffice.org Spezial''' covering the release of OpenOffice.org 3.0. The article "Klebezettel ade! Die neuen Notizen im Writer" covers both the new features and the development. Here is the [http://wiki.services.openoffice.org/wiki/Image:Notes2_Ooo-Spezial_Noack_notes2.pdf article (PDF)], and more information about the magazine [http://www.ooo-spezial.de www.ooo-spezial.de].<br />
* The I-Team presented the '''Making the New Notes - Community, Cooperation, Concepts''' at the '''OpenOffice.org Conference 2008'''. Please have a look at [http://marketing.openoffice.org/ooocon2008/programme/friday.html OOoCon2008 Program Friday] where you can find the:<br />
** [http://marketing.openoffice.org/ooocon2008/programme/friday_abstracts.html#a1397 session abstract]<br />
** [http://marketing.openoffice.org/ooocon2008/programme/friday_1422.odp presentation slides in OpenOffice.org Impress format] (to be viewed best in the OpenOffice.org Impress presentation mode)<br />
** [http://marketing.openoffice.org/ooocon2008/programme/friday_1397.pdf presentation slides in PDF format]<br />
** [http://users2.ooodev.org/~ooocon2008/friday_1397.avi video coverage in AVI format]<br />
* http://www.jonobacon.org/2009/01/28/writing-a-book-with-free-software/: ''In terms of editing, it is also performed in OpenOffice.org, making use of its change tracking and notes features. This functionality is more than suitable for working on a book. I am also pleased to see this is being further refined in OpenOffice.org 3.1 with the ability to reply to notes. Interestingly, one of my editors uses Microsoft Word to edit. This means that my original .odt file is converted to a .doc file, edits are made in Word (using change tracking and notes) and then saved back in .odt format. ...Pretty nifty, eh?''<br />
* I'm really encouraged by these developments. With the ability to add notes to Writer it is one small step to move the right hand margin in for the main text and put the notes just outside the margin. That makes a great tool for minutes of meetings as you can use the note for recording who has an action against the relevant sentence in the minute. That's important because when you later edit a minute early in the minutes then all the "Action: Joe" notes will move correctly and stay with their relevant lines. Export to pdf and that's a handy minute taking tool.<br />
''Do you have additional stuff for us? Tell us or add it to this section. Thanks!''<br />
<br />
==Extensions related to Notes==<br />
<br />
For related OpenOffice.org Extensions, see [[Notes2_Extensions]]<br />
<br />
[[Category:Writer/Effort/Completed]][[Category:Notes2]]</div>Oshttps://wiki.openoffice.org/wiki/Mercurial_PilotMercurial Pilot2009-10-01T06:10:42Z<p>Os: /* Tips and Tricks */</p>
<hr />
<div>[[Category:Mercurial]][[Category:SCM]]<br />
=Mercurial Pilot=<br />
__TOC__<br />
<br />
{{Template:Documentation/Caution|The mercurial pilot has been successfully concluded, we are now in the implementation phase. The information presented here is still useful and will be part of the upcoming OOo Mercurial documentation}}<br />
<br />
==Purpose==<br />
To find out if our DSCM tool of choice can stand the harsh realities of the OOo development process. The OOo code base is pretty huge and we want to know if there are scalability and usability issues before we commit ourselves to a (hopefully) very long lasting SCM tool for OpenOffice.org. After all, SCM migrations are no fun, indeed.<br />
<br />
==Timeline==<br />
<br />
The OOo Mercurial pilot is expected to last at least 2 month. This is necessary to ensure that some "old enough" hg hosted child workspaces exist to check out how well the updating/re-sychronizing/re-basing works, something which is considerably painful with SVN. If we find that the pilot doesn't expose some hidden Mercurial problems, we will switch over completely to Mercurial as soon as the necessary infrastructure for a full scale hg usage is in place. If substantial problems surfaces during the pilot we might extend the pilot time frame to see if the problem can be overcome, or, in an extreme case, even set up a new pilot with git or bazaar.<br />
<br />
==The details==<br />
<br />
* The pilot will only cover the main development code line (aka trunk in SVN).<br />
* The master repository can be found here: http://hg.services.openoffice.org/hg/DEV300. This repository is hourly updated from the SVN trunk.<br />
* After a CWS is finished and QAed, it is necessary to pull/merge/push one final time in case of expected conflicts to help the REs with clean patches. <br />
* RE will pull from your hg hosted CWS, export your changes and apply it as patch to the SVN trunk.<br />
<br />
==What the pilot covers==<br />
* The complete CWS life cycle from a developers point of view, this includes cloning, committing, pulling, merging, re-basing and transplanting changesets from other CWSs <br />
* Parts of the CWS handling for RE: pulling from developers, merging.<br />
<br />
==What is not covered==<br />
* Transplanting changesets from a release code line to the main development line. RE will set up separate tests for this. <br />
* Integration is done via SVN, thus pushing to the master repository on the remote server by REs is not tested. No problem, this repository is only by convention different from developer repositories.<br />
<br />
==Caveats==<br />
<br />
Since every change will still have to be integrated via SVN there are a few caveats which you will need to consider if you plan to participate in the pilot:<br />
<br />
* Your changesets will loose their identity during integration. This might create problems if you cross merged changesets between several CWSs.<br />
* Integration will lump all changeset of your CWS into one single changeset. <br />
* The single changeset commit during integration will contain of course only a single commit log. If possible I'll lump all commit logs of your changesets together via scripting. Alternatively REs will accept a detailed commit log supplied by you.<br />
* The author of the hg changesets is lost as well because a RE engineer will do the integration commit to SVN. We include the original author info into the commit log.<br />
* Be careful with renaming files. If you do a lot of renaming you'll most probable cause a lot of stress on the integrator. Remember, the RE integrate your CWS via diff and patch. Also, all restrictions of SVN regarding the renaming of files/directories still apply. If you plan to do a lot of renaming please do it on a SVN based repository. Or better, if possible, wait for the final switch to hg.<br />
* Tinderboxes, buildbots etc will have to be adapted to Mercurial, this will take some time.<br />
<br />
==I'm an OOo domain developer, what to do if I want to participate?==<br />
<br />
The "outgoing" repositories on hg.services.openoffice.org will be created automatically for all child workspaces which are:<br />
* in state 'new' or later<br />
* flagged for Mercurial<br />
The Mercurial flag can be set either via the EIS webinterface or, starting with DEV300 m57, with the "--hg" switch to "cws create" (see example). If you are an OOo domain developer vou can use your SVN public key to access the outgoing repositories via SSH. It's not mandatory that you use this server to publish your changes, but at integration time latest the RE will naturally need to access your repository somewhere. It is planned to offer this server as central publishing point for OOo CWSs to ease the development coordination. An overview of active hg hosted CWSs can be found here: http://hg.services.openoffice.org/hg/cws.<br />
<br />
==Example==<br />
<br />
Let's say you want to participate in the pilot with your new CWS 'mycws', based on DEV300 m58. The steps are:<br />
(see [[OOo and Mercurial]] for general mercurial workflow info)<br />
<br />
* clone the repository the repository from the master server into a pristine local copy (best w/o tree)<br />
$ hg clone -U -r DEV300_m58 http://hg.services.openoffice.org/hg/DEV300 local_DEV300<br />
<br />
Why use '-r DEV300_m58' here? During the pilot the master repository will be hourly updated from /trunk of the SVN server. Since /trunk may not be buildable all the time (for instance during integrations) as a consequence the tip of the master hg repository may not be buildable as well. The situation will change after the switch to Mercurial. RE will only push complete milestones to the repository so you can safely use the tip of the repository. Why use a pristine intermediate local repository? Well, the current repository size is ~1.3 GB, cloning from the server can take from about 50min up to several hours if you are unlucky. You want to do this only once.<br />
* clone into your cws<br />
$ hg clone local_DEV300 mycws<br />
Note: don't use '-r DEV300_m58 here, if you followed the steps above you know that the tip of repository local_DEV300 is at milestone DEV300_m58. Without '-r' hg can use optimizations like hard linking the repository if both repository are on the same disk.[http://blogs.sun.com/GullFOSS/entry/mercurial_pilot_fast_local_clones] Besides you'll encounter the one unintuitive <strike>bug</strike>[http://www.selenic.com/mercurial/wiki/GitConcepts#head-944e6c0ababa25ffb900b7a5d4f6ee0391cb9092 feature] I have seen so far in Mercurial. If the clone is created with -r <tag>, the clone will contain everything up to the tag ''excluding'' the tag itself.<br />
* configure, etc<br />
* register your CWS with EIS<br />
$ cws create --hg -m m58 DEV300 mycws<br />
* A server side clone of the master repository will be automatically created to serve you as your outgoing repository. This saves time (270000 changesets less to push over the line) and server diskspace (hard links).<br />
* hack away in your CWS ...<br />
* push your changes to the outgoing repository for publishing if you feel so<br />
$ hg push ssh://hg@hg.services.openoffice.org/cws/mycws<br />
* pull/merge from the repository to re-synchronize your CWS from a new milestone, say DEV300_m62<br />
$ hg pull -r DEV300_m62 http://hg.services.openoffice.org/hg/DEV300<br />
$ hg merge<br />
Note: if you haven't published your repository yet, 'hg rebase' is an option to consider.<br />
* QA,fixes, etc<br />
* final pull/merge/push cycle to ease the integration into SVN trunk.<br />
<br />
==Mercurial documentation and other resources==<br />
<br />
* Main Mercurial web presence: http://www.selenic.com/mercurial/wiki/<br />
* Mercurial book: [http://hgbook.red-bean.com/ Mercurial: The Definitive Guide]<br />
<br />
==Mercurial extensions you might want to have look at==<br />
<br />
* hg rebase: git style rebasing instead of pull/merge<br />
* hg transplant: cherry picking of changsets from other CWSs<br />
* hg win32text: if you work on windows<br />
* hg purge: purge your tree from untracked items<br />
* hg mq: quilt style patch queue, also enables the 'hg strip' command<br />
* hg collapse: collapses a sequence of commits in one<br />
<br />
==Things that may surprise former CVS or SVN users==<br />
<br />
* hg clone -r <tag> <from_rep> <to_rep> will clone everything up to the tagged revsion ''excluding'' the tag. See explanation above.<br />
* hg resolve does a very different thing than svn resolve. Read the man page.<br />
* Many hg commands work on the whole repository if no path is specified, no ''matter where your current working directory is inside the source tree''. Example: "cd DEV300/sw/source; hg status" will compute the status of the whole source tree (which might take a minute depending on IO bandwidth). Use "cd DEV300/sw/source; hg status ."<br />
<br />
==Tips and Tricks==<br />
<br />
* After pulling/merging: <br />
** "hg log --follow-first" is a convenient way to display your CWS changes at the top of the log.<br />
** "hg log --follow-first -P <original_clone_milestone>" will show only the CWS changesets.<br />
** "hg outgoing" works as well, of course.<br />
* Combined diff which contains exactly the changes of your CWS without anything pulled from master<br />
** If your current milestone tag is locally available: hg diff -r <current_milestone_tag><br />
** If your current milestone tag is *not* locally available (might happen, see above): <br />
***search for your last pull/merge from the master with "hg log -m"<br />
*** "hg diff -r <second_parent_of_last_master_merge>"<br />
** Alternatively, if you just did a pull/merge from master: "hg export --switch-parent tip"<br />
* The default output format of some commands does not suit you? Use styles:<br />
**"hg log --style=compact"<br />
**"hg outgoing --style=changelog"<br />
* Or create your own format with the template engine:<br />
**"hg outgoing --template '{date|shortdate} {author|person} {desc}\n' --newest-first"<br />
* Accessing repository over ssh using Tortoise Hg on Windows<br />
** If you want to use the ssh client of your Cygwin shell (and also ssh-agent), add the following to the [ui] section of your mercurial.ini: ssh = ssh<br />
<br />
==Mercurial on Windows/cygwin (experimental)==<br />
<br />
Using hg from cygwin sometimes fails miserably. TortoiseHg (http://bitbucket.org/tortoisehg/stable/wiki/Home) is worth a try.<br />
<br />
To get rid of cygwin's hg rename /usr/bin/hg.<br />
<br />
You can decide which ssl support you use in the cygwin shell. <br />
If you want to use the ssh-agent then you have to specify the following line in the [ui] section of ~/.hgrc:<br />
ssh = <Windows path to cygwin>\bin\ssh.exe<br />
<br />
For the graphical interface of TortoiseHg Putty's pageant is the default ssh key provider. TortoiseHg allows easy graphical access to hg repositories within the Windows Explorer.</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-05-29T05:32:32Z<p>Os: /* Identified Hotspots */</p>
<hr />
<div>{{Performance}}<br />
== Investigation and Profiling of Writer Load/Save Performance ==<br />
We started a systematic profiling of the load/save performance on a current milestone (DEV300_m45). Im using Intel vTune on Windows and cachegrind (with cache-simulation) on Linux. On each platform each of the four documents (see Testdocuments below) is profiled for a load and a save procedure. Each measurement is done twice. In total, this will result in:<br />
2 (platforms/profilers) * 4 (documents) * 2 (load/save) * 2 (measurements) = 32 measurements in total<br />
The analysis of the data should:<br />
* show, if the profiler output is stable and reproducable<br />
* show, if there are any differences between platforms and profiler (and for cachegrind: does cache-simulation result in meaningful additional accuracy?)<br />
* identify the hotspots of the current implementation<br />
* be a basis to evalute the progress by optimizations<br />
<br />
=== Birdsview Callgrind (Contributions To StoreToUrl DEV300_m45) ===<br />
{|class="wikitable" style="text-align:center"<br />
|+ Contributions to StoreToUrl<br />
|-style="font-weight: bold;"<br />
| Document||Lib||Method||Instructions fetched||Cycle Cost Est.||Instructions fetched||Cycle Cost Est.<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| ScienceThesis 1||libsfx||SfxBaseModel::StoreAsUrl||11592161864||12700226484||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||4567386009||5193895809||39.40%||40.90%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||2336567459||2525183559||20.16%||19.88%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||4535296593||4794222283||39.12%||37.75%<br />
|-<br />
| ||||other||152911803||186924833||1.32%||1.47%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| ScienceThesis 2||libsfx||SfxBaseModel::StoreAsUrl||11144535020||12250598830||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||4575357778||5189916868||41.05%||42.36%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||1925216412||2116790032||17.27%||17.28%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||4535327879||4792981289||40.70%||39.12%<br />
|-<br />
| ||||other||108632951||150910641||0.97%||1.23%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Manual 1||libsfx||SfxBaseModel::StoreAsUrl||2019415631||2238108511||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||1239343887||1399947937||61.37%||62.55%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||607055773||642691573||30.06%||28.72%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||140565338||157789035||6.96%||7.05%<br />
|-<br />
| ||||other||32450633||37679966||1.61%||1.68%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Manual 2||libsfx||SfxBaseModel::StoreAsUrl||2043487795||2268710495||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||1262263271||1428106081||61.77%||62.95%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||608153207||644497207||29.76%||28.41%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||141902391||154627511||6.94%||6.82%<br />
|-<br />
| ||||other||31168926||41479696||1.53%||1.83%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Spec 1||libsfx||SfxBaseModel::StoreAsUrl||17995716426||20443306916||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||16713230798||19057592468||92.87%||93.22%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||619569536||652497286||3.44%||3.19%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||605712688||663294858||3.37%||3.24%<br />
|-<br />
| ||||other||121121223||133131403||0.67%||0.65%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Spec 2||libsfx||SfxBaseModel::StoreAsUrl||18062701503||20537474953||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||16681477861||19055676231||92.35%||92.78%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||654539677||685122457||3.62%||3.34%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||605562742||663544862||3.35%||3.23%<br />
|-<br />
| ||||other||121121223||133131403||0.67%||0.65%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| MailMerge 1||libsfx||SfxBaseModel::StoreAsUrl||40777455765||53626856665||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||23118516732||34395921752||56.69%||64.14%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||17475120469||19021058129||42.85%||35.47%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||161846924||179849124||0.40%||0.34%<br />
|-<br />
| ||||other||21971640||30027660||0.05%||0.06%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| MailMerge 2||libsfx||SfxBaseModel::StoreAsUrl||34409955066||45317878876||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||22931786501||33059699851||66.64%||72.95%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||11294919676||12054180246||35.03%||26.60%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||161097067||173977107||0.47%||0.38%<br />
|-<br />
| ||||other||22151822||30021672||0.06%||0.07%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| std. deviation||libsfx||SfxBaseModel::StoreAsUrl||||||9.31%||9.21%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||||||1.17%||2.53%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||||||23.04%||23.30%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||||||0.61%||2.21%<br />
|-<br />
| ||||other||||||16.88%||12.56%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|}<br />
<br />
=== Callgrind Save XML-Generation (Contributions To SaveAsOwnFormat DEV300_m45) ===<br />
The following profiling files have been generated with:<br />
valgrind --tool=callgrind "--toggle-collect=*SaveAsOwnFormat*" ./soffice.bin<br />
Rerunning the save procedure shows them to have a high reproducability (~1% deviation for SaveAsOwnFormat as a whole).<br />
* [[media:Callgrind_DEV300_m45_Save1.tgz]] MailMerge, Manual<br />
* [[media:Callgrind_DEV300_m45_Save2.tgz]] ScienceThesis<br />
* [[media:Callgrind_DEV300_m45_Save3.tgz]] Odf Spec<br />
To analyse these files open them with kcachegrind or callgrind_annotate.<br />
<br />
=== Callgrind Load XML-Parsing (Contributions To LoadOwnFormat DEV300_m45) ===<br />
The following profiling files have been generated with:<br />
valgrind --tool=callgrind "--toggle-collect=*LoadOwnFormat*" --callgrind-out-file=OdfLoad1.cg ./soffice.bin<br />
* [[media:Callgrind_DEV300_m45_Load1.tgz]] MailMerge, ScienceThesis<br />
* [[media:Callgrind_DEV300_m45_Load2.tgz]] Manual, Odf Spec<br />
To analyse these files open them with kcachegrind or callgrind_annotate.<br />
<br />
=== Benchmarking with vTune ===<br />
In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document.<br />
== Optimizations ==<br />
=== Implemented Optimizations ===<br />
==== Save time index entries ====<br />
[http://qa.openoffice.org/issues/show_bug.cgi?id=57008 Issue 57008]<br />
<br />
Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.<br />
<br />
==== Conversion of Hyperlinks ====<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=100683 issue 100683]<br />
<br />
* Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
<br />
Even worse: <br />
[http://www.openoffice.org/issues/show_bug.cgi?id=50983 issue 50983]<br />
<br />
* Saving a file with a lot of fragment URLs "#bookmarkname" to a network share takes a lot of time. In comparision: DEV300 m41 takes about 1:55 min while os128 takes only 28s to save the document to a network share. <br />
<br />
To make sure the file URLs are correctly normalized the dialog code to insert all kinds of links has to call the normalization. This applies to Insert/Hyperlink, Insert/Picture from File and others.<br />
<br />
==== String Indexed Access of PropertySets ====<br />
<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=99568 issue 99568]<br />
<br />
* Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
* The replacement for the SfxItemPropertyMap that uses an std::hash_map is ready. After changing a lot of code in the applications as well as in svtools, sfx2, svx and others I started to compare the load/save times.<br />
<br />
*The result is not as expected. In [[Media:Odfsave_withhash.ods]] you can see that SfxItemPropertyMap::getByName() takes longer than before. The new function takes about 5.3 s totally. These are about 1.7 s more than it's predecessor SfxItemPropertyMap::GetByName() required. The time is consumed mostly in the _M_Find<::rtl::OUString> method of the hash_map implementation.<br />
* One of the probable reasons is the fact that the sorted access to properties eliminated a lot of string comparisons.<br />
<br />
==== Iteration over Frame Collections ====<br />
[http://qa.openoffice.org/issues/show_bug.cgi?id=101084 Issue 101084]<br />
<br />
The methods SwDoc::GetFlyCount and SwDoc::GetFlyNum contribute more than 13 % of the instructions to SaveAsOwnFormat for the MailMerge document. The iteration over the frames array is O(n^2). Suggested solution:<br />
* make the frames collections support XEnumerationAccess<br />
<br />
==== Compressed files do not need to be compressed again in Storage ====<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=100722 Issue 100722]<br />
* The large contribution of SfxMedium::Commit for documents "ScienceThesis" and "Manual" to StoreAsUrl in the callgrind analysis are attributed to the pictures in the document. We are investigating, if it might help to store image files that are already compressed (JPEG for examples) directly to the storage without trying to compress again in vain.<br />
<br />
=== Identified Hotspots ===<br />
<br />
==== Using XMultiPropertySet where XTolerantMultiPropertySet might suffice and be more performant ====<br />
* To decide which properties have to be saved xmloff uses the interface methods css::beans::XPropertyState::getPropertyStates() and css::beans::XMultiPropertySet::getPropertyValues(). It could also use the interface css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant() which is not implemented for Writer's UNO objects. <br />
* Saving Writer's text content is done by iterating over the paragraphs and iterating over so-called text portions within the paragraphs. Text portions are parts of the paragraph that have a single attribute set, text fields, redline portions, inline anchored frames etc. It might make sense to detect their properties at construction time and preset their css::uno::XTolerantMultiPropertySet interface. And the moment a text portion is created that adds a bookmark to remember it's position. The impact on real documents is not yet checked.<br />
* A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.<br />
==== Font Fallback ====<br />
The huge contribution of GenerateAndStoreThumbnail in the callgrind measurements for some of the documents is attributed to substitution matching for missing fonts. This might be an issue to investigate.<br />
<br />
== Test Documents ==<br />
<br />
* http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt (long specification document)<br />
* http://www.fonaso.com/translations/FonasoVersatil-German-V1.8.doc (short document with images)<br />
* https://eldorado.uni-dortmund.de/bitstream/2003/21520/2/Jackler.doc (~150 pages scientific paper with tables and images)<br />
* [[media:MailMerge.odt]] MailMerge Document (~600 pages)<br />
Microsoft Word Documents where loaded and saved as odt before profiling.<br />
<br />
[[Category:Performance]]</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-04-03T05:34:48Z<p>Os: /* Conversion of Hyperlinks */</p>
<hr />
<div>{{Performance}}<br />
== Investigation and Profiling of Writer Load/Save Performance ==<br />
We started a systematic profiling of the load/save performance on a current milestone (DEV300_m45). Im using Intel vTune on Windows and cachegrind (with cache-simulation) on Linux. On each platform each of the four documents (see Testdocuments below) is profiled for a load and a save procedure. Each measurement is done twice. In total, this will result in:<br />
2 (platforms/profilers) * 4 (documents) * 2 (load/save) * 2 (measurements) = 32 measurements in total<br />
The analysis of the data should:<br />
* show, if the profiler output is stable and reproducable<br />
* show, if there are any differences between platforms and profiler (and for cachegrind: does cache-simulation result in meaningful additional accuracy?)<br />
* identify the hotspots of the current implementation<br />
* be a basis to evalute the progress by optimizations<br />
<br />
=== Birdsview Callgrind (Contributions To StoreToUrl DEV300_m45) ===<br />
{|class="wikitable" style="text-align:center"<br />
|+ Contributions to StoreToUrl<br />
|-style="font-weight: bold;"<br />
| Document||Lib||Method||Instructions fetched||Cycle Cost Est.||Instructions fetched||Cycle Cost Est.<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| ScienceThesis 1||libsfx||SfxBaseModel::StoreAsUrl||11592161864||12700226484||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||4567386009||5193895809||39.40%||40.90%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||2336567459||2525183559||20.16%||19.88%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||4535296593||4794222283||39.12%||37.75%<br />
|-<br />
| ||||other||152911803||186924833||1.32%||1.47%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| ScienceThesis 2||libsfx||SfxBaseModel::StoreAsUrl||11144535020||12250598830||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||4575357778||5189916868||41.05%||42.36%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||1925216412||2116790032||17.27%||17.28%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||4535327879||4792981289||40.70%||39.12%<br />
|-<br />
| ||||other||108632951||150910641||0.97%||1.23%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Manual 1||libsfx||SfxBaseModel::StoreAsUrl||2019415631||2238108511||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||1239343887||1399947937||61.37%||62.55%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||607055773||642691573||30.06%||28.72%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||140565338||157789035||6.96%||7.05%<br />
|-<br />
| ||||other||32450633||37679966||1.61%||1.68%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Manual 2||libsfx||SfxBaseModel::StoreAsUrl||2043487795||2268710495||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||1262263271||1428106081||61.77%||62.95%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||608153207||644497207||29.76%||28.41%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||141902391||154627511||6.94%||6.82%<br />
|-<br />
| ||||other||31168926||41479696||1.53%||1.83%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Spec 1||libsfx||SfxBaseModel::StoreAsUrl||17995716426||20443306916||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||16713230798||19057592468||92.87%||93.22%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||619569536||652497286||3.44%||3.19%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||605712688||663294858||3.37%||3.24%<br />
|-<br />
| ||||other||121121223||133131403||0.67%||0.65%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Spec 2||libsfx||SfxBaseModel::StoreAsUrl||18062701503||20537474953||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||16681477861||19055676231||92.35%||92.78%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||654539677||685122457||3.62%||3.34%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||605562742||663544862||3.35%||3.23%<br />
|-<br />
| ||||other||121121223||133131403||0.67%||0.65%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| MailMerge 1||libsfx||SfxBaseModel::StoreAsUrl||40777455765||53626856665||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||23118516732||34395921752||56.69%||64.14%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||17475120469||19021058129||42.85%||35.47%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||161846924||179849124||0.40%||0.34%<br />
|-<br />
| ||||other||21971640||30027660||0.05%||0.06%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| MailMerge 2||libsfx||SfxBaseModel::StoreAsUrl||34409955066||45317878876||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||22931786501||33059699851||66.64%||72.95%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||11294919676||12054180246||35.03%||26.60%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||161097067||173977107||0.47%||0.38%<br />
|-<br />
| ||||other||22151822||30021672||0.06%||0.07%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| std. deviation||libsfx||SfxBaseModel::StoreAsUrl||||||9.31%||9.21%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||||||1.17%||2.53%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||||||23.04%||23.30%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||||||0.61%||2.21%<br />
|-<br />
| ||||other||||||16.88%||12.56%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|}<br />
<br />
=== Callgrind Save XML-Generation (Contributions To SaveAsOwnFormat DEV300_m45) ===<br />
The following profiling files have been generated with:<br />
valgrind --tool=callgrind "--toggle-collect=*SaveAsOwnFormat*" soffice.bin<br />
Rerunning the save procedure shows them to have a high reproducability (~1% deviation for SaveAsOwnFormat as a whole).<br />
* [[media:Callgrind_DEV300_m45_Save1.tgz]] MailMerge, Manual<br />
* [[media:Callgrind_DEV300_m45_Save2.tgz]] ScienceThesis<br />
* [[media:Callgrind_DEV300_m45_Save3.tgz]] Odf Spec<br />
To analyse these files open them with kcachegrind or callgrind_annotate.<br />
=== Benchmarking with vTune ===<br />
In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document.<br />
== Optimizations ==<br />
=== Implemented Optimizations ===<br />
==== Save time index entries ====<br />
[http://qa.openoffice.org/issues/show_bug.cgi?id=57008 Issue 57008]<br />
<br />
Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.<br />
<br />
=== Identified Hotspots ===<br />
==== Conversion of Hyperlinks ====<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=100683 issue 100683]<br />
<br />
* Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
<br />
Even worse: <br />
[http://www.openoffice.org/issues/show_bug.cgi?id=50983 issue 50983]<br />
<br />
* Saving a file with a lot of fragment URLs "#bookmarkname" to a network share takes a lot of time. In comparision: DEV300 m41 takes about 1:55 min while os128 takes only 28s to save the document to a network share. <br />
<br />
To make sure the file URLs are correctly normalized the dialog code to insert all kinds of links has to call the normalization. This applies to Insert/Hyperlink, Insert/Picture from File and others.<br />
<br />
==== String Indexed Access of PropertySets ====<br />
<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=99568 issue 99568]<br />
<br />
* Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
* The replacement for the SfxItemPropertyMap that uses an std::hash_map is ready. After changing a lot of code in the applications as well as in svtools, sfx2, svx and others I started to compare the load/save times.<br />
<br />
*The result is not as expected. In [[Media:Odfsave_withhash.ods]] you can see that SfxItemPropertyMap::getByName() takes longer than before. The new function takes about 5.3 s totally. These are about 1.7 s more than it's predecessor SfxItemPropertyMap::GetByName() required. The time is consumed mostly in the _M_Find<::rtl::OUString> method of the hash_map implementation.<br />
* One of the probable reasons is the fact that the sorted access to properties eliminated a lot of string comparisons.<br />
<br />
==== Using XMultiPropertySet where XTolerantMultiPropertySet might suffice and be more performant ====<br />
* To decide which properties have to be saved xmloff uses the interface methods css::beans::XPropertyState::getPropertyStates() and css::beans::XMultiPropertySet::getPropertyValues(). It could also use the interface css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant() which is not implemented for Writer's UNO objects. <br />
* Saving Writer's text content is done by iterating over the paragraphs and iterating over so-called text portions within the paragraphs. Text portions are parts of the paragraph that have a single attribute set, text fields, redline portions, inline anchored frames etc. It might make sense to detect their properties at construction time and preset their css::uno::XTolerantMultiPropertySet interface. And the moment a text portion is created that adds a bookmark to remember it's position. The impact on real documents is not yet checked.<br />
* A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.<br />
==== Font Fallback ====<br />
The huge contribution of GenerateAndStoreThumbnail in the callgrind measurements for some of the documents is attributed to substitution matching for missing fonts. This might be an issue to investigate.<br />
==== Compressed files do not need to be compressed again in Storage ====<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=100722 Issue 100722]<br />
* The large contribution of SfxMedium::Commit for documents "ScienceThesis" and "Manual" to StoreAsUrl in the callgrind analysis are attributed to the pictures in the document. We are investigating, if it might help to store image files that are already compressed (JPEG for examples) directly to the storage without trying to compress again in vain.<br />
==== Iteration over Frame Collections ====<br />
The methods SwDoc::GetFlyCount and SwDoc::GetFlyNum contribute more than 13 % of the instructions to SaveAsOwnFormat for the MailMerge document. The iteration over the frames array is O(n^2). Suggested solution:<br />
* add a method SwXFrames::getAllFrames returning a Sequence<XFrame><br />
* get rid of the GetByIndex accesses in xmloff<br />
<br />
== Test Documents ==<br />
<br />
* http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt (long specification document)<br />
* http://www.fonaso.com/translations/FonasoVersatil-German-V1.8.doc (short document with images)<br />
* https://eldorado.uni-dortmund.de/bitstream/2003/21520/2/Jackler.doc (~150 pages scientific paper with tables and images)<br />
* MailMerge Document (~600 pages, to be uploaded)<br />
Microsoft Word Documents where loaded and saved as odt before profiling.<br />
<br />
[[Category:Performance]]</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-04-01T13:01:16Z<p>Os: /* Conversion of Hyperlinks */</p>
<hr />
<div>{{Performance}}<br />
== Investigation and Profiling of Writer Load/Save Performance ==<br />
We started a systematic profiling of the load/save performance on a current milestone (DEV300_m45). Im using Intel vTune on Windows and cachegrind (with cache-simulation) on Linux. On each platform each of the four documents (see Testdocuments below) is profiled for a load and a save procedure. Each measurement is done twice. In total, this will result in:<br />
2 (platforms/profilers) * 4 (documents) * 2 (load/save) * 2 (measurements) = 32 measurements in total<br />
The analysis of the data should:<br />
* show, if the profiler output is stable and reproducable<br />
* show, if there are any differences between platforms and profiler (and for cachegrind: does cache-simulation result in meaningful additional accuracy?)<br />
* identify the hotspots of the current implementation<br />
* be a basis to evalute the progress by optimizations<br />
<br />
=== Birdsview Callgrind (Contributions To StoreUrl DEV300_m45) ===<br />
{|class="wikitable" style="text-align:center"<br />
|+ Contributions to StoreToUrl<br />
|-style="font-weight: bold;"<br />
| Document||Lib||Method||Instructions fetched||Cycle Cost Est.||Instructions fetched||Cycle Cost Est.<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| ScienceThesis 1||libsfx||SfxBaseModel::StoreAsUrl||11592161864||12700226484||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||4567386009||5193895809||39.40%||40.90%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||2336567459||2525183559||20.16%||19.88%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||4535296593||4794222283||39.12%||37.75%<br />
|-<br />
| ||||other||152911803||186924833||1.32%||1.47%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| ScienceThesis 2||libsfx||SfxBaseModel::StoreAsUrl||11144535020||12250598830||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||4575357778||5189916868||41.05%||42.36%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||1925216412||2116790032||17.27%||17.28%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||4535327879||4792981289||40.70%||39.12%<br />
|-<br />
| ||||other||108632951||150910641||0.97%||1.23%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Manual 1||libsfx||SfxBaseModel::StoreAsUrl||2019415631||2238108511||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||1239343887||1399947937||61.37%||62.55%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||607055773||642691573||30.06%||28.72%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||140565338||157789035||6.96%||7.05%<br />
|-<br />
| ||||other||32450633||37679966||1.61%||1.68%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Manual 2||libsfx||SfxBaseModel::StoreAsUrl||2043487795||2268710495||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||1262263271||1428106081||61.77%||62.95%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||608153207||644497207||29.76%||28.41%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||141902391||154627511||6.94%||6.82%<br />
|-<br />
| ||||other||31168926||41479696||1.53%||1.83%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Spec 1||libsfx||SfxBaseModel::StoreAsUrl||17995716426||20443306916||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||16713230798||19057592468||92.87%||93.22%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||619569536||652497286||3.44%||3.19%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||605712688||663294858||3.37%||3.24%<br />
|-<br />
| ||||other||121121223||133131403||0.67%||0.65%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Spec 2||libsfx||SfxBaseModel::StoreAsUrl||18062701503||20537474953||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||16681477861||19055676231||92.35%||92.78%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||654539677||685122457||3.62%||3.34%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||605562742||663544862||3.35%||3.23%<br />
|-<br />
| ||||other||121121223||133131403||0.67%||0.65%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| MailMerge 1||libsfx||SfxBaseModel::StoreAsUrl||40777455765||53626856665||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||23118516732||34395921752||56.69%||64.14%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||17475120469||19021058129||42.85%||35.47%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||161846924||179849124||0.40%||0.34%<br />
|-<br />
| ||||other||21971640||30027660||0.05%||0.06%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| MailMerge 2||libsfx||SfxBaseModel::StoreAsUrl||34409955066||45317878876||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||22931786501||33059699851||66.64%||72.95%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||11294919676||12054180246||35.03%||26.60%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||161097067||173977107||0.47%||0.38%<br />
|-<br />
| ||||other||22151822||30021672||0.06%||0.07%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| std. deviation||libsfx||SfxBaseModel::StoreAsUrl||||||9.31%||9.21%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||||||1.17%||2.53%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||||||23.04%||23.30%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||||||0.61%||2.21%<br />
|-<br />
| ||||other||||||16.88%||12.56%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|}<br />
<br />
=== Benchmarking with vTune ===<br />
In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document.<br />
== Optimizations ==<br />
=== Implemented Optimizations ===<br />
==== Save time index entries ====<br />
[http://qa.openoffice.org/issues/show_bug.cgi?id=57008 Issue 57008]<br />
<br />
Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.<br />
<br />
=== Identified Hotspots ===<br />
==== Conversion of Hyperlinks ====<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=100683 issue 100683]<br />
<br />
* Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
<br />
==== String Indexed Access of PropertySets ====<br />
<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=99568 issue 99568]<br />
<br />
* Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
* The replacement for the SfxItemPropertyMap that uses an std::hash_map is ready. After changing a lot of code in the applications as well as in svtools, sfx2, svx and others I started to compare the load/save times.<br />
<br />
*The result is not as expected. In [[Media:Odfsave_withhash.ods]] you can see that SfxItemPropertyMap::getByName() takes longer than before. The new function takes about 5.3 s totally. These are about 1.7 s more than it's predecessor SfxItemPropertyMap::GetByName() required. The time is consumed mostly in the _M_Find<::rtl::OUString> method of the hash_map implementation.<br />
* One of the probable reasons is the fact that the sorted access to properties eliminated a lot of string comparisons.<br />
<br />
==== Using XMultiPropertySet where XTolerantMultiPropertySet might suffice and be more performant ====<br />
* To decide which properties have to be saved xmloff uses the interface methods css::beans::XPropertyState::getPropertyStates() and css::beans::XMultiPropertySet::getPropertyValues(). It could also use the interface css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant() which is not implemented for Writer's UNO objects. <br />
* Saving Writer's text content is done by iterating over the paragraphs and iterating over so-called text portions within the paragraphs. Text portions are parts of the paragraph that have a single attribute set, text fields, redline portions, inline anchored frames etc. It might make sense to detect their properties at construction time and preset their css::uno::XTolerantMultiPropertySet interface. And the moment a text portion is created that adds a bookmark to remember it's position. The impact on real documents is not yet checked.<br />
* A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.<br />
==== Font Fallback ====<br />
The huge contribution of GenerateAndStoreThumbnail in the callgrind measurements for some of the documents is attributed to substitution matching for missing fonts. This might be an issue to investigate.<br />
==== Compressed files do not need to be compressed again in Storage ====<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=100722 Issue 100722]<br />
* The large contribution of SfxMedium::Commit for documents "ScienceThesis" and "Manual" to StoreAsUrl in the callgrind analysis are attributed to the pictures in the document. We are investigating, if it might help to store image files that are already compressed (JPEG for examples) directly to the storage without trying to compress again in vain. <br />
== Test Documents ==<br />
<br />
* http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt (long specification document)<br />
* http://www.fonaso.com/translations/FonasoVersatil-German-V1.8.doc (short document with images)<br />
* https://eldorado.uni-dortmund.de/bitstream/2003/21520/2/Jackler.doc (~150 pages scientific paper with tables and images)<br />
* MailMerge Document (~600 pages, to be uploaded)<br />
Microsoft Word Documents where loaded and saved as odt before profiling.<br />
<br />
[[Category:Performance]]</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-04-01T12:59:37Z<p>Os: /* String Indexed Access of PropertySets */</p>
<hr />
<div>{{Performance}}<br />
== Investigation and Profiling of Writer Load/Save Performance ==<br />
We started a systematic profiling of the load/save performance on a current milestone (DEV300_m45). Im using Intel vTune on Windows and cachegrind (with cache-simulation) on Linux. On each platform each of the four documents (see Testdocuments below) is profiled for a load and a save procedure. Each measurement is done twice. In total, this will result in:<br />
2 (platforms/profilers) * 4 (documents) * 2 (load/save) * 2 (measurements) = 32 measurements in total<br />
The analysis of the data should:<br />
* show, if the profiler output is stable and reproducable<br />
* show, if there are any differences between platforms and profiler (and for cachegrind: does cache-simulation result in meaningful additional accuracy?)<br />
* identify the hotspots of the current implementation<br />
* be a basis to evalute the progress by optimizations<br />
<br />
=== Birdsview Callgrind (Contributions To StoreUrl DEV300_m45) ===<br />
{|class="wikitable" style="text-align:center"<br />
|+ Contributions to StoreToUrl<br />
|-style="font-weight: bold;"<br />
| Document||Lib||Method||Instructions fetched||Cycle Cost Est.||Instructions fetched||Cycle Cost Est.<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| ScienceThesis 1||libsfx||SfxBaseModel::StoreAsUrl||11592161864||12700226484||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||4567386009||5193895809||39.40%||40.90%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||2336567459||2525183559||20.16%||19.88%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||4535296593||4794222283||39.12%||37.75%<br />
|-<br />
| ||||other||152911803||186924833||1.32%||1.47%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| ScienceThesis 2||libsfx||SfxBaseModel::StoreAsUrl||11144535020||12250598830||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||4575357778||5189916868||41.05%||42.36%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||1925216412||2116790032||17.27%||17.28%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||4535327879||4792981289||40.70%||39.12%<br />
|-<br />
| ||||other||108632951||150910641||0.97%||1.23%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Manual 1||libsfx||SfxBaseModel::StoreAsUrl||2019415631||2238108511||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||1239343887||1399947937||61.37%||62.55%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||607055773||642691573||30.06%||28.72%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||140565338||157789035||6.96%||7.05%<br />
|-<br />
| ||||other||32450633||37679966||1.61%||1.68%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Manual 2||libsfx||SfxBaseModel::StoreAsUrl||2043487795||2268710495||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||1262263271||1428106081||61.77%||62.95%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||608153207||644497207||29.76%||28.41%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||141902391||154627511||6.94%||6.82%<br />
|-<br />
| ||||other||31168926||41479696||1.53%||1.83%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Spec 1||libsfx||SfxBaseModel::StoreAsUrl||17995716426||20443306916||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||16713230798||19057592468||92.87%||93.22%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||619569536||652497286||3.44%||3.19%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||605712688||663294858||3.37%||3.24%<br />
|-<br />
| ||||other||121121223||133131403||0.67%||0.65%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| Spec 2||libsfx||SfxBaseModel::StoreAsUrl||18062701503||20537474953||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||16681477861||19055676231||92.35%||92.78%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||654539677||685122457||3.62%||3.34%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||605562742||663544862||3.35%||3.23%<br />
|-<br />
| ||||other||121121223||133131403||0.67%||0.65%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| MailMerge 1||libsfx||SfxBaseModel::StoreAsUrl||40777455765||53626856665||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||23118516732||34395921752||56.69%||64.14%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||17475120469||19021058129||42.85%||35.47%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||161846924||179849124||0.40%||0.34%<br />
|-<br />
| ||||other||21971640||30027660||0.05%||0.06%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| MailMerge 2||libsfx||SfxBaseModel::StoreAsUrl||34409955066||45317878876||100.00%||100.00%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||22931786501||33059699851||66.64%||72.95%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||11294919676||12054180246||35.03%||26.60%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||161097067||173977107||0.47%||0.38%<br />
|-<br />
| ||||other||22151822||30021672||0.06%||0.07%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|-<br />
| std. deviation||libsfx||SfxBaseModel::StoreAsUrl||||||9.31%||9.21%<br />
|-<br />
| ||libsfx||SfxObjectShell::SaveAsOwnFormat||||||1.17%||2.53%<br />
|-<br />
| ||libsfx||SfxObjectShell::GenerateAndStoreThumbnail||||||23.04%||23.30%<br />
|-<br />
| ||libsfx||SfxMedium::Commit||||||0.61%||2.21%<br />
|-<br />
| ||||other||||||16.88%||12.56%<br />
|-<br />
|colspan="7" style="background:#444444;"|<br />
|}<br />
<br />
=== Benchmarking with vTune ===<br />
In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document.<br />
== Optimizations ==<br />
=== Implemented Optimizations ===<br />
==== Save time index entries ====<br />
[http://qa.openoffice.org/issues/show_bug.cgi?id=57008 Issue 57008]<br />
<br />
Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.<br />
<br />
=== Identified Hotspots ===<br />
==== Conversion of Hyperlinks ====<br />
<br />
* Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
==== String Indexed Access of PropertySets ====<br />
<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=99568 issue 99568]<br />
<br />
* Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
* The replacement for the SfxItemPropertyMap that uses an std::hash_map is ready. After changing a lot of code in the applications as well as in svtools, sfx2, svx and others I started to compare the load/save times.<br />
<br />
*The result is not as expected. In [[Media:Odfsave_withhash.ods]] you can see that SfxItemPropertyMap::getByName() takes longer than before. The new function takes about 5.3 s totally. These are about 1.7 s more than it's predecessor SfxItemPropertyMap::GetByName() required. The time is consumed mostly in the _M_Find<::rtl::OUString> method of the hash_map implementation.<br />
* One of the probable reasons is the fact that the sorted access to properties eliminated a lot of string comparisons.<br />
<br />
==== Using XMultiPropertySet where XTolerantMultiPropertySet might suffice and be more performant ====<br />
* To decide which properties have to be saved xmloff uses the interface methods css::beans::XPropertyState::getPropertyStates() and css::beans::XMultiPropertySet::getPropertyValues(). It could also use the interface css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant() which is not implemented for Writer's UNO objects. <br />
* Saving Writer's text content is done by iterating over the paragraphs and iterating over so-called text portions within the paragraphs. Text portions are parts of the paragraph that have a single attribute set, text fields, redline portions, inline anchored frames etc. It might make sense to detect their properties at construction time and preset their css::uno::XTolerantMultiPropertySet interface. And the moment a text portion is created that adds a bookmark to remember it's position. The impact on real documents is not yet checked.<br />
* A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.<br />
==== Font Fallback ====<br />
The huge contribution of GenerateAndStoreThumbnail in the callgrind measurements for some of the documents is attributed to substitution matching for missing fonts. This might be an issue to investigate.<br />
==== Compressed files do not need to be compressed again in Storage ====<br />
[http://www.openoffice.org/issues/show_bug.cgi?id=100722 Issue 100722]<br />
* The large contribution of SfxMedium::Commit for documents "ScienceThesis" and "Manual" to StoreAsUrl in the callgrind analysis are attributed to the pictures in the document. We are investigating, if it might help to store image files that are already compressed (JPEG for examples) directly to the storage without trying to compress again in vain. <br />
== Test Documents ==<br />
<br />
* http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt (long specification document)<br />
* http://www.fonaso.com/translations/FonasoVersatil-German-V1.8.doc (short document with images)<br />
* https://eldorado.uni-dortmund.de/bitstream/2003/21520/2/Jackler.doc (~150 pages scientific paper with tables and images)<br />
* MailMerge Document (~600 pages, to be uploaded)<br />
Microsoft Word Documents where loaded and saved as odt before profiling.<br />
<br />
[[Category:Performance]]</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-03-12T14:52:42Z<p>Os: </p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL><br />
<P>A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.</P><br />
<br />
<P>In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document. I will concentrate on two issues here:</P><br />
<UL><br />
<LI><P>Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
</P><br />
</LI><br />
<LI><br />
<P>Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
</P><br />
</LI><br />
</UL><br />
<P>The replacement for the SfxItemPropertyMap that uses an std::hash_map is ready. After changing a lot of code in the applications as well as in svtools, sfx2, svx and others I started to compare the load/save times.</P><br />
<br />
<P>The result is not as expected. In [Media:Odfsave_withhash.ods] you can see that SfxItemPropertyMap::getByName() takes longer than before. The new function takes about 5.3 s totally. These are about 1.7 s more than it's predecessor SfxItemPropertyMap::GetByName() required. The time is consumed mostly in the _M_Find<::rtl::OUString> method of the hash_map implementation. </P><br />
<P>One of the probable reasons is the fact that the sorted access to properties eliminated a lot of string comparisons. </P><br />
<br />
[[Category:Performance]]</div>Oshttps://wiki.openoffice.org/wiki/File:Odfsave_withhash.odsFile:Odfsave withhash.ods2009-03-12T14:49:35Z<p>Os: uploaded a new version of "Image:Odfsave withhash.ods": New revision, _M_find<::rtl::OUString> was missing in svl</p>
<hr />
<div>Results of loading/saving using a hash_map implementation of SfxItemPropertyMap.</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-03-12T14:48:18Z<p>Os: (checkpoint save)</p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL><br />
<P>A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.</P><br />
<br />
<P>In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document. I will concentrate on two issues here:</P><br />
<UL><br />
<LI><P>Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
</P><br />
</LI><br />
<LI><br />
<P>Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
</P><br />
</LI><br />
</UL><br />
<P>The replacement for the SfxItemPropertyMap that uses an std::hash_map is ready. After changing a lot of code in the applications as well as in svtools, sfx2, svx and others I started to compare the load/save times.</P><br />
<br />
<P>The result is not as expected. In [Media:Odfsave_withhash.ods] you can see that SfxItemPropertyMap::getByName() takes longer than before. The new function takes about 5.3 s totally. These are about 1.7 s more than it's predecessor SfxItemPropertyMap::GetByName() required. The time is consumed mostly in the _M_Find<::rtl::OUString> method of the hash_map implementation. </P><br />
<br />
<br />
[[Category:Performance]]</div>Oshttps://wiki.openoffice.org/wiki/File:Odfsave_withhash.odsFile:Odfsave withhash.ods2009-03-12T14:36:40Z<p>Os: Results of loading/saving using a hash_map implementation of SfxItemPropertyMap.</p>
<hr />
<div>Results of loading/saving using a hash_map implementation of SfxItemPropertyMap.</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-03-12T14:35:40Z<p>Os: (checkpoint save)</p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL><br />
<P>A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.</P><br />
<br />
<P>In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document. I will concentrate on two issues here:</P><br />
<UL><br />
<LI><P>Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
</P><br />
</LI><br />
<LI><br />
<P>Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
</P><br />
</LI><br />
</UL><br />
<P>The replacement for the SfxItemPropertyMap that uses an std::hash_map is ready. After changing a lot of code in the applications as well as in svtools, sfx2, svx and others I started to compare the load/save times.</P><br />
<br />
<P>The result is not as expected. </P><br />
<br />
[[Category:Performance]]</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-02-27T08:13:43Z<p>Os: </p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL><br />
<P>A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.</P><br />
<br />
<P>In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document. I will concentrate on two issues here:</P><br />
<UL><br />
<LI><P>Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
</P><br />
</LI><br />
<LI><br />
<P>Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
</P><br />
</LI><br />
</UL><br />
<P>At the moment I'm creating a replacement of SfxItemPropertyMap that uses a std::hash_map. This requires a lot of changes not only in svl but also in sfx2, svx and all application library.</P><br />
<br />
[[Category:Performance]]</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-02-19T12:35:14Z<p>Os: </p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL><br />
<P>A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save speed.</P><br />
<br />
<P>In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document. I will concentrate on two issues here:</P><br />
<UL><br />
<LI><P>Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
</P><br />
</LI><br />
<LI><br />
<P>Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
</P><br />
</LI><br />
</UL><br />
<P>At the moment I'm creating a replacement of SfxItemPropertyMap that uses a std::hash_map. This requires a lot of changes not only in svl but also in sfx2, svx and all application library.</P></div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-02-19T07:02:31Z<p>Os: </p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL><br />
<P>A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save times.</P><br />
<br />
<P>In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document. I will concentrate on two issues here:</P><br />
<UL><br />
<LI><P>Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
</P><br />
</LI><br />
<LI><br />
<P>Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
</P><br />
</LI><br />
</UL><br />
<P>At the moment I'm creating a replacement of SfxItemPropertyMap that uses a std::hash_map. This requires a lot of changes not only in svl but also in sfx2, svx and all application library.</P></div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-02-19T06:57:05Z<p>Os: (checkpoint save)</p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL><br />
<P>A test implementation of the XTolerantMultiPropertySet in Writer's text portion objects didn't result in increased save times.</P><br />
<br />
<P>In the spreadsheet [[Media:odfsave.ods]] you can find a list of the top consumers from some selected libraries (sw, xo, svl, svt, sal3, sfx2) when saving the ODF specification document. I will concentrate on two issues here:</P><br />
<UL><br />
<LI><P>Conversion of hyperlinks takes a lot of time. To bring hyperlinks into the correct form and to make them relative to the target URL the methods from svt's URIHelper are used. This is done for _all_ URLs independent of the protocol. In the given document URLs are mostly http while the document is stored to file. One approach to solve this issue is to convert only if the protocols are the same. Another approach is to manage the hyperlinks within the document and to keep them in the correct form all the time. So they only have to be converted at the time the target URL changes (saveAs/storeToURL). On the given system this would save about 4 s from a total of 12 s save time. (stopwatch estimation)<br />
</P><br />
</LI><br />
<LI><br />
<P>Another rather big part of processing time is consumed to access the members of the implementations of css::beans::XPropertySet, XPropertyState, XPropertySetInfo. To find the requested element by it's name the methods from SfxItemPropertySet, SfxItemPropertySetInfo etc. iterate over an array of structs that define a property (SfxItemPropertyMap). This can be seen by the numbers from SfxItemPropertyMap::getByName, rtl::OUString::equalsAsciiL, SfxItemPropertySetInfo::hasPropertyByName in the svl library. <br />
</P><br />
</LI></div>Oshttps://wiki.openoffice.org/wiki/File:Odfsave.odsFile:Odfsave.ods2009-02-19T06:39:40Z<p>Os: This spreadsheet contains the top functions in the libraries sw,svx,svl,svt,xo,sal3 (mi.dll).</p>
<hr />
<div>This spreadsheet contains the top functions in the libraries sw,svx,svl,svt,xo,sal3 (mi.dll).</div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-02-09T07:17:52Z<p>Os: </p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL></div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-02-09T07:16:53Z<p>Os: </p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008">http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
</LI><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
</LI><br />
</UL><br />
</UL></div>Oshttps://wiki.openoffice.org/wiki/Performance/WriterLoadSavePerformance/WriterLoadSave2009-02-09T07:15:48Z<p>Os: New page: <P>Started investigation of load/save issues in Writer using VTune. The first test document is the ODF 1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1...</p>
<hr />
<div><P>Started investigation of load/save issues in Writer using VTune.<br />
The first test document is the ODF<br />
1.1 specification document (http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1.odt)</A>. <br />
</P><br />
<UL><br />
<LI><P>Already done:</P><br />
<UL><br />
<br />
<LI><P>Issue 57008 related to save time index entries ( http://qa.openoffice.org/issues/show_bug.cgi?id=57008">http://qa.openoffice.org/issues/show_bug.cgi?id=57008 ). It saves est. 10% of the save time.</P><br />
</UL><br />
<LI><P>Already identified issues:</P><br />
<UL><br />
<LI><P>To decide which properties have to be saved xmloff uses the<br />
interface methods css::beans::XPropertyState::getPropertyStates()<br />
and css::beans::XMultiPropertySet::getPropertyValues(). It could<br />
also use the interface<br />
css::beans::XTolerantMultiPropertySet::getDirectPropertyValuesTolerant()<br />
which is not implemented for Writer's UNO objects. <br />
</P><br />
<br />
<LI><P>Saving Writer's text content is done by iterating over the<br />
paragraphs and iterating over so-called text portions within the<br />
paragraphs. Text portions are parts of the paragraph that have a<br />
single attribute set, text fields, redline portions, inline<br />
anchored frames etc. It might make sense to detect their properties<br />
at construction time and preset their css::uno::XTolerantMultiPropertySet interface. <BR>And<br />
the moment a text portion is created that adds a bookmark to<br />
remember it's position. The impact on real documents is not yet<br />
checked.</P><br />
<LI><P></P><br />
</UL><br />
</UL></div>Oshttps://wiki.openoffice.org/wiki/PerformancePerformance2009-02-09T07:07:02Z<p>Os: /* Work in progress */</p>
<hr />
<div>== The performance project ==<br />
<br />
This wiki page is the collaboration entry point for the [http://performance.openoffice.org/ OOo Performance project].<br />
<br />
For questions and discussions, please use the new mailing list from the performance project: [mailto:dev@performance.openoffice.org dev@performance.openoffice.org]<br />
<br />
When you plan to file an issue with regard to performance, please make sure to add the keyword <tt>performance</tt><br />
<br />
The official (incubator) project was started February 2009, but there have already been a lot of activities in this area in the past, so some of the information here (especially on the linked wiki pages) might be outdated.<br />
<br />
This page will be cleaned up soon.<br />
<br />
== ToDo's ==<br />
<br />
Probably a lot - please [http://wiki.services.openoffice.org/wiki/Performance/ToDo%27s continue here].<br />
<br />
== Work in progress ==<br />
What's going on in Writer load/save performance [http://wiki.services.openoffice.org/wiki/Performance/WriterInProgress see here ].<br />
<br />
== User Experience Index (UEI) ==<br />
[http://wiki.services.openoffice.org/wiki/UEI UEI and Case Documents]<br />
<br />
== Helpful Tooling and Resources ==<br />
<br />
=== Test Environment ===<br />
Popular reference configuration(use similar env on Hamburg and Beijing)<br />
CPU: Core2 Duo 1.5~2G<br />
Memory: 512M<br />
OS: WindowsXP HOME<br />
<br />
=== Test & Tuning Tools ===<br />
<br />
* [[cold-start-simulator|Linux/cold-start]] simulator<br />
* [[relocstat]] - dump relocation data<br />
* [http://tools.openoffice.org/profiling/profiling-howto.html RTL_LOGFILE] - export RTL_LOGFILE=/tmp/foo<br />
* [[valgrind]] - debug / trace memory usage<br />
* [[callgrind]] - call graph profiler<br />
* [[speedprof]] - a quick / hackish sampling full-stack-trace profiler & visualisation tool<br />
* [[APPR]] - a performance regression tool suite<br />
* [[Performance Related Test Documents]]<br />
* [[Performance/Tools/TrueTime|TrueTime]] - profiler for windows, commercial product<br />
<br />
=== Benchmarks & Description ===<br />
If you have some benchmarks, share them here!<br />
<br />
* [http://www.oooninja.com/2008/09/fastest-openofficeorg-edition-benchmark.html The Fastest OpenOffice.org Edition] comparison of four 2.4.1 editions on Fedora 9<br />
* [http://www.oooninja.com/2008/05/openofficeorg-getting-faster-benchmark.html Is OpenOffice.org Getting Faster?] Comparison of 11 versions from 1.1.5 to 3.0.0 alpha<br />
<br />
== Performance Test Results Distribution ==<br />
<br />
=== Test results for different versions ===<br />
* [http://wiki.services.openoffice.org/wiki/Performance/Performance_Work_2005 Performance 2005]<br />
* [http://www.oooninja.com/2008/05/openofficeorg-getting-faster-benchmark.html Is OpenOffice.org Getting Faster?] benchmarks 5 common operations for OpenOffice.org versions 1.1.5 through 2.4.0 and DEV300_m3<br />
* [http://wiki.services.openoffice.org/wiki/Performance/OOo31_LibrariesOnStartup List of libraries loaded on OpenOffice.org 3.1 (DEV300m40) startup]<br />
<br />
== Integrated improvements == <br />
* [[Integrated_Performance_Improvements]]<br />
<br />
== Other Pages == <br />
This Wiki page should become the ultimate authority on performance issues. Meanwhile, here are links to performance informationat other places<br />
* [http://tools.openoffice.org/performance/ OpenOffice.org Performance Tuning page]<br />
* [http://sw.openoffice.org/drafts/optimization.html Writer (SW, XML, Math, Linguistic) specific Optimizations]<br />
* [http://tools.openoffice.org/performance/performance-activities-overview.html Current & Planned Performance Activities]<br />
<br />
[[Category:Development]]<br />
[[Category:Performance]]<br />
[[Category:Database]]<br />
[[Category:Effort]]<br />
[[Category:Build_System]]</div>Oshttps://wiki.openoffice.org/wiki/PerformancePerformance2009-02-09T07:04:43Z<p>Os: /* ToDo's */ (checkpoint save)</p>
<hr />
<div>== The performance project ==<br />
<br />
This wiki page is the collaboration entry point for the [http://performance.openoffice.org/ OOo Performance project].<br />
<br />
For questions and discussions, please use the new mailing list from the performance project: [mailto:dev@performance.openoffice.org dev@performance.openoffice.org]<br />
<br />
When you plan to file an issue with regard to performance, please make sure to add the keyword <tt>performance</tt><br />
<br />
The official (incubator) project was started February 2009, but there have already been a lot of activities in this area in the past, so some of the information here (especially on the linked wiki pages) might be outdated.<br />
<br />
This page will be cleaned up soon.<br />
<br />
== ToDo's ==<br />
<br />
Probably a lot - please [http://wiki.services.openoffice.org/wiki/Performance/ToDo%27s continue here].<br />
<br />
== Work in progress ==<br />
<br />
== User Experience Index (UEI) ==<br />
[http://wiki.services.openoffice.org/wiki/UEI UEI and Case Documents]<br />
<br />
== Helpful Tooling and Resources ==<br />
<br />
=== Test Environment ===<br />
Popular reference configuration(use similar env on Hamburg and Beijing)<br />
CPU: Core2 Duo 1.5~2G<br />
Memory: 512M<br />
OS: WindowsXP HOME<br />
<br />
=== Test & Tuning Tools ===<br />
<br />
* [[cold-start-simulator|Linux/cold-start]] simulator<br />
* [[relocstat]] - dump relocation data<br />
* [http://tools.openoffice.org/profiling/profiling-howto.html RTL_LOGFILE] - export RTL_LOGFILE=/tmp/foo<br />
* [[valgrind]] - debug / trace memory usage<br />
* [[callgrind]] - call graph profiler<br />
* [[speedprof]] - a quick / hackish sampling full-stack-trace profiler & visualisation tool<br />
* [[APPR]] - a performance regression tool suite<br />
* [[Performance Related Test Documents]]<br />
* [[Performance/Tools/TrueTime|TrueTime]] - profiler for windows, commercial product<br />
<br />
=== Benchmarks & Description ===<br />
If you have some benchmarks, share them here!<br />
<br />
* [http://www.oooninja.com/2008/09/fastest-openofficeorg-edition-benchmark.html The Fastest OpenOffice.org Edition] comparison of four 2.4.1 editions on Fedora 9<br />
* [http://www.oooninja.com/2008/05/openofficeorg-getting-faster-benchmark.html Is OpenOffice.org Getting Faster?] Comparison of 11 versions from 1.1.5 to 3.0.0 alpha<br />
<br />
== Performance Test Results Distribution ==<br />
<br />
=== Test results for different versions ===<br />
* [http://wiki.services.openoffice.org/wiki/Performance/Performance_Work_2005 Performance 2005]<br />
* [http://www.oooninja.com/2008/05/openofficeorg-getting-faster-benchmark.html Is OpenOffice.org Getting Faster?] benchmarks 5 common operations for OpenOffice.org versions 1.1.5 through 2.4.0 and DEV300_m3<br />
* [http://wiki.services.openoffice.org/wiki/Performance/OOo31_LibrariesOnStartup List of libraries loaded on OpenOffice.org 3.1 (DEV300m40) startup]<br />
<br />
== Integrated improvements == <br />
* [[Integrated_Performance_Improvements]]<br />
<br />
== Other Pages == <br />
This Wiki page should become the ultimate authority on performance issues. Meanwhile, here are links to performance informationat other places<br />
* [http://tools.openoffice.org/performance/ OpenOffice.org Performance Tuning page]<br />
* [http://sw.openoffice.org/drafts/optimization.html Writer (SW, XML, Math, Linguistic) specific Optimizations]<br />
* [http://tools.openoffice.org/performance/performance-activities-overview.html Current & Planned Performance Activities]<br />
<br />
[[Category:Development]]<br />
[[Category:Performance]]<br />
[[Category:Database]]<br />
[[Category:Effort]]<br />
[[Category:Build_System]]</div>Oshttps://wiki.openoffice.org/wiki/Writer/PerformanceWriter/Performance2009-02-05T07:10:00Z<p>Os: /* CWS swautomatic01 */</p>
<hr />
<div>=== CWS swautomatic01 ===<br />
<br />
The XML export of Writer will be improved by the introduction of automatic styles in the Writer core. This was done in OOo 2.1 by [http://eis.services.openoffice.org/EIS2/cws.ShowCWS?Id=3074 CWS swautomatic01]. [[Writer_swautomatic01|More details.]]<br />
<br />
<br />
[[Category:Writer]]</div>Oshttps://wiki.openoffice.org/wiki/OOo_and_SubversionOOo and Subversion2008-10-15T10:56:18Z<p>Os: /* Ongoing Maintenance */</p>
<hr />
<div>=OpenOffice.org migrates to Subversion=<br />
<br />
After a long discussion and preparation time it finally happens, OpenOffice.org migrates with milestone DEV300 m31 to a new SCM (<u>S</u>oftware <u>C</u>onfiguration <u>M</u>anagement) system. The new SCM system will be Subversion (SVN) for now, which might disappoint the one or other who had hoped for a distributed SCM like Git, Bazaar or Mercurial. Please be assured that we'll keep our mind open and we'll reevaluate the fast moving SCM tooling situation periodically. We were very careful to make certain that our move now to Subversion does not preclude a move to a DSCM in the future.{{note|why}}<br />
<br />
==Trunk only migration==<br />
<br />
Milestone DEV300 m31 is the last milestone which has been integrated via CVS and is the first milestone which is available in SVN. We did a trunk migration only. The trunk (DEV300) code line will be maintained via Subversion only starting with milestone DEV300 m32. Old code lines, '''including the OOo-3.0 release codeline(OOO300)''' will still be maintained via CVS.<br />
<br />
Please let me repeat this: we did not migrate branches and tags to SVN. There are a number of advantages (and some disadvantages as well) to this approach. A discussion of the merits of different approaches to the actual repository conversion can be found [[Scm_migration_scope|here]].<br />
<br />
The most important reasons for choosing the "trunk only" approach were:<br />
* No quiet time for developers<br />
* The conversion of historical workspaces from CVS to SVN would be fragile at best due to some liberties we took in structuring our CVS repository. Having no historical branches and tags is preferable to having inaccurate branches and tags IMHO.<br />
<br />
The "trunk only" approach enabled us to optimize the resulting SVN repository, it shrunk from 90 GBytes to about 6 GBytes overall size. The following rules have been applied during conversion<br />
* Only modules which have active content in DEV300 m31 are included in the conversion.<br />
* All files in "Attic" have been removed.<br />
* All but the last revisions of binary files have been dropped - after all you can't diff them anyway<br />
* All but the last revisions of "localize.sdf" files have been dropped - saves an incredible amount of repository space without loosing much<br />
<br />
==Required tools==<br />
<br />
* subversion-1.5.1 or later; [http://subversion.tigris.org/getting.html#binary-packages download...]<br />
* ssh for committers, preferably OpenSSH 2.4 or later<br />
<br />
We make heavy use of the new SVN merge tracking feature. So please do not use a SVN client older than subversion-1.5.1 if you work on a child workspace (CWS).<br />
<br />
Windows users most certainly want to take a look at [http://tortoisesvn.tigris.org/ TortoiseSVN] and lots of documentation available at http://tortoisesvn.net/<br />
<br />
==Subversion documentation==<br />
<br />
The excellent Subversion documentation can be found here: http://svnbook.red-bean.com. Please use the nightly build documentation for 1.5 and later, it covers new features like merge tracking.<br />
<br />
<font color=red size=5><center>Please read at least the "fundamental concepts" and "basic usage" part of the documentation!</center></font>.<center>CWS tooling is only a very thin layer around the SVN client hiding the lengthy URLs from sight plus some bookkeeping. Everything else is plain Subversion.</center><br />
<br />
==Server access methods==<br />
<br />
===Read only SVN access===<br />
<br />
svn checkout svn://svn.services.openoffice.org/ooo/trunk<br />
<br />
or<br />
<br />
svn checkout http://svn.services.openoffice.org/ooo/trunk<br />
<br />
===Read only browser access===<br />
<br />
Point your browser to <br />
http://svn.services.openoffice.org/ooo<br />
<br />
===Read/write access===<br />
<br />
Write access to the repository requires a ssh setup.<br />
<br />
svn co svn+ssh://svn@svn.services.openoffice.org/ooo/trunk<br />
<br />
==Basic read/only SVN operations==<br />
<br />
===Repository structure===<br />
<br />
The OOo SVN repository is structured like this:<br />
<br />
../trunk <= the main development line, currently master workspace DEV300 is on trunk<br />
../cws <= child workspaces live here, names need to be unique<br />
../cws/foo <= for example CWS ''foo''<br />
../cws/...<br />
../branches <= future master workspaces which are not on trunk<br />
../branches/OOO310 <= for example MWS ''OOO310''<br />
../branches/...<br />
../tags <= milestone tags, release tags<br />
../tags/DEV300_m32 <= for example milestone ''DEV300 m32''<br />
../tags/..<br />
../dist <= space for distribution specific stuff<br />
../patches <= space for globally useful patches<br />
../contrib <= do we need this?<br />
<br />
Examples:<br />
<br />
Check out a milestone ''DEV300 m32'' with:<br />
<br />
svn checkout svn://svn.services.openoffice.org/ooo/tags/DEV300_m32<br />
<br />
Switch from milestone ''DEV300_m32'' to ''DEV300_m35'' (safes potentially a lot of checkout time):<br />
<br />
cd DEV300_m32<br />
svn switch svn://svn.services.openoffice.org/ooo/tags/DEV300_m35<br />
cd ..<br />
mv DEV300_m32 DEV300_m35<br />
<br />
Check out a child workspace ''foo'':<br />
<br />
svn checkout svn://svn.services.openoffice.org/ooo/cws/foo<br />
<br />
Update working copy of ''foo'':<br />
<br />
cd foo<br />
svn update<br />
<br />
Find out the base (milestone or cws) of working copy ''wc'':<br />
<br />
cd wc<br />
svn info<br />
<br />
List all available milestone and release tags:<br />
<br />
svn list svn://svn.services.openoffice.org/ooo/tags<br />
<br />
List all available child workspaces:<br />
<br />
svn list svn://svn.services.openoffice.org/ooo/cws<br />
<br />
View commit log on a single file:<br />
<br />
svn log file.cxx<br />
<br />
Include all paths affected by change sets in log command:<br />
<br />
svn log --verbose file.cxx<br />
<br />
Attribute blame:<br />
<br />
svn annotate file.cxx<br />
<br />
==SSH Setup==<br />
<br />
===SSH key generation===<br />
<br />
On Unix or Windows/cygwin a SSH identity (private/public key pair) is generated with:<br />
<br />
ssh-keygen -t dsa<br />
<br />
You'll be asked for pass phrase which protects your private key. Per default the public key is named <tt>$HOME/.ssh/id_dsa.pub</tt>.<br />
<br />
Note that SSH keys created between September 2006 and May 2008 on Debian or Debian derived systems like Ubuntu may be insecure. Please use recent SSH tools.<br />
<br />
If you want a dedicated identity for just OOo SVN usage, generate a specially named key pair<br />
<br />
ssh-keygen -t dsa -f ~/.ssh/id_dsa_ooo_svn <br />
<br />
and instruct Subversion to use this identity (key pair) with the following configuration entry in <tt>$HOME/.subversion/config</tt>:<br />
<br />
[tunnels]<br />
ssh = ssh -i /home/jr93709/.ssh/id_dsa_ooo_svn<br />
<br />
===Initial Migration of Keys===<br />
<br />
Please query IssueZilla for your issue with the attachment holding your key and add it as a dependency to [http://www.openoffice.org/issues/show_bug.cgi?id=94002 issue 94002]. If you are a new svn user, create the issue yourself.<br />
<br />
We will then install those keys for DomainDevelopers for read/write access to the SVN repository.<br />
<br />
Up to now only few code committers made use of the ssh key upload for the OpenOffice.org SVN repository. Please note that your key will not be migrated without your intervention.<br />
<br />
===Ongoing Maintenance===<br />
<br />
We will continue to use IssueZilla for managing and tracking key additions and changes.<br />
Please see [http://www.openoffice.org/docs/ddSSHGuide.html#sendingkey Submitting Your Certificate (Public Key)].<br />
<br />
===Windows ssh setup with putty ===<br />
<br />
<p>To use ssh on Windows putty's ssh can be used. You need plink.exe, puttygen.exe and pageant.exe. All are available from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html]</p><br />
<br />
<br />
<p>As putty uses a different ssh key format you have to convert the private key using puttygen.exe.</p> <br />
<p>Save this file for example as:</p><br />
<p><system drive>:\Documents and settings\<user name>\Application Data\id_dsa_ooo_svn.ppk</p><br />
<br />
<br />
<p>In the [tunnels] section of the file <system drive>:\Documents and settings\<user name>\Application Data\Subversion\Config</p><br />
<p>You have to add the line:</p><br />
<p>ssh = plink.exe -i <path to you converted private key file></p><br />
<p>Please note that spaces in the path have to be escaped using a backslash.</p><br />
<br />
<br />
To be able to call an svn command you need to have the pageant.exe running. Start this and import your private key file. <br />
<br />
<br />
On the first connection to the server the server's host key has to be added to the registry. This doesn't work within the svn command. <br />
<p>The following command helps to accept this key:</p><br />
<p>plink.exe -i <path to private pkey> svn.services.openoffice.org svnserve -t</p><br />
<br />
==CWS tooling==<br />
<br />
The CWS tooling has been reworked to adapt to SVN. The basic CWS tool is now simply called <tt>cws</tt> and is invoked as <tt>cws <subcommand></tt> in the style of the SVN client, there are no longer separate tools like <tt>cwsadd</tt>, <tt>cwsresync</tt> etc etc.<br />
<br />
Note: the <tt>cws</tt> script will probably change over time as we learn how the usage patterns evolve. If you got a problem with the <tt>cws</tt> script, it might be worthwhile to rebase it to the latest version<br />
$ cd <your_CWS>/ooo/solenv/bin<br />
$ svn merge svn+ssh://svn@svn.services.openoffice.org/ooo/trunk/solenv/bin/cws.pl<br />
<br />
The most important change in the way CWSs are handled now is that there is no longer a need nor possibility to add modules. Well, there is still a <tt>sw</tt> and <tt>config_office</tt> but all modules of old are now just top level directories, at least as far as SVN is concerned.<br />
<br />
===CWS config file===<br />
<br />
Please keep all your entries in your <tt>$HOME/.cwsrc</tt> for now, you'll need them if you want to work with CWSs for OOo-3.0. Just add the following line to refer to the OpenOffice.org SVN server:<br />
<br />
SVN_SERVER=svn+ssh://svn@svn.services.openoffice.org/ooo<br />
<br />
Example of a complete <tt>.cwsrc</tt> file:<br />
<br />
[CWS_CONFIG]<br />
# Network proxy for EIS connections. <br />
# Comment in if you need to access EIS via a proxy.<br />
# Note: This is just for accessing the EIS database. For SVN have a look here:<br />
# http://svnbook.red-bean.com/nightly/en/svn.advanced.confarea.html#svn.advanced.confarea.opts.servers<br />
# Example: PROXY=http://myproxy.company.com:8080<br />
# PROXY=<br />
<br />
# CWS Database server (SOAP). It's possible specify to several backup server,<br />
# currently there are none.<br />
CWS_DB_SERVER_1=https://eis.services.openoffice.org/soap/servlet/rpcrouter<br />
<br />
# Your CVS login (for authentication with the CWS database).<br />
CVS_ID=<your_ooo_name><br />
<br />
# Your scrambled CVS password (for authentication with the CWS database).<br />
# Take this one from .cvsrc<br />
CVS_PASSWORD=<your_current_cvs_scrambled_ooo_password><br />
<br />
# OOo CVS tunnel<br />
# Example: CVS_SERVER_ROOT=pserver:cvs_id@mytunnel.mydomain.de:/cvs<br />
CVS_SERVER_ROOT=<br />
<br />
# Path to the cvs binary (optional)<br />
# Example: CVS_BINARY=/usr/bin/cvs<br />
#CVS_BINARY=<br />
<br />
# SVN server for OOo-3.1 child workspaces<br />
# r/w access via svn+ssh pprotocol<br />
SVN_SERVER=svn+ssh://svn@svn.services.openoffice.org/ooo<br />
# r/o access via svn protocol<br />
#SVN_SERVER=svn://svn.services.openoffice.org/ooo<br />
<br />
===Working with the new <tt>cws</tt> tool===<br />
<br />
Help can be invoked with:<br />
<br />
cws help<br />
<br />
which yields<br />
<br />
cws -- version: 1.1.2.10<br />
usage: cws <subcommand> [options] [args]<br />
Type 'cws help <subcommand>' for help on a specific subcommand.<br />
<br />
Available subcommands:<br />
help (h,?)<br />
create<br />
fetch (f)<br />
rebase (rb)<br />
analyze (an)<br />
query (q)<br />
task (t)<br />
integrate *** release engineers only ***<br />
eisclone *** release engineers only ***<br />
help (h, ?): Describe the usage of this script or its subcommands<br />
usage: help [subcommand]<br />
<br />
The subcommands do more or less what their CVS predecessors did.<br />
<br />
====Create a CWS====<br />
<br />
Create a CWS with the <tt>cws create</tt> subcommand. Please note that this commands only creates a CWS branch on the OOo server and registers it with EIS. It changes nothing on your hard disk.<br />
<br />
Example: create CWS ''foo'' on the latest published milestone of master DEV300<br />
<br />
cws create DEV300 foo<br />
<br />
Of course you can specify which milestone you want to use, here is the complete command reference:<br />
<br />
cws -- version: 1.1.2.10<br />
create: Create a new child workspace<br />
usage: create [-m milestone] <master workspace> <child workspace><br />
-m milestone: Milestone to base the child workspace on. If ommitted the<br />
last published milestone will be used.<br />
--milestone milestone: Same as -m milestone.<br />
<br />
Your CWS branch is then available as <br />
<br />
svn://svn.services.openoffice.org/ooo/cws/foo<br />
<br />
<br />
for check out.<br />
<br />
====Check out a CWS====<br />
<br />
$ cws fetch -c foo <br />
<br />
Currently this simply checks out your cws ''foo'' just as if you had issued the command<br />
<br />
$ svn checkout svn+ssh://svn@svn.services.openoffice.org/ooo/cws/foo<br />
<br />
A milestone can be fetched with<br />
<br />
$ cws fetch -m DEV300_m32<br />
<br />
this is equivalent to <br />
<br />
$ svn checkout svn+ssh://svn@svn.services.openoffice.org/ooo/tags/DEV300_m32<br />
<br />
Over time I'll add a number of convenience functions to <tt>cws fetch</tt>, like configurable partial checkouts, use of prebuild solvers, switching CWSs etc etc.<br />
<br />
Please note that if you have already a OOo SVN working copy lying around, you can save considerable checkout time with the svn switch command.<br />
<br />
cd <wc><br />
svn switch svn+ssh://svn@svn.services.openoffice.org/ooo/cws/foo<br />
<br />
will fetch the differences between your working copy in <wc> and the CWS ''foo''<br />
<br />
====Rebasing a CWS====<br />
<br />
The successor of the <tt>cwsresync</tt> tool is called <tt>cws rebase</tt>. rebasing is now a two step operation:<br />
<br />
[Tbd.]<br />
<br />
<br />
====How to migrate still active CVS based child workspaces destined for DEV300 to SVN?====<br />
<br />
Let's assume your CVS based CWS is called 'foo42'.<br />
<br />
=====CVS tasks=====<br />
<br />
1) Find out the CVS branch tag and anchor tag of your CWS.<br />
<br />
$ cd <added_module><br />
$ cvs log <some_file_which_already_existed_before_your_cws> | grep -i foo42<br />
CWS_SRC680_FOO42_ANCHOR: 1.36<br />
cws_src680_foo42: 1.36.0.44<br />
<br />
The first tag (<tt>CWS_SRC680_FOO42_ANCHOR</tt>) is your anchor tag, the second tag (<tt>cws_src680_foo42</tt>) is your branch tag. Note them down.<br />
<br />
2) Rebase your CVS based CWS to milestone DEV300 m31 with the old CWS tools<br />
<br />
$ cwsresync -m m31 all<br />
... solve conflicts ...<br />
$ cwsresync -c all<br />
<br />
3) Create a patch containing all modules. It is recommend to use the "rdiff" CVS command. A "rdiff" is quite handy for this purpose, because it allows the creation of one patch for all "added" modules. Use the "-u" switch for creating a so called "unified context patch".<br />
<br />
$ cvs -d <your_tunnel_setup> rdiff -u -rCWS_SRC680_FOO42_ANCHOR -rcws_src680_foo42 `cwsquery modules` > foo42.patch<br />
<br />
4) Check the patch for consistency. It should contain only stuff you did on your CWS. Throw out trivial changes.<br />
<br />
5) Search the patch for lines which starts with<br />
<br />
diff -u /dev/null <some_file>:<some_revision><br />
<br />
These files have been added to your CWS. Note the filenames down, you'll need them later.<br />
<br />
6) Search the patch for lines which end with <tt>:removed</tt><br />
<br />
diff -u <some_file>:<some_revision> <some_file>:removed<br />
<br />
These files have been removed in your CWS. Note the filenames down, you'll need them later.<br />
<br />
=====SVN tasks=====<br />
<br />
7) Check out the OOo trunk with<br />
<br />
$ svn checkout svn+ssh://svn@svn.services.openoffice.org/ooo/trunk my_working_copy<br />
<br />
8) Configure and source the configured script as usual<br />
<br />
$ cd my_working_copy/config_office<br />
$ configure<br />
...<br />
$ cd ..<br />
$ source LinuxX86Env.Set.sh # YMMV<br />
<br />
9) Create the CWS branch in SVN<br />
<br />
$ cws create --migration -m m32 DEV300 foo42<br />
<br />
This command creates the CWS branch in Subversion based on milestone DEV300 m32 and updates the "current milestone" information in EIS. Milestone DEV300 m32 is essentially the same as DEV300 m31 (just a few fixes for SVN and CWS tooling).<br />
<br />
The <tt>--migration</tt> switch tells the <tt>cws create</tt> command that an existing CWS is migrated. Without the switch, EIS will complain that your CWS already exists.<br />
<br />
10) Switch your working copy to your CWS branch<br />
<br />
$ cd my_working_copy<br />
$ svn switch svn+ssh://svn@svn.services.openoffice.org/ooo/cws/foo42<br />
...<br />
$ svn info<br />
Path: .<br />
URL: svn+ssh://svn@svn.services.openoffice.org/ooo/cws/foo42<br />
Repository Root: svn+ssh://svn@svn.services.openoffice.org/ooo<br />
Repository UUID: 8da58560-a4e7-4996-a0c2-a735b94b261c<br />
...<br />
<br />
Check the URL: line of the output of <tt>svn info</tt>. It should point to your CWS branch. '''Please be extra certain that you never commit anything on trunk.''' You can always check the branch on which you are working with <tt>svn info</tt>.<br />
<br />
11) Apply the patch<br />
<br />
$ cd my_working_copy<br />
$ patch -p1 < foo42.diff<br />
<br />
Since DEV300 m31 and DEV300 m32 are almost identical, your patch should apply cleanly. If you get rejects you'll need to fix them manually. The patch tool handles added and removed files quite nicely, the first ones are created by <tt>patch</tt>, the latter ones appear as zero byte sized files in the tree.<br />
<br />
12) Handle added and removed files<br />
<br />
This is pretty straight forward. You need to tell SVN which files have been added or removed by your patch.<br />
$ svn add <list_of_added_files><br />
$ rm <list_of_removed_files><br />
$ svn remove <list_of_removed_files><br />
<br />
13) Commit the changes to your CWS branch<br />
<br />
Review all changes with<br />
$ svn status<br />
$ svn diff<br />
<br />
and if everything is OK, commit them to your branch<br />
$ svn commit -m"#iXXXXX#: migrate CWS foo42 to SVN."<br />
<br />
Remember, always check that your working copy has been switched to the CWS branch before committing.<br />
<br />
=====Do I really need to resync first to DEV300 m31 in CVS? Can I use a later milestone than DEV300 m32 in SVN?=====<br />
<br />
No and yes resp. If you are certain that your patch applies cleanly or you are prepared to handle the rejects, you can skip the resync part of the recipe or/and start of from a newer SVN based milestone. Working with DEV300 m31 (CVS) and DEV300 m32 (SVN) almost ensures that the patch applies cleanly which is why we recommend to work with them.<br />
<br />
== Using plain SVN ==<br />
=== Showing the differences between a milestone on the master and a cws ===<br />
$ svn diff http://svn.services.openoffice.org/ooo/tags/DEV300_m32/ http://svn.services.openoffice.org/ooo/cws/os120/<br />
=== Showing the differences in a module or directory between a milestone on the master and a cws ===<br />
$ svn diff http://svn.services.openoffice.org/ooo/tags/DEV300_m32/sw http://svn.services.openoffice.org/ooo/cws/swrefactormarks2/sw<br />
This shows the differences in module sw between the milestone DEV300_m32 (a tagged version in svn-speech) and the most current checkin on cws swrefactormarks2. Since modules are only directories in svn (not in any way special as they where with CVS), this can be used to compare any directory too. For example:<br />
$ svn diff http://svn.services.openoffice.org/ooo/tags/DEV300_m32/sw/source/core http://svn.services.openoffice.org/ooo/cws/swrefactormarks2/sw/source/core<br />
A specific revision can be compared to the master like this:<br />
$ svn diff http://svn.services.openoffice.org/ooo/tags/DEV300_m32/sw/source/core http://svn.services.openoffice.org/ooo/cws/swrefactormarks2/sw/source/core@4711<br />
This compares the cws at revision 4711 to the milestone m32<br />
<br />
==Tips and tricks==<br />
<br />
<br />
===Ignoring output trees===<br />
<br />
Platform output trees (<tt>unxlngi6, unxsols4, wntmsci12</tt> etc) will show up as unversioned directories marked with a "?" in the output of <tt>svn status</tt> and other SVN commands. This is inconvenient and reduces the usefulness of these commands. In principle it's possible to deal with this in two ways:<br />
* attach and commit a <tt>svn:ignore</tt> property to every top level directory listing all possible platforms<br />
* add all the platforms which are usually build to the so called global ignore list<br />
<br />
Actually the global ignore list is more of a kind of "personal" ignore list which is applied to all svn commandos which access your personal subversion configuration.<br />
<br />
Since maintaing <tt>svn:ignore</tt> properties in about 200 directories for about 20 platforms is pretty clumsy, I suggest that we use the global ignore list for this purpose. Here is my personal ignore list, YMMV.<br />
<br />
$ cat $HOME/.subversion/config<br />
<br />
...<br />
[miscellany]<br />
...<br />
global-ignores = common* unxsoli4* unxsols4* unxlngi6* unxlngx6* unxubti8* wntmsci12* unxlngi6* unxmacxi* <br />
...<br />
<br />
Detailed instructions on how to use ignore lists can be found [http://svnbook.red-bean.com/en/1.5/svn.advanced.props.special.ignore.html here].<br />
<br />
===Ignoring contents of .svn when using grep===<br />
<br />
To avoid being driven insane by getting grep hits on the .svn dirs contents you can use<br />
<br />
$ export GREP_OPTIONS="--exclude=\*.svn\*"<br />
<br />
===Speeding up svn+ssh: access by caching ssh connections===<br />
<br />
You might have notice that nearly all SVN operations are somewhat slower via svn+ssh: access than via plain (and read-only) svn: access. This is due to the ssh handshake, some SVN operations even suffer several times from the handshake overhead.<br />
<br />
There is an easy and convenient way to get rid of this overhead by using a ssh connection cache.<br />
<br />
The following description is taken more or less verbose from the the GCC [http://gcc.gnu.org/wiki/SSH_connection_caching wiki]:<br />
<br />
SSH connection caching works with OpenSSH 4.0 on all Unix look alikes, but not on Windows/cygwin. You can check your OpenSSH version with<br />
<br />
$ ssh -V<br />
OpenSSH_4.6p1, OpenSSL 0.9.8e 23 Feb 2007<br />
<br />
Create a <tt>ControlPath</tt> entry for <tt>svn.services.openoffice.org</tt> in <tt>$HOME/.ssh/config</tt><br />
<br />
$ cat $HOME/.ssh/config<br />
...<br />
Host svn.services.openoffice.org<br />
ControlPath /tmp/ssh_ooo_control<br />
...<br />
<br />
Set the permissions <tt>0600</tt> on <tt>$HOME/.ssh/config</tt>, otherwise SSH refuses to work.<br />
<br />
$ chmod 0600 $HOME/.ssh/config<br />
<br />
Set up up a permanent connection to the ooo SVN server, for example right after login:<br />
<br />
$ ssh -fMN svn@svn.services.openoffice.org<br />
<br />
Subsequent svn commands will use the already established connection, resulting in noticeably less lag for short commands. It's also an alternative to using <tt>ssh-agent</tt> if you have protected your private key with a pass phrase.<br />
<br />
===Create a OOo repository mirror===<br />
<br />
The repository can be replicated with the svnsync tool. No special server side setup is necessary (read-only access is sufficient, please use either the svn: or http: method), but you need to make certain that the target repository can't be modified by other means than svnsync.<br />
<br />
* First create an empty target repository:<br />
$ svnadmin create /absolute/path/to/rep<br />
* Implement the pre-revprop-change and start-commit hooks<br />
$ cat /absolute/path/to/rep/hooks/pre-revprop-change<br />
<pre><br />
#!/bin/sh <br />
<br />
USER="$3"<br />
<br />
if [ "$USER" = "syncuser" ]; then exit 0; fi<br />
<br />
echo "Only the syncuser user may change revision properties" >&2<br />
exit 1<br />
</pre><br />
$ cat /absolute/path/to/rep/hooks/start-commit<br />
<pre><br />
#!/bin/sh <br />
<br />
USER="$2"<br />
<br />
if [ "$USER" = "syncuser" ]; then exit 0; fi<br />
<br />
echo "Only the syncuser user may commit new revisions" >&2<br />
exit 1<br />
</pre><br />
<br />
* Initialize the target repository:<br />
<br />
$ svnsync init --username syncuser file:///absolute/path/to/rep svn://svn.services.openoffice.org/ooo<br />
<br />
* And finally synchronize the target repository with the source repository<br />
<br />
$ svnsync synchronize file:///absolute/path/to/rep<br />
<br />
The full details for replicating SVN repositories can be found [[http://svnbook.red-bean.com/en/1.4/svn.reposadmin.maint.html#svn.reposadmin.maint.replication here]].<br />
<br />
===Reporting Problems with the Repository===<br />
<br />
The list [mailto:dev@tools.openoffice.org dev@tools.openoffice.org] is used to coordinate the initial migration and ongoing work on the repository and related tools. Please raise problems there first.<br />
Issues which need the attention of repository administrators should be filed in IssueZilla against the component [http://www.openoffice.org/issues/enter_bug.cgi?component=www&version=current&subcomponent=openoffice.org%20SVN www/openoffice.org SVN].<br />
<br />
===Read/Write access with TortoiseSVN===<br />
<br />
Please have a look [[Tortoisesvn|at this page]].<br />
<br />
[More SVN Tips and tricks will appear here]<br />
<br />
==Notes==<br />
# {{note|why}}Why not just wait until one of the DSCM systems emerges as a clear front runner? Well, first, this might never happen because each of the mentioned DSCM system emphasizes certain strength, leaving no one clear winner in all categories (ease of use, performance, integration possibilities and platform support). Second, and this is way more important, the OpenOffice.org CWS/CVS system is crumbling under the heavy weight of 8 years worth of OOo coding.<br />
<br />
[[Category:SVN]]</div>Oshttps://wiki.openoffice.org/wiki/Community_Innovation_Program/proposalsCommunity Innovation Program/proposals2008-05-05T06:04:46Z<p>Os: </p>
<hr />
<div>=Proposals=<br />
==Writer==<br />
<H3>Enhanced usability of working with page attributes and properties</H3><br />
<P>Currently changing page attributes or adding page properties like<br />
headers and footers is hard to understand. Users that don't know<br />
about hard page breaks, page styles and follow styles will have a<br />
hard time to format pages to their likings. A nice UI that hides the<br />
technical details and uses terms of the user space would help a lot. <br />
</P><br />
<P>Tasks <br />
</P><br />
<UL><br />
<LI><P STYLE="margin-bottom: 0cm">learn about Writer's page styles<br />
and their (UNO) API <br />
</P><br />
<LI><P>design and code the new user interface based on this API <br />
</P><br />
</UL><br />
<P>Required skills/knowledge: C++; UI design <br />
</P><br />
<DL><br />
<DT>Contact <br />
</DT><DD><br />
dev@sw.openoffice.org <br />
<DD STYLE="margin-bottom: 0.5cm"><br />
Oliver.Specht at sun dot com <br />
</DL><br />
<P><br />
<BR><BR><br />
</P><br />
<br />
==Calc/Chart==<br />
<br />
===Component to check spreadsheet documents for potential errors===<br />
<br />
Write an extension that checks a spreadsheet document for potential errors in formulas, presents a list of the findings to the user, and allows to correct them.<br />
<br />
Things to look for could include<br />
* Cells that contain numbers stored as text and are used in formulas,<br />
* Number cells that are not used in any formulas,<br />
* Empty cells that are used in single-cell formula references.<br />
The list is not meant to be exhaustive.<br />
<br />
* Required skills/knowledge: Java or C++, spreadsheet usage<br />
<br />
; Contact<br />
: dev@sc.openoffice.org<br />
: Niklas.Nebel at sun.com<br />
<br />
===Watch Window===<br />
<br />
A Watch Window is a separate, small window that remains "on top" and enables users to monitor a selected set of cells, see [http://www.openoffice.org/issues/show_bug.cgi?id=28386 issue 28386].<br />
<br />
This could be implemented as an add-on component with a modeless dialog containing the list of watched cells as well as the UI to add or remove cells.<br />
<br />
* Required skills/knowledge: Java or C++<br />
<br />
; Contact<br />
: dev@sc.openoffice.org<br />
: Niklas.Nebel at sun.com<br />
<br />
==Impress/Draw==<br />
==Base==<br />
<br />
==Misc==<br />
=== OXT Extension wizard for templates and XSLT based filter ===<br />
<br />
OXT extensions provide a nice way to bundle a group of templates and share them with other users. Templates are normally created with the office and the idea is to provide a wizard that helps on a higher abstraction level to collect templates and all necessary oxt specific information and build an oxt extension package.<br />
The second part is to extend the XSLT filter dialog to export new created XSLT based filters as an oxt extension as well. Currently it is only possible to create a jar file. The new feature should collect some more oxt related info and should create a well formed oxt extension package.<br />
The goal is to simplify these tasks and make it as easy as possible to create oxt extension packages that can be easy deployed and can be uploaded in the extensions repository to share them with many other users. <br />
<br />
<br />
* Required skills/knowledge: Java or C++ <br />
<br />
; Contact<br />
: juergen.schmidt at sun.com<br />
<br />
=== ODT/PDF export from OpenOffice.org Wiki ===<br />
<br />
OpenOffice.org documentation is being moved to the OpenOffice.org Wiki. This makes it easier for people to contribute to the content of the documentation, but producing a PDF of ODT of the entire Wiki document or book is not yet possible. The OpenOffice.org Wiki needs a MediaWiki extension developed that will provide a way to convert an aggregation of wiki pages (a "book") to ODT and/or PDF.<br />
<br />
* Required skills/knowledge: PHP and XML, familiarity with ODF a plus<br />
<br />
; Contact<br />
: [mailto:dev@documentation.openoffice.org dev@documentation.openoffice.org]<br />
: ccornell at openoffice.org, juergen.schmidt at sun.com<br />
<br />
<br />
===Netbeans OOo API Plugin: UNO IDL language support===<br />
<br />
Currently, the IDL language support consists only of syntax highlighting. With NetBeans 6 there is a project to support a generic language syntax. This can be used to create a full support including e.g. auto-completion and tool-tips in IDL files.<br />
<br />
[http://wiki.services.openoffice.org/wiki/OpenOffice_NetBeans_Integration OpenOffice.org API plugin for NetBeans]<br />
<br />
[http://languages.netbeans.org Generic Language Framework in NetBeans (Project Schliemann)]<br />
<br />
The work for this project consists of creating the description of the static elements of IDL files. But to provide a meaningful auto completion, the currently available keywords have to be taken dynamically from OpenOffice.org. With this, not only the build-in UNO types of OOo, but also added types from registered extensions can be offered for auto-completion.<br />
<br />
Since there may be performance issues regarding the usage of the Generic Language Framework, this project includes some evaluation work.<br />
<br />
* Required skills/knowledge: Java, OpenOffice.org API, ideally NetBeans API<br />
<br />
; Contact<br />
: [mailto:dev@api.openoffice.org dev@api.openoffice.org]<br />
: steffen.grund at sun.com<br />
<br />
=New Tasks=<br />
<br />
If you have a task that is challenging enough, and there is an OpenOffice.org Project Member willing to mentor the task, feel free to coordinate with the appropriate project lead. If the appropriate project has been found and it supports the task, add it below and make sure the mentor applies with the web app.<br />
<br />
Each entry should contain the task description, required skills, project mailing list for discussion and personal contact. Links to the [[To-Dos]] are appreciated provided that the task is well described there ;-)<br />
<br />
===Example entry===<br />
<br />
This is just an example - the real entry must not start with a space at the beginning<br />
of the line.<br />
A full description of the task should be here; one that will help to see that this task<br />
is important, and interesting to hack on. Alternatively it could be an exact link to<br />
the description that is already in [[To-Dos]].<br />
<br />
* Required skills/knowledge: Language, technology1, technology2, ...<br />
<br />
; Contact<br />
: dev@project.openoffice.org<br />
: The.Mentor at organization com</div>Oshttps://wiki.openoffice.org/wiki/Community_Innovation_Program/proposalsCommunity Innovation Program/proposals2008-05-05T06:02:20Z<p>Os: /* Writer */</p>
<hr />
<div>=Proposals=<br />
==Writer==<br />
<H3>Enhanced usability of working with page attributes and properties</H3><br />
<P>Currently changing page attributes or adding page properties like<br />
headers and footers is hard to understand. Users that don't know<br />
about hard page breaks, page styles and follow styles will have a<br />
hard time to format pages to their likings. A nice UI that hides the<br />
technical details and uses terms of the user space would help a lot. <br />
</P><br />
<P>Tasks <br />
</P><br />
<UL><br />
<LI><P STYLE="margin-bottom: 0cm">learn about Writer's page styles<br />
and their (UNO) API <br />
</P><br />
<LI><P>design and code the new user interface based on this API <br />
</P><br />
</UL><br />
<P>Required skills/knowledge: C++; UI design <br />
</P><br />
<DL><br />
<DT>Contact <br />
</DT><DD><br />
dev@sw.openoffice.org <br />
</DD><DD STYLE="margin-bottom: 0.5cm"><br />
Oliver.Specht at sun dot com <br />
</DD></DL><br />
<P><br />
<BR><BR><br />
</P><br />
<br />
==Calc/Chart==<br />
<br />
===Component to check spreadsheet documents for potential errors===<br />
<br />
Write an extension that checks a spreadsheet document for potential errors in formulas, presents a list of the findings to the user, and allows to correct them.<br />
<br />
Things to look for could include<br />
* Cells that contain numbers stored as text and are used in formulas,<br />
* Number cells that are not used in any formulas,<br />
* Empty cells that are used in single-cell formula references.<br />
The list is not meant to be exhaustive.<br />
<br />
* Required skills/knowledge: Java or C++, spreadsheet usage<br />
<br />
; Contact<br />
: dev@sc.openoffice.org<br />
: Niklas.Nebel at sun.com<br />
<br />
===Watch Window===<br />
<br />
A Watch Window is a separate, small window that remains "on top" and enables users to monitor a selected set of cells, see [http://www.openoffice.org/issues/show_bug.cgi?id=28386 issue 28386].<br />
<br />
This could be implemented as an add-on component with a modeless dialog containing the list of watched cells as well as the UI to add or remove cells.<br />
<br />
* Required skills/knowledge: Java or C++<br />
<br />
; Contact<br />
: dev@sc.openoffice.org<br />
: Niklas.Nebel at sun.com<br />
<br />
==Impress/Draw==<br />
==Base==<br />
<br />
==Misc==<br />
=== OXT Extension wizard for templates and XSLT based filter ===<br />
<br />
OXT extensions provide a nice way to bundle a group of templates and share them with other users. Templates are normally created with the office and the idea is to provide a wizard that helps on a higher abstraction level to collect templates and all necessary oxt specific information and build an oxt extension package.<br />
The second part is to extend the XSLT filter dialog to export new created XSLT based filters as an oxt extension as well. Currently it is only possible to create a jar file. The new feature should collect some more oxt related info and should create a well formed oxt extension package.<br />
The goal is to simplify these tasks and make it as easy as possible to create oxt extension packages that can be easy deployed and can be uploaded in the extensions repository to share them with many other users. <br />
<br />
<br />
* Required skills/knowledge: Java or C++ <br />
<br />
; Contact<br />
: juergen.schmidt at sun.com<br />
<br />
=== ODT/PDF export from OpenOffice.org Wiki ===<br />
<br />
OpenOffice.org documentation is being moved to the OpenOffice.org Wiki. This makes it easier for people to contribute to the content of the documentation, but producing a PDF of ODT of the entire Wiki document or book is not yet possible. The OpenOffice.org Wiki needs a MediaWiki extension developed that will provide a way to convert an aggregation of wiki pages (a "book") to ODT and/or PDF.<br />
<br />
* Required skills/knowledge: PHP and XML, familiarity with ODF a plus<br />
<br />
; Contact<br />
: [mailto:dev@documentation.openoffice.org dev@documentation.openoffice.org]<br />
: ccornell at openoffice.org, juergen.schmidt at sun.com<br />
<br />
<br />
===Netbeans OOo API Plugin: UNO IDL language support===<br />
<br />
Currently, the IDL language support consists only of syntax highlighting. With NetBeans 6 there is a project to support a generic language syntax. This can be used to create a full support including e.g. auto-completion and tool-tips in IDL files.<br />
<br />
[http://wiki.services.openoffice.org/wiki/OpenOffice_NetBeans_Integration OpenOffice.org API plugin for NetBeans]<br />
<br />
[http://languages.netbeans.org Generic Language Framework in NetBeans (Project Schliemann)]<br />
<br />
The work for this project consists of creating the description of the static elements of IDL files. But to provide a meaningful auto completion, the currently available keywords have to be taken dynamically from OpenOffice.org. With this, not only the build-in UNO types of OOo, but also added types from registered extensions can be offered for auto-completion.<br />
<br />
Since there may be performance issues regarding the usage of the Generic Language Framework, this project includes some evaluation work.<br />
<br />
* Required skills/knowledge: Java, OpenOffice.org API, ideally NetBeans API<br />
<br />
; Contact<br />
: [mailto:dev@api.openoffice.org dev@api.openoffice.org]<br />
: steffen.grund at sun.com<br />
<br />
=New Tasks=<br />
<br />
If you have a task that is challenging enough, and there is an OpenOffice.org Project Member willing to mentor the task, feel free to coordinate with the appropriate project lead. If the appropriate project has been found and it supports the task, add it below and make sure the mentor applies with the web app.<br />
<br />
Each entry should contain the task description, required skills, project mailing list for discussion and personal contact. Links to the [[To-Dos]] are appreciated provided that the task is well described there ;-)<br />
<br />
===Example entry===<br />
<br />
This is just an example - the real entry must not start with a space at the beginning<br />
of the line.<br />
A full description of the task should be here; one that will help to see that this task<br />
is important, and interesting to hack on. Alternatively it could be an exact link to<br />
the description that is already in [[To-Dos]].<br />
<br />
* Required skills/knowledge: Language, technology1, technology2, ...<br />
<br />
; Contact<br />
: dev@project.openoffice.org<br />
: The.Mentor at organization com</div>Oshttps://wiki.openoffice.org/wiki/Import_of_Hindi_numbers_from_Microsoft_Word_documentsImport of Hindi numbers from Microsoft Word documents2008-04-16T11:31:26Z<p>Os: /* Configuration */</p>
<hr />
<div>{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
<br />
|- align="left"<br />
| colspan="2" bgcolor="#cccccc" | '''Specification Status''' <br />
|-<br />
| width="150" | '''Author''' || Henning Brinkmann<br />
|-<br />
| width="150" | '''Last Change''' || 17.09.2007<br />
|-<br />
| width="150" | '''Status''' || Preliminary [[Specification_Template_Help#Status|Help]]<br />
|-<br />
|}<br />
<br />
== Abstract ==<br />
<br />
Microsoft Word marks numbers with the script to use by a<br />
hint. Furthermore there is an option to display numbers as Hindi,<br />
Arabic, by Context or determined by the System. This specification<br />
defines how the script hint and the display option shall be handled on<br />
import of Microsoft Word documents.<br />
<br />
__TOC__<br />
<br />
== References ==<br />
<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| width="300" bgcolor="#dddddd" | '''Reference Document''' || bgcolor="#dddddd" | '''Check''' || bgcolor="#dddddd" | '''Location (URL)'''<br />
|-<br />
| |'''[http://wiki.services.openoffice.org/wiki/Category:Specification#Before_Writing_a_Software_Specification_--_What_Else_Do_I_have_to_Do.3F Specification Process Entry Check]'''<br />
| passed<br />
| n/a<br />
|-<br />
| '''Product Requirement, RFE, Issue ID''' (required)<br />
| available<br />
| [http://www.openoffice.org/issues/show_bug.cgi?id=81577]<br />
|-<br />
| Product Concept Document<br />
| not available<br />
| <br />
|-<br />
| '''[[Test case specification]]''' (required)<br />
| not available<br />
| <PLEASE ENTER LOCATION HERE><br />
|-<br />
| IDL Specification<br />
| not available<br />
| <br />
|-<br />
| [http://wiki.services.openoffice.org/wiki/The_Three_Golden_Rules_for_Writing_OpenOffice.org_Specifications '''Software Specification Rules''']<br />
| n/a<br />
| n/a<br />
|-<br />
| Other, e.g. references to related specs<br />
|<br />
| <br />
|-<br />
|}<br />
<br />
== Contacts ==<br />
<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| width="300" bgcolor="#dddddd" | '''Role''' || bgcolor="#dddddd" | '''Name''' || bgcolor="#dddddd" | '''E-Mail Address'''<br />
|-<br />
| '''Developer'''<br />
| Henning Brinkmann<br />
| Henning.Brinkmann@sun.com<br />
|-<br />
| '''Quality Assurance'''<br />
| Michael Rüß<br />
| Michael.Ruess@sun.com<br />
|-<br />
| '''Documentation'''<br />
| Uwe Fischer<br />
| Uwe.Fischer@sun.com<br />
|-<br />
| '''User Experience'''<br />
| <First Name, Last Name><br />
| <User@openoffice.org><br />
|-<br />
|}<br />
<br />
== Acronyms and Abbreviations ==<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| bgcolor="#dddddd" | '''Acronym / Abbreviation''' || bgcolor="#dddddd" | '''Definition'''<br />
|-<br />
| <WYSIWYG><br />
| <What You See Is What You Get><br />
|-<br />
|}<br />
<br />
== Detailed Specification ==<br />
<br />
When a digit is marked to have CTL script in the imported Word<br />
document it shall be imported as Hindi digit. This feature shall only<br />
be activated if and only if the configuration item RegardHindiDigits<br />
(see below) is true.<br />
<br />
If the configuration item RegardHindiDigits is set the following<br />
mapping between Arabic and Hindi characters applies:<br />
<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| width="300" bgcolor="#dddddd" | '''Arabic (Unicode)''' || bgcolor="#dddddd" | '''Hindi (Unicode)'''<br />
|-<br />
| 0 (U+0030)<br />
| &#x0660; (U+0660)<br />
|-<br />
| 1 (U+0031)<br />
| &#x0661; (U+0661)<br />
|-<br />
| 2 (U+0032)<br />
| &#x0662; (U+0662)<br />
|-<br />
| 3 (U+0033)<br />
| &#x0663; (U+0663)<br />
|-<br />
| 4 (U+0034)<br />
| &#x0664; (U+0664)<br />
|-<br />
| 5 (U+0035)<br />
| &#x0665; (U+0665)<br />
|-<br />
| 6 (U+0036)<br />
| &#x0666; (U+0666)<br />
|-<br />
| 7 (U+0037)<br />
| &#x0667; (U+0667)<br />
|-<br />
| 8 (U+0038)<br />
| &#x0668; (U+0668)<br />
|-<br />
| 9 (U+0039)<br />
| &#x0669; (U+0669)<br />
|-<br />
|}<br />
<br />
[[Specification_Template_Help#Detailed_Specification|Help]] | [[UI-Elements|User Interface Element Templates]] | [[Specification_Example|Example Spec]]<br />
<br />
== Migration ==<br />
<br />
The specified features improves interoperability with Microsoft Word.<br />
<br />
== Configuration ==<br />
<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| width="300" bgcolor="#dddddd" | '''Configuration''' || bgcolor="#dddddd" | '''Group''' || bgcolor="#dddddd" | '''Setting''' || bgcolor="#dddddd" | '''Type''' || bgcolor="#dddddd" | '''Default''' || bgcolor="#dddddd" | '''Comment''' | <br />
|-<br />
| Writer.xcs<br />
| FilterFlags/WinWord<br />
| RegardHindiDigits<br />
| xs:long<br />
| false<br />
| If true yields to digits marked as CTL script to be imported as Hindi digits.<br />
|}<br />
<br />
[[Specification_Template_Help#Configuration|Help]] | [[Configuration-Table|Configuration Table Template]]<br />
<br />
== File Format ==<br />
<br />
This specification covers import only and thus has no consequences<br />
regarding the file format.<br />
<br />
[[Specification_Template_Help#File_Format|Help]]<br />
<br />
[[Specification_Template_Help#File Format|Help]] | [[File Format Table|File Format Table Template]]<br />
<br />
== Open Issues ==<br />
<State a bulleted list of issues Issue here><br />
<br />
[[Category:Specification]]</div>Oshttps://wiki.openoffice.org/wiki/Import_of_Hindi_numbers_from_Microsoft_Word_documentsImport of Hindi numbers from Microsoft Word documents2008-04-16T11:24:12Z<p>Os: /* Configuration */</p>
<hr />
<div>{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
<br />
|- align="left"<br />
| colspan="2" bgcolor="#cccccc" | '''Specification Status''' <br />
|-<br />
| width="150" | '''Author''' || Henning Brinkmann<br />
|-<br />
| width="150" | '''Last Change''' || 17.09.2007<br />
|-<br />
| width="150" | '''Status''' || Preliminary [[Specification_Template_Help#Status|Help]]<br />
|-<br />
|}<br />
<br />
== Abstract ==<br />
<br />
Microsoft Word marks numbers with the script to use by a<br />
hint. Furthermore there is an option to display numbers as Hindi,<br />
Arabic, by Context or determined by the System. This specification<br />
defines how the script hint and the display option shall be handled on<br />
import of Microsoft Word documents.<br />
<br />
__TOC__<br />
<br />
== References ==<br />
<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| width="300" bgcolor="#dddddd" | '''Reference Document''' || bgcolor="#dddddd" | '''Check''' || bgcolor="#dddddd" | '''Location (URL)'''<br />
|-<br />
| |'''[http://wiki.services.openoffice.org/wiki/Category:Specification#Before_Writing_a_Software_Specification_--_What_Else_Do_I_have_to_Do.3F Specification Process Entry Check]'''<br />
| passed<br />
| n/a<br />
|-<br />
| '''Product Requirement, RFE, Issue ID''' (required)<br />
| available<br />
| [http://www.openoffice.org/issues/show_bug.cgi?id=81577]<br />
|-<br />
| Product Concept Document<br />
| not available<br />
| <br />
|-<br />
| '''[[Test case specification]]''' (required)<br />
| not available<br />
| <PLEASE ENTER LOCATION HERE><br />
|-<br />
| IDL Specification<br />
| not available<br />
| <br />
|-<br />
| [http://wiki.services.openoffice.org/wiki/The_Three_Golden_Rules_for_Writing_OpenOffice.org_Specifications '''Software Specification Rules''']<br />
| n/a<br />
| n/a<br />
|-<br />
| Other, e.g. references to related specs<br />
|<br />
| <br />
|-<br />
|}<br />
<br />
== Contacts ==<br />
<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| width="300" bgcolor="#dddddd" | '''Role''' || bgcolor="#dddddd" | '''Name''' || bgcolor="#dddddd" | '''E-Mail Address'''<br />
|-<br />
| '''Developer'''<br />
| Henning Brinkmann<br />
| Henning.Brinkmann@sun.com<br />
|-<br />
| '''Quality Assurance'''<br />
| Michael Rüß<br />
| Michael.Ruess@sun.com<br />
|-<br />
| '''Documentation'''<br />
| Uwe Fischer<br />
| Uwe.Fischer@sun.com<br />
|-<br />
| '''User Experience'''<br />
| <First Name, Last Name><br />
| <User@openoffice.org><br />
|-<br />
|}<br />
<br />
== Acronyms and Abbreviations ==<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| bgcolor="#dddddd" | '''Acronym / Abbreviation''' || bgcolor="#dddddd" | '''Definition'''<br />
|-<br />
| <WYSIWYG><br />
| <What You See Is What You Get><br />
|-<br />
|}<br />
<br />
== Detailed Specification ==<br />
<br />
When a digit is marked to have CTL script in the imported Word<br />
document it shall be imported as Hindi digit. This feature shall only<br />
be activated if and only if the configuration item RegardHindiDigits<br />
(see below) is true.<br />
<br />
If the configuration item RegardHindiDigits is set the following<br />
mapping between Arabic and Hindi characters applies:<br />
<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| width="300" bgcolor="#dddddd" | '''Arabic (Unicode)''' || bgcolor="#dddddd" | '''Hindi (Unicode)'''<br />
|-<br />
| 0 (U+0030)<br />
| &#x0660; (U+0660)<br />
|-<br />
| 1 (U+0031)<br />
| &#x0661; (U+0661)<br />
|-<br />
| 2 (U+0032)<br />
| &#x0662; (U+0662)<br />
|-<br />
| 3 (U+0033)<br />
| &#x0663; (U+0663)<br />
|-<br />
| 4 (U+0034)<br />
| &#x0664; (U+0664)<br />
|-<br />
| 5 (U+0035)<br />
| &#x0665; (U+0665)<br />
|-<br />
| 6 (U+0036)<br />
| &#x0666; (U+0666)<br />
|-<br />
| 7 (U+0037)<br />
| &#x0667; (U+0667)<br />
|-<br />
| 8 (U+0038)<br />
| &#x0668; (U+0668)<br />
|-<br />
| 9 (U+0039)<br />
| &#x0669; (U+0669)<br />
|-<br />
|}<br />
<br />
[[Specification_Template_Help#Detailed_Specification|Help]] | [[UI-Elements|User Interface Element Templates]] | [[Specification_Example|Example Spec]]<br />
<br />
== Migration ==<br />
<br />
The specified features improves interoperability with Microsoft Word.<br />
<br />
== Configuration ==<br />
<br />
{| border="2" cellpadding="4" cellspacing="0" style="margin: 1em 1em 1em 0; border: 1px #cccccc solid; border-collapse: collapse; width: 100%"<br />
|-<br />
| width="300" bgcolor="#dddddd" | '''Configuration''' || bgcolor="#dddddd" | '''Group''' || bgcolor="#dddddd" | '''Setting''' || bgcolor="#dddddd" | '''Type''' || bgcolor="#dddddd" | '''Default''' || bgcolor="#dddddd" | '''Comment''' | <br />
|-<br />
| Writer.xcs<br />
| FilterFlags/WinWord<br />
| RegardHindiDigits<br />
| Boolean<br />
| false<br />
| If true yields to digits marked as CTL script to be imported as Hindi digits.<br />
|}<br />
<br />
[[Specification_Template_Help#Configuration|Help]] | [[Configuration-Table|Configuration Table Template]]<br />
<br />
== File Format ==<br />
<br />
This specification covers import only and thus has no consequences<br />
regarding the file format.<br />
<br />
[[Specification_Template_Help#File_Format|Help]]<br />
<br />
[[Specification_Template_Help#File Format|Help]] | [[File Format Table|File Format Table Template]]<br />
<br />
== Open Issues ==<br />
<State a bulleted list of issues Issue here><br />
<br />
[[Category:Specification]]</div>Oshttps://wiki.openoffice.org/wiki/Developers_FAQDevelopers FAQ2007-07-16T09:17:36Z<p>Os: /* Application, Writer */</p>
<hr />
<div>The Developers FAQ is a collection from different questions asked by Developers who are new to OpenOffice.org:<br />
<br />
This is work in progress, the developer mentioned behind the question will write the answer soon.<br />
<br />
<br />
==Framework==<br />
* Description of the SFX2 framework? (MBA)<br />
* There are the roles of the different shells and views? See [[Framework/Article/Implementation_of_the_Dispatch_API_In_SFX2|Implementation of the Dispatch API In SFX2]]<br />
* Explanation of the dispatch/interceptor mechanism / command handling? See [[Framework/Article/Implementation_of_the_Dispatch_API_In_SFX2|Implementation of the Dispatch API In SFX2]]<br />
* How is the slot mechanism implemented, and how is the listener and broadcast mechanism implemented? See [[Framework/Article/Implementation_of_the_Dispatch_API_In_SFX2|Implementation of the Dispatch API In SFX2]]<br />
* Generic user interface: How does the framework implement the interface management, how to load toolbar, and how to implement berth area? See [[Framework/Article/General_Architecture_Of_The_Framework_User_Interface_Implementation|General Architecture of the Framework User Interface]]<br />
* Filter configuration<br />
** [http://api.openoffice.org/docs/DevelopersGuide/OfficeDev/OfficeDev.xhtml#1_2_4_Integrating_Import_and_Export_Filters Developer's Guide 7.2.4 Integrating Import and Export Filters]<br />
** Note: Description of the used configuration format in XML is outdated.<br />
* API for accessing OOo package format<br />
** http://api.openoffice.org/docs/common/ref/com/sun/star/embed/Storage.html<br />
* Embedding (MAV)<br />
* Templates (MAV)<br />
* Add-ons / Extensions support<br />
** [[Framework/WorkInProgress/Addon_Menu_Toolbar_Merging|Support merging into the Office menu bar/toolbars.]]<br />
** [[Framework/Article/Generic_UNO_Interfaces_for_complex_toolbar_controls|Complex toolbar controls for add-ons.]]<br />
** [http://api.openoffice.org/docs/DevelopersGuide/Components/Components.xhtml#1_7_3_Add-Ons Developer's Guide 4.7.3 Add-ons]<br />
** Add-on examples with sources (http://framework.openoffice.org/servlets/ProjectDocumentList?folderID=226&expandFolder=226&folderID=72) <br />
* The general part of the document storing/loading process (MAV)<br />
<br />
==[http://udk.openoffice.org UDK]==<br />
* Mechanisms of Multi-Platform support ([[KR]]) <br />
:I assume this means multiple language support, please see [[Uno/Article/Working_with_Environments%2C_Mappings_%26_Objects|Working with Environments, Mappings & Objects]] .<br />
* How to communicate with other OOo components, and how to implement a bridge ([[KR]]) <br />
: There is a tutorial for implementing C++ bridges, please see http://udk.openoffice.org/cpp/man/cpp_bridges.html . Most [[Uno/Binary|Binary Uno]] bridges are implemented in the [[Uno/Binary/Module/bridges|bridges]] module.<br />
* [[Uno/PyUno|Python Uno]] components may be run within the OOo process - how can they be called from [[Java]], [[C++]] or [[BASIC|StarBasic]]? ([[KR]])<br />
: [[Uno/PyUno|Python Uno]] components may be accessed like any other component, e.g. by doing a "createInstance" with a service which is implemented in [[Uno/PyUno|Python Uno]].<br />
* The relationship between [[Uno]], COM+, CORBA ([[KR]])<br />
: For a comparison between [[Uno]] and CORBA please see http://udk.openoffice.org/common/man/comparison_uno_corba.html , though [[Uno]] nowadays does support multiple inheritance. Also there was an effort, which unfortunately did not complete, do implement a [[Uno]] CORBA bridge, please see http://udk.openoffice.org/common/man/concept/uno_corba.html .<br />
: There is no document regarding [[Uno]] and (D)COM+. [[Uno]] has some similarities with COM+:<br />
:* "queryInterface"<br />
:* Language agnostic<br />
:* "Apartments", please see the [[Uno/Spec/Threading-Model|Uno Threading-Model]] for details.<br />
:* Components<br />
:* Remote transparency<br />
<br />
==Application, Writer==<br />
* Structure of tables? [[Writer_Core_And_Layout]]<br />
* Mechanism/Implementation of Undo? [[Writer_Core_And_Layout]]<br />
* Mechanism/Implementation of Frames, relationship? [[Writer_Core_And_Layout]]<br />
* Class structure of SwNode, SwNodes, SwNodeIndex, ..., relationship? [[Writer_Core_And_Layout]]<br />
* The relationship between the class ViewShell and SwDoc (OS)<br />
The ViewShell is created as indirect member of the SwPagePreview or as base class of SwWrtShell, SwFEShell, SwEditShell, SwCrsrShell as member of the SwView. This stack of shells provides access from SwView/SwPagePreview to the document. The ViewShell mostly for output (printing, painting) and the others for cursor travelling and content modification.<br />
<br />
==Application, Draw/Impress==<br />
* The mechanism of slide show and animation (what is the difference between before and after)?<br />
** The slideshow itself, and all animation functionality are implemented in the [[Slideshow]] module. (What is the meeaing of "before" and "after" in the context of this question?)<br />
* Copy&Paste: Memory structure of clipboard in SD, difference between coping objects inside slides and coping objects from or into other applications like SW, SC? <br />
** Clipboard between the same application type is done by using core implementations. Clipboard between different application types is done by using xml.<br />
* Mechanism of designing and implementation of master page and layout style in presentation (CL)<br />
** Master pages and layout styles are already implemented in OOo.<br />
<br />
* Flash Export: (CL)<br />
** Difference between the export of the simple character and the complex character?<br />
*** Simple characters are exported using the font tags of the flash format, complex characters are exported as polygons.<br />
** The implementation of the Polygon in VCL. Such as the EM square,Body,Ascent and Descent to realize of the glyph?<br />
*** This question needs more data as the implementation of the Polygon in VCl itself has nothing to do with glyphs. It is only used to represend glyphs.<br />
** The custom animation will be lost after the slides being exported into the swf. What cause it? There is no support to export the animation, is there? Do you have any plan to consummate it? And what's your purpose in this area?<br />
*** It is caused by the absence of an implementation that exports this particular feature. There is currently no assigned developer for the task to add this feature to the flash export. One developer from Redflag is activly contributing to the flash export.<br />
<br />
==Application, Calc==<br />
* How are the areas for charts recognized, and how does the update notification work? (NN)<br />
** Note that much of this has changed for the new chart. Don't make any changes based on older versions (before SRC680m213).<br />
** The chart implementation uses the UNO API (XDataProvider) to get data sequence objects for each data series, and registers itself as modify listener with these objects.<br />
** For the time when a chart object hasn't been loaded yet, Calc has its own internal listener (ScChartListener) to load the chart if its data is changed.<br />
* Relationship between cells, sheets and workbooks (class diagram) (NN)<br />
** The core implementation of the document is class ScDocument. It has an array of pointers to ScTable (implementation of a sheet). Each sheet has a fixed-size array of ScColumn objects. Each ScColumn object holds the cells in one column.<br />
** The cells are implemented by base class ScBaseCell and derived classes:<br />
*** ScNoteCell (empty except for note and listeners)<br />
*** ScValueCell (constant numeric value)<br />
*** ScStringCell (simple text)<br />
*** ScEditCell (formatted or multi-paragraph text)<br />
*** ScFormulaCell (formula)<br />
* Copy&Paste: What's happening on Copy directly, and later if someone pastes into same spreadsheet or other application (NN)<br />
** Implementation of copy for cells is at ScViewFunc::CopyToClip. The cells are copied into an internal object (ScDocument), then a ScTransferObj is created as the interface to the clipboard.<br />
** Pasting into Calc copies the cells directly from the internal ScDocument object. Clipboard data in other formats for external applications is created on demand by ScTransferObj (ScTransferObj::GetData).<br />
<br />
==Application, Database==<br />
* Why was the dbase driver engineered into file_based driver? And compared with the text/csv driver, what are the advantages of it in OOo's database, and how it was being designed in OOo's database?<br />
** The "file" driver is not an own driver, it's a framework for, well, file-based databases, where a single file represents a table, and all such files in a given directory comprise the database. Both the text/csv driver and the dBase driver use this framework.<br />
** The advantage of the dBase driver over the text/csv driver is that the former can read dBase files, while the latter can't.<br />
** What do you mean with "was being designed"?<br />
* What are the project's short-term plan or the direction of development now?<br />
** There are two areas which we currently mainly focus on: The new report designer which will be part of OpenOffice.org 2.3, and improved programmability of Base, where the first step is to allow embedding macros in database document.<br />
* In the project named "Native, cross-platform access to MS Access databases", what is the current solution? Is there anyone beginning to re-engineer the MDB format himself?<br />
** The current solution is employing [http://mdbtools.sourceforge.net/ MDB-Tools], a third-party product which allows access to .mdb files. The driver based on MDB-Tools is in [http://eis.services.openoffice.org/EIS2/cws.ShowCWS?logon=true&Id=2959&Path=SRC680%2Fmdbdriver02 alpha state], and still has serious shortcomings. The project itself is currently stalled, nobody is actively working on it. Also, nobody is working on doing a reverse-engineering of the MSA file format. If somebody wants to do this, we should seriously investigate whether it makes more sense to support and improve existing reverse engineering efforts, namely MDB-Tools.<br />
<br />
==Application, Chart2==<br />
* Description of the Chart2 module: [http://graphics.openoffice.org/chart/chart2codestructure.html Code Structure of chart2]<br />
<br />
==General Questions==<br />
* Which are the common threads started at start up time and used by all OOo applications at runtime, and what are their functions? ([[KR]])?<br />
: A first over regarding the threads can be found here: [[Effort/Implement_Basic_Threading-Architecture#Threads]] More to come ...<br />
* Structure of VCL? See [[Sort_of_documentation_about_VCL_around_Native_Mac_OS_X_port]]<br />
** A general overview about VCL was created by Eric Bachard, please have a look [http://wiki.services.openoffice.org/wiki/User:Ericb#Sort_of_documentation_about_VCL_around_Native_Mac_OS_X_port here]<br />
* [[TextRenderingProcess|Text rendering process]]<br />
* Memory Management (MT/MHU)<br />
* How to improve stability, and where to start? Beside doing thorough QA, you might use tools like [http://tools.www.openoffice.org/debugging/usingvalgrind.sxw Valgrind] or [http://tools.openoffice.org/files/documents/15/872/BoundsChecker.html BoundsChecker]<br />
* How to improve performance, and where to start? Use a [http://en.wikipedia.org/wiki/Performance_analysis profiler] to analyze the runtime behavior of the code that you want to improve.<br />
* Which code of OOo is responsible for the VBA Macro parse? How are they implemented?<br />
** This is done by a hard coded parser in basic/source/comp<br />
* How is the code implemented to fulfill the specification of Unicode and gb2312 (SB): Follow the source code links for &ldquo;UTF7&ldquo; and &ldquo;UTF8&ldquo; (Unicode) and &ldquo;EUC_CN&rdquo; (GB2312) in [http://porting.openoffice.org/source/browse/porting/sal/textenc/textenc.cxx?rev=1.6&view=markup sal/textenc/textenc.cxx:1.6].<br />
* How/where is the Boost library used? See [[boost]] for an overview.<br />
* The process of SVG protraction and SVG parse. The relationship between controller protraction and the resource of the controller. (SJ)<br />
* Mozilla and ActiveX plugins (MAV)<br />
* Which internet protocols are supported in OOo, and what can be improved to the network ability of OOo? (KSO)<br />
<br />
==Feature Questions==<br />
* Currently a document can only be processed by one of the OOo applications - what about having just one application processing all kind of documents?<br />
* Are there any plans to integrate a desktop search engine? (Could be done as an extension)<br />
* Any plans to introduce classes comparable to CObject and CWinApp in MFC? (???)<br />
<br />
<br />
[[Category:FAQ]]<br />
[[Category:Developer_Documentation]]</div>Os