Performance/Meetings/2009 03
From Apache OpenOffice Wiki
< Performance | Meetings
Revision as of 07:13, 16 March 2009 by Yugq@redflag2000.cn (Talk | contribs)
Contents
2009-03-06
Brief meeting summary
Overall status
| Work | Status Description | Member
(IRC Nick) |
| OOo Performance Test System | A set of tools in Java for analysis binary raw data, and will create and update the CWS next week. | liheng |
| Changing SfxItemPropertyMap SfxItemPropertySet | It required changes in almost all applications. Everything works well now but I have not had time to create data about the results in load/save time. They will be ready by the end of next week. | so_ooo |
| Merging UNO dlls | yugq | |
| Analyzing Load/Save | yugq | |
| Asynloading Impress | Created CWS asynloadingimpress, based on m42
Put the modified code into on the latest version of the m42,but there some problems neet to check. |
xiuzhi, zhangyuwei |
| Improvement of compiling spreadsheet formulas | erAck | |
| configmgr | Created and Updated to CWS: svn://svn@svn.services.openoffice.org/ooo/cws/configtune00. | liangjun |
| Tool for Collecting RTL_LOG data into a spreadsheet | ooo_mib | |
| Analyzing the start up performance | That most time is lost waiting for file I/O. Library loading is the first to blame, next reading data files.
Another problem is that we lose about 10% with reading internal file system file (for NTFS this is $MFT (master file table) and $Directory. |
oo_cd |
| I/O around PDF export | Working on buffered file I/O around PDF export. | tora-japan |
Details
Test System
- The new test system seems almost like the RTL_LOGFILE, but it will create a web system to trace hotspots defined in performance diagram for long-term development. Maybe RTL_LOGFILE and text system can use a all-purpose offline report template (a set of spreadsheet)
Save
- Saving changed parts is a well known good idea.
Test on other OS
- ericb2 Can be helpful
Improvement for UX(User Experience)
- Saving in the background
- Native splash screen would also be good
- Do something for it and discuss again
Action Item
- liheng to write the MM.
URIs mentioned
- Calc/Performance/string_handling_in_formula_compiler (erAck)
- Configmgr_Refactoring/Design (liangjun)
- Performance_Benchmark_Tool_Set (liheng)
Log of meeting
| (4:33:32 PM) liheng: Agenda: |
| (4:33:33 PM) liheng: 1.Communication with overall status. |
| (4:33:33 PM) liheng: 2.To discuss ideas of Saving. |
| (4:33:33 PM) liheng: 3.To distinguish user experience works from real optimizing works. |
| (4:33:43 PM) peter13j: liheng: was much longer than I expect the flowing ones to be |
| (4:35:13 PM) liheng: We can improve it step by step |
| (4:35:39 PM) FrankL [n=fl@nat/sun/x-9526a11c2d026787] entered the room. |
| (4:35:55 PM) peter13j: I guess the length was a result of my proposals at the previous meeting |
| (4:36:06 PM) liheng: OKay we start with our agenda, 1.Communication with overall status. |
| (4:36:12 PM) yugq [n=Administ@218.249.75.106] entered the room. |
| (4:38:21 PM) yugq: Hi, all |
| (4:38:31 PM) liheng: I am just completing test system for performance of OOo |
| (4:38:53 PM) liheng: yugq:1.Communication with overall status. |
| (4:39:01 PM) os_ooo: I'm almost done with the changes of SfxItemPropertyMap/SfxItemPropertySet. It required changes in almost all applications. Everything works well now but I have not had time to create data about the results in load/save time. They will be ready by the end of next week. |
| (4:39:03 PM) os_ooo: But I wouldn't expect miracles ;-) |
| (4:39:25 PM) yugq: LiHeng: OK. I'm working for merging UNO dlls and analyzing load and save these days. |
| (4:39:32 PM) xiuzhi: CWS asynloadingimpress has been created, based on m42, |
| (4:40:22 PM) liheng: os_ooo: We need useful works,but not miracles. |
| (4:40:39 PM) erAck: some small 5% improvement when compiling spreadsheet formulas, see http://wiki.services.openoffice.org/wiki/Calc/Performance/string_handling_in_formula_compiler |
| (4:40:49 PM) liangjun: Write the wiki on http://wiki.services.openoffice.org/wiki/Configmgr_Refactoring/Design about configmgr and sent our new configmgr source on svn://svn@svn.services.openoffice.org/ooo/cws/configtune00 |
| (4:41:10 PM) odf-mib: I have created a tool that collects data from the RTL_LOGFILE in a spreadsheet, and allows to a) compare to logfiles, and b) entries within a log file. |
| (4:41:33 PM) odf-mib: More details can be found in the Wiki |
| (4:41:39 PM) Malte: Not too much updates from my side - just many discussions with other people, and Intel support wrt VTune issue with OOo 3 |
| (4:42:59 PM) cd_oo: I finished my first step analyzing the start up performance of OpenOffice.org under Windows. What I found out is that most time is lost waiting for file I/O. Library loading is the first to blame, next reading data files. Another problem is that we lose about 10% with reading internal file system file (for NTFS this is $MFT (master file table) and $Directory. Next I want to try out if start... |
| (4:43:01 PM) cd_oo: ...up performance improves when all libraries are located next to the executable (first test look promising). I also want to reorder object files within libraries to improve sequential access to the library pages. |
| (4:44:07 PM) FrankL left the room (quit: Read error: 104 (Connection reset by peer)). |
| (4:44:32 PM) FrankL [n=fl@nat/sun/x-fa823f4f77055f99] entered the room. |
| (4:44:38 PM) tora-japan: I have been working on buffered file I/O around PDF export. |
| (4:44:38 PM) liheng: odf-mib:We are just building a tools in Java to analysis our binary raw data, and we will update a CWS next week. |
| (4:44:38 PM) tora-japan: In this time, no change would be made in SAL as Eike suggested. |
| (4:44:38 PM) tora-japan: |
| (4:45:47 PM) zhangyuwei: I have just put the modified code into on the latest version of the m42 about Asynchronous Loading. |
| (4:45:47 PM) zhangyuwei: but, I found that loading file is so slowly in the latest version, and i'm checking the code. |
| (4:46:13 PM) liheng: Can we discuss a all-purpose template in a shreadsheet? |
| (4:46:14 PM) odf-mib: liheng: What is "binary raw data", and what is the output of the tool? We may follow-up on this on the list, actually. |
| (4:46:52 PM) Malte: And what is "all-purpose template in a shreadsheet"? |
| (4:47:51 PM) erAck: tora-japan: for the read/write statistics it would be good to know from which implementation the calls came, e.g. UCB, SvStream, "manual" file access, ... |
| (4:48:16 PM) tora-japan: erAck: Yeap! |
| (4:49:03 PM) liheng: odf-mib:To measure mainly hotspots like load/save and startup we define a performance diagram and to map it into code and get User Experience Index and Techincal performance Index, that log in a binary format. |
| (4:49:21 PM) yugq: ALL: I want to explain about the CWS DEV300/configtune00. Because this CWS was planned a long time ago(about a half year), we chose milestone DEV300/m26 that time. And now it is not changed to newer milestone. We will complete the code gradually and solve the current problem. We can't compile and use the configtune00 correctly by normal and automatic way for current state. So if someone want to check and use it, please contact with liangjun(zenglian |
| (4:50:18 PM) odf-mib: liheng: Sounds similar to RTL_LOGFILE. |
| (4:50:50 PM) liheng: Malte:We do almost same work as odf-mib,but we build system on server,we want use a common template for offline report |
| (4:51:16 PM) Malte: liheng: Got it, thanks :) |
| (4:51:44 PM) liheng: odf-mib:Yes,but our goals are trace in long-term development,... |
| (4:52:47 PM) odf-mib: liheng: I think we should discuss on the list where the differencies and similarities are, and where we can share something. |
| (4:53:54 PM) liheng: odf-mib:Yes, some details in http://wiki.services.openoffice.org/wiki/Performance_Benchmark_Tool_Set |
| (4:54:54 PM) liheng: odf-mib:next week, YuGuoqiang will update CWS of it for me |
| (4:55:40 PM) odf-mib: lihengs: Thanks. I will have a look at this, |
| (4:56:26 PM) liheng: OK,can we turn next "To discuss ideas of Saving."? |
| (4:58:53 PM) liheng: We are discussing asynchronously save for user experience when they are saving a document. |
| (5:00:43 PM) liheng: anything else for saving ? |
| (5:01:41 PM) tora-japan: What does the term 'asynchronously' mean here? |
| (5:01:41 PM) tora-japan: Saving a document without interrupting user's action? |
| (5:01:41 PM) tora-japan: |
| (5:01:51 PM) odf-mib: We should in any case consider to save on changed parts, where a part may be for instance a drawing page. |
| (5:02:00 PM) odf-mib: on -> only |
| (5:02:32 PM) erAck: Niklas showed that incremental save can speed things up by 30-50% if only small changes were done. |
| (5:02:34 PM) Malte: And making saving faster at all, even w/o only saving parts, or saving in the background |
| (5:03:03 PM) odf-mib: Sure:-) |
| (5:03:29 PM) ericb2: I see there are test under Windows, and I'd like to know whether other OS are compared. e.g. is somebody working on Mac OS X ? |
| (5:03:39 PM) ericb2: or Linux or whatever |
| (5:03:50 PM) liheng: odf-mib:us too, consider samething, we can save changed parts to temprary file and UnlockUI, make a new thread to complete saving work |
| (5:04:30 PM) JackieSun: As I discussed with others, in some conditions, the user would NOT use the high-resolution images, we can save the downscaled ones only |
| (5:04:51 PM) cd_oo: ericb2: Stephan Bergmann is working on start up performance on Linux. |
| (5:05:03 PM) ericb2: cd_oo: thank you :) |
| (5:05:33 PM) liheng: tora-japan:asynchronously means do more work of save with another thread |
| (5:05:36 PM) Malte: ericb: I thought Mac OSX would be your part? ;) |
| (5:06:41 PM) ericb2: Malte: Mac OS X is not "my" part, but I'm interested to participate to the improvement and if I can be helpfull, I'll do ;-) |
| (5:06:49 PM) liheng: tora-japan:At least,the task of creating zip file can move out |
| (5:07:37 PM) tora-japan: liheng: Thanks! In that case, 'concurrently' would be better? |
| (5:08:05 PM) liheng: tora-japan:thank you:) |
| (5:08:54 PM) liheng: last topic:To distinguish user experience works from real optimizing works |
| (5:09:28 PM) tora-japan: liheng: Just an example. User make an action, then OOo synchronously do the command. Without any action made, OOo asynchronously start some task. |
| (5:09:51 PM) liheng: Yes:) |
| (5:10:19 PM) liheng: As Malte's suggestions, some work can make OOo "seem good"... |
| (5:10:47 PM) Malte: Saving in the background is already such a task |
| (5:10:59 PM) Malte: native splash screen would also be good for this |
| (5:11:35 PM) Malte: native spash would helping "seeing" OOo immediately, even when it then takes 20s to start |
| (5:12:10 PM) Malte: Very helpful e.g. on double clicking an email attachment, where you always wonder "did I start OOo now?", because it takes ages to see anything |
| (5:12:35 PM) liheng: so, we redefined the checkpoint in our performance diagram for distinguishing the user experience and technical value |
| (5:14:40 PM) liheng: Malte:We really need more works for rapid response |
| (5:15:31 PM) liheng: FrankL:Do you have some idea for it? |
| (5:15:44 PM) liheng: FrankL: Do you have some idea for it? |
| (5:16:02 PM) FrankL: I have to think about it. |
| (5:17:32 PM) liheng: I think we can create a sub-project for improving response |
| (5:18:39 PM) FrankL: Immediate feedback is very important. So the application feels alive and responsive. So having a sub project for this sounds good for me. |
| (5:19:00 PM) tora-japan: In general, it is said that if a machine need to take more than approx. 0.7 seconds to respond to user's command, do something to show a sign "I have started it and been working on." |
| (5:19:15 PM) Malte: I don't like to term "sub project". |
| (5:19:33 PM) Malte: It is _one_ performance project, and we shouldn't split up into several sub projects, IMHO |
| (5:20:08 PM) Malte: It's simply up to UX to also have this topic in mind |
| (5:20:12 PM) Malte: So we also work on this |
| (5:20:19 PM) FrankL: But we have two kinds of issues. Technical and those UX related ones. |
| (5:20:35 PM) Malte: UX is also technical |
| (5:21:01 PM) Malte: UX also doesnt work w/o any implementation |
| (5:21:02 PM) FrankL: One part art one par science ;-) |
| (5:21:19 PM) liheng: Malte:Yes but,it need different methods of measuring performance |
| (5:21:47 PM) Malte: Does it? |
| (5:21:56 PM) Malte: It simply adds one more data point |
| (5:22:12 PM) Malte: Which is before the action really ends... |
| (5:22:33 PM) liheng: Yes,we need change viewpoint from user feeling |
| (5:23:33 PM) liheng: Malte:We can do something for it and discuss again :) |
| (5:24:15 PM) liheng: That'all for me, it's my turn to make meeting minutes. |
| (5:25:12 PM) liheng: Anything else? |
2009-3-13
Overall Status
| Work | Status Description | Member(IRC nick) |
| OOo Performance Test System | Just finished the first version of test system, and will create CWS and update it next week... | liheng |
| Investigating Calc's area broadcasters | Started investigating Calc's area broadcasters, see http://wiki.services.openoffice.org/wiki/Calc/Performance/Refactoring_Area_Broadcasters, but got interrupted by 3.1 blocker issues. | erAck |
| Runtime loader instructions improvements on Linux | Got substantial improvements of instructions burned in runtime loader on Linux through use of -Bsymbolic-functions/-Bsymbolic (issue 85679) together with latest GCC/binutils; now looking at whether those changes are indeed correct (esp. -Bsymbolic, which can cause different bindings be used for one data symbol across libraries). | sb |
| Reorder code to reduce file I/O | I want to pack the code which is needed during start up to sequential pages. This should significantly reduce file I/O and help start up performance. I want to create a little helper library which traces what symbols are used during startup. This file can be used with the help of another tool to find the mangled names in the map file. The mangled names must to be put into an order file which can be used by the Microsoft linker to reorder the code. | cd_oo |
| Incremental save | Trying to finish an incremental storage model for XML files. | yugq |
| Speedup drawing | I have submitted "downscaledimageshow " in CWS. Which can increase the drawing speed by using the downscaled bitmap of high-resolution images only to draw or "slide show" in the screen | JackieSun |
| Asynchronous load Impress | I'm fix the bugs of Asynchronous Loading | zhangyuwei |
| Store refactoring
Buffered file I/O |
Works on module 'store' refactorings for better startup performance; milestone 1 of these refactorings is in CWS mhu17, now nominated for integration. Also, I'm evaluating extensions to osl/file for buffered file I/O, similar to what Tora suggested. Maybe I can propose something next week. | mhu |
| I/O issue | I have no accomplishment on the task of performance improvements this week . | tora-japan |
| Test load/save time with vtune | Using a hash_map as basis of SfxItemPropertyMap/Set doesn't help for load/save performance. Alphabetical sorting of property names in xmloff in the past - which was required by the previous implementation - helped to prevent unnecessary string comparisons. I didn't yet compare the load/save times in Calc and Impress as it takes a lot of time to get data out of vtune. | os_ooo |
| Wiki structure and project logo | I am currently working on the wiki structure and on a project logo. I have added a frame on the top right of any performance related wiki page to give the project a visual identity and offer a project navigation to the user. The logo will be shown as soon as it becomes available. | FrankL |
URIs mentioned
Calc/Broadcasters (erAck created)
Ternary search trees (mhu mentioned)
Boost/spirit/symbols (erAck mentioned)
Svn/performance/trunk (liheng mentioned)
Log of meeting
| (16:32:09)liheng: The agenda of IRC Mar. 13th |
| (16:32:09)liheng: 1.Update overall status |
| (16:32:09)liheng: 2.Discuss how to organize the source tree in performance module |
| (16:32:09)liheng: * Move all tooling code into performance module |
| (16:34:19)liheng: I just finished the first version of test system, and will create CWS and update it next week... |
| (16:34:37)liheng: Some detail document will post on WIKI page |
| (16:35:10)erAck: I started investigating Calc's area broadcasters, see http://wiki.services.openoffice.org/wiki/Calc/Performance/Refactoring_Area_Broadcasters, but got interrupted by 3.1 blocker issues. |
| (16:37:28)sb: My status: Got substantial improvements of instructions burned in runtime loader on Linux through use of -Bsymbolic-functions/-Bsymbolic (issue 85679) together with latest GCC/binutils; now looking at whether those changes are indeed correct (esp. -Bsymbolic, which can cause different bindings be used for one data symbol across libraries). |
| (16:38:08)cd_oo: I want to pack the code which is needed during start up to sequential pages. This should significantly reduce file I/O and help start up performance. I want to create a little helper library which traces what symbols are used during startup. This file can be used with the help of another tool to find the mangled names in the map file. The mangled names must to be put into an order file which... |
| (16:38:10)cd_oo: ...can be used by the Microsoft linker to reorder the code. |
| (16:39:10)yugq: I'm trying to finish an incremental storage model for XML files. |
| (16:39:48)JackieSun: I have submitted "downscaledimageshow " in CWS. Which can increase the drawing speed by using the downscaled bitmap of high-resolution images only to draw or "slide show" in the screen |
| (16:39:49)zhangyuwei: I'm fix the bugs of Asynchronous Loading |
| (16:41:54)***mhu works on module 'store' refactorings for better startup performance; milestone 1 of these refactorings is in CWS mhu17, now nominated for integration. Also, I'm evaluating extensions to osl/file for buffered file I/O, similar to what Tora suggested. Maybe I can propose something next week. |
| (16:43:12)liheng: Okay, I also arrange a report of comparison between several officesuite products,it will post in WIKI in two weeks |
| (16:44:06)tora-japan: My status: I have no accomplishment on the task of performance improvements this week since I have been working on some macro functions such as StrConv(), MidB(), LenB(), the Excel's legacy macro functions. The functions have been successfully and experimentally implemented. |
| (16:45:22)os_ooo: My status: Using a hash_map as basis of SfxItemPropertyMap/Set doesn't help for load/save performance. Alphabetical sorting of property names in xmloff in the past - which was required by the previous implementation - helped to prevent unnecessary string comparisons. I didn't yet compare the load/save times in Calc and Impress as it takes a lot of time to get data out of vtune. |
| (16:46:53)erAck: os_ooo: maybe a perfect hashmap would help? |
| (16:47:21)liheng: os_ooo:We can help you to get the data in CH2000, you can tell how/where to do them. |
| (16:50:14)liheng: os_ooo:Do you need someone to help you get some compared data? |
| (16:50:32)liheng: os_ooo: Do you need someone to help you get some compared data? |
| (16:53:07)liheng: Okay, we go to next topic, How to organize the source tree in performance module. |
| (16:53:12)FrankL: My status: I am currently working on the wiki structure and on a project logo. I have added a frame on the top right of any performance related wiki page to give the project a visual identity and offer a project navigation to the user. The logo will be shown as soon as it becomes available. |
| (16:53:45)FrankL: Do plan to have something like a monthly status presentation? We make this in project Renaissance and this allows us to promote the outcome of the project. This is also something we can show the press to get some more visibility for the project. |
| (16:54:14)erAck: FrankL: present where to whom? |
| (16:54:49)FrankL: Virtually |
| (16:55:08)FrankL: Today it not that easy to get an overview about the current status of the project. |
| (16:55:42)mhu: os_ooo: your comment about "alphabetical sorting" reminded me of "ternary search trees" by Robert Sedgewick; search google/wikipedia for "ternary tree" or see http://www.cs.princeton.edu/~rs/strings/ |
| (16:56:22)os_ooo: The code changes are in cws os128. So everybody who wants to create data can do that. At the moment I have only a few documents for Calc, Impress and Draw that I can use. |
| (16:58:27)liheng: os_ooo:Can you send the documents to me, email: liheng@redoffice.com |
| (17:00:05)liheng: FrankL:Do you want to do something like a status table of CWSes and works? |
| (17:00:16)liheng: : Do you want to do something like a status table of CWSes and works? |
| (17:00:18)os_ooo: liheng: You will get them in a minute. |
| (17:00:41)liheng: os_ooo: Thank you! |
| (17:01:00)FrankL: liheng: I think more like a status report for non technical people. |
| (17:02:06)liheng: FrankL: I see, it very important for UX test case definition. |
| (17:03:35)FrankL: liheng: I think it is important for all of us to have an overview about the improvements. Then we would have something were we can point interested people to. |
| (17:04:20)liheng: FrankL: Yes, and we can show some value about them:) |
| (17:04:57)liheng: 2nd topic is "How to organize the source tree in performance module". |
| (17:05:52)liheng: I think at first we can move all tooling code about performance to performance module, like RTL_LOG |
| (17:07:42)mhu: liheng: with 'performance module' you mean a (top level) directory 'performance' in OOo SVN repository, right ? |
| (17:08:04)liheng: Yes, |
| (17:08:18)mhu: okay |
| (17:09:20)mhu: ... I currently dont know of a better place; can we ask someone else, maybe Martin H. or Heiner R. ? should we ask at all ? |
| (17:10:18)liheng: Matthias: We have a repository of performance |
| (17:11:09)mhu: hmm, an own repository or with OOo svn repository ? |
| (17:11:14)liheng: it's own repository, which is not part of the other OOo repository. |
| (17:11:40)mhu: ahh, then all is fine, and I simply missed something ... |
| (17:12:17)liangjun: +1 |
| (17:12:49)***mhu asks himself what the repository url might be, or where it can be found ... |
| (17:14:17)liheng: http://performance.openoffice.org/svn/performance/trunk |
| (17:15:00)erAck: os_ooo: boost's spirit library's class symbols uses a ternary search tree, see http://www.boost.org/doc/libs/1_38_0/libs/spirit/classic/doc/symbols.html |
| (17:17:09)erAck: liheng: that's not the OOo SVN repository, plus it needs an account and password?!? |
| (17:18:59)liheng: erAck:I use my account in OOo |
| (17:19:49)sb: What is the reason to have such an additional repository? |
| (17:20:07)liheng: erAck:You can check page http://performance.openoffice.org/source/browse/performance/ |
| (17:20:14)***erAck tried OOo account and works. |
| (17:21:04)erAck: liheng: but I think we should use svn.services.oo.o instead. |
| (17:21:43)liheng: sb: All code of tools are only for benchmark or test OOo. |
| (17:22:21)erAck: it doesn't have to be the .../ooo repository there, as the code is not needed to build OOo, but .../performance or some such may do. |
| (17:22:53)sb: liheng: but why not use something like svn.services.openoffice.org/ooo/contrib/performance then? |
| (17:23:49)erAck: sb: I think it does not need to be in the main repository, no need to check it out just for building OOo. |
| (17:24:12)***mhu also tried OOo account and works, too. |
| (17:25:30)liheng: sb: I will confirm them with Dieter :) |
| (17:25:39)yugq: I think we can use that repository to store other files except the code. Such as some test report or related files. |
| (17:26:10)erAck: whoa, "Powered by Subversion version 1.4.3" ... a version we wouldn't even touch with a pair of pliers ;-) |
| (17:26:14)sb: erAck: that's why I wrote "/contrib/" |
| (17:26:15)yugq: I can see mt have performance-startup-2005 odt on it now. |
| (17:26:55)mhu: maybe Malte can tell some details when he is back from vacation next week |
| (17:27:00)erAck: sb: ah, ok, same repository but different branch; yes, might be best. |
| (17:27:39)liheng: Okay,that all :) |
| (17:27:39)yugq: mhu: yes! It seems Malte know this repository well:) |
| (17:27:54)mhu: yugq: :-) |
| (17:28:08)yugq: LiHeng: I will make up and publish the MM of this meeting. |
| (17:28:26)mhu: ...and yes, maybe a contrib branch maybe the method of choice |
