Performance/Meetings/2008 07
Performance/Meetings/2008/07/31
Meeting Minutes
IRC Meeting of Sun Microsystems (StarOffice) with RedFlag2000 and the Community ??
Performance Project
Date: 2008/07/31
Time: 15:38– 17:12
Meeting No.:
Agenda:
(15:58:35) liheng: hi xiuzhi
(15:59:11) xiuzhi: liheng:hi afternoon
(16:00:16) xiuzhi: I suggested that we can have bi-weekly meeting if we have no more issues to discuss.
(16:02:01) liheng: xiuzhi:OK, i am agree :)
(16:03:04) mhu: Hi all
(16:03:08) xiuzhi: liheng::)
(16:03:22) xiuzhi: mhu: moin
(16:03:24) liheng: hi: matthias
(16:03:28) liangjun: mhu: hi:)
(16:03:34) kuangliang: hi,mhu
(16:03:44) mhu: moin :-)
(16:05:20) yugq: hi
(16:05:20) liheng: mhu:xiuzhi suggested that we can have bi-weekly meeting in the future , what is your opinion?
(16:05:21) mhu: Okay, I have read the agenda; you have a new team member? will he join our meeting also?
(16:06:19) liheng: Guo yongshi, jion this meeting next time, today, he have another meeting of our company
(16:06:19) liheng: :)
(16:06:20) mhu: liheng: I think, having a meeting every two weeks only is fine when we are all busy working.
(16:07:25) mhu: (also: I will be out on vacation for the next two weeks; so next meeting would be in 3 weeks)
(16:07:51) liheng: I can have a meeting every week, yugq , kuangliang, liangjun also, but xiuzhi maybe so busy :)
(16:08:23) xiuzhi: liheng: not true. I means if we have not too many issues to discuss
(16:08:36) mhu: When we still have so much to discuss, as we did in the past, I think it is okay to have a meeting every week.
(16:09:00) xiuzhi: liheng: weekly meeting is ok for me
(16:09:09) mhu: xiuzhi: yes, that is what I also mean.
(16:09:53) liheng: Okay, we keep weekly meeting
(16:10:43) liheng: xiuzhi: Okay, and the proposal of paper "Improving and Maintaining Performance of OO in Long-term Development" has been passed selection,
(16:11:14) xiuzhi: xiuzhi: great
(16:11:29) xiuzhi: liheng: great
(16:11:36) mhu: yes, I think that is great also
(16:11:54) xiuzhi: liheng: and OO -> OOo
(16:12:00) liheng: we need more discussion for tools and analysis of performance of OOo
(16:12:05) liheng: Yes, OOo:)
(16:12:27) liheng: Peter has mentioned to me:)
(16:13:59) mhu: okay, discusssion of benchmarking tools... do have any ideas?
(16:14:13) mhu: s/do have/do you have/
(16:15:00) liheng: mhu:In your opinion, what are urgent improvement of Tuning and Benchmark tools?
(16:16:33) mhu: well, I think we once discussed that first of all we need a set of "standard and reproduceable" measurements / tests.
(16:17:15) mhu: ...say: how long it takes to load (and display?) a particular reference document.
(16:18:30) mhu: ...this would be the macroscopic / user relevant measurement.
(16:18:50) liheng: Yes, for a long time most raw-data will be comparability and explicit
(16:20:07) liheng: please explain "how long it takes to load (and display?) a particular reference document." :(
(16:20:29) mhu: what I am thinking about is something like: Y=f(x1, x2, ...,xN); where Y is the user visible / macroscopic value, and the x's are the variables that cause it
(16:20:58) mhu: liheng: okay, I will explain...
(16:22:31) mhu: I said: "...set of reproduceable measurements"; one measurement out of that set of measurements could be a measurement of the time it takes to load one of the (yet to be defined) reference documents.
(16:22:42) mhu: does this make it clearer?
(16:24:10) liheng: you mean the individual measurements in specified parts?
(16:24:36) liheng: for load a reference documents?
(16:25:01) mhu: not yet, measurements of specified parts may be measurements of those x1, ...,xN's
(16:25:46) mhu: in this example, I meant a simple number, say: 5 seconds
(16:25:51) yugq: mhu: about Y=f(x1, x2, ...,xN), may I understand that like: Y=speed, x1=startup speed, x2=load/save document speed, x3=menu shown speed ... ?
(16:25:59) liheng: If we have enough manpower we can not only define the f(x1.......xN) but also paint a big picture for key paths and hotspots
(16:27:48) mhu: yugq: well, almost; Y would the total load time, say 5 seconds, and x1 may be xml parsing speed, x2 memory allocation speed, x3 locking speed, and so on; f() would basically tell how these factors contribute to the end result Y.
(16:28:09) mhu: liheng: yes, that is the direction that I am thinking :-)
(16:30:23) liheng: after them, we must define effectual and comparability benchmarks for all node (x1.....xN)
(16:30:28) liheng: right?
(16:31:37) mhu: yes, probably; but hopefully not for so many, but only the most important ones; that is, for those that contribute to many Y's (like xml parsing, memory alloc, ...).
(16:34:18) liheng: For a long time, I think for create a set of tools to automatically analyze data of benchmark and compare them between several versions
(16:35:12) liheng: mhu:Network seems slowly:(
(16:35:31) mhu: so, yugq already mentioned: startup speed, load/save speed, menu open speed; that would be the different macroscopically observable Y's in my world; they may all depend on different x's, but may also depend on a set of same x's.
(16:36:18) mhu: liheng: I can wait, dont worry too much about your network
(16:37:26) liheng: mhu:For a long time, I think of create a set of tools to automatically analyze data of benchmark and compare them between several versions
(16:37:35) mhu: ...that set of same x's (contributing to many Y's) would be the ones that I would create individual benchmarks for, to start with.
(16:38:07) mhu: oh yes, I forgot to answer that one, stay tuned...
(16:38:25) mhu: ..set of tools to automatically analyse...
(16:39:35) mhu: ...I think of that can be done with a "T-Test" called statistical method (testing statistically difference between two datasets), do you mean someting like this?
(16:41:37) mhu: sorry for all this math; I have had a course in "six sigma methodology", and in my previous life I have been a physicist :-)
(16:41:44) liheng: Yes, most of, and all data can be traced in difference versions
(16:42:02) mhu: yes
(16:44:14) liheng: Yes, now we need a start point to try which tools can do that
(16:44:14) liheng: :)
(16:45:24) mhu: well, tooling would be two parts: one part to collect the data (maybe that is what you have from Malte?), the other part to analyze it.
(16:46:02) liheng: Three parts in my mind, testcase, benchmarks, method of report
(16:46:52) mhu: if you can produce a set of measurements, say: measure a documents load time 10 times in a row, for 2 versions of OOo, than I can provide you with the code to do a T-Test of that datasets.
(16:47:58) mhu: yes, you right; but at this high level, I think report can initially be "the diff is 5% and it is statistically significant" or so.
(16:48:17) liheng: i am not clear of "than I can provide you with the code to do a T-Test of that datasets."
(16:48:20) liheng: :)
(16:48:52) mhu: I have written a small set of Java methods some years ago; I can give you the code.
(16:49:27) mhu: meaning you dont need to invent the math yourself
(16:50:05) liheng: oh, yes thank you,
(16:50:22) mhu: you are welcome
(16:50:46) liheng: i have a mistake for 'code' and 'datasets'
(16:50:47) liheng: :)
(16:51:37) mhu: TTest.java = code, {5,4,4,5,...7,5} = dataset
(16:51:42) mhu: :-) :-)
(16:52:39) liheng: i mean the report must can lead more developers to notice some problem of performance
(16:53:51) liheng: one target is tuning for every version ,and another is refactoring in the future.
(16:54:01) mhu: well, yes. on the other hand, the numbers of course must be able to speak for them selves; than we can think of how to present them; but yes, presentation is important also.
(16:55:14) mhu: yes, we agreed on that already: tuning for a version is short-term only; the real goal is indeed long-term (refactoring) improvements.
(16:57:37) liheng: will we discuss a example in OOo performance (maybe common hotspot), to define a set of benchmark
(16:58:51) mhu: back to tooling: maybe, for reporting we can make a small extension / addin for Calc to do the T-Test between two datasets; then we would have an initial presentation also ? just an idea.
(16:59:28) mhu: sorry, I needed to finish that idea first; what was your question wrt "example..."
(17:00:17) liheng: a sample for a hotspot or a key path
(17:01:17) mhu: and you want to discuss what about that sample? sorry, I have not yet understood
(17:01:47) xiuzhi: mhu: study a case in formance
(17:02:02) xiuzhi: liheng: does that your meaning?
(17:02:24) mhu: oh yes, we should study a case to learn what is going on and how to analyze and present it, yes.
(17:03:05) liheng: xiuzhi:thank you, i just try to find clear words:)
(17:03:33) mhu: maybe, for practical purposes, we start with a configmgr example?
(17:04:31) liheng: mhu:Of course!!!;) back to tooling: I want to setup a server for datasets:)
(17:04:34) mhu: that could be: Y=configmgr contribution to startup time, and we could try to find the major x's (hotspots) contributing to that Y by some f().
(17:05:23) mhu: yes, that server to store the measurement results was a good idea.
(17:06:48) mhu: ...we could also so a "synthetic" configmgr benchmark, not use startup time of entire office; I think yugq already wrote some "unit test" or "benchmark" ?
(17:07:26) yugq: mhu: yes
(17:07:46) liheng: so, it's good for persisting all data of teat
(17:07:57) liheng: sorry teat/test
(17:08:37) mhu: yes, if we want to learn something from the data, and other developers shall learn something of it, then it is good to store that data.
(17:09:07) liheng: okay, :)
(17:09:30) mhu: okay
(17:10:08) liheng: mhu:that all for me.
(17:10:27) mhu: yes, that's all from me also
(17:10:33) liheng: I wish you have a nice holiday
(17:10:41) mhu: thank you
(17:11:10) liheng: you are welcome, and see you at Aug 21th
(17:11:36) mhu: see you in three weeks, august 21.
(17:11:45) mhu: bye all
(17:11:47) xiuzhi: see you
(17:11:51) liheng: bye:)
(17:12:04) kuangliang: bye
(17:12:27) liangjun: bye
Performance/Meetings/2008/07/03
Meeting Minutes
IRC Meeting of Sun Microsystems (StarOffice) with RedFlag2000
Performance Project
Date: 2008/07/03
Time: 15:00– 15:58
Meeting No.:
Agenda:
(15:00:42) LiHeng: hi,xiuzhi, how about our ID?
(15:01:46) xiuzhi: Liheng: matthias is not in
(15:02:02) xiuzhi: he will be here in several minutes
(15:02:15) LiHeng: okay,
(15:02:38) xiuzhi: 9:00 is a little early to him ,maybe
(15:02:46) LiHeng: xiuzhi:thank you for your translation
(15:03:30) LiHeng: xiuzhi:how about our ID? we have no ID like xx@openoffice.org
(15:03:33) xiuzhi: LiHeng:my pleasure
(15:04:08) LiHeng: where did you apply it ?
(15:04:27) xiuzhi: LiHeng: that is liheng@openoffice.org and yugq@openoffice.org. you can log in openoffice.org using that account
(15:04:51) yugq: xiuzhi, hi. hi all.
(15:05:05) LiHeng: ahh,:-(, we didn't know it
(15:05:06) xiuzhi: yugq:hi ,afternoon
(15:05:33) LiHeng: xiuzhi:you are in morning:)
(15:05:47) xiuzhi: Please see :http://www.openoffice.org/servlets/Login?cookieCheck=on
(15:06:23) yugq: xiuzhi is in Germany?
(15:06:32) LiHeng: liangjun:what is your ID on OpenOffice.org?
(15:06:49) liangjun: :)
(15:07:15) LiHeng: yugq:yes he went to harnbag last week
(15:07:37) liangjun: zengliangjun@RedOffice.com.cn
(15:07:39) yugq: wow, :-D
(15:07:40) xiuzhi: LiHeng: and Matthias want to know the meaning of "same platform" and "does the reconstruction mean a long-term plan"
(15:07:59) xiuzhi: yugq:sure
(15:08:23) xiuzhi: LiHeng: Please see my yesterday's email
(15:08:41) xiuzhi: LiHeng: that is one of the topics today
(15:09:08) yugq: xiuzhi, it seems I can't use yugq@openoffice.org to log in openoffice.org.
(15:09:41) xiuzhi: yugq:really? I have tell Martin this account. Please try again
(15:09:46) liangjun: your mail has been changed
(15:09:48) xiuzhi: or regist again
(15:10:01) liangjun: yugq:your mail has been changed
(15:10:35) yugq: I'll try it again.
(15:10:43) LiHeng: I have already read your translation. "same platform" means we want to build a lab for performance on internet, all of OO developers can submit his version and CWS, and that get the report based same platform
(15:11:12) LiHeng: xiuzhi:can you understand my explaination
(15:11:59) xiuzhi: LiHeng: not really
(15:12:15) LiHeng: not really?
(15:12:47) xiuzhi: same platform: the same OOo milestone?
(15:13:02) xiuzhi: that is my and Mhu's understanding
(15:13:35) xiuzhi: mhu is in now. we can start the meeting soon
(15:14:54) LiHeng: :)
(15:15:49) xiuzhi: or same platform: the same OOo Master workspace milestone., the performance evluation of every CWS should based on one same OOo milestone.
(15:16:05) LiHeng: mhu:hi, good morning
(15:16:08) mhu: Hi all, sorry for being so late (traffic :-( )
(15:16:16) yugq: mhu, hi!
(15:16:23) liangjun: mhu:hi:)
(15:16:31) LiHeng: mhu:so long time to see you again
(15:16:43) mhu: Hi LiHeng, yugq, liangjun
(15:16:54) mhu: yes, 2 weeks?
(15:17:04) LiHeng: 2 :)
(15:18:00) LiHeng: mhu:we just talk about "same platform";)
(15:19:01) mhu: yes, xiuzhi and I had some discussion yesterday; some words seem to be unclear ti us.
(15:19:17) mhu: s/ti us/to us/
(15:20:10) xiuzhi: yugq: try" yugh" to log on http://www.openoffice.org/servlets/Login?cookieCheck=on
(15:20:24) yugq: I think LiHeng means the "same platform" mostly like "unitive platform".
(15:20:30) yugq: xiuzhi, OK.
(15:20:30) LiHeng: In my opipion,to measure performance of OO, we should run all testing cases on same machine, and same state within
(15:21:25) mhu: Ah, I see. "same platform" = "same machine + state"
(15:21:43) xiuzhi: LiHeng: same hardware and software environment/context?
(15:22:34) mhu: this sounds like a "virtual machine" like "VirtualBox".
(15:22:35) LiHeng: we want build a lab to provide it on internet for OO
(15:23:07) LiHeng: mhu: :) ,yes, cache snapshot and some state we need
(15:23:28) yugq: xiuzhi, what is my password?
(15:23:36) mhu: yes, that's what I mean :-)
(15:23:54) xiuzhi: yugq: that is yours, I donot know
(15:24:50) LiHeng: xiuzhi:mhu and I think the "same platform=same machin + state"
(15:25:15) xiuzhi: LiHeng:ok,
(15:26:13) yugq: xiuzhi, are you sure my id is "yugh"? I can't login:-(
(15:26:57) mhu: yugq: can you also try "yugq@openoffice.org" ?
(15:27:10) xiuzhi: yugq: that is your account. I do not know . if you cann't log in, please regist again
(15:27:44) LiHeng: mhu:and "the reconstruction mean a long-term plan" means refactor
(15:29:23) yugq: xiuzhi, mhu, I know now. I thought you assigned a new id for me. Sure, I can login with "yugq", which is my id always used.:-D
(15:29:52) LiHeng: i means refactor need a lot of work in following version and make OO framework well and delicate step by step
(15:31:03) mhu: LiHeng: Ah, I see. "refactor" to me sounds better than "reconstruction" (where I often think "chart2").
(15:32:06) mhu: yes, refactoring can be a lot of work. But I also do not see much alternative. Many code cannot stay (for long time) as it is now. It needs refactoring.
(15:33:34) LiHeng: Yes, i am agree, but in another aspect, refactor can make OO delicate but fast
(15:34:29) LiHeng: so for performance we must confirm every refactoring planing make OO faster (at least not slower) ;)
(15:35:04) mhu: sure, we want to use refactoring to make it faster (but also "better" in terms of functionality, quality, maintainability,. ...)
(15:35:36) mhu: yes, it shall not get slower, of course :-)
(15:37:13) LiHeng: so, i think refactor of OO need discuss in a large scale to make hotspot static
(15:38:23) mhu: yes, I agree.
(15:40:17) LiHeng: lots of work maybe don't effect to performance directly, but they can make hotspot locating easily and stably
(15:41:36) LiHeng: and that ,all people can find problem of performance and deal it individual
(15:42:33) mhu: yes, that may be. Do you have some ideas how to achieve that?
(15:44:34) LiHeng: some, so i am experimenting for them
(15:46:27) mhu: okay, maybe we can discuss these idea when you're experiments have shown some results.
(15:47:17) LiHeng: so, at least we must improve performance tools of testing for CWS, comparability is important things for it
(15:48:17) mhu: yes, that is true. these tests need to function properly.
(15:49:01) LiHeng: that comparabiliy is not only for a CWS but also can do comparing between servsal
(15:49:46) mhu: ...between several... what ? releases ? like 2.3.0, 2.3.1, ...
(15:50:19) LiHeng: and Performance Group mark some statistics for report some problem would be existing
(15:51:13) LiHeng: mhu:may be, but several CWS for 2.3.0 or a major version
(15:51:41) mhu: yes, problem detection and reporting is indeed important.
(15:52:29) mhu: ah, okay. ...between several... can also be several CWS, yes.
(15:55:23) LiHeng: yes, next week we discuss some idea of tools and feasbility of lab on internet
(15:57:02) mhu: okay, fine.
(15:57:18) LiHeng: i will send some details of them to you (maybe yugq will send it to you, i am gloomy for mailbox recently )
(15:57:37) mhu: sorry, but xiuzhi and I are now expected to go to the OOo ESC meeting here in Hamburg.
(15:57:51) mhu: LiHeng: yes, please send it via email.
(15:57:57) LiHeng: okay , that all
(15:58:33) mhu: okay, then see you next week. have a nice afternoon / evening.
(15:58:43) mhu: bye all
(15:58:44) LiHeng: thank you,:)
(15:58:45) LiHeng: bye
(15:58:50) yugq: mhu, bye.
