Education ClassRoom/Previous Logs/MercurialMigration

From Apache OpenOffice Wiki
Jump to: navigation, search

Soon on your screens :-)

[16:59] <ericb2> blauwal_: hello. thanks for accepting the presentation

[16:59] <ericb2> blauwal_: you can start :)

[16:59] <blauwal_> OK then let's go.

[17:00] <b_michaelsen> ericb2: huh? wiki root? anyway - later. blauwal_'s show ;)

[17:00] <blauwal_> My name is Jens-Heiner Rechtien and I'm with the OpenOfiice.org release engineering team

[17:01] <blauwal_> Lately we switched our Source Code Management (SCM) system from SVN to Mercurial and I'm going to tell a little about that

[17:02] <blauwal_> First I should say a bit about the WHY we did this

[17:03] <blauwal_> About a year ago we switched to SVN as an intermediate step and promised to re-evaluate the SCM situation again within a year.

[17:05] <blauwal_> This is because many developer wanted a distributed SCM (DSCM) system, but we found the than existing ones lacking.

[17:05] <blauwal_> All the three "big" DSCM evolved a lot over the last year.

[17:06] <blauwal_> Git, Bazaar and Mercurial (hg)

[17:07] <blauwal_> And early this year we found that Mercurial and Git are able to host the OOo source code and Bazaar nearly so.

[17:07] <blauwal_> Why not just stay with SVN?

[17:09] <blauwal_> Well, SVN is not a distributed SCM, which is what many developers prefer nowadays. But then SVN also disappointed in several areas.

[17:09] <blauwal_> The new merge tracking inplemented in SVN-1.5 proved to be very buggy (at least in the beginning) and quite slow.

[17:11] <blauwal_> Also the implementation of file renames is quite limited, it's possible to view the history of renamed files, but it's not possible to forward changes made to file under the old name to the new name.

[17:12] <blauwal_> In the end SVN stopped our suffering from CVS but was not the tool we wanted to stay with for a long time.

[17:13] <blauwal_> So let's say a few words about DSCM before we go into the details of Mercurial.

[17:14] <blauwal_> All DSCM keep the complete history in a storage on the local disk.

[17:14] <blauwal_> That's why they are called "distributed", all copies of the repositories have the complete history.

[17:15] <blauwal_> There are a number of consequences which follow immediately.

[17:15] <blauwal_> a) Revisionnumbers can't be numerical anymore.

[17:16] <blauwal_> Consider two repositories with 100 changes in there. Developer A add change in his copy, developer B another change in her copy.

[17:17] <IZBot> News from cws: ooo32gsl03: nominated

[17:17] <blauwal_> Both would have a local revision number 101 but the changes are not identical.

[17:18] <blauwal_> So all DSCM use some kind of hash to identify changes, or more specifically changesets. For mercurial it's SHA1 hashes which are used.

[17:20] <blauwal_> Mercurial allows still local numerical revisions numbers, but they are valid for the local repository only. For exchange purposes always use the hashes.

[17:21] <blauwal_> b) All the history must be copied over and over

[17:21] <blauwal_> This is no problem with small repositories, but the OOo one is not small at all

[17:22] <blauwal_> So any setup which uses a DSCM for OOo source code hosting has to be very careful about not requiring developers to copy the same old changesets over the net again and again.

[17:22] <blauwal_> I'll come to that later.

[17:23] <blauwal_> c) Almost all operations on the repository are done locally. This is just great for speed. You'll notice it when you do merge work for instance.

[17:24] <blauwal_> Actually all DSCMs are great with regard to merges, this is because this operation is at the heart of the workflow with DSCMs.

[17:25] <blauwal_> d) You can do most of your work without Net access - which is great if you are on a flaky line

[17:26] <blauwal_> e) There is no central repository.

[17:27] <blauwal_> This means any DSCM setup requires a number of conventions. The nice thing about this that you can tailor the conventions to your needs

[17:28] <blauwal_> So how does working with Mercurial looks like? Let's explore this with an example. You can find the whole documentation in the OOo wiki.

[17:29] <ericb2> blauwal_: got this page : http://wiki.services.openoffice.org/wiki/OOo_and_Mercurial

[17:29] <blauwal_> First you'll have to create a clone of the repository which is provided by OOo release engineering. Let's call this the master repository, its name is DEV300

[17:29] <blauwal_> ericb2: yes, that's the entry point

[17:30] <blauwal_> If you have time and band-width just do:

[17:30] <blauwal_> hg clone -U http://hg.services.openoffice.org/DEV300 my_local_DEV300

[17:31] <blauwal_> There are faster ways to do this, please have a look at the documentation.

[17:32] <blauwal_> Now you have got a treeless copy of the DEV300 repository on your disk. Update it regularly with:

[17:32] <blauwal_> cd my_local_DEV300

[17:32] <blauwal_> hg pull

[17:32] <blauwal_> You could implement a cron-job for that.

[17:33] <blauwal_> Now, you'll need a working copy to actually do something with it.

[17:33] <blauwal_> hg clone -r DEV300_m66 my_local_DEV300 my_working_copy

[17:34] <blauwal_> This will create a working repository - with source tree - which contains everything up to DEV300_m66 or whatever milestone you specify.

[17:35] <blauwal_> cd my_working_copy

[17:35] <blauwal_> ... hack away ...

[17:35] <blauwal_> hg commit -m"#4711# my changes to fix issues 4711"

[17:36] <blauwal_> The nice thing is that you don't need to be a domain developer, or even be known to the OOo community to do this.

[17:36] <blauwal_> Everything just happens on the local disl

[17:36] * Matt1 has quit (Nick collision from services.)

[17:36] <blauwal_> s/disl/disk/

[17:37] * matoosss is now known as Matt1

[17:38] <blauwal_> Say, you have worked a while, committed a number of changes and now want to merge new changes from the master repository into your working tree

[17:39] <blauwal_> You can do this by pulling the new changes from your (updated) copy "my_local_DEV300", or by pulling the changesets directly from http://hg.services.openoffice.org/DEV300

[17:39] <blauwal_> Let's assume we do the latter:

[17:39] <blauwal_> cd my_working_copy

[17:40] <blauwal_> hg pull http://hg.services.openoffice.org/DEV300

[17:40] <blauwal_> ... fetches new changes

[17:40] <blauwal_> hg merge

[17:40] <blauwal_> If you got conflicts, resolve them:

[17:40] <blauwal_> and mark them as resolved

[17:41] <blauwal_> hg resolve -m <resolved_file>

[17:41] <blauwal_> and finally commit the result of the merge

[17:41] <blauwal_> hg commit -m"merge with DEV300 m70"

[17:42] <blauwal_> Your working copy is now up-to-date with the latest changes.

[17:42] <blauwal_> Mercurial allows you to configure your merge-tool of choice.

[17:44] <blauwal_> One merge-tool which has a very good reputation is "kdiff3". Or you can just fall back to the style of auto-merging with conflict markers which are known to all CVS and SVN users. Your choice.

[17:45] <blauwal_> So, you got a fix for OOo and want to contribute it, how to do this?

[17:46] <blauwal_> You could export your change as patch via hg export and send it to the mailing lists for review.

[17:47] <blauwal_> If the patch is accepted it will be integrated with all the Meta-data (author name, date, commit message) intact.

[17:48] <blauwal_> If you got larger changes to contribute have a look at the "patchbomb" extension of Mercurial.

[17:49] <blauwal_> Domain developers can contribute directly to OOo without the detour via the mailing lists.

[17:51] <blauwal_> When a domain developer create a so called child workspace (CWS - that's the major vehicle for OOo development), a clone of the master repository with all the changes up to the "creation milestone" of the CWS is created on the OOo repository server

[17:52] <blauwal_> We call these repositories "outgoing repositories". It's the central collaboration point for developers.

[17:53] <blauwal_> Domain developer can "push" their changes to that server, RE will take the changes from there for integration.

[17:54] <blauwal_> OK, I think that's enough for a first overview. Questions?

[17:55] <ericb2> yes, got some. First, I'd like to say it was very clear, and you pointed the essential. I didn't know more than this the tool, and now, I think I got the concept. Thank you very much  !

[17:55] <ericb2> the first question coming to my mind : reading what you explained, I'm wondering about the QA : will it be made "transparently "?

[17:55] <ericb2> or is it the same process, and so on ?

[17:56] <blauwal_> The setup with regard to QA is practically unchanged. But the good and fast merge capabilities of Mercurial

[17:57] <blauwal_> allows us to do more than we do now.

[17:57] <blauwal_> Child workspaces will be much more up-to-date because it's so easy to merge new changes from the master into the CWS.

[17:58] <blauwal_> The result will be a better overall quality of long running child workspaces.

[17:59] <blauwal_> We plan to have automated merge runs when the status of a CWS is toggled from "ready for QA" to "nominated", this gives the developer time to fix conflicts etc.

[18:01] <blauwal_> In the long run I expect that build bots will automatically create the install sets for QA purposes, this eliminates a number of problems we have now, like uncommitted changes etc

[18:01] <blauwal_> Does this answer your question?

[18:02] * Twarz (i=Tao44@37.12.103-84.rev.gaoland.net) has joined #education.openoffice.org

[18:02] <ericb2> blauwal_: completely, thanks :)

[18:03] <Rail> blauwal_: I've just noticed that hgweb is not in UTF-8...

[18:04] <blauwal_> I think, the best part of the switch to Mercurial for a casual or beginning OOo developer is the ease of making changes to the source code and still be current the change flow.

[18:04] <blauwal_> For domain developers it's the easy merging.

[18:05] <blauwal_> Rail: yup, known bug

[18:05] <Rail> k

[18:05] <blauwal_> The notification mails needs a better content-type maker as well :-)

[18:06] <blauwal_> will be fixed RSN

[18:06] <blauwal_> More questions?

[18:07] <Rail> we want more complicated examples :D

[18:07] <Rail> maybe next time

[18:07] * ericb2 would like to mention the archives of dev@tools.openoffice.org becasue they contain lot of information, about the mercurial migration, including extremely detailed and intersting discussions too.

[18:07] <ericb2> URL is : http://tools.openoffice.org/servlets/SummarizeList?listName=dev


[18:08] <blauwal_> more complicated examples? Fortunately it's not complicated at all for most developers. What do you have in mind?

[18:08] <Rail> usual CWS commands, for example

[18:09] <blauwal_> Rail: easy!

[18:09] <blauwal_> cws create --hg DEV300 mycws

[18:09] <blauwal_> This will create the CWS in EIS. You can do it directly in EIS as well.

[18:10] <blauwal_> Than just clone and hack away. You could use the "cws fetch" command to get the CWS, like

[18:10] <blauwal_> cws fetch -c mycws mycws

[18:10] <blauwal_> but that's just for convenience.

[18:10] <blauwal_> After updating your CWS to a new milestone do:

[18:11] <blauwal_> cws setcurrent -m DEV300_<new_milestone>

[18:11] <blauwal_> and that's it.

[18:11] <Rail> a piece of cake :)

[18:12] <blauwal_> Actually I plan to create a python plugin which will do away the need for the "cws"-tool

[18:12] <b_michaelsen> pssst: s/python/mercurial/

[18:12] <blauwal_> That is to say a python hg plugin

[18:12] <b_michaelsen> hrhr

[18:13] <Rail> ah

[18:13] <Rail> it would be great

[18:13] <Rail> mercurial's API is not so hard

[18:13] <blauwal_> Yep, the more difficult part is the EIS API :-)

[18:13] <Rail> hehe

[18:14] <b_michaelsen> Rail: http://wiki.services.openoffice.org/wiki/Mercurial/Cws <- this has pretty much every CWS-related that is still needed. As you see not much, as much is handled directly with hg.

[18:15] <blauwal_> Yep, good riddance to the non standard tools :-)

[18:16] <Rail> blauwal_: thanks a lot for the presentation

[18:17] <blauwal_> Be welcome

[18:17] <Rail> congratulations, we've finally switched to DSCM! :D

[18:17] <blauwal_> We took our time, but well, you know, in the end ...

[18:19] <blauwal_> More questions?

[18:19] <blauwal_> More questions?

[18:21] <blauwal_> ericb2: I think we are through with the topics today.

[18:21] <blauwal_> Thank you very much for your attention.

[18:21] <ericb2> Indeed. Looks like there are no more questions. That's normal : you clarified everything, and we have material for work and practice for a while :-) Thank you very very much for your time. And BRAVO for your work !

[18:21] b_michaelsen blauwal_

[18:22] <ericb2> blauwal_: you did it :)

Personal tools