Forum OpenACS Q&A: How do you separate project customisations from improvements?

Hi all,

I can think of a number of alternative ways of doing this in CVS - I
wondered if there was any experience to share here?

Suppose you're working on a project for a client - you're going to
make a bunch of customisations. Some will be specific to the client,
and some will be things that might be general improvements or bug
fixes. What's the best way to use CVS to manage the difference?

One suggestion is to keep two checked out versions of the tree - two
different branches. One is the project branch and one the main trunk.
Customisations are made in the project branch and checked back in -
other improvements/bug fixes are made in the other branch, checked
in, and then merged with the project branch (and possibly other
projects).

Sounds clunky but I can see some difficulties with alternatives.
Suppose you are working on a file, making customisations, and
checking them in to the project branch. Then you notice a bug that
should be fixed from the original code. Can you check that bug fix
back into the main branch without also checking in the customisation
code? Possibly with some fancy use of CVS versions? But that would be
pretty clunky too.

I think aD used to do this with some shell scripts they wrote to do
all the clever CVS stuff and give a reasonable interface to the
programmer. Any experiences to share?

thanks

steve

Well, the projects I was on at aD had a very simple procedure for submitting bug fixes back to the core - we didn't. Now, my projects all happened to be late rather than early adopters of new ACS toolkits, so that may not be representative. But it did seem to be the standard procedure, unfortunately...

So, when it comes to how best to make changes so as to make it easy to contribute fixes back to OpenACS core, I'm offering my own speculations here, not any true hands-on experience.

And anyway, aD did not do this with any "clever shell scripts". Yes, there were some very simple shell scripts floating around for different projects, and there may have been some others I wasn't aware of, but there's nothing here you can just wave a shell-script wand at to make it go away.

FYI, you can find both some of aD CVS shell scripts and a bunch of info on CVS my personal web site. I wrote that CVS Policy and Howto document because I found it useful; I sure wouldn't mind hearing if other folks find it useful, too. :)

Changes can logically be thought of as two different types: A customization just to my codebase, or a change that I think should be rolled back into OpenACS - I will call these "customizations" and "OpenACS fixes". If you're careful to make changes of only one type of change in a single commit, you should always be able to go back in later and sort things out, in order to prepare a patch against the core OpenACS.

What you really, really want to avoid, is where you commit something extra by mistake, or otherwise mix different changes into a single commit. Because then, in order to fix bugs or prepare a patch, you have to figure stuff out at a finer grain than the CVS commits, and that is a royal PITA. You want to avoid this. So the standard CVS religion applies - always use cvs diff on your files before commiting, just to be sure what you're really committing is what you think you're commiting, etc. etc.

CVS is kind of a blunt instrument in some ways for doing distributed development, and some folks (Larry McVoy of BitMover) make fairly convincing arguments that their tools are technically superior. But CVS is what we have, and what OpenACS uses, and it can certainly do the job, so we'll all just have to learn to cope with it.

Note that if you've modified 5 files as part of a single logical change, you should always commit those 5 files with a single cvs commit command, and a single (descriptive) commit comment. This is because although CVS doesn't have genuine atomic commits, varius add-on tools like cvs2cl.pl will use the fact that the commit comments are identical to simular atomic commits when generating GNU-stle changelogs and the like. So do your commits as if CVS commits are atomic, even though they're not.

Steve, your two branches suggestion makes sense to me. This might be most useful if you are doing a lot of changes, both customizations and OpenACS bugfixes.

I think, ideally, one way or another you must have your own separate "OpenACS fixes" branch. I say "ideally", because if you haven't made many of your own changes you can clearly get away without it, but let's say you've customized your own codebase quite a lot for some reason. Now, you find an OpenACS bug and fix it - great, it works! Except, how can you really be sure your fix also works in a stock OpenACS? The answer is, you can't, not without testing it on a stock OpenACS. So you need to set up a stock OpenACS for testing.

Merging aross CVS branches is definitely better than manually applying patches, so you certainly want to use branches. (If you did a proper CVS vendor import of the OpenACS code when starting your own project, this should be easy to do.)

So either you make changes to your own customized branch and then merge them into your "stock plus my fixes" branch, or you make changes to your "stock plus my fixes" branch, and merge them into your own customized branch. I don't see any particular reason to prefer one direction of merging over the other.

Clearly, if you are using a completely stock OpenACS and just adding your own packages, then maintaining a separte "OpenACS fixes" branch is a waste of time - you already have that. Thing is, everyone's going to have some customizatins, even if only to www/default-master.adp The more customizations you make, and the more OpenACS fixes you make which haven't made it back into the core, and then back out to you again in the form of a new OpenACS release, the more likely the separate "OpenACS fixes" branch is going to be worth having. I don't where that trade-off falls, though.

Now, on getting your OpenACS fixes back into OpenACS proper, I've still not contributed any code to OpenACS, so I can't say anything useful about that.

Your local "OpenACS fixes" branch is also the obvious place to frist merge in new releases of OpenACS, before you merge them into your customized codebase.

Merging aross CVS branches is definitely better than manually applying patches, so you certainly want to use branches. (If you did a proper CVS vendor import of the OpenACS code when starting your own project, this should be easy to do.)

OK, so we're just getting started. I checked out openacs-4 from the openacs repository and imported it into my repository with

cvs import myservice openacs openacs4-cvs-20011029

So what should I do to implement the two-branch vision discussed here? Let's say I want to make changes that I want rolled back into the toolkit. I'm going to change two files: packages/acs-subsite/www/admin/site-map/parameter-set.tcl and packages/acs-tcl/tcl/defs-procs.tcl. Do I make the changes and tag those files only? Do I checkout all of myservice and tag the whole thing with a tag that will be for changes that we want to get added to the toolkit, and then change those files and merge the changes back into my main trunk? That scenario looks complicated--you would be forever checking in, merging, then checking out so you could test on your trunk.

C.R., did you read Ron's article, and the other links I give at the top of my document? Ron's article is probably the best introduction to this stuff. But, here we're not talking about testing periodic Dev -> Staging -> Production releases, we're talking about creating two ongoing parallel lines of development, one on the trunk, one on a branch. So the mechanics are what Ron describes as the "two branch" method.

You always want to place branch tags across the whole project, never just a few individual files. If you want, you can tag only a few individual files in order to do merge operations across the two branches, but when creating the branch you'd better tag all the files.

One thing I'm foggy on, is how to best keep track of what changes you've already merged from one branch to the other, and which ones you haven't. (Incidentally, I say "branch" all the time when I mean "branch or trunk". The trunk is really just another branch.)

If you make a change on your OpenACS Fixes branch, then presumably you're always going to want to merge all those changes back to your Custom branch. So my gut tells me you probably just want to periodically tag everything on the OpenACS Fixes branch with a myproject-openacs-fixes-2001-11-01 tag or something like that, and merge it all into your Custom branch.

Where it gets hairy is merging changes from Custom to OpenACS Fixes, because presumably you only want to merge some of those changes across. I don't have a good answer for that, I think I'd have to do it for real for a while in order to decide how to cope.

But when merging from Custom to OpenACS Fixes, I suspect that one way or another, you're going to end up with a whole bunch of random tags saying "fix-4239-blah-blah" scattered all over various files. I think you'd want to place tags just on the individual affected files in that case, not the whole checkout, but I don't remember for sure. All these random tags everywhere do sound kind of ugly, though. This might be a good reason to instead make all your bug-fix changes in your OpenACS Fixes branch first, and then merge them into your Custom branch, rather than vice versa. That way you'd only ever be merging in one direction, from OpenACS Fixes back to Custom, which should help keep things simpler. But again, my thinking is kind of foggy on this, and in practice, no matter how much you plan to only do merges in one direction, eventually you're going to modify something on the "wrong" branch and have to merge in the other direction, too. But limiting the number of such "wrong direction" merges may help to keep things manageable.