Forum OpenACS Improvement Proposals (TIPs): TIP #35: (Approved in Part) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate

1: TIP #35: (Approved in Part) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate

Posted by Joel Aufrecht on 12/04/03 01:35 PM

Preamble

The current Milestone Criteria for OpenACS 5.0 has many criteria. Two of them stand out as still incomplete and still risky:

Translation server data synchronized to branch
all the core packages have upgrade scripts

The upgrade script is actually a code-complete blocker - ie, we should not have produced the first alpha and begun alpha testing without having the 4.6.3 to 5.0 upgrade scripts for all core packages.

Purpose of this TIP

This TIP has two purposes. The first is to ask for a decision to defer or to not defer the two milestone criteria listed above. The second is to experiment with a non-binary TIP. I considered a "TIP + Amendments" process but that seems really complicated here. What I propose is that we skip "two yes and no no" and have a straight two-thirds vote on each of two decisions. After one week, we count responses. If a decision does not have a two thirds positive majority (and at least one vote), the status quo remains. In this case, the status quo is that the milestone criterion still applies. (More information on TIP process alternatives and Debian methodology)

OCT members must those vote in two parts, according to this template:

"Tranlations" criterion for 5.0.0 is RELAXED/CONFIRMED
"Upgrade" criterion for 5.0.0 is RELAXED/CONFIRMED

A relaxed vote means that we can produce OpenACS 5.0.0 release candidates and final releases without meeting the criterion.

2: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Jeff Davis on 12/04/03 02:08 PM

What are the time estimates for the two tasks? Who is working on them now? It seems like if we are going to make a decision it should be framed in terms of 5.0 now vs 5.0 in one week with translations or two weeks with both or something like that.

Also, if it's a resource constraint can we solicit some volunteers for just those tasks?

3: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Caroline Meeks on 12/04/03 02:29 PM

I suggest we do a call for volunteers first.

I contacted MIT yesterday and asked if they could dedicate resources in the next week or two to work on Upgrade. AL seemed positive as he feels this is work MIT has to do eventaully anyway. They are assessing thier human and hardware resources before they can give us a definitive answer on who can help and on what time frame.

Can we think of, and contact directly, other organziations that plan to upgrade and might be willing to put some effort in now, prerelease?

4: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Joel Aufrecht on 12/04/03 03:18 PM

Peter and Lars are working on translations for the rest of this week, but then must return to client work until January. There is a chance they will not complete the work, which includes: Enhancing the acs-lang module to make synchronization easier and safer; (backing up and) upgrading the translation server to have this new functionality; using the functionality to get new translations from the translation server db to the file system to CVS.

On upgrade, Daveb is the point man and unless he protests, I recommend that all new upgrade work be funnelled through him so that we don't duplicate work.

My reason for suggesting this is that we have several different constituencies for OpenACS. We have the option of satisfying those people who want to use the new functionality, are not upgrading from existing sites, and don't need translations, and showing some concrete progress, by removing the top remaining sources of risk and uncertainty in 5.0 and going to 5.0rc1 this week. Otherwise, we can expect another one week to one month before we can release 5.0rc1. (I.e., between roughly Dec 8 and Jan 2.)

5: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Janine Ohmer on 12/04/03 04:20 PM

I don't like the idea of releasing without complete upgrade scripts (though I'm not on the OCT so this is just MHO). Can Dave give us an idea of the current situation and how much still needs to be done?

6: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Dave Bauer on 12/04/03 04:33 PM

The upgrade process needs to be tested. One way to test would be to create a base 4.6.3 install and update the code to 5.0 then perform an upgrade through the APM. There are existing issues with the authentication package that are being investigated.

There is a forums post and a bug report about the upgrade process:
https://openacs.org/forums/message-view?message_id=143497
https://openacs.org/bugtracker/openacs/bug?bug%5fnumber=1233

I believe Don Baccus is looking into adding code to the acs-bootstrap-installer to detect a potential 5.0 upgrade and run enough of the upgrade to allow the administrator to login to finish the process through the APM.

Another more interesting test would be to upgrade an existing 4.6.3 site with data.

I was specifically working on creating a script that would automatically upgrade from 4.6.3 to 5.0. It is not clear this is a good idea due to the potential for problems with existing data. We don't want to automatically introduce errors into a database.

7: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Caroline Meeks on 12/04/03 05:24 PM

I agree with Janine and I want to clarify the situation based on my converstations with Dave.

Dave had agreed to work on an automated upgrade system. Although Don has some ideas and will continue to work on it, we (OCT) have decided that will definitely *not* be part of 5.0 release criteria.

So now we are discussing whether we will have working, tested manual process avaiable as part of the release criteria.

Dave happens to also be working as a funded resource on file storage as part of his webdav work. There is concern that the 5.0 file storage may not be scalable at this point and he wants to look into that before release.

Thus we have Dave working on two release critical issues. File storage scalability and upgrade scripts. For file storage he is a funded resource, for upgrade scripts he is a volunteer.

What I'm hoping is that Sloan or another organization can find a funded resource to take a leadership role in testing and *fixing* the upgrade scripts so that they run for a full database (eg SloanSpace).

Dave will certainly want to stay in the loop but I think he needs to put his funded work, which is also of critical importance, first.

8: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Janine Ohmer on 12/04/03 05:31 PM

Well, we at Sloan have an urgent need for working upgrade scripts so I hope we will be able to help, but I'm not the one who gets to decide these things. I'll see what I can do.

9: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Don Baccus on 12/04/03 11:45 PM

Correct me if I'm wrong but what we're talking about is whether or not to provide an automated hands-off upgrade for this release, or require some manual steps. In either case there will be either scripts or detailed directions.

Originally Dave and I were going to collaborate as mentioned above, with my offering to hack the bootstrap installer to "fix" your installation so the sysadmin can log in and do the rest. That's the primary issue here - if you install 5.0 on top of 4.6.3 you can't log in until an upgrade script is run, but can't run the upgrade script until you log in.

Or you run the script by hand ... then log in and move forward.

In our last OCT meeting I was questioning the wisdom of running this script automatically when rebooting after the files are upgraded ... among other things people may be less likely to fully backup before rebooting thinking they'll have to run the APM upgrades before any changes are made? And if our boot upgrade fails you'd be stuck with an unbootable system perhaps which would really suck ...

So a combination of some manual steps followed by the traditional APM upgrade steps seems safer to me at the moment.

Any comments?

Are there other packages that don't have complete upgrade scripts? We have diddled the way relsegs and groups are managed by the new UI pages slightly between beta and today (bug fix, really) but hopefully people haven't been upgrading their production systems to 5.0 beta expecting stability to be guaranteed?

Am I wrong about this?

I haven't checked to see if Lars provided upgrade scripts to create the new "admin" relseg in all existing subsites but if not we DO need to do that...

10: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 9)

Posted by Joel Aufrecht on 12/05/03 07:05 AM

I interpret the milestone criterion to mean, there is an automated upgrade from any version back to 4.5, and it works by 1) updating some files on the hard drive, 2) clicking Upgrade in the APM. We do not quite meet this criterion in 4.6.3, because you usually have to upgrade the kernel first and restart and upgrade everything else in a second pass.

A looser interpretation would be, there exists a successfully tested process for upgrading which does not require manual, site-specific manipulation of the database. I think we're not here in 5.0.0 either because we have only one successful report of upgrade.

11: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 9)

Posted by Malte Sussdorff on 12/05/03 03:06 PM

Would it be possible to have a script that checks, if the upgrade has been run for 5.0 and if not, display one page at http://<yourserver>/ stating you have to run the upgrade script and are advised to do and backup before the upgrade, and skipping the whole initialization of packages ?

Is this the only "manual" task that needs to be done?

In general I'm in favour of offering a manual upgrade script with documentation for the time beeing and just *aim* for an automatic upgrade in future releases.

12: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Dave Bauer on 12/05/03 03:45 PM

Matle,

Showing the instructions to upgrade in the browser is a good idea.

Now we just need to get some users with existing sites to test the upgrade as soon as we have the process documented.

13: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Dave Bauer on 12/05/03 03:49 PM

Here is a possible workaround. I am not sure how this would work, do we have any way to execute a tcl procedure ourside a running OpenACS?

https://openacs.org/bugtracker/openacs/bug?bug%5fnumber=1129

14: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Caroline Meeks on 12/10/03 02:49 AM

I vote to relax "Upgrade" criteria.

I went to a meeting at Sloan today. The meeting decided that .LRN wants to decouple the release of 2.0 from OACS 5.0.

Based on what happened in the meeting I believe that the .LRN community will eventually provide/fund upgrade scripts. No specific time was set and I don't think it will be complete for oracle and postgres and tested next week or two.

Although I think tested upgrade scripts for all modules are desirable and important for OACS in general I feel there are also important benefits for releasing 5.0 for new sites. Indeed, even in the .LRN meeting it was considered that the .LRN governance structure might decide to they don't want to hold the release of 2.0 for upgrade scripts so new sites can launch with 2.0. However, as of now no such decision has yet been made.

Thus think its better for both existing OACS users and .LRN if we are clear that we don't have the scripts yet, nor can we make a good risk free estimate as to when we will have tested scripts. This will allow the .LRN governance structure to plan accordingly.

15: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 14)

Posted by Alfred Essa on 12/10/03 11:44 PM

I would like to qualify the statement that Caroline made. An item that we discussed at Sloan was whether in our opinion it was desirable to decouple .LRN releases from OACS 5.0. Our view was yes.

However, the people attending the meeting do not have decision making authority for either .LRN or OACS. The decision can only be made by the folks involved in governance for the two organizations and, as far as I am aware, neither have made a ruling on this issue.

16: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Caroline Meeks on 12/11/03 03:08 AM

Sorry to have spoken inaccurately. Though my reasoning stands whether .LRN is actaully decoupling or going to consider decoupling we should provide them with accurate information and 5.0 has a great deal of value without the upgrade.

This TIP has not passed yet and though my vote was pragmatic, I really really hope we do not have to release without a well tested upgrade path.

Any one who has a site they can try upgrading and has time in the next week please do. Dave will be publishing new instructions probably tomorrow.

BTW: I consider decoupling of the .LRN release from OACS releases part of a move away from monolithic releases. I do not belive its a negative thing at all.

17: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 16)

Posted by Malte Sussdorff on 12/11/03 08:19 AM

Hi Caroline, I think the ultimate goal in the end is to release an OpenACS core and have packages evolve independently, though we always ship stable versions of each package with the core.

As .LRN after all is *just* a package (technically spoken), this would allow a decoupling.

Anyway, I might be wrong and it would be good to have a TIP about both the decoupling and the package evolvement *after the release(s)*.

18: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 17)

Posted by Joel Aufrecht on 12/17/03 11:33 PM

I am calling for a full vote by the OCT on the first part of this tip, "Tranlations" criterion for 5.0.0. If you are an OCT member, please vote by posting in this forum. This is a strictly binary vote, so your options are YES (the criterion should be relaxed) or NO (the current criterion should be retained). According to our rules, if this TIP accumulates 2/3 YES votes in two weeks, it passes. Otherwise, it is rejected.

19: Re: TIP #35: YES: Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Don Baccus on 12/18/03 02:38 AM

We discussed this in today's OCT chat an if I understand correctly, Lars doesn't feel that delaying 5.0 to synchronize the translation server will speed up that synchronization work. It's not going to be completed until sometime in January anyway.

After synchronization and further translation work we can continue to make catalog files available that will work with 5.0 as the work gets done.

If I totally misunderstood Lars I'll change my vote :)

20: Re: TIP #35: (Full Vote) YES Relax Translations Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Dave Bauer on 12/18/03 05:11 AM

YES, relax translations criterion.

21: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 18)

Posted by Lars Pind on 12/18/03 10:25 AM

YES

22: Re: TIP #35: (Full Vote) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 1)

Posted by Peter Marklund on 12/18/03 11:59 AM

YES, relax translation criterion.

23: Re: TIP #35: (Proposed) Relax Two Milestone Criteria for OpenACS 5.0.0 Release Candidate (response to 18)