Forum OpenACS Development: Remaining work for 5.0.0 release

Collapse
1: Remaining work for 5.0.0 release
Posted by Joel Aufrecht on 11/20/03 09:41 AM
We are currently in beta testing. The following things have to happen before we can have 5.0.0rc1: When we achieve this, we can release 5.0.0rc1. The criteria for judging and rc release to be Final are a bit skimpy; two were discussed in the last OCT meeting, including an OCT vote of approval (rejected) and requiring that an RC tarball be up for a week without any new Sev 1 bugs found before it is declared Final.
Collapse
2: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Joel Aufrecht on 11/26/03 11:47 AM
After our very fun Hamburg Bug Bash and the synchronous efforts of others, we are much closer to releasing OpenACS 5.0.0rc1:
  • We have no open or untested critical bugs
  • 5 open Sev 2 bugs and 5 resolved Sev 2 bugs remain
  • We have 44 open Sev 3 bugs, of which 3 are show-stoppers.
  • We need to re-synchronize with the translation server. This has been a risky and time-consuming task in the past; we are working on new code to make it much easier and won't sync until that code is complete.

At the current rate, I expect we will meet our minimal criteria the middle of next week. Note that we will still have perhaps 40 known Sev 3 bugs unfixed. This is acceptable according to our release criteria, but it is certainly not desirable. If you have been waiting for the right time to fix a Sev 3 bug, that time is now.

Collapse
3: Re: Remaining work for 5.0.0 release (response to 2)
Posted by Jeff Davis on 11/26/03 09:12 PM
Here is an overview of the blocking bugs...

1-Critical

1231: Notifications fails to install on oracle (found today - I am working on this one now)

2-Major
  • 1144: users duplicated in cc_users (Lars, Don, Dirk looking at it).
  • 1226: .LRN class members can create/edit/delete calendar items (assigned to Dirk)
  • 1117: www/admin/permissions-user-add.tcl does not scale (Lars looking at it)
  • 857: Logger broken on Oracle (maybe shouldn't be blocking).
3-Normal/Priority-1
  • 1215: class event not found in calendar.
  • 1172: Creating a guest user no longer works.
  • 936: acs-lang Package documentation is confusing. Probably should not be treated as blocking.
  • 929: Relative Linking does not work in Versionview (file storage bug). Not sure what the status on this one is.
Collapse
4: 12 Dec 2003 Update (response to 3)
Posted by Joel Aufrecht on 12/12/03 11:07 AM
The following criteria are not yet met for releasing an RC (I have omitted trivial and administrative criteria):
  • all the core packages have upgrade scripts. I interpret this to mean that any site back to 4.5 can be upgraded to 5.0.0 via the APM. If there is an alternate inpretation, please post it as a TIP. We have not met this criterion; Daveb is currently working on it.
  • All Severity 1 and Severity 2 bugs fixed or postponed and Test team verifies no blocking bugs. Blocking bugs are those designated Priority 1. (This is not a good system. We'll fix it after we upgrade 5.0.) We have 4 remaining blocker bugs:
    • #1265. Security: Somebody managed to get a ; in a page title.
    • #1226. .LRN class members can create/edit/delete calendar items
    • #1144. Users duplicated in views registered_users and cc_users
    • #1172. Creating a guest user no longer works
  • Translation server data synchronized to branch. Collaboraid has committed code for an improved synchronization process and is testing this code; it still remains to upgrade the translation server so that it has this new code, and then to use it to synchronize the data on the translation server to the branch.
Collapse
5: Re: 12 Dec 2003 Update (response to 4)
Posted by Jeff Davis on 12/12/03 12:06 PM
On the upgrade criteria, it was discussed in the OCT meeting and the consensus was that it was not a blocker if the upgrade had a manual step but that it had to be well documented. Of course, if an automated update was done and robust that would be better...

(We did not have a formal vote so that is of course my interpretation, but you can look at the irc log).

Collapse
6: Re: 12 Dec 2003 Update (response to 4)
Posted by Malte Sussdorff on 12/12/03 12:56 PM
If I'm not mistaken, the TIP 35 has been approved (or should have been). Isn't this the two yes and no no. Gosh, I can't remember these TIP rules...

Anyway, I second Jeff in saying that the OCT has been in the meetings in favour of a manual upgrade from 4.6.3 to 5.0 to get the release and not treat the non existance of an automatic upgrade as a blocker for release.

Collapse
7: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Dave Bauer on 12/12/03 01:23 PM
Anyone who can help out with upgrade testing would be greatly appreciated. There is some information about what I have learned so far: https://openacs.org/forums/message-view?message_id=152200

Anyone with an existing 4.6.x site who can try an upgrade would be great! We want to test a base 4.6.3 install upgrade to 5.0, but testing with real data would also be helpful.

Collapse
8: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Ola Hansson on 12/12/03 02:40 PM
If I'm not mistaken openacs.org is running 4.5b or something like that so it will have to be upgraded to 4.6.3 before it can be upgraded to 5.0, right? And since I have a few sites running the same beta version (or is it final - I forget) I'm curious if the upgrade to 4.6.3 is straight-forward... Has that line of upgrade been documented somewhere already, or is the one that is going to upgrade openacs.org to 5.0 planning on making his/her notes publically available?

Thanks.

Collapse
9: Re: Remaining work for 5.0.0 release (response to 8)
Posted by Jeff Davis on 12/12/03 02:46 PM
Ola, the 4.5->4.6 step is documented here:
https://openacs.org/doc/openacs-4-6-3/upgrade-4.5-to-4.6.html
Collapse
10: Re: Remaining work for 5.0.0 release (response to 9)
Posted by Jeff Davis on 12/12/03 03:26 PM
What should we say about pg7.4 support in 5.0.0? I have installed everything and run through most packages and have not encountered any problems with it so am inclined to say it's "verified" per the Version compatibility chart.

Also, I don't imagine it's that relavant but I am not aware of any testing on 8.1.6 so maybe that should say "Untested".

Collapse
11: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Don Baccus on 12/12/03 09:44 PM
#1144 is fixed and I believe I marked it so yesterday (it was fixed a bit earlier but I forgot to mark it in bugtracker)

#1172 has been bounced back to Peter Marklund for either some explanation or a fix depending on what he wants to do.  The code's a bit convoluted, apparently because it is used elsewhere, not just by the add-one-user code, and I figured he could tell me what's going on much more efficiently than I can try to figure out what's going on in that code.  The bug's obvious, what's not obvious is how to fix it without breaking other chunks of code that call upon it.

I think Dirk was going to revisit the calendar problem, which he thought was fixed.

Collapse
12: Re: Remaining work for 5.0.0 release (response to 11)
Posted by Jeff Davis on 12/12/03 09:51 PM
I committed a fix for the calendar problem (1226), Dirk set direct permissions but did not revoke inherit so students ended up with create by virtue of create on the parent.

Hopefully that's it and once the test servers get rebuilt Lars can confirm it's ok.

Collapse
13: Re: Remaining work for 5.0.0 release (response to 12)
Posted by Joel Aufrecht on 12/17/03 02:20 PM
The following items still block the release of 5.0.0 release candidate 1:
  • Creating a guest user no longer works.. A fix was committed today; if it is not to the submitter's satisfaction, I suggest we demote the bug to not being an OpenACS blocker (it's a dotlrn bug anyway).
  • Translation server. The code to make upgrades easy and safe has been committed. Remaining work is
    • an intermittent failure in the automatic tests must be tracked down and fixed
    • The translation server has to be updated with the new code and then new translations have to be merged back.

    A conservative ETA for this work is early January.

  • Upgrades. As far as I know, several people have upgraded successfully; in each case it has taken several days and many manual steps plus some bug fixes. I would say that we have not met our upgrade criterion yet.
Collapse
14: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Don Baccus on 12/17/03 05:56 PM
Many thanks to Peter for fixing 1172 ...

As far as upgrading goes ... has anyone tested Oracle or is it just Postgres that people have upgraded?

I'll probably ask that in today's OCT chat :)

Collapse
15: Re: Remaining work for 5.0.0 release (response to 14)
Posted by Joel Aufrecht on 01/06/04 02:48 PM
Remaining and recently met criteria for releasing an RC:
  • all the core packages have upgrade scripts. I have defined five different levels of strictness for this criterion, and will put this on the table for the next OCT meeting, followed by a vote or TIP.
  • All Severity 1 and Severity 2 bugs fixed or postponed and Test team verifies no blocking bugs. Done. (Thanks everybody!)
  • All core packages have automated tests, not necessarily complete, and pass the tests. The number of failed test cases has been creeping up, and as of today we have 24 failed Oracle test cases and 6 failed PostGreSQL test cases. Due to cascading and related errors, these probably represent 4 and 3 different issues, respectively. I believe that we can punt on all of these:
    • One test is for bug 1144, which is punted
    • ad_html_to_text_clipped_link fails, but it doesn't seem critical
    • acs_templating group_tag fails, but it doesn't seem critical. Bug 428.
    • Some of the acs_lang upgrade tests fail on Oracle but not PostGreSQL. Initial inspection suggests that these would require refactoring acs_lang to add a better multi-field primary key to one of the upgrade tables. Failure to fix this shouldn't block release, especially since we've explicitly voted that translation server isn't a blocking issue.
Collapse
16: Re: Remaining work for 5.0.0 release (response to 15)
Posted by Guan Yang on 01/06/04 03:18 PM
So what is blocking 5.0.0?
Collapse
17: Re: Remaining work for 5.0.0 release (response to 16)
Posted by Joel Aufrecht on 01/06/04 06:02 PM
kernel upgrade from 4.6.3 to 5.0.0 is the only remaining blocker.
Collapse
18: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Dave Bauer on 01/07/04 06:12 PM
I posted a procedure for upgrading from 4.6.3 to 5.0: https://openacs.org/projects/openacs/5.0/upgrade-463-50

It has been tested by at least one person. It would be good if we could get some more tests, especially on actual sites that have some data in them.

Feedback on the instructions for the procedure would also be helpful.

Collapse
20: Re: Remaining work for 5.0.0 release (response to 18)
Posted by Malte Sussdorff on 01/07/04 07:32 PM
If at least one more person verifies that the instructions mentioned by Dave work on a life site, we will release RC1.

Help is especially needed for testing Oracle upgrades, though we are not stopping the release if this remains untested.

Collapse
21: Re: Remaining work for 5.0.0 release (response to 20)
Posted by Vinod Kurup on 01/08/04 04:08 AM
I just tried upgrading my site from 4.6.4ish to 5.0.0 using DaveB's instructions and it worked. AOLserver4, PG 7.4. I had a couple minor stumbling blocks:
  1. the acs-kernel 4.6.5-4.6.6 upgrade script gave 2 errors which seem unimportant: 
    ERROR:  duplicate key violates unique constraint "users_pk"
ERROR:  relation "admin_rels" already exists
  2. the acs-kernel upgrade 5d7 to 5d9 script gave me a couple errors because it couldn't drop 2 tables. Moving the 2 'drop table' commands to the end of the file fixed that.
  3. The first time I tried to login, the login form bombed with the error: "can't read 'defaults': no such variable" inside template::form::create. Restarting the server again mysteriously solved the problem.
  4. Upgrade failed for static-pages (can't insert duplicate key into apm_package_versions) and acs_messaging (couldn't find a function acs_message__new to drop). I didn't look into fixing the static-pages error, but commenting out the 'drop function' in acs-messaging's upgrade 4.62-4.63 script allowed it to upgrade. Looking further, I think my site was at acs-messaging version 4.6.3 in the DB, but the .info file reported 4.5, thus causing this error.
I haven't tested things in detail, but the upgrade process seemed to work for me. I actually did the upgrade process in full twice.

I also note that I didn't need to load the postgresql.sql file. pg_dump seems to be fixed, at least in 7.4.

Collapse
19: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Dave Bauer on 01/07/04 06:14 PM
Anyone who is testing upgrading and has any questions please post to the forums or visit the IRC channel https://openacs.org/irc/
Collapse
22: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Peter Marklund on 01/08/04 10:10 AM
I just wanted to brief people on what is involved in bringing translations on the translation server over to the 5.0 branch:

1. First we need to fix the broken acs-lang upgrade test case on the Oracle test server. My suspicion is that the cause of the failure is a violation of the lang_messages_audit_pk constraint which is on (package_key, message_key, locale, overwrite_date). I suppose it is the overwrite_date that is identical for two inserts although I would have thought that sysdate would be granular enough for this not to happen. A fix would be to make lang_messages_audit use an integer primary key instead. Making this change should be straightforward and shouldn't be more than 1 work day.

2. Merge the latest message catalog files on the 5.0 branch with the translations on the translation server. This involves importing the 5.0 catalog files to the translation server, resolve any conflicts in the acs-lang admin UI, and then export the result and commit it on the 5.0 branch. After that we should run my catalog consistency check script on the 5.0 branch and see if any keys should be removed or are missing. I think it's best if I do this and I think we're looking at another work day.

Collapse
23: Re: Remaining work for 5.0.0 release (response to 22)
Posted by Joel Aufrecht on 01/11/04 01:50 PM
Based on the last OCT discussion, I think we are ready to release 5.0 RC1. I tidied up the (still incomplete) upgrade instructions. In addition to upgrade code and docs being very preliminary, I still haven't dealt with all of the doc bugs and errata. Anyway, I went to build the RC, following these instructions, and got: 
[joela@samoyed joela]$ cd /var/tmp
[joela@samoyed tmp]$ cvs -d /cvsroot checkout -r oacs-5-0 openacs-4
cvs checkout: in directory openacs-4:
cvs [checkout aborted]: *PANIC* administration files missing
[joela@samoyed tmp]$
Collapse
24: Re: Remaining work for 5.0.0 release (response to 23)
Posted by Jeff Davis on 01/11/04 03:02 PM
Checkout worked for me (although there were some files owned
by the web group rather than the cvs group which I fixed).
See if it works now.
Collapse
25: Re: Remaining work for 5.0.0 release (response to 24)
Posted by Joel Aufrecht on 01/11/04 04:16 PM
That fixed the OpenACS cvs error, so I went ahead and made the tarball. I've posted it on the front page in addition to beta 4; once I test it and get some feedback, i'll take down Beta 4.

I hit this error for dotlrn, so I didn't make dotLRN 2 RC1: 

[joela@samoyed dotlrn-packages]$ cvs -d /dotlrn-cvsroot checkout -r dotlrn-2-0 dotlrn-all
cvs checkout: Updating bboard-portlet
cvs checkout: failed to create lock directory for `/dotlrn-cvsroot/bboard-portlet' (/dotlrn-cvsroot/bboard-portlet/#cvs.lock): Permission denied
cvs checkout: failed to obtain dir lock in repository `/dotlrn-cvsroot/bboard-portlet'
cvs [checkout aborted]: read lock failed - giving up
Collapse
26: Re: Remaining work for 5.0.0 release (response to 25)
Posted by Jade Rubick on 01/12/04 07:28 PM
Does the OpenACS core team have a policy on upgrades with release candidates? If there was a policy of supporting upgrades to and from release candidates, that might encourage more people to try the release candidates.
Collapse
27: Re: Remaining work for 5.0.0 release (response to 1)
Posted by Malte Sussdorff on 01/13/04 10:22 AM
If I remember our talks a while back the official line is: Alpha and Beta do not have upgrade support but Release Candidates and Releases have upgrade support, meaning: Upgrade scripts from RC need to be available for the release to come out. OTOH I seriously hope we do not find anything in the RC that requires us to write upgrade scripts ....