Forum OpenACS Development: OpenACS.org Priorities
I think that there are some issues with OpenACS.org that we need to prioritize on the same level as getting bugs fixed and releases out. It's a real bummer to position the OACS as a scaleable, high-performance toolkit when the website is so slow.
This is not to say that those who are spending their valuable time getting that work out are doing something wrong. But I feel we're doing a disservice not addressing this stuff on the same level.
The two critical issues are:
- Forums scalability This is really horrid. Personally, I'm finding that the amount of time to get a post confirmed is a discouragement to responding to posts.
- Download area We need both kernel, official and contrib package APM area.
Again, I understand people are overwhelmed getting these things out, but I really think that these should be critical showstoppers for getting at least v5.1 out.
Talli, do you have rights to add the download areas? If so, why not just do it? If it can be made better later, it will.
BTW, it's great to see more forum activity on the weekend. I haven't seen this much Saturday activity in some time.
(i'm going to take a nap now, while I await posting completion)
I agree on your every point. One more thing I (and at least a few others) would like to see in the download area is the availability to wget the tarballs. The current form solution makes it impossible. (Assuming you want the packet to your server) It's a real nuisance to launch lynx and try to find your way through openacs.org when you could as well copy the location from Mozilla to your ssh window and wget it.
OK, this was just one (though IMHO quite typical) scenario. Making this work would probably need some tweaking on file-storage package but I think it'd be worth it.
if instead we simply sorted the threads by most recent reply, the thread index page could be cached globally. you could provide another page sorted by thread creation date if you liked, but from experience users seem to prefer the first by a wide margin.
Anyway, as you are all aware, the source code the site runs on is freely available. The site would improve much more quickly if more than one or two people helped to maintain and improve it. Besides improving this site, and fixes or improvements can also be added to the toolkit.
I don't mind discussing ways to improve openacs.org in the forums, but let's also keep track of what needs to be fixed in the bugtracker https://openacs.org/bugtracker/openacs.org setup specifically for the web site.
Maybe someone will look in there and possibly even submit a patch. I am sure we can find someone to review these and apply then as needed.
We have two ways to get code into openacs and openacs.org. Either get commit privileges on CVS or submit a patch. The process does work. If there are outstanding patches in the openacs.org bugtracker I will handle them this week. I admit I still have some outstanding patches for the survey package. (I peronsally wouldn't mind a weekly reminder of these things.)
Anyway, if you have a problem with a certain package email the maintainer. Yes, I know, how do you find out? We all know that perhaps everything isn't ideal with openacs, but if we work together it can improve.
I was going to add myself as maintainer of the openacs.org bugtracker, but it uses the all 3000 users is a dropdown box interface. I'll do it in psql tomorrow. Beyond that I will look at all the open bugs this week and see what I can do. If anyone wants to volunteer to help out let me know.
The problem as I see it is that there is no official decision about who can make decisions like providing a direct download link. Maybe someone decided once that every download had to be logged to a community member, and adding a direct download link would ruin those stats.
If there was a smallish team who could together make such decisions and act on them I think it wouldn't be a lot of work (I would be happy to be on such a team).
The forums scalability is of course another matter. Do we have any information about where the bottleneck is? I guess the way forward is to replicate openacs.org onto someone's dev box and start testing with the real data set like collaboraid did with the bug tracker recently.
Has anyone done this, or does anyone have rough ideas about where the scalability problem is? I know Lars has suggested we remove the CVS commit forum, but it is not the biggest forum by a long shot, so I don't see that it is the main problem.
Yes, a week or two ago I replicated the site on a dev server and identified a couple of bottlenecks. One is mentioned on this thread:
It has to do with a scheduled proc in the notifications package which runs every minute and is a *huge* resource hog.
Creating new objects is also slow (fyi, creating a new post creates 3 objects I think). Specifically, there is a query in the "acs_objects_insert_tr" trigger which seems to take a while:
where context_id is null;
Don says he has ideas on how to improve this. However, the bigger gain would be fixing the notifications query I mentioned earlier. There may be other issues, I don't know.
I apologize to the group that I haven't made the time to work on these issues. I will try to do so this week.
This is my primary concern with the current process.
Would it help to simply reduce the scheduled proc to only run every 5 minutes - or is it the actual bulk of notifications rather than just the querying that is expensive?
If it would help, i think we can all wait a few minutes to be emailed new posts.
I suggest the following:
1/ I will write up a page listing the known issues contributing to the performance problems in openacs.org forums and suggested avenues for solution
2/ we can discuss that page via comments, and it can be opened via etp for the people working on the problem
3/ the different problems can be allocated to different people to fix as takes their fancy
4/ when we agree on solutions we can test them on Robert's copy of openacs.org (Is this possible Robert? otherwise I can set up a test server, but it seems a waste to do it all again)
5/ Don (or someone) can vet our solutions and decide which should go into the cvs tree and which might be openacs.org specific fixes (and of course which are too ugly for public consumption at all...)
> copy of openacs.org (Is this possible Robert?
But ofcourse! The machine's at home and I'm behind a semi-unreliable DSL connection, but I'd even be willing to give SSH access as needed.
This sounds like a great application of bugtracker. We should list the problems at https://openacs.org/bugtracker/openacs.org/
The items can be assigned, and commented on. Patches can be submited, or the person could be given commit to the openacs.org tree.
That's not going to help notifications, though, not much, as it queues things up first then mails 'em later even if the notification is "immediate". Well, it will help by getting rid of one insert per message, actually, that is a good thing.
I'm going to start a new thread on forums scalability under PG ... stay tuned.
Especially since a number of the tasks won't be bugs per se - just things that should be done.
Maybe i'm wrong actually...