Forum OpenACS Development: Re: Towards OpenACS 5.9

Collapse
2: Re: Towards OpenACS 5.9 (response to 1)
Posted by Don Baccus on
"The OpenACS data model for PostgreSQL has many places, where dependent tuples are deleted manually instead of using proper foreign key constraints. This happens in acs_object__delete() but as well on other types such as the content repository types."

This is partially due to my losing an argument with ArsDigita oh, what, 15 years ago? For whatever reason, the arsDigita folk were unwilling to use proper foreign key constraints in their Oracle data models, and at that time, for simplicity of tracking ongoing rapid changes by aD, we based our PG datamodel strictly on the aD Oracle data model (helped by automated conversion tools that did much of the original conversion of ACS 4 to PG).

Then it turned out that the PG RI implementation (gulp, I helped with the first version) had serious locking contention issues making it very slow and even susceptible to deadlock. Another blow against the full use of SQL.

Over the years the RI implementation got more robust, but resources to maintain Oracle datamodels were slim (they were named "Don Baccus", mostly). So an overhaul wasn't really practical.

If you want to make the move, do it, it's the right thing to do and has been ever since I first proposed it to aD back in the ACS 3 days (ignoring PG bugs).

And quit supporting Oracle, geez.

Getting rid of tree_sortkey is the right thing to do, even though I wrote most of the code and came up with the datamodel design and key representation. It was before recursive queries, of course, and we always used "connect by" in Oracle (except for some bizarre Ben Adida code).

Quit supporting Oracle, geez. Oh, I already said that :) Given that almost no one uses OpenACS any more, why support it? I'm guessing support is weak and much is broken anyway.

Collapse
3: Re: Towards OpenACS 5.9 (response to 2)
Posted by Antonio Pisano on
Don Baccus opens a very interesting point: who is using Oracle with OpenACS?

This could be the right for its eventual userbase to let their voices be heard, because dropping Oracle is very inviting and would free a big deal of resources for people working on the project.

Maybe we should open a poll?

Collapse
4: Re: Towards OpenACS 5.9 (response to 2)
Posted by Neophytos Demetriou on
Getting rid of tree_sortkey is the right thing to do, even though I wrote most of the code and came up with the datamodel design and key representation. It was before recursive queries, of course, and we always used "connect by" in Oracle.

I'm not using Oracle with OpenACS anymore and, for the record, I have used recursive queries in PostgreSQL already (in a different context) and found them to be quite handy. That said, I think the tree_sortkey implementation is interesting as a concept as it implies different architectural/design constraints on the system itself i.e. to some extent it does not rely on the underlying RDBMS. Either way, keeping it or not, it would not make much of a difference for me.

Collapse
5: Re: Towards OpenACS 5.9 (response to 2)
Posted by Gustaf Neumann on
Concerning tree_sortkeys: for 5.9 my intention above was not to through these out completely, since this will break a many (custom) packages. The tree_sortkeys are a cool idea and they work surprisingly well. The suggestion for 5.9 was just to remove it on places, where they are completely useless, and where the maintenance costs are high (i.e. in acs-objects) and start phasing it out on other places. There are cases where tree_sortkeys have advantages over recursive queries (e.g. in views, at least up to pg 9.4).

Concerning Oracle: we for our part have no intention to drop Oracle in total. We would rather try not to damage it, but we do not have the resources for actively testing it. There are several OpenACS installations (or tailored clones) based on Oracle we don't want to be left alone. We should try to get in touch with these to assess their situation. A possible road-map could be to keep the oracle "support" as it is for 5.9 and reconsider it for 6.0.

Collapse
6: Re: Towards OpenACS 5.9 (response to 5)
Posted by Don Baccus on
Right, I assumed you just meant for acs_objects, not the auxillary keys defined and used by various packages. Those would take a lot of work.

If you can't test oracle, by definition it no longer works.

The Oracle-based people have, for over a decade, screamed at the mention of dropping Oracle but have never offered significant resources to keep support current (though one company, when they learned I was doing so for core single-handedly on a volunteer basis, did give me a small gift as a token of appreciation. Not enough for even one decent dinner but I did appreciate it).

"We should try to get in touch with these to assess their situation" - post a thread stating "we're talking about dropping Oracle again!!!!" and see if anyone screams. If not, you're in the clear. If so, tell them to step up and maintain the damned thing.

I'm guessing that the various legacy users of oracle-based .lrn, for instance, haven't upgraded in a long time. I'm 99% certain that's true of UNED (Emmanuelle's no longer there, and in her last years there had nothing to do with the maintenance of the .LRN installation there - despite encouragement to upgrade, last I heard they were stuck on a version that's about five years old).

So if you investigate I think you'd find that any existing oracle usage is legacy usage, and already orphaned.

I could be wrong but it would simplify what's left of the project, and given the lack of resources now that most of us have moved on, would seem like a smart thing to do.

Collapse
8: Re: Towards OpenACS 5.9 (response to 6)
Posted by Andrew Piskorski on
We use OpenACS on Oracle where I work, and have since 2001, prior to upgrading from ACS 4.2 to OpenACS.  We indeed, have not upgraded our OpenACS in a very long time, but certainly hope and plan to do so eventually.  It's merely that we haven't had any burning need for an upgrade, and limited time and manpower, so we haven't yet.
Collapse
9: Re: Towards OpenACS 5.9 (response to 8)
Posted by Jim Lynch on
Andrew, what would you guys ngals think of having to move to postgres soonish? do you have the ability in manpower?
Collapse
10: Re: Towards OpenACS 5.9 (response to 9)
Posted by Antonio Pisano on
Well, even if I am VERY biased towards open-source, I think throwing away Oracle would be a loss if people using it could invest a bit of efforts at least for testing.

All recent development on OpenACS passed through the uneasy task of considering Oracle people still around, so it would be better if such struggle could end up useful.

Collapse
11: Re: Towards OpenACS 5.9 (response to 9)
Posted by Andrew Piskorski on
Andrew, what would you guys ngals think of having to move to postgres soonish?
Not going to happen, not anytime soon, and maybe never, for a variety of reasons.

Folks, keep in mind that we don't use our RDBMS just for a website, and I suspect this is pretty common for other small firms out there.

In our particular case, we use Oracle extensively for other non-OpenACS non-web data management. We take advantage of OpenACS to view, help maintain, etc. that data. Essentially, in addition to the other ways we use it OpenACS provides a UI to that data. It all lives in the same Oracle instance, in different table spaces.

When we were starting with this back in 2001, PostgreSQL wasn't quite ready for our needs and ACS/OpenACS 4.x didn't yet support PostgreSQL anyway. I'm told that PostgreSQL now supports the SQL:2003 OLAP (windowing, lead/lag, etc.) features, which are incredibly useful when dealing with financial and other time-series data. So if we were starting from scratch today, perhaps we'd go with PostgreSQL rather than Oracle. But not necessarily, it depends.

Many third parties - like data vendors - still support Oracle but not PostgreSQL in ways that can matter. E.g, in my experience most data vendors essentially just give you obnoxious, giant, completely de-normalized CSV files. The better ones (or perhaps the ones with the most complicated data) pre-specify a reasonable relational schema, and often provide their own loader tool to keep it updated. We use one which only supports Oracle and MS SQL Server. (Could they add support for PostgreSQL if they wanted to, or even SQLite? Probably, but so far they haven't.)

With a data vendor like that, do you have to use their loader? Often, no, but if you don't, you get to handle all the work of processing their internal-format transactional update files multiple times every day, etc. etc. None of that is insurmountable. But it is an example of why it is valuable to have a nice toolkit that just so happens to support the RDBMS that you also want to use for other reasons.

Should you care about this? About our specific needs, perhaps not. But I hope it least illustrates how the "legacy" feature of good support for both Oracle and PostgreSQL may be of more value than the typical PostgreSQL-only OpenACS user appreciates.

By the way, Oracle has ended up working well for us, but we didn't know that it would ahead of time, nor could we have, since we didn't even know many of the details of our own needs yet. (Heck, I didn't even know that lead, lag, and all the other SQL:1999 and SQL:2003 OLAP functions existed until 2001, and now we critically depend on them.)

In that situation, which I think is pretty common, the right approach is to bias your choice of tools towards flexibility. So which would sound like a better bet to you? "We only support one RDBMS, period." Or, "We support two of the most powerful and full-featured RDBMSs available, PostgreSQL and Oracle."

Back then, I was certainly thinking about the fact that if we ran into some unexpected major roadblock with Oracle, we might be able to get around it with PostgreSQL. The same feeling in the opposite direction might be more common.

Collapse
14: Re: Towards OpenACS 5.9 (response to 11)
Posted by Don Baccus on
How much support to the project are you giving? How much money to pay people to continue oracle support?

None, is my guess.

Back when I was maintaining Oracle support, did you ever offer to help pay for my time? Or, when the subject was ever raised, did you just post as you have posted now: "we need oracle support, don't drop it!"

"Should you care about this? About our specific needs, perhaps not. But I hope it least illustrates how the "legacy" feature of good support for both Oracle and PostgreSQL may be of more value than the typical PostgreSQL-only OpenACS user appreciates."

Sure, valuable to you and other freeloaders. But the reality is that for over a decade, almost all of the financial and programmer resource support has come from those using PG.

Collapse
13: Re: Towards OpenACS 5.9 (response to 5)
Posted by Andrew Piskorski on
Concerning Oracle: we for our part have no intention to drop Oracle in total. We would rather try not to damage it, but we do not have the resources for actively testing it.
That sounds like the right approach to me.
Collapse
15: Re: Towards OpenACS 5.9 (response to 13)
Posted by Antonio Pisano on
Best effort Oracle maintenance is what has and will be done FOR NOW, but I am almost sure people trying to upgrade now their Oracle installation will face issues.

Dear Andrew, we of course understand the reasons you raised for using Oracle, but be aware that this kind of support community has provided is fragile and has had almost no testing whatsoever, so if your company plans to enjoy new development on OpenACS you should seriously consider coding or other forms of contributions.

Something I think I can suggest to people using Oracle around is trying a fresh install of current OpenACS and give feedbacks, or even better go for an upgrade in a safe environment and report issues. Somebody could offer him/herself to address them.

Anyway, at risk of repeating myself, without concrete contribution from Oracle userbase all of you will eventually be left alone.

Collapse
23: Re: Towards OpenACS 5.9 (response to 15)
Posted by Dave Bauer on
What level of testing is being done on Oracle now? Are new installs for each new version tested before release?
Collapse
24: Re: Towards OpenACS 5.9 (response to 23)
Posted by Maurizio Martignano on
Dear Dave,
I cannot talk for the distribution in general.
When the standard distribution (from the repositories) is compiled for Windows using MS tool chain, nsoracle does not even compile.
I am continously checking naviserver code
(http://sonarsrv.spazioit.com/dashboard/index/42350) as support to Gustaf in his development/maintainace activities
and, as far as I can see, new versions are only compiled (and tested?) only on specific platforms (e.g. Linux and not Windows).

Hope it helps,
Maurizio

Collapse
7: Re: Towards OpenACS 5.9 (response to 2)
Posted by Andrew Piskorski on
This is partially due to my losing an argument with ArsDigita oh, what, 15 years ago? For whatever reason, the arsDigita folk were unwilling to use proper foreign key constraints in their Oracle data models
Really? That seems weird and suprising. I don't remember hearing about that back then. I wonder what their reasons were, and if they made any sense.
Collapse
12: Re: Towards OpenACS 5.9 (response to 7)
Posted by Don Baccus on
Well, you probably don't remember hearing it because it was me, not you, proposing it.

The argument was that with proper RI operators, someone might enter the DB, delete a class of objects, and wipe out the whole contents.

Where by using "foreign key" without "on delete", manual attempts of this sort would be aborted by the db.

Now, as to why one might delete such things manually, at that time. aD's code was really horrible at cleaning itself up.