Forum OpenACS Development: Content Repository problems on Oracle

Our first couple of tests with Assessment on Oracle were a small fiasco due to the content repository and the fact that most upgrade scripts have only been written for PostgreSQL and not for Oracle. Oh, and acs_objects was also incomplete... Either way, Timo can comment more on this and will submit the fixes today. Let me encourage people though to either write upgrade scripts for both databases *at the same time* (if they fail, we realize it on upgrade, if they are not there they upgrade works but the package is broken), or make the change only on HEAD.

Be it as it is, we are now facing a different problem, which we might fix by using varchar(4000) instead of LOB, but maybe this is rooted somewhere deeper and anyone else has a clue:

When calling as:assessment::new, which is essentially a call for content::revision::new, we get the following error:

nsoracle.c:3904:ora_tcl_command: error in `OCIStmtExecute ()': ORA-25008: no
implicit conversion to LOB datatype in instead-of trigger

SQL: insert into as_assessmentsi
(revision_id, object_type, creation_user, creation_date,
creation_ip, title, description, item_id, text, mime_type , creator_id,
instructions, run_mode, anonymous_p, secure_access_p, reuse_responses_p,
show_item_name_p, entry_page, exit_page, consent_page, return_url,
start_time, end_time, number_tries, wait_between_tries, time_for_response,
ip_mask, show_feedback, section_navigation, survey_p)
values
(:revision_id, :content_type, :creation_user, :creation_date, :creation_ip, :title, :description, :item_id, :content, :mime_type , :creator_id, :instruction !>>>!, :run_mode, :anonymous_p, :secure_access_p, :reuse_responses_p, :show_item_name_p, :entry_page, :exit_page, :consent_page, :return_url, :start_time, :end_time, :number_tries, :wait_between_tries, :time_for_response, :ip_mask, :show_feedback, :section_navigation, :survey_p)

Any help is highly appreciated.

Collapse
Posted by Timo Hentschel on
I just committed my fixes. Apparently, there are several postgres upgrade scripts for acs-content-repository after version 5.1.2, but those were never transfered to oracle (grrrr!!!). So I had to add "security_inherit_p" to content_folder.new, content_item.new and acs_object.new (get that!!! nobody thought about upgrading the acs_object-package either!!!). In addition to that, content_folder.new needed the parameter package_id for it to work with the new content-repository code.

Still, after those changes I got the error mentioned by Malte which points to a problem deeply rooted in the content-repository: how to deal with lob-values on insert/update of a revision through the nice tcl-interface. I could work around the problem for now by changing the appropriate procs in assessment not to provide a value for lob-columns in a call to content::revision::new and to update them later in the normal way to do for lobs, but I think this will not fix the deeper problem in the content-repository and since this is the foundation on which openacs is build nowadays, I think it needs fixing asap.

Collapse
Posted by Dave Bauer on
First, thank you for finding this error and looking for a solution.

It is not the case that work occurs on PostgreSQL while neglecting Oracle. It is possible that not every developer has the resources to maintain an Oracle installation for testing purposes. This is why it is so important for those who use Oracle to kindly volunteer to test before a release. With this information we can improve OpenACS.

On the LOB issue. It is interesting that the code orginally written for ACS does not work. It appears that content::revision::new can not use the view to set the content of the CLOB column. In this case we should change the procedure to use a two step process where the content is set as an update to the revision. Thank you for tracking this down.

Collapse
Posted by Malte Sussdorff on
Dave, I understand that not everyone has an oracle installation to test the scripts and this is fine. What is not fine though is the fact that upgrade scripts have been forgotten, which have been developed for the postgres version. The minimum a developer can do is copy the upgrade scripts from the postgres directory to the Oracle directory. With luck, it works, if not, one can fix the problem. But if you are not aware that you should have run upgrade scripts, finding the errors is hard.

Therefore, for the future, it would be good if developers at least copy the postgres upgrade scripts over to the oracle directory.

Collapse
Posted by Dave Bauer on
Malte,

Of course, I agree completely that upgrade scripts need to be kept in synch for both databases. And that is my usualy procedure. I suspect a mistake in this case, and that is why I suggested better testing would catch such a mistake.

Collapse
Posted by Dirk Gomez on
What is the point of copying something over that is *guaranteed* to be broken. It may and probably will just stun another developer in the future.

Please only create a placeholder file that clearly states that work has to be done here.

Collapse
Posted by Don Baccus on
It sounds as though the basic CR API paradigm might be slightly broken, since CLOB/LOB columns can not be directly inserted (regardless of whether or not you're trying to do so using the auto-generated view for a content type).

It is necessary to do a two-step process, as Dave's saying perhaps is necessary. Yes, it is. Check out the old cr revision code that inserts a file into the database (or file system) to see this.

In order to make the writing of db-agnostic code possible the Tcl API should offer the two-step (INSERT of non-[C]LOB data followed by an UPDATE that adds the [C]LOB data) approach for both databases.

Collapse
Posted by Malte Sussdorff on
Dirk, the whole point is, that the upgrade on Oracle (or postgres) breaks. This way you will realize something is wrong, otherwise you will never detect this. And it is considerably easier to fix if you have a file that contains all the things that should be done, albeit in a different database.

A placeholder file will have to break the upgrade as well, otherwise it won't do any good.

Collapse
Posted by Dave Bauer on
Unless someone else volunteers I'll fix content::revision::new tonight.

Malte,

Having a best practice for development is great. We also need developers to keep an eye on CVS and do code reviews as well as help with testing a release. All the rules won't help if no one tests before a release.

Collapse
Posted by Jeff Davis on
I think in an ideal world we would have something which loaded an old version, upgraded, dumped the schema in some canonical form, and a clean install dump which we could diff in order to pick up missing upgrades. Of course I am full of good ideas but not full of time to implement them.

Basically though, it's hard to catch missing or incorrect updates and any solution which relies on the diligence of a large set of developers is pretty much doomed to failure.

Collapse
Posted by Dirk Gomez on
Malte - if you want your code to break, then put in an error statement, *not* code that may work or may not work or may make a developer think somebody else put it there on purpose.

It is much clearer and a lot less time-consuming for the person doing the port if he knows what is supposed to happen. If you write this:

error;

// This code needs to be ported from the file foo.sql

than:

insert into ....

which may fail with a bogus Oracle error message. Maybe worse: it does not fail, but it has weird side effects.

The bottom line is: don't commit code to a non-private branch that has not been tested (without a warning and/or commitment to fix the bugs yourself) and if you want to indicate future work, write that in prose, not in code which may or may not work.

Collapse
Posted by Malte Sussdorff on
Dave, you mentioned you wanted to fix content::revision::new on Oracle. Did you have a chance to work on this as the error still exists on 5.1 and therefore prevents a clean 5.1.5 release.

If you do not have the time to do it, maybe you can at least outline what you had in mind so someone else can work on it.

Collapse
Posted by Dave Bauer on
This should be fixed on the oacs-5-1 branch. I modified the code to do a seperate update on the content column.
Collapse
Posted by Malte Sussdorff on
Hmm, still no luck. As advised I rerun the test cases and here are the results:

content-item-test number 12:

FAILED

content_item (body 0): Error during execution: Setup failed with error syntax error in expression " == 0": unexpected operator ==

syntax error in expression " == 0": unexpected operator ==
while executing
"expr [content::item::delete -item_id $evil_item_id] == 0"
invoked from within
"aa_true "evil_name item deleted" [expr [content::item::delete -item_id $evil_item_id] == 0]"
("uplevel" body line 70)
invoked from within
"uplevel 1 $transaction_code "

syntax error in expression " == 0": unexpected operator ==

syntax error in expression " == 0": unexpected operator ==
while executing
"expr [content::item::delete -item_id $evil_item_id] == 0"
invoked from within
"aa_true "evil_name item deleted" [expr [content::item::delete -item_id $evil_item_id] == 0]"

content-revision-test number4:

content_revision (body 0): Error during execution: Setup failed with error nsoracle.c:3904:ora_tcl_command: error in `OCIStmtExecute ()': ORA-01465: invalid hex number

SQL:
update cr_revisions set content=:content !>>>!where
revision_id=:revision_id


nsoracle.c:3904:ora_tcl_command: error in `OCIStmtExecute ()': ORA-01465: invalid hex number

SQL:
update cr_revisions set content=:content !>>>!where
revision_id=:revision_id