Forum OpenACS Q&A: Postgres Problem -again

Collapse
Posted by Grzegorz Mucha on
I mentioned the problem with deleting from Postgres - I still haven't
resolved it, I cannot delete from ec_user_classes,
ec_user_class_user_map and some other tables. It doesn't work with
7.0.2, 7.0.3 and 7.1devel (yesterday snapshot).

As I am trying the 7.1devel with outer joins support, I attacked the
ec_recommendations_cats_view - though I made the query work, the view
doesn't want to execute. Anyway, as it is used only twice, I
reconstructed the query, getting the recommendations to work. And
again, once I have made a recommendation, deleting it is impossible.

Any further ideas on PG behavior (it deletes from ec_items or
ec_orders, while doesn't want to work with the above mentioned tables)

Thanks.

Collapse
Posted by Roberto Mello on
What do you mean by "I cannot delete"? What's the error returned?
Collapse
Posted by Grzegorz Mucha on
The problem is, that PG behaves all right, except that it doesn't.
Everytime it returns

DELETE 0

(and surely it has records to delete and even without WHERE (DELETE FROM ec_product_recommendations) clause it is still DELETE 0) - It does not complain about referential integrity if that's what you'are about.

Collapse
Posted by Don Baccus on
You should probably make sure the PG hackers list is aware that outer joins aren't working within a view.  Outer joins are new and views have been rewritten, and this is pre-beta software - perfect timing for a bug report.

As for the delete problem, I'm still stumped.  Since thousands of PG users can delete rows from their tables with no error, I'm certain it's some sort of "cockpit error", i.e. something you've done.  For the life of me, though, nothing comes to mind.

Collapse
Posted by Dan Wickstrom on
Maybe it is a referential integrity problem and postgresql is not complaining about it properly. Referenial integrity is new for pg 7.x, so it's possible that there might be some bugs in it. Try dropping all of the referential inegrity triggers from one of the tables that you are having trouble deleting from, and see if you can then delete from that same table. An easy way to do this would be to save the table data in a tmp table and then drop the original table. Then recreate the original table from the tmp table by either using create table followed by insert-select or by using create table as select *...

for example (you might want to run pg_dump first, in case you need to restore the original data):

-- save the original data in a tmp table.
create table tmp_user_classes as
select * from ec_user_classes;

-- this will also drop the referential integrity triggers.
drop table ec_user_classes;

-- do this or...
create table ec_user_classes as
select * from tmp_user_classes;

-- use the original table definition
create table ec_user_classes...

-- followed by 
insert into ec_user_classes
select * from tmp_user_classes;

-- now try and delete from ec_user_classes
delete from ec_user_classes;

The only difference should be that ec_user_classes now has no referential integrity triggers enabled.
Collapse
Posted by Grzegorz Mucha on
Well, yes, it worked... So it is a postgres bug after all...

I wanted to ammend I tried it with 7.0.1,7.0.2,7.0.3 and 7.1.devel on
4 different machines and boo...Now I'll try to add those foreign keys
and see what happens.
Have you tried the same thing on your ACS installation?

Collapse
Posted by Grzegorz Mucha on
Again on the subject:

It works even though I recreated those tables with appropriate foreign
keys. So it looks that when you load the data model from SQL scripts
when installing or after dump it won't work, while manual recreation
of the above does. Strange.

Collapse
Posted by Dan Wickstrom on
How did you go about re-adding the foreign key constraints after you re-created the table?  It seems likely that you have data in other tables that reference ec_user_classes, and the foreign key constraints are kicking in, but not giving an error message.  If this is the case, you are seeing normal operation of except for the lack of an error message.  To see how to add the foreign key constraints, go through your dump file and search for "create constraint trigger" statements which reference "ec_user_classes".  It would be nice to be able to isolate this down to a specific constraint trigger.  You could also try initiating the deletion from a .tcl script.  I think I now remeber coming across this before, where deleting from psql would only give DELETE 0, but doing the same deletion from a .tcl script would come back with a referential integrity violation error message.
Collapse
Posted by Grzegorz Mucha on
Well, it started when I couldn't delete from ec_user_classes and ec_user_class_user_map inside Tcl scripts, that seemed to work perfectly OK, (except for not deleting). Then I started checking things in Postgres. AFAIK there are no constraints referencing the latter table. I had the same problem with ec_sale_prices (I mostly work on the ecommerce module).

The ec_user_classes is referenced only from within ec_user_class_user_map and it references users, while ec_user_class_user_map references ec_user_classes and users, while not being referenced by another table. Almost the same for ec_sale prices.

If you can, please check this operations on your installation and report if it works.

Collapse
Posted by Dan Wickstrom on
Sorry, but I don't have ecommerce set-up on my machine.  Other people have used the ecommerce module without reporting problems similar to yours, so I doubt that I would be able to recreate it by entering random data.

When you first noticed the problem when deleting from a .tcl script did you look in the aolserver log file for any error messages?

Looking at the ecommerce data model, I see that it is refereneced from: 1) ec_product_user_class_prices 2) ec_product_recommendations 3) ec_user_class_user_map 4) ec_spam_log.  Do any of those four tables contains data?  Based on the fact that you were able to delete from the table with all of the referential integrity triggers dropped, I'm sure that you are looking at some kind of problem with referential integrity.  I think you need to focus on one table such as ec_user_classes and see if you can isolate the problem down to a particular referential integrity constraint trigger.  Each of the four tables mentioned above will add a constraint trigger to ec_user_classes, and you need to see if you can isolate it to one of those triggers.

Collapse
Posted by Don Baccus on
pg_dump adds the foreign key constraints after the data is COPY'd into  the tables.  So you might try dumping/restoring some small examples to see if you can correct the problem.

Did you have any problems reloading the dump?  Did you edit the dump file at all before reloading it?  Did you use any switches to pg_dump?

I dump and restore my PG 7.0-based site all the time, and have never seen this behavior (I dump my live site to load into my development site for testing changes).

There were some strange things with referential integrity and permissions,  i.e. with different permissions on the referenced and referencing tables, but I can't imagine you've stumbled down that path .  Not by doing a dump/restore...

Collapse
Posted by Grzegorz Mucha on
<p>Well, I do not edit the dump files - the only thing I do is running the postgres.sql script first, if I don't have the dual, sysdate plpgsql etc., then go with the dump file. The only encountered problems while loading it were with duplicating definitions already done in postgres.sql

<p>When I dropped the table I did it simply with
<pre>drop table ec_user_class_user_map</pre>
(I copied it into another table with SELECT), then reloaded with another select and added foreign keys:

<pre>ALTER TABLE ec_user_class_user_map ADD CONSTRAINT uid FOREIGN KEY (last_modifying_user) REFERENCES users(user_id);
ALTER TABLE ec_user_class_user_map ADD CONSTRAINT u_class_id FOREIGN KEY (user_class_id) REFERENCES ec_user_classes(user_class_id) ON DELETE CASCADE;
ALTER TABLE ec_user_class_user_map ADD CONSTRAINT uid2 FOREIGN KEY (user_id) REFERENCES users(user_id);</pre>
<p>After adding the above constraints, I can delete the records, AFAIR the only missing check is for the value of 'active_p' in ('t','f'). I have to check what about the ec_sale_prices, what are the FK's there
<p>Do you know if there are any RULEs referencing ec_user_class_user_map (or triggers, beside the audit one)?

Collapse
Posted by Dan Wickstrom on
I was able to recreate your problem on ec_user_class_user_map.  After dropping the table and recreating it, you need to also re-add the indexes and the triggers on ec_user_class_user_map.  The problems seems to be with the ec_user_class_user_audit_tr trigger.  I don't have time to follow through on it right now, but you might want to try changing the audit trigger from a before to an after trigger.
Collapse
Posted by Dan Wickstrom on
I had a chance to try it, and changing the trigger to an after trigger allows you to delete from ec_user_class_user_map.
Collapse
Posted by Grzegorz Mucha on
Perfectly correct - no problems after altering the SQL, though I would appreciate the chance to do a CREATE OR REPLACE TRIGGER statement - it would save me some work (but thanks to Emacs, I managed to do it in 10 minutes).

Thanks, Dan- I owe you one!

Collapse
Posted by Dan Wickstrom on
ec_user_classes now also works after moving the audit trigger from a before to an after trigger, so I'm converting all of the audit triggers in ecommerce from before to after triggers.  The new version is checked in to cvs now.