Forum OpenACS Development: Re: Compatibility Matrix

Eduardo,

You are right about the changes in PostgreSQL, which became worse for largish OpenACS/DotLRN sites after 8.3; for smallish sites, 8.4+ is ok. Therefore that's not a question about compatibility, but about recommended versions depending on the site's needs.

Starting from PostgreSQL 8.4, the postgres optimizer prefers in more cases sequential scans (at least with pretty standard pg-config files). This is especially bad for permissions (and search) in large sites with dotlrn. For this reason, we kept our production site a long time in PostgreSQL 8.3.*, and we hoped that the postgres optimizer in newer versions would behave better for these cases.

During this summer, we lost our patience and changed in our main installations the permission management to recursive queries [1] and switched to PostgreSQL 9.0.*. The change involved as well changes in the basic data-model (the huge materialized transitive tables). The work was done mostly by Victor Guerra. As a result, permission queries are now for our site significantly faster (often 4 times) than with 8.3. I think, there is still some need for cleanup in our installation (Victor is currently on vacation), but we will commit our changes back to CVS when ready.

-gustaf neumann

[1] http://www.postgresql.org/docs/9.0/static/queries-with.html

Hi Dave,

As Gustaf mentioned, we made some modifications in our installation in order to have a better performance when moving from PG 8.3 to PG 9.0. The changes mostly affect permissions.

Basically one has to run a couple of scripts and change some xql files. So the code is not in reality much. Everything is packaged in its own package so applying the changes should not be hard for a stand alone openacs installation; the only thing is that steps for upgrading need to be well stablished so people dont get lost. So, Ill try to dedicate a couple of hours this and next week and do some clean up and docu ( and some testing ;) ).

Best,

Hi,

I have applied Victors changes and it works fine except when trying to update the context hierarchy or set permissions inheritance since it then fires the triggers to maintain the tables.

The code I got from Victor did not include any data model changes or changes to the triggers so I am not sure where to go from here.

I know Victor is on vacation but if someone from Vienna has some information, or anyone else, and I am leaving this message as a record of my progress.

Thanks
Dave

Here is where I am experiencing some issues

2.5 seconds:
update acs_objects
set security_inherit_p = 'f'
where object_id = '1290509'

1.6 seconds
select acs_permission__grant_permission(
'182879',
'27310',
'read'
);

So it appears that these are probably updating acs_object_party_privilege_map and related tables via triggers with this which will get really slow if you need to update more than 1 object. Its already pretty slow, several seconds to change permissions on 1 object.

Very interesting results.

If you try to change the access policy on a bugtracker instance it calls set_inherit and grant_permission on every bug.

This can be hundreds of objects. The first ones took 1-2 seconds per query as the page runs by the end the last objects take 1 ms.

It appears Postgresql is loading the tables/indexes into memory and as that happens to seems to help.

Maybe this is not a signifigant issue unless you try to issue hundreds of small queries in one submission.

Try reducing random_page_cost to something lower than the default value 4, and raising the value of effective_cache_size to the size of your shared buffers + something reasonably large for the OS filesystem cache (and remember, modern disk drives cache too!).

We were having problems with unwanted sequential scans where indexed scans were obviously 50x faster in a production system I'm working on (NOT OpenACS/dotLRN) and by fiddling with these parameters I was able to persuade the planner to use indexed scans where it seemed to me it should.

I set random_page_cost to 1.0 ...