Forum OpenACS Q&A: Backups with OACS4-5

Collapse
Posted by defunct defunct on
Folks,

Has anyone successfully restored a backed-up oacs-4-5 database?

If so, what form of pg_dump did you use and what procedure?

The reason I ask is that when trying to restore from a pg_dump I get
only a partially functioning server. I get some of the main pages, but
when I attempt to navigate to admin pages lets say, I just get an
error from the system which appears to be looking up site node
mappings.

Suspiciously I get an error early on in the restore from backup that
complains:


Function 'tree_ancestor_keys(varbit, int4)' does not exist
        Unable to identify a function that satisfies the given
argument types
        You may need to add explicit typecasts

Which maybe related...

I've tried a couple of alternative such as a straight

pg_dump openacs >outfile

pg_dump -Fc -o -b >outfile

and so forth, but similar results each time...

Any advice for this Oracle-poisoned mind?

Thanks Simon

Collapse
Posted by Dave Bauer on
Try psql -f packages/acs-kernel/sql/postgresql.sql

I think I had to do that when I dumped and restored my database. It looks like some things where exported in an order that causes problems with openacs.

Also which version of postgresql are you using?

Collapse
Posted by Dave Bauer on
I forgot. I just did pg_dump databasename.
Collapse
Posted by Don Baccus on
Yes, it looks like the old ordering bug problem with pg_dump, grrr.  Simon, did you try this with PG 7.2?  It might do a better job though if you're using PG 7.2 obviously it didn't do a better job ...

We'll be able to make this a bit easier on folks when we abandon PG 7.1 because we can then rewrite all of postgresql.sql to use "create or replace function" and then arbitrarily tell people to run it before restoring backups without worrying about errors later.

That is ... assuming PG 7.2's dump/restore is smart enough to use "create or replace".  Hmmm...come to think of it, it's probably not.  Drat.

Collapse
Posted by defunct defunct on
Many thanks... executing the postgres.sql file has done the trick it would seem 😊

Don - Yes this is on PG 7.2, Mandrake 8.1

Thanks
Simon

Collapse
Posted by Jun Yamog on
This is a real pain.  Especially if you don't know why is this happening.  It puts OpenACS in a bad position of not being a restorable system.  Although this is not postgres specific I believe.  In ACS 3.4.10 I get 3 broken views using export and import of Oracle

Can we have backup and restore as part of testing?  I think its very important.  Also maybe some body is kind enough to place in the docs a "How to backup and restore OpenACS sites".  Maybe this should include some explanation of the content-repository-content-files dir.  That this dir needs to be restored too.  This would help a lot of new users.

Collapse
Posted by defunct defunct on
Actually thays a very salient point. Backup/restore is critical for any real service, so I think your quite correct in that it needs properly testing.

I shall add it in to the work the test team are currently doing.

Are we saying there is a bug at present though? I.e. should this be logged in the SDM?

I also agree a backup doc would be good. I would have a go myself, but that may be the blind leading the blind as I'm no Postgres DBA..

Collapse
Posted by Jade Rubick on
I agree that this should be a part of the documentation, but until someone steps up and volunteers, we're not going to have it. I think there is something in the docs by Don about how he set his backups up. It could be fleshed out.

We'd probably need a separate doc for PostgreSQL and Oracle. I always found Oracle backups problematic. I could never find step by step documentation on how to set up backups -- and the export/import never worked for me.

I actually do it now by brute force -- I have a huge hard drive that backs up the entire /ora8 directory. And then I also run a tape backup of the entire computer. There is some history there, but cleaning up if the Oracle hosed my data  would be a pain. Not good dba practice!

Collapse
Posted by Tilmann Singer on
I think it is not the pg_dump ordering problem that shows up here (well, maybe that too) but the fact that in PG the function tree_ancestor_keys(varbit, integer) is not restorable. postgresql.sql executes create_tree_ancestor_keys(), so after calling it the function exists and the restore works fine.

Not that this would change anything - psql -f postgresql.sql does the trick.

Collapse
Posted by Don Baccus on
Oh, right, I wrote that and I didn't even remember it working that way :(

It's done this way to kludge the thing to work for both PG 7.1 and PG 7.2.  When we eventually drop PG 7.1 then I can simply do a "create or replace" of the function inline (and PG 7.3 will probably allow the recursive definition anyway meaning even that level of kludge won't be necessary).  If pg_dump's smart enough to issue "create or replace" rather than "create" then restoring would work fine for PG 7.2 in that case ...

Collapse
Posted by Jun Yamog on
I dont think it should be in the SDM since this is not a OpenACS bug but a limitation of the database.  I still believe just placing a small secton on the doc is good enough.

Hint Hint someone who has access to the docs please just put a small section regarding the Backup and Restore of OpenACS 4.5

Collapse
Posted by Jun Yamog on
FYI only.

The view party_approved_member_map view is not dumped correctly.  So
what I did I just run the
packages/acs-kernel/sql/postgresql/rel-segments-create.sql.

Although this code is a few days before the beta code.  I use also
used PG 7.1.3

Collapse
Posted by Ola Hansson on
Hi Jun!

I did as you suggested and ran rel-segments-create.sql after the db had been restored. That is what you meant, right?

Workflow's index page and thus glossary's index page is nonetheless inaccessable after I resore. Do you too get this?

Cheers,

Collapse
Posted by Jun Yamog on

Hi Ola,

Using OpenACS 4.5 beta 1 with Postgres 7.1.3. The steps that I use to backup and restore.

Backup

  1. pg_dump -d [db_name] > [dump_file]

Restore

  1. createdb [db_name]
  2. createlang plpgsql [db_name]
  3. psql -d [db_name] -f [acs_root]/packages/acs-kernel/sql/postgresql/postgresql.sql
  4. psql -d [db_name] -f [dump_file]
  5. psql -d [db_name] -f patch.sql

My patch.sql contains this:

create view party_approved_member_map
as select distinct segment_id as party_id, member_id
   from rel_seg_approved_member_map
   union
   select distinct group_id as party_id, member_id
   from group_approved_member_map
   union
   select party_id, party_id as member_id
   from parties;

create view party_element_map
as select distinct group_id as party_id, element_id
   from group_element_map
   union
   select distinct segment_id as party_id, party_id as element_id
   from rel_segment_party_map
   union
   select party_id, party_id as element_id
   from parties;

create view wf_user_tasks as
select distinct ta.task_id,
       ta.case_id,
       ta.workflow_key,
       ta.transition_key,
       tr.transition_name,
       ta.enabled_date,
       ta.started_date,
       u.user_id,
       ta.state,
       ta.holding_user,
       ta.hold_timeout,
       ta.deadline,
       ta.estimated_minutes
from   wf_tasks ta,
       wf_task_assignments tasgn,
       wf_cases c,
       wf_transition_info tr,
       party_approved_member_map m,
       users u
where  ta.state in ( 'enabled','started')
and    c.case_id = ta.case_id
and    c.state = 'active'
and    tr.transition_key = ta.transition_key
and    tr.workflow_key = ta.workflow_key
and    tr.trigger_type = 'user'
and    tr.context_key = c.context_key
and    tasgn.task_id = ta.task_id
and    m.party_id = tasgn.party_id
and    u.user_id = m.member_id;

Before doing restore step #4 you want to check the "connect" string of the dump file and make sure it the correct user. You may also want to use "psql -d [db_name] -f [dump_file] 2> error.log". The after the restore do a "grep -i error error.log | grep -v "already exists", you should get 3 errors which the patch will fix. Good luck I hope this has been helpful to you and others with the same problem.

Collapse
Posted by Ola Hansson on
Jun, thanks, it is superplusgood!

Maybe it's a good idea to have this in the "next steps" section of the install docs?

Collapse
Posted by Jun Yamog on
Hi Guys,

I hope those who have access to documentation would put in the post above. It would really help.

As for testing I have done this to several OpenACS 4.5 sites.  This sites contains, ETP, file-storage, bboard, news, and subsites.  I have not tested it for others packages.

Maybe add on the fact that [acs_root]/content-repository-content-files needs to be restored too.

Anyway the above info should be very helpful to others and hope that its Documented.

Collapse
Posted by David Geilhufe on
I added a reference to this thread in the comments on the documentation at /doc/openacs-4/acs-admin.html

I don't have time to do the research and draft a skeleton of what the backup chapter/ section might contain, but anyone else with a few free moments could draft something and post it in the comments to the documentation. Remember - the comment functionality allows you to attach a file, so you can write whatever you want of whatever length, in whatever format!
Collapse
Posted by Jun Yamog on
Thanks David for the link, I have put in my comment there hopefully Roberto or Vinod can cut and paste into the docs.  Maybe we can have the patch.sql as part of the distro.
Collapse
Posted by Rafael Calvo on
How about the BLObs? I posted this in another thread here