Forum OpenACS Q&A: Delaying Aolserver startup until network/postgresql are ready?

I've go two OpenACS installations, and neither will startup correctly when the server reboots -- you have you restart aolserver after restarting the server or you get the "OpenACS Installation: Error ... Possible causes might include:" page. This means I have to worry about leaving the server unattended.

On one server, which has both OACS and postgresql on it, it's because postgresql isn't up yet when OACS first tries to connect to it. That doesn't surprise me because their startup priorities are both 98. postgresql's should be a lower number -- any suggestions?

However, the second server is puzzling me. It connects to postgresql running on the first server. I'm not restarting them both at the same time, just the second one, but what I get is this:

[20/Mar/2003:14:00:26][286.16384][-main-] Notice: dbdrv: opening database 'postgres:pekoe::bma'
[20/Mar/2003:14:00:26][286.16384][-main-] Notice: Opening bma on pekoe, port
[20/Mar/2003:14:00:26][286.16384][-main-] Error: Ns_PgOpenDb(postgres):  Could not connect to pekoe::bma: 
could not connect to server: Network is unreachable
        Is the server running on host pekoe and accepting
        TCP/IP connections on port 5432?
[20/Mar/2003:14:00:26][286.16384][-main-] Error: dbdrv: failed to open database
'postgres:pekoe::bma'
[20/Mar/2003:14:00:26][286.16384][-main-] Notice: Couldn't allocate a handle from database pool "pool1".

If I then restart aolserver, all is fine. It's as if the network isn't up yet when aolserver first tries to connect to postgresql. The answer to "Is the server running on host pekoe and accepting TCP/IP connections on port 5432?" is "yes," this server just can't see that yet. How can I delay aolserver startup until the network and/or postgresql are ready?

my stupid solution when running both server progs on one machine is to let aolserver wait for 10 seconds in the startup file :) i do all the auto startup things with daemontools.
i think it should not be too difficult to modify the startup file so that it checks pg-connectivity every 5 secs and let aolserver start when a pg-connection is available.
On one server, which has both OACS and postgresql on it, it's because postgresql isn't up yet when OACS first tries to connect to it. That doesn't surprise me because their startup priorities are both 98. postgresql's should be a lower number -- any suggestions?
Yeah, change the /etc/rc?.d/ priorities so Postgres starts first. What else is there to ask?

Andrew is right, either raise the OACS to 1:99 or lower pg. AOLserver should start up after everything else.

Also, make sure your ssh is one of the last to go down before networking is removed. That way when the shutdown doesn't go right, you can login and fix the problem.

The problem I've been having seems to be with daemontools and supervise-scripts. supervise-scripts (http://untroubled.org/supervise-scripts/) is a set of utilities that make it easier to start and stop services running under daemontools, but they also seem to cause those services to start up before everything else.

I've also added this as a documentation bug. This seems pretty critical for production environments, yet our default documentation doesn't take this into account. Even for developmental servers, it's an annoyance.

See: https://openacs.org/bugtracker/openacs/bug?bug%5fnumber=706