Forum OpenACS Development: APM auto-mount feature

Posted by Lars Pind on
On a related note (to removal of acs-workflow causing install on HEAD to break), I have a plan to enhance the APM with a feature that will let it automatically mount relevant packages during installationg. For example, the notifications package could be auto-installed at /notifications, general-comments at /gp or /general-comments, workflow at /workflow, etc.

The info about where to auto-mount singleton packages would be included in the .info file, and so interdependencies like this could be avoided in the future.



2: Re: APM auto-mount feature (response to 1)
Posted by Malte Sussdorff on
I'd suggest adding a page to the APM installation process which offers a form with default values taken from the info file for the to be installed packages. This should only be true for packages that suggest a default value and should be changeable by the user (obviously, why else to have a form :)).

Furthermore, if a package requires the user to have a look at it's parameters for it to be useful, we should link this up from the installation process as well.

Other than that, cool idea Lars !

3: Re: APM auto-mount feature (response to 1)
Posted by Ola Hansson on
Clearly a useful feature.

The .info files could provide default locations to auto-mount packages and maybe a simple form should appear with those defaults pre-filled (in text fields) next to each package in the package installation list (for non-core packages, at least). We would certainly like to have the freedom to choose our own locations, too, no?

4: Re: APM auto-mount feature (response to 1)
Posted by Ola Hansson on
It would of course be very nice if it played well with the "auto-install" way of installation, too (I am talking about the scripts Peter M developed as well as the one Jonathan M created - /auto-install.tcl) ...
5: Re: APM auto-mount feature (response to 1)
Posted by Don Baccus on
There's already an "initial-install-p" flag in the .info file which causes packages to be mounted during installation, using the package key as the mount point.

What exactly do you want the new feature to do that's different?  Naming the mount point would be useful (perhaps a "default-mount-point" tag?).  Better than mounting during installation would be to auto-mount when any package requiring the utility package is installed and enabled.

BTW I've just tracked down a bug in the code that autoinstalls packages during OpenACS 4 installation.  The package instantiation callback routine apparently isn't honored.  This is why you have to visit the "groups" admin page in acs-substite to get the main site's membership group created.  There's a hack on that page to configure the subsite if it's not been configured - which should be impossible because the callback does the configuration.


I'm going to look into fixing this bug and will then post some information about a big can of worms I've opened by inspecting this code and the old aD 4.1.1->4.2 upgrade packages found in sql/[postgresql | oracle]/upgrade.  For a good time take a look yourself :)

6: Re: APM auto-mount feature (response to 1)
Posted by Don Baccus on
Oh, damn.  Auto-mounting is controlled by acs-kernel/sql/*/acs-install.sql rather than the .info file.

And this of course is why the package instantiation callback is not invoked.  There's no way to call a Tcl proc from SQL.

If you're talking about controlling auto-mounting as well as auto-installation via the .info file from the APM rather than hardwired SQL files, then by all means yes, this should be done.

And it should be done by following the full mounting drill including calling the post instantiation callback proc for the package being auto mounted.

(actually "auto instantiated" though for a singleton package they're the same thing since you can't mount such an instance at multiple points, or at least shouldn't be able to)

7: Re: APM auto-mount feature (response to 1)
Posted by Lars Pind on

The main difference is that I was talking about auto-mounting packages that you install manually through the APM *after* the initial bootstrap-installation.

Synchronizing the two would be ideal, though.

The reason is that I'm sick of, every time I've reinstalled my development server and want to install and mount the packages I'm going be working on, there's too much work.


8: Re: APM auto-mount feature (response to 1)
Posted by Don Baccus on
Oh, OK, I see ...if you're going to be looking at this problem, though, it would be great if you'd look at the problem I've brought up.  "<initial-mount-p>" along with "<initial-install-p>" in the .info file, rip out acs-install, and do a proper APM mount as well as install so the post-instantiation code gets run.

Especially given that in 4.7 I want to complete the work that aD 50% completed to get rid of the registered users group hack, which was meant to be replaced with the subsite Membership group.  If that group's not built at mount time we can't complete the transition.  If we can complete the transition and add some decent UI for defining subgroups of a subsite group we'll be fairly far down the path of providing dotLRN-like user management in the acs-subsite context.

9: Re: APM auto-mount feature (response to 1)
Posted by Lars Pind on
Okay, now that I've looked at it a little closer, what I have in mind is this:

- Add 3 tags to the .info file and the apm_package_version table:


This is the name of a Tcl procedure, which, if defined, will get called after the package's data model, parameters, *-procs.tcl, etc., have been loaded, but before the *-init.tcl files are loaded. -init files are loaded once post-install and auto-mount is done. (We need to have sourced the -procs files, because that's where the post-install procedure itself will be defined.)


This is a replacement for the current "magic" post_instantiation proc. I figured it was cleaner to explicitly name both the post-install proc and the post-instantiation proc in the .info file. For backwards compatibility, we'll still look for a package_key_post_instantiation procedure. Also, the APM will, when generating XML files, check if there is such a proc defined, and if so, output the name using the <post-install-proc> tag (unless another post-install proc is already defined.


This would be the name of a folder to be created off of the root of the site-map for the new package instance created. If a folder by that name already exists, we just crap out. We're not trying to be fancy here, just make installation a bit more convenient.

Install sequence:

1) Install data model, parameters, etc.
2) Load -procs files
3) Run post-install proc
4) Create new instance if neceesary singleton or automount
5) Mount new instance if automount
6) Load -init files

In order to implement this, here's the code we'll have to touch:

- acs-kernel data model for both PG and Oracle, plus upgrade scripts for both. Bump up the version number.

- Add the extra parameters to PL/SQL proc, and to apm_package_install_version Tcl proc.

- apm_generate_package_spec to output these 3 extra tags

- apm_read_package_info_file should read those new tags.

- apm_package_install should add the 3 new parameters to its call to apm_package_install_version.

- apm_package_install should call post-install-proc.

- apm_package_install should instantiate and mount if auto-mount is present.

- Add the 3 new parameters to the acs-admin/www/apm/version-edit.tcl.

- apm_post_instantiation_tcl_proc_from_key should first look in apm_package_versions for the name of the proc to call, then if there's nothing there, try the normal package_key_post_instantiation proc.

- Fix the bootstrap-installer to use the normal APM procs to install the packages it installs. I haven't looked at this in detail yet, but I think we should get rid of acs-install.sql completely, and instead use auto-mount and post-install proc to take care of mounting, setting permissions, and starting the DBMS job to gather statistics. (packages-install.tcl, auto-install.tcl)

This is a larger effort than I thought it would be, so I'll postpone implementing this until next week when we're back.

Meanwhile, I'd appreciate your comments. I have very likely overlooked something.


Posted by Don Baccus on
My quick read is that this looks very good, Lars.  If you've missed something minor I'm sure it will be obvious when you're implementing it but I don't think you've missed anything big.

Probably need to do some error checking, i.e. if someone's named a post-instantiation proc that's not the magic name yet one of the magic name also exists you probably don't want to blithely continue on but to should give an error message and crap out.

I agree this doesn't need to be fancy, though - some simple sanity checking and an explanation as to why the package install and mount failed is sufficient.  Presumably we won't release packages to users in such a broken state ...

Posted by Don Baccus on
Lars ... when adding the capability to autogenerate trees of mounted acs-subsite packages to the "populate" package I've been working on, I ran into an issue that will need addressing if you're going to do the above work.

acs-subsite, at least, expects its "post-intantiate" proc to be called AFTER MOUNTING, not simply after instantiation.  It fails if you call the apm package installation proc, you have to call the site node package's install and mount proc.

We really need both "post-instantiate" and "post-mount" procs, as these are separate actions and packages may reasonably want  to execute certain code only after the package is mounted.  acs-subsite, for instance, finds its parent subsite in order to properly construct the composition relation that relates members of the just-mounted subsite to the members of the enclosing subsite.

Not many packages have post-instantiate procs so there won't be much package-whacking needed if you write the code to call both procs at the appropriate time.  If you do so, I volunteer to fix acs-subsite at least (just as soon as you get done) and maybe the rest of the packages, too (which might not require any work).