Forum OpenACS Development: Do we need to duplicate file listings of APM packages in info files and db?

1: Do we need to duplicate file listings of APM packages in info files and db?

Posted by Peter Marklund on 02/18/03 10:18 AM

It just occured to me that I don't quite understand why we enumerate the files of an APM package in our info files. Our file types are always determined by the file path and the files of the package are simply the files its root dir contains.

It happens quite a lot that file listings in info files are out of sync with the file system. Either the programmer will forget to add/rermove files in the APM or he might forget to generate the XML file afterwards.

So when is synching an issue? Well in particular when installing/upgrading a package only sql files and message catalog files listed in the info file are sourced. Recently I added loads of message catalog files to a range of packages but those files were not added to the info files and thus do not get sourced on installation (adding them via the APM UI would have been quite tedious).

I think we should change the APM to always get files from the filesystem directly and use those files that match our clearly defined filepath patterns for different file types.

What do people think?

2: Re: Do we need to duplicate file listings? (response to 1)

Posted by Don Baccus on 02/18/03 08:26 PM

This has been an annoyance to me, too, because some of the code digs through the filesystem (when sourcing the Tcl library) while others ignore anything not in the .info file (as you point out)

At minimum things should be consistent, either the .info file significance should be dumped or everything should be driven by it.

I'd label this a triage issue, i.e. there's a fair amount of code to whack away at, and it's not near the top of my list of things to whack away at in the near future.

On the other hand I really appreciate all the improvements to the APM you're making. If folks agree with the idea of driving things from the filesystem depending on the standard file patterns and you're sufficiently annoyed by the current state of affairs that you'll go ahead and rewrite it ... sure!

3: Do we need to duplicate file listings of APM packages in info files and db? (response to 1)

Posted by Peter Marklund on 02/20/03 09:34 AM

Don,
yes, consistency is another argument for the change that I am proposing.

As the saying goes, silence gives consent... As a first step I'll make the APM get message catalog files from the file system and try to do the same for sql files. If I find the time and inspiration I might attempt to remove the file listings in the info files altogether.

I'll post any progress in this thread.

4: Re: Do we need to duplicate file listings of APM packages in .... (response to 3)

Posted by Jeff Davis on 02/20/03 10:39 AM

Peter, you might modify my qa tool to check that the automatically generated file manifest for apm matches what is in the existing info files just to make sure the current code does not require any hand tweaking to get things right. Also, you should check and make sure it properly picks up upgrade files in both the sql/DB/ and sql/DB/upgrade/ directories.

At some point I would like to add mapping url stubs to different files which would need something like a manifest but I think until we do provide some useful functionality via a file manifest it's better not to have it at all.

5: Re: Do we need to duplicate file listings of APM packages in info files and db? (response to 1)

Posted by Peter Marklund on 02/22/03 02:42 AM

Here is what I've done so far (not committed to cvs yet). I've added a new proc called apm_get_package_files -file_types $file_types -package_key $package_key that finds file paths in the filesystem and is supposed to be the canonical proc for that. It relies solely on ad_find_all_files, apm_include_file_p (exlcludes CVS and backup files), apm_guess_file_type and apm_guess_db_type.

I changed sourcing of sql files (install and upgrade) to use this proc instead of the info file. I also made catalog files be sourced directly from the filesystem and everything seems to be working ok.

I guess I should conclude this change then by removing all file tags in all info files, change the info file xml procs to not read/write files, remove all file db procs we don't need anymore, remove the apm_package_files table, and change the file manage page in the APM.

I feel pretty confident that all this stuff is superfluous now and can be removed. It's just that in terms of number of lines it's such a massive change that I'm a little reluctant to do it...