Forum OpenACS Development: Re: Possible issue in file storage

The files are uploaded using a custom procedure which takes a big zip file and extract songs from it.

I could reproduce the situation in the regular gui putting some files which names contained spaces into a zip file and then using the 'official' zip upload feature.

The problem comes from the fact that sanitization takes place for files uploaded singularly by the form (it's a form feature to sanitize names), but nothing happens if we 'inject' files using a zip. They make it to the file storage untampered.

The two strategies we have are either to sanitize every file before it enters the system, moving sanitization into the file-saving procs, or let them be saved however they are, remembering that decoding should take place when they are retrieved (this could be put into procs too).

In my humble opinion I think the second approach is the most feasible, because it is easier, backward compatible and gives us the less constraints about filenames format.

What do you think about it?

As I suspected ad_conn path_info is urldecoded automatically.

I created a test on one of my sites like this.

url.vuh

set path_info [ad_conn path_info]
ad_return_complaint 1 "Path Info = '${path_info}'"

Then i visited /url/this is a test %26 %26 (foo)

and the result is

We had a problem with your input:

    Path Info = '/this is a test & & (foo)'
Please back up using your browser, correct the above error, and resubmit your entry.
Thank you.

The contents of ad_conn path info are url decoded properly.

What version of OpenACS are you using? My test is a version of 5.7, I believe.

The situation is more complicated as it seems at first sight.

a) The behavior should be the same for uploading files via zip files, or a single files. Therefore, one should sanitize always or never.

b) I had a hard time to find, where the "view" method is used. On OpenACS.org, all file-links are "download" links.

c) you say, that "download" works everywhere, but "view" has problems. Why are the files accessed differently in "view" and "download"?

d) urlencoding: the RFC 3986 defines url-encoding differently for the "path" and the "query" part of the URL. Aolserver ignores the difference, and performed everywhere (incorrectly) the "query" variant. In OpenACS+naviserver, the urls (and therefore the path_info) are decoded correctly, but not, if someone encodes the path-part of the URL with the query-part encoding. Therefore, the place, where the "view" link is generated, should use the "path" encoding.

How do you get the "view" link? is this a custom code?

Antonio,

Thank you for these pointers. Normally, i try to avoid file-storage. I've added a function ad_urlencode_path to handle the differences between aolserver and naviserver and added these to the folder_chunk in file-storage (btw., the encoding was not done consistently there either, some branches used the encoding for $title, some not).

This version is no supposed to work with both servers for viewing sanitized and non-sanitized names.

I've not touched the inconsistency of the names for uploading single files vs. uploading via zipped files. It seems to me, as if the original intention was to permit arbitrary names in the name attribute of the content repository, but to convert these when downloading via fs::get_file_system_safe_object_name, but folder-chunk does it the other way round, sanitize the file_upload_name with fs::remove_special_file_system_characters, which is a helper of fs::get_file_system_safe_object_name. This would mean that the zip-behavior is as intended.

i would not wonder if one could get as well different names when downloading single files vs. downloading via zip.