Forum .LRN Q&A: LRN Application Status Discussion - File Storage

I will be starting a thread for each of the current applications in the .LRN distribution. This is to provide feedback to the user advisory board. The first application is "file storage" What is the state of file storage? What improvements need to be made?
Collapse
Posted by Malte Sussdorff on
Nice Idea Al. Okay, here is what I envision:
  • All files shall be indexed when uploaded and their content stored as clear text along with the object created, thereby allowing full text search within documents using tsearch2 or intermedia, regardless whether the files are stored in the datbase or not.
  • Use bittorrent technologies. The idea of using Bittorrent hash sums to save some diskspace and link files together has been in existance for some time now. To sum it up:
    • Each file will store an additional bittorrent hash sum.
    • If a hash is already in existance, there will be only a link to that content instead of uploading a new file.
    • OpenACS acts as a bittorrent client for this file. This way a file can be retrieved using P2P methods.
    • It is not necessary to upload a file to the server. Giving the server the bittorent link will enable OpenACS to download the file from the file sharing network or just promote the link. This way you can have closed user groups who can share the file.
    • The hash value can be checked against lists from the RIAA or the Movie Association to detect illegal content. Same can be done the other way round (if we detect illegal content deliver the HASH value back to the RIAA).
  • Use more icons for the file types.
  • Use color coding in various dimensions (this is actually something we might consider in other packages as well).
    • Rating (if the rating module was installed)
    • Permissions (public, group wide, some people, private)
    • Relevance (e.g. calculated how often the file is reused in communitites that I'm a member of)
    • Last time changed
    Keep in mind though, that color coding is seperate from sorting.
  • Easy integration with SMB shares (I know WebDAV is cool, but I imagine most sysadmins would like to have the ability to link OpenACS files into an existing SMB Domain and link the domain controller together with the WebDAV shares).
  • Last one, though I did not look into it to clearly (but got as feedback in one training): Easy way to switch between live revisions and the posibility to restrict setting a version live only to the owner of the file
Sadly for the UAB these recommendations are mostly in functionality along with technical implications, not at all from a User experience point of view.
Hello Al,
this is really good idea, well, probably ain't the best person to suggest UI improvements, though I dont like that much the UI of FS, probably I'll need to look at other UI's for comparission.

I expect to get some end users together and link them to the UAB so we can get feedback from them, which usually are the most valuable comments.

Collapse
Posted by Alfred Essa on
Malte, some great ideas. I encourage you to take a look at the "Microgrants".
Collapse
Posted by Bruce Spear on
I've just sorted a few dozen files in a class where students had not followed consistent naming conventions, where a half dozen files are being added each week, and where I faced a basic problem of organizing group work. I created some directories and, to move things around, ended up doing a lot of clicking and dreaming of drag-and-drop, or at least of selecting bunches of files and then making one move. Is drag-and-drop difficult to do with openacs? Are there other simply sorting principles/methods we might consider?

I also think notifications are the cat's meow, and I'm wondering if others think adding a notification feature to the file storage makes sense: that when you add a file you are given a choice to post a notice in an existing forum, for example, so you don't have to go to all the trouble of copying the url, then going into the forum and building a link. I also like this idea as it suggests further integration of existing communications functionality, and specifically, with notifications, the encouragement of timely responses as a conversation moves forward.

Collapse
Posted by Matthew Geddert on

the UI for fastmail.fm's file-storage aspects is pretty good if you ask me. The three things I think are best that could easily be included in file-storage are:

  1. it allows renaming of files en-mass (there is a form field in the file list with the name in it - you can edit it directly and then update the whole list)
  2. it also allows easy copying/moving of files between folders. We could use listbuilder for this, so there can be checkboxes on the left for all files or selecting a few and then there could be a drop down list somewhere so you can select which folder to move/copy to
  3. you can go to an upload page that lets you upload 10 files at once (that aren't zipped together)... i.e. bulk upload.

I think Malte's ideas are great. Instead of bitorrent we could use md5sum (only because its more common)... but which one we use doesn't matter to me. I really like the idea of storing a text version for searching.

The most sorely missing technical thing if you ask me is quotas. I think there should be three ways of doing this, by the:

  • file-storage instance
  • file-storage folder (which might be the way we want to do it since an instance has a root folder, and we can implement limits on that folder)
  • user on the system... i.e. no matter where you upload the files john doe can only own up to 15MB of files or however else we want ot set that up. This would be a sysem wide configuration not a per instance, and it would require the ability to ascribe custom levels for different users so it might not be worth the effort. It might be easier to just say in a users "my files" folder they may only have 15MB of files or whatever the limit you want is.
Collapse
Posted by Dave Bauer on
The latest release of file-storage 5.1.2 support bulk moving and copying. I will see if there is a publicly available site for a demo.

The new file-storage also uses listbuilder so that is already taken care of. It also support WebDAV which allows for drag and drop operations from your desktop.

Thanks to MIT Sloan for supporting these improvements to the file-storage package.

Collapse
Posted by Deirdre Kane on
Sloan has file storage notifications in place, to which users can opt in/out.  Though the notification format needs a lot of work, a notification is sent for each file upload.  One problem with this occurs with the upload of a zipped file - a notification is sent for each file, not just one.

Another great improvement of our upgrade is that admins can select all files in a folder to be deleted or delete a folder and its files.  What a relief that was.

Collapse
Posted by Matthias Melcher on
We have discussed the filestore UI in the UAB, and my
conclusion is as follows.

The main source of end user confusion is the application's
conflicting pursuit of two objectives:

1. Lightweight content delivery / presentation. This is

  - why filestore is using so called "titles" in addition
    to file names;

  - why its default action (when clicked) is displaying the
    content in a dotLRN framing (of navigation bars) rather
    than in a clean window with its own layout, and
    suppressing the original html title of html files;

  - why filestore's focus is far away from the files'
    originals, e. g., URLs without their proper locations,
    and files with unfamiliar long mime-type names
    ("application/vnd.ms-powerpoint") rather than their
    proper extensions (".ppt") and icons;

  - why filestore uses its own version administration,
    confusing not only people who use their word processors'
    built-in revision tracking for collaborative editing, but
    also ordinary users asked to enter revision descriptions.

2. Storage, and interfacing to more sophisticated content
    delivery and presentation, which needs

  - relative LINKING (e. g., from a lesson overview html file
    to indvidual html pages and embedded pictures,

  - granular addressability of the reusable objects (e. g.
    instead of just entire pdf dumps, individual slides to be
    discussed in the forums during the constructive
    understanding process, or for interactive exploration
    of slides being clickable maps).

  - shorter URLs, to have at least a chance to handle them
    without intervening line-breaks.

The conflict between these two approaches is not only reflected
in lots of terminological and other confusion in the user
interface (e. g., names vs. titles,
https://openacs.org/forums/message-view?message_id=135270
but especially in various bugs like
https://openacs.org/bugtracker/openacs/com/dotlrn/bug?bug%5fnumber=929
https://openacs.org/bugtracker/openacs/com/dotlrn/bug?bug%5fnumber=2072
where the designed behaviour is controversially disputed.

If the two above approaches cannot be combined in a common
behaviour design, it is IMO necessary to separate them at site
setup time using a configuration parameter where the
administrator could specify "Switch all the confusing title and
versions handling completely off and use native filenames and
relative linking".

This principal conflict is IMO so important that I wanted to
post the issue before we turn to other and minor problems, among
which there is another cluster of problems grouping around
permissions (questionable defaults and wording for the group's
and uncovered folders, and permission setting dialogue).
Minor UI distortions (like line breaks in byte counts) are
not yet enumerated.

Collapse
Posted by Alfred Essa on
Matthias, Thanks for the feedback. I want to try and see if I understand each of your comments.

"- why filestore is using so called "titles" in addition
    to file names;"

Isn't this a standard in many applications? For example, I use flickr.com for photographs. At the time of upload, it asks me for a title for the file, which I can choose to be the same or not as the filename. One of the useful UI improvements with 2.1 (perhaps it appeared earlier) is that I can see the both the title as well as the actual name of the file (in grey immediately underneath).

Some users consider this feature useful i.e. to be able to present a more "human readable" version of the filename in the title without having to alter the original filename.

Collapse
Posted by Alfred Essa on
Matthias: "why filestore uses its own version administration,
    confusing not only people who use their word processors'
    built-in revision tracking for collaborative editing, but
    also ordinary users asked to enter revision descriptions."

Collaborative editing is not the same as version control. It's a strength of OpenACS | .LRN that it provides version control i.e. the ability to manage multiple versions of arbitrary objects, including document, through the content repository. Collaborative editing is something different entirely: you can see who made which changes but you are still working with a single document.

IMHO this is not a UI issue, but a user education issue. We don't see this as a problem among our users.

<blockquote> Some users consider this feature useful
</blockquote>

I do not doubt that. But others find it confusing.

Collapse
Posted by Dave Bauer on
Please install or try out the same version of the file-storage package. Perhaps we can install a demostration somewhere so we are all talking about the same thing.
Sorry to tune in so late - I've been swamped with other work lately.

Anyway, lots of good ideas & surely the issue with naming is one we need to sort out.

As per Dave's idea, I'll install a site with the latest oacs-5-1 filestorage which we can use as a 'reference platform' for filestorage. Note that the site won't be very fast, as it'll run on an already very busy dev server - so for performance related testing this site will be no good.

I'll post the url here so interested parties can register.

Rgds,
Jeroen

All,

At http://oacs-fs.baasbartels.org/file-storage/ you'll find a brand new, clean oacs-5-1 install, with a file-storage instance up for testing.

Rgds,
Jeroen

Collapse
Posted by Michael Hebgen on
Up to now we have many interesting views on how file
storage should be improved.

Let me add a naive view of an author generating data and
putting these data into the file storage.

How should I do it? What do I have to consider for

- the generation of data,
- the uploading process,
- a (partially) replacement of files,
- the using part (by my students) and
- the download feature?

As long as we speak about single (maybe large) files
almost everything is working. The problem starts with
a series of interconnected files where interlinking
is a key feature needed in file storage.

Authors do create the data on there own computer in a
(sub)tree with relative linking - this is the way to
publish it on the web. And we have a mass of these trees
and we must support them within .LRN.

How to upload this tree into .LRN or to replace parts of
it within .LRN keeping the relative linking in good order?
Should I use single file upload? or zip-file upload? or
web-dav? which upload feature can GUARANTEE the relative
linking to work - one or all? Similar problems apply to
the usage and "reverse" problems to the download process.

We need consistency between the external view and the
internal databased management of files - and an easy way
for authors to handle to above processes.

Michael

Collapse
Posted by Malte Sussdorff on
I think the content creation as it was mentioned by Michael should not be part of File Storage. Either you can create content using e.g. the Reload Editor and stuff the result into LORS, which should keep track of the relative linking.

Alternatively it might be nice for LORS to be able to import a ZIP file of HTML content, relatively linked, and generate the SCORM package that is generated out of this, automatically.

A totally different approach would have to be taken if you plan to use this not as course material but e.g. for your personal website. In this case, it might make sense to get ETP (or whatever CMS you fancy) to take the contents of the Zipfile, upload all images into the content repository, store the HTML internally and provide the linking this way.

Last but not least, I think the naming of file storage in 5.1.4 already supports this. If you use relative linking it should not break if you upload a ZIP file. If it does, this is a bug :).

*to give a name of an upload is important, if not entered should put the name of file, therefore, the name of the upload should be shown bigger than original filename (but this will be always a confusion for a lot of users, so better to have the ability to turn off/on to show both at the same time).
*version is important, but most of the time misleading to casual users (not advanced ones), then I suggest a parameter to just show the last one, or turn on the versioning UI for advanced users.

The two above, in real life are quite a trouble for many users.

*relative linking is achieved through lors (ims cp), in my opinion we shall not reinvent the wheel, ims-cp/scorm rte was created of it and more ...
*shorter urls in part will be achieved when we change the way how the class instances are created (using keys instead of names), we have already that a UG and will contribute it on january

Collapse
Posted by Ernie Ghiglione on
<blockquote> Alternatively it might be nice for LORS to be able to import a ZIP file of HTML
content, relatively linked, and generate the SCORM package that is generated
out of this, automatically.
</blockquote>

Although possible, this could be a bit tricky. Say for instance you have the following files in a zip file that you want to share with your students:

[ root folder]
|-> default.html
|-> intro.html
|-> flash_animation.swf
|-> xmlfile.xml
|- [ images folder ]
    |-> randompic1.jpg
    |-> randompic2.jpg

Given this structure, LORS can't really guess or know where the entry points are. We could say, well, maybe the default.html might be the entry point, but again, that's an assumption that we would make.

Now Michael has a very good point here and probably we should clarify the difference between learning objects and plain content.

LORS is good to handle learning objects aggregations (courses) and/or individual learning objects. But if the zip file represented above isn't a learning object neither a course, but a piece of content that I want to share with my students, I shouldn't have to convert it into a learning object by using Reload, creating a content packaging, edit metadata, etc, etc... that'll be rather tedious if I have to do this for each hand out in my class.

Probably an example can ilustrate this better. Say for instance I'm a teacher for a class and on the way to uni on the bus I read this article that relates to the subject I'm teaching today. I might want to add this article as "suggested readings" for the subject. This is not necesarily a learning object as such, but just a random article that might content various pieces that you want to share with your class.

I know -for sure- that File-storage works well linking different pieces of content together using its index.vhu. As a matter of fact LORSm relied on it until recently for all its content delivery -now LORSm has its own.

So summarizing, I think it's good that file-storage still maintains its index.vhu for linking random content that might not necesarily need to be part of a course. It's always good to have that as a choice.

Thanks,

Ernie

Collapse
Posted by Michael Hebgen on
thanks to earnie who made the point. we have non-course
material in a community system, e.g. a presentation or
suggested/additional material. this kind of material
is usually html and created on a pc (windows, linux, mac),
it consists of files and folders interlinked.

it works on the local pc file system and does NOT work
when uploaded into the openacs/dotlrn file store. i do
not think this is what we want to have.

as far is i understood the design of the file-store
(it should work like the internet explorer) and the
new web-dav access this is a major bug when relative
linking across directories does not work.

because we have masses of this kind of material it is
at least what we need TODAY.

michael

Collapse
Posted by Dirk Gomez on
From the UI perspective I would agree that it is a major bug.

Technically it is a rather minor issue - something like two mandays. All you need to do is to add the "j" option to the unzip comment (which was there at some point, but then commented out) and then declare all the files in file-storage.

I had some code that was halfway finished but threw it away. Somebody else may have solved this problem though. Anybody out there?

Collapse
Posted by Ernie Ghiglione on
<blockquote> I had some code that was halfway finished but threw it away. Somebody else may
have solved this problem though. Anybody out there?
</blockquote>

Yeah, I'll do it. I've done a function in LORS that basically given a directory it adds all the directories and files recursively into the CR.

Show I add this to file-storage on HEAD?

Ernie

Collapse
Posted by Malte Sussdorff on
<blockquote> Yeah, I'll do it. I've done a function in LORS that basically given a directory it adds all the directories and files recursively into the CR.
Show I add this to file-storage on HEAD?
</blockquote>

As this is a bug (not working recursive directories in file uploads) please fix it in 5-1 as well.

Furthermore we need to make sure that File-Storage and WebDAV work exactly like Windows Explorer to avoid user confusion. This means if you add a file in webdav that depends on other files in a subfolder (e.g. a large word document, HTML pages) it should work regardless if you add the file cum directories via WebDAV or File-Storage and the view should work as well. This includes Bug #2157 (https://openacs.org/bugtracker/openacs/bug?bug%5fnumber=2157) which seems to me to be a leftover from the name/title misconceptions and changes that happened within FS.