Forum OpenACS Development: from File System to Content Repository

Collapse
1: from File System to Content Repository
Posted by Iuri Sampaio on 04/24/10 04:48 PM
I am trying to upload a structure of directories and files from a file system (linux debian) to content repository.

The way of manipulating folder is very straight forward. No problems with that. However, creating files is a bit tricky since i don't entirely understand the format of it.

I am using file-storage/www/file-add.tcl as a guide to write my script.
file-add.tcl copies the new file to a temporary directory /tmp/, treats it, and then upload it to CR.

ad_form -html { enctype multipart/form-data } -export { folder_id lock_title_p name } -form {
file_id:key
{upload_file:file {label \#file-storage.Upload_a_file\#} {html "size 30"}}
}

What is in upĺoad_file field? I printed it out but it only shows the filename.
how does it actually get the file and manipulate it to insert it into CR?

ps. i searched APIs to do the job but i haven't found any. There only APIs to do the other way around. From CR to FS

Collapse
2: Re: from File System to Content Repository (response to 1)
Posted by Nathan Lunt on 04/25/10 03:09 AM
content::folder::new and content::item::new are all you need. You need a recursive procedure that reads a directory tree, then you do this:

1) Add root folder using content::folder::new using the file-storage root folder and package id if you want the files to go to a file-storage instance.

2) Get all the child folders and files under that folder and add each item with content::folder::new or content::item::new as required, again specifying the correct root folder and package id.

3) If you come across a folder, go to step 2 on that folder.

4) Repeat this process until all folders and files are added (no more child folders or files exist).

That's an over-simplification perhaps, but that's the basic idea.

API Docs you need:
content::item::new

content::folder::new

Collapse
3: Re: from File System to Content Repository (response to 1)
Posted by Dave Manginelli on 04/25/10 05:03 PM
No need to reinvent the wheel here.  This functionality already exists in folder-add.tcl as referred to in this thread:
https://openacs.org/forums/message-view?message_id=467566

I don't have my notes in front of me but I seem to recall that all I had to do was directly type in the URL of
http://mysite/myfilestorageinstance/folder-add and I was greeted with an input box in which to input the directory I wanted to import.  I loaded almost 10 gigs in one whack.  (The procedure does assume that the files to be imported are in a local filesystem on the server.  However, if you
have enough time on your hands I would wager that something like sshfs would be acceptable if the files must be loaded remotely.)