Forum OpenACS Development: File-storage upload slow in OpenACS 4.6.3

We're experiencing very very slow file storage file upload on collaboraid.biz, which runs 4.6.3.

The slowness occurs in the call to

<fullquery name="new_lob_file">
      <querytext>
      select file_storage__new_file (
                :title,                -- title
                :folder_id,            -- parent_id
                :user_id,              -- creation_user
                :creation_ip,          -- creation_ip
                true                    -- indb_p
                );
      </querytext>
</fullquery>

in file-add-2.tcl. I've tried calling this proc manually in psql, and it virtually never ends.

This only happens once the DB has some data in it. It used to never happen.

Before I start digging into this, I'd like to know if anyone is aware of this problem, and have already dug into or even fixed this in a later version.

Thanks,
/Lars

Collapse
Posted by Peter Holzleitner on
Have the same problem here.  Did anyone ever figure this out?  Lars?

Thanks
--Peter

Collapse
Posted by Lars Pind on
I didn't take the time out, no.

DaveB, are you load-testing file-storage now that you're working on it?

Maybe you could get Don and/or Jeff to help you with that.

/Lars

Collapse
Posted by Dave Bauer on
Lars,

Thanks for the pointer. I will look into it, and check with Don. I hadn't realized the upload speed issue was a database problem.

Collapse
Posted by Dave Bauer on
What is the setting of the parameter  StoreFilesInDatabaseP ?
Collapse
Posted by Lars Pind on
StoreFilesInDatabaseP = 1
Collapse
Posted by Dave Bauer on
Very strange.

I know that Jeff Davis put in a few fixes to speed up object creation. I wonder if this would affect file-storage uploads. Those changes are in OpenACS 5.0. I will try to do a comparison to see if it has any effect.

Collapse
Posted by Jeff Davis on
the changes I made for object creation have not been done for the content repository (which has the same tree_sortkey without a max_child_sortkey field as the old acs_objects table). I will make the changes in 5.1 (at the time 5.0 seemed to close to release to have it go in...ah well).

Otoh, It should not matter unless the content repository is big and mostly flat (i.e. single nodes with lots of children).

is it on 7.2 or 7.3? Also, I assume it's been analyzed but if not do that and see what it says.

Collapse
Posted by Dave Bauer on
If you search the AOLserver archives, you can see messages saying that upload is generally slow on AOLserver. I have tested this previously just using a plain old form with a file  widget.

Here is something from the list. Can anyone test or confirm this?

"Earlier this year (sometime in February) I described a problem I had with file uploads taking excessively long time within an intranet environment, with effective speeds 750-800Kbit/sec on a 100MBit/s ethernet.

This week I spent some time investigating the problem further. My initial suspicion was that IOBuffers are not really allocated and we end- up reading only a few characters at a time. After checking the socket read/write code in AOLServer (using printf statements) my initial hypothesis was overulled (buffer allocation works OK!).

To cut a long story short, I found out that the culprit was the Silly Window Syndrome (SWS; see Stevens, Vol.1) exhibited by the TCP stack in WinNT4 clients when sending bulk data to the web server (the IE and Netscape browsers may also be at fault here).

The work-around was to change the tcp_deferred_ack_interval parameter in Solaris 8; this parameter indicates the delay in milliseconds before sending a deferred ACK (the default for windows is 200ms, for Solaris 8 is 100 ms and for most linux distributions 50ms; I set it to 10ms and the file uploads are 10 times faster!).

I believe the new low value of 10ms is OK for LANs (where RTT for TCP packets are less than 10ms) but it may not work for WANs.

I hope that anybody who would like to speed-up file uploads to AOLServer may find this information useful. "

Collapse
Posted by Don Baccus on
That's all very interesting information, Dave, and worth exploring, but it doesn't explain why the standalone call to file_storage__new_file takes such a long time.

It does a create_item first off ... first guess is the tree sortkey stuff as has been discussed above.  When Jeff has time to munge 5.1 perhaps he can introduce his changes to file storage first, and we can see if this fixes the problem?

Of course the 5.0 basic object creation tree sortkey speed-up would also need to be applied to Collaboraid's system, just speeding up the creation of one of the two tree sortkeys involved won't solve the problem.

The other troublesome call in file_storage__new_file is the call to update the last modified attribute of the object and all its context_id parents.  For one thing this doesn't seem right, OF sort of snuck it in while working on .LRN.  The last modified attribute in the original aD object model was intended to track the last direct modification of an object, not those who inherit permissions from the object.  For a second thing, the recursive nature of the implementation might be slow ...

But I think the first place to look is the sortkey stuff.  When Jeff gets the mods made to the CR (which should help the performance of several packages) we can do some performance testing to see if the update last modified procedure is also a problem.