Forum OpenACS CMS: Re: content repository physical folders keeps growing

Collapse
13: Re: content repository physical folders keeps growing (response to 1)
Posted by Gustaf Neumann on 12/27/13 10:45 AM
Hi Antonio,

it seems that you did not use the version from 12/25/13 05:18, before you last two messages

http://cvs.openacs.org/changelog/OpenACS?cs=oacs-5-8%3Agustafn%3A20131225161031

that version uses the external "find" command, and receives as well "-mtime" as argument to avoid scans over everything (see above site with 2.5 mio files). Maybe, one can pass "mtime" as filtercmd to globfind, and maybe we could incorporate it in openacs, but for large sites i doubt it can reach the speed of find.

for unix + mac os x/ports, "find" is readily available, for windows the findutils are contained in MSYS.

We could incorporate both approaches (tcl globfind and find), but that's one more code bloat. The easier approach is to register a cr_check_orphaned_files as scheduled proc only when "find" is available, and behave in the other cases as OpenACS did over the last 10 years.

How many files do you have on your system that grew to 17GB? how long takes a full run of cr_check_orphaned_files on that system?

-g

Collapse
14: Re: content repository physical folders keeps growing (response to 13)
Posted by Antonio Pisano on 12/27/13 11:39 AM
Files can grow as many as 17GB of leftovers because playlists are at least 500MB big. It became a problem after at least 3 months of usage, which included early test days of the feature, but my customer is really averse to technology (a very good tester), and there is no day he doesn't mess up something (must admit that sometimes the blame is on me tough)...

On my production server, which is very narrow in resources, the proc takes around 1,5 seconds to return from /ds/shell. I have around 1000 files and it returns around 250 orphans.