Forum OpenACS CMS: Re: content repository physical folders keeps growing

Posted by Gustaf Neumann on
Hi Antonio,

it seems that you did not use the version from 12/25/13 05:18, before you last two messages

that version uses the external "find" command, and receives as well "-mtime" as argument to avoid scans over everything (see above site with 2.5 mio files). Maybe, one can pass "mtime" as filtercmd to globfind, and maybe we could incorporate it in openacs, but for large sites i doubt it can reach the speed of find.

for unix + mac os x/ports, "find" is readily available, for windows the findutils are contained in MSYS.

We could incorporate both approaches (tcl globfind and find), but that's one more code bloat. The easier approach is to register a cr_check_orphaned_files as scheduled proc only when "find" is available, and behave in the other cases as OpenACS did over the last 10 years.

How many files do you have on your system that grew to 17GB? how long takes a full run of cr_check_orphaned_files on that system?


Posted by Antonio Pisano on
Files can grow as many as 17GB of leftovers because playlists are at least 500MB big. It became a problem after at least 3 months of usage, which included early test days of the feature, but my customer is really averse to technology (a very good tester), and there is no day he doesn't mess up something (must admit that sometimes the blame is on me tough)...

On my production server, which is very narrow in resources, the proc takes around 1,5 seconds to return from /ds/shell. I have around 1000 files and it returns around 250 orphans.