Forum OpenACS Development: Re: Image resizing, exec and server problems

3: Re: Image resizing, exec and server problems (response to 1)

Posted by Tom Jackson on 08/29/07 07:14 PM

From the AOLserver list:

 Periodically some of our AOLserver installations get into a mode where

 all calls to "exec" just hang, not really taking up any processor time

 but eating up a thread.  Doesn't seem to be any memory problems

 coinciding, which I had originally suspected being a limiting factor on

 the forks.  Besides, we usually get an error message when there’s not

 enough memory to fork a child process.  Haven't yet figured out anything

 in particular that causes this to start happening, but once it starts

 happening, it keeps going eating up more and more threads until we

 restart the nsd.  Any ideas what the cause might be and what I can do to

 investigate further?



 We run AOLserver 4.0.10 on RHELAS 3, and CentOS 4, w/ Tcl 8.4.11.

This is a known issue/conflict with pthreads and fork. Tcl does not yet have the pthread_at_fork functionality in place because that requires a careful marshaling of mutexes over the fork. You will find several core bug reports on this in the Tcl SF db area. It is possible to solve, its just that noone has invested the time yet.

And nsproxy was the suggested fix.

4: Re: Image resizing, exec and server problems (response to 3)

Posted by Malte Sussdorff on 08/29/07 08:30 PM

Which would explain why on Solaris this problem does not seem to exist (at least to Patrick G. who mentioned this to me). Thanks, will look into how to setup ns_proxy then. Any pointers are welcome 😊

5: Re: Image resizing, exec and server problems (response to 4)

Posted by Tom Jackson on 08/29/07 08:48 PM

I'm not sure how much help is available for ns_proxy, but this should be fixed. Obviously you need AOLserver 4.5, and there is a new pools.tcl file in CVS which can be used to adjust the base number of threadpools.

I just posted the htmlized man page for nsproxy:

http://junom.com/document/aolserver/nsproxy/ns_proxy.html

6: Re: Image resizing, exec and server problems (response to 5)

Posted by Malte Sussdorff on 08/31/07 09:06 AM

I stuffed the pools.tcl into acs-tcl/packages/tcl/pools-init.tcl as I am fairly certain most people will forget that when they download AOLserver and most wont download the latest CVS checkout anyway, but the release.

Alternatively I can use the one provided by AOLserver now and put it into my install directions, which is probably preferred. Tom, if you look at http://cognovis.de/developer/en/aolserver_install would it be included by default or would I need to get it separately (which is what I assume)? If yes, where from and where to put it to? Additionally does it need special treatment in the config.tcl file?

8: Re: Image resizing, exec and server problems (response to 6)

Posted by Tom Jackson on 09/06/07 08:07 AM

I don't think the ns_pools has anything to do with ns_proxy. I haven't used ns_proxy, but just provided a link to the manpage.

I have some scripts for image resizing if there is an interest.

9: Re: Image resizing, exec and server problems (response to 8)

Posted by Malte Sussdorff on 09/06/07 09:43 AM

Just for clarification. The pools-init.tcl has nothing to do with the resizing issue. It was just a reply to Tom's comment and badly connected as such. Just in case people get confused.

Tom, have you had a look at the image:: procedures written by Dave? I think a comparison what they do and at which level (inside TCL or outside with imagemagick, alternatives) they interface would be interesting.

11: Re: Image resizing, exec and server problems (response to 9)

Posted by Tom Jackson on 09/06/07 05:58 PM

Malte, Dave:

I haven't seen how the image procedures work here. I wrote a module for AOLserver which allows upload of a gzip file of original images and extracts them. Then creates a series of smaller versions of each image. You can specify the sizes and a number of other options.

Most of it is not useful outside of the package, but I noticed a single procedure which takes some options for convert and creates a conversion string. There are a lot of options and you can't just plug them in, they have to be calculated. It also detects the orientation of the image.

The proc is ::qphoto::photo::convertOptions and is available in this page, near the bottom:

http://rmadilo.com/m2/servers/rmadilo/modules/tcl/twt/packages/quick-photo/tcl/quick-photo-procs.tcl

When working on this I tried Imagemagick loaded into Tcl. It was a shared library and didn't require exec. However I fould it to be significantly slower at conversions than simply exec'ing convert.

An example of the data used as input is in:
http://rmadilo.com/albums/tom/IHD2005/series.rdl

The output data is in:
http://rmadilo.com/albums/tom/IHD2005/photo.rdl

There is a form for testing out the conversion proc here:
http://rmadilo.com/albums/tom/coptions-form

7: Re: Image resizing, exec and server problems (response to 3)

Posted by Malte Sussdorff on 09/06/07 07:50 AM

What is the prefered way of doing this:

a) Edit the sourcecode for image generation and use ns_proxy there. Use a specific ns_proxy pool for image generation only

b) Write a generic "exec_proxy" procedure which works just as exec, but executes through an ns_proxy. I could probably add background capabilities to that (so you could push it to the background and have errors reported in the error.log or as user messages). Replace exec with exec_proxy in the places where it matters.

If I go with "b" is it fine to just use the default configuration for the "exec" proxy or should I provide parameters to tune that? Should I put this in acs-tcl?