Forum OpenACS Development: Re: ANNOUNCE: NaviServer 4.99.8 available

Thanks Gustaf, this is an impressive list to begin with -- and there are a lot of details to appreciate here. (The non-select() note alone is very welcome.)

I'd be interested in a few examples of how ns_connchan works, and the first thing I wondered was whether a "ns_connchan copy $fd" would be helpful to replace a lot of code to run background delivery of files?

Collapse
Posted by Gustaf Neumann on
Hi Steffen,

With recent versions of NaviServer, most of the usages of bg-delivery can be replaced by "ns_writer submitfile ..." when writer-threads are in use (look for use_writerThread in [1]). The writer threads are roughly the equivalence of the bgdelivery thread, but implemented in C, not relying on select(), are lightweight in memory use and fast (using the common io-machinery of NaviServer). For the introduction to the async support in NaviServer, see section "2.3. Asynchronous Architecture" in [2].

In fact, when one is using a recent version of xotcl-core, ad_returnfile_background uses automatically the writer threads, no application code changes are needed. we use this since a while on our production systems an on openacs.org.

Actually, the only place were we are still using - and that might be the source of your interest - is the h264 streaming code. It should be actually not hard to replace there the tcl/io (e.g. replace "puts -nonewline $channel ..." by "ns_connchan write $channel ...", use "ns_connchan callback ...", etc.). This has as well the advantage that it will work without changes with tls. When one wants to copy the fd from a channel, this would defeat this purpose. So far, i had no time to look into the details to implement h264 delivery via ns_connchan.

To see ns_connchan in action, look at the WebSocket code in [3].

-g

[1] https://openacs.org/xotcl/show-object?show_methods=2&show_source=1&object=bgdelivery
[2] https://next-scripting.org/xowiki/docs/misc/naviserver-connthreadqueue/index1
[3] https://bitbucket.org/naviserver/websocket/src/c9da8e104075ed8bb9a1d3e6a244f99fd730f0fc/websocket-procs.tcl?at=default

Good to know! I really need to look into the project of switching our core over to to NaviServer. It is complicated by the fact that we use a fairly old and fairly modified version of OpenACS.

We use the h264 delivery mechanism less and less these days ourselves, but we never actually used ad_background_returnfile for the purspose. Instead we've simply got a tcl thread handling the reading and writing. HTML5 and byte ranges have decreased the use of this drastically though.

Collapse
Posted by Gustaf Neumann on
The same with us (that's why a rewrite of h264 is not top prio). i just looked up our figures for video deliveries for our production site: factor h264/html5 = 1/9, with peaks on htm5: 5.500 per hour.

Concerning ad_returnfile_background: it has the same interface like ns_returnfile. The changes for NaviServer in OpenACS are mostly due to using features of NaviServer not available in aolserver. If you get a recent version of OpenACS (e.g. the tar file), just grep for NaviServer, these should be pretty easy applicable for every OpenACS mutant.