Hi Jeff, nice to hear from you.
No, currently there is no such feature, but it certainly would make sense.
Concerning spooling the content to memory, NaviServer has a limit (maxupload) to decide to write the content into memory or to a spool file. In the latter case, just the disk space is the limit. ns_http (which is used by default by util::http::get) has as well a limit for spooling (spoolsize), takes the tmp directory from the configuration file, and is trying to make sensible defaults. However, when someone transfers a huge file (say in the TB range), you do not want to fill up all the disk space. Therefore, limiting makes certainly sense.
Limiting the size would be in some cases easy: When the content-size is available, the transmission can be stopped without the need to transfer all data.
However, to address this perfectly is not so easy. There are at least the following complications:
- Many request return no content-length (streaming HTML, chunked transfer encoding)
- compressed content (the transfer size will not be the same as the content on disk).
In these cases, it is necessary to transfer the content and to make the decision based on the transferred and/or decoded content.
What i have checked, both nginx and Apache offer some options. NGINX and Apache rely on Content-Length, buffer limits, and timeouts for controlling data size, but they struggle with compressed data, chunked transfers, and streaming because they can't easily determine the uncompressed or total size of data at the start of the transfer.
What is your major concern? As said, spooling to a file works with existing NaviServer, the measuring approach requires some extension.
