Forum OpenACS Development: How to get gzip/brotli transfer compression working?
We are in the ridiculous situation right now of serving eg. 2.5Mb HTML sections of a TCL/ADP page as a 2.5MB uncompressed transfer. Instead of that being compressed down to a fraction of the size before transmission (https://tools.paulcalvano.com/compression.php)
The only mention of it I can find is an old post here from 2013 (https://openacs.org/forums/message-view?message_id=1205589) but it is a confusing because it appears to be about previous AOLServer setup/config and also the newer Naviserver is talking about large static files which we don't care about for now. I tried adding the various parameters to config file but it didn't work. Is the "preauth filter" method still the only way? I tried adding that but that code appears to be obsolete and non working too.
What is the current correct way to enable brotli transfer compression on Windows based Naviserver/OpenACS? (assume latest https://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/) Any instructions/examples anywhere?
gzip compression is implemented in NaviServer for static content and dynamic content. Activating gzip compression for static content works exactly like for brotli static compression (see e.g. ), dynamic content is compressed, when "compressenable" is set as a parameter in the per server settings in the config file (see as well ).
For compression of static content, external programs are used (as configured in the config file). For dynamic content, NaviServer has to be compiled with gzip support (which is the case on all *nix platforms). Maurizios download page mentions "zlib.1,2,8), therefore i would think, that it contains gzip support.
Try it with a command like the following
curl -H "Accept-Encoding: gzip" -I https://www.amazon.com/If you see in the result "Content-Encoding: gzip", then the result is sent gzipped (make sure to you a request without redirects)
Hope this helps
I installed the latest Windows naviserver to my local machine and added the "compressenable" parameter (so that the server should compress HTML files coming back to browser) but it doesn't do anything. HTML (from TCL/ADP pages) is still coming back uncompressed.
No matter what I do, I can't get it to do the dynamic compression (of TCL/ADP files). As far as I can tell, this is done by simply adding the extra "compressenable" parameter to config file. We are not so much interested in the static compression of other files right now, so don't need the fastpath stuff.
I can see in source code of naviserver in server.c where it uses this parameter and dumps an ns_log line warning of "init server %s: compress is enabled, but no zlib support built in" is there is some zlib problem, but this line doesn't appear in log so I guess there is no problem with zlib.
I would like to be able to add more log lines myself and recomplile to test, but I am not familiar enough with Visual Studio to do this.
Has anyone here used the Windows version of Naviserver and got any compression working?
I can see in source code of NaviServer in server.c where it uses this parameter and dumps an ns_log line warning of "init server %s: compress is enabled, but no zlib support built in"notice that the change  was added on Oct 17, while the last build of Maurizio is from Oct 12th. So, the absence of the message is in your case no indicator.
So any idea why the compress parameter appears to be getting ignored? Its impossible for me to tell without adding extra logging myself which I don't have the ability to do.
the previous versions of my distribution (up to 4.4.9) did not support compression.
I'm currently uploading (it will be available in few hours) version 4.5.0 with compression support included (via zlib 1.2.11).
The thing compiles and links properly and at startup does not print any message of the type "no zlib support built in".
But I did not have the time to do proper testing on this functionality.
I hope it helps, even though I personally believe that compression support is not going to improve dramatically performances.
Yes it may not speed it up much, but its better than none I think and is really an industry standard to to transfer compression of HTML and other easily compressible formats.
ps. the "no zlib support built in" message is something Gustaf added in later source version as he mentioned above. It is probably not in the source you used.
A few sentences above confuse me: Why can't you use the version of Maurizio in your product? You are mentioning a "custom built" the first time: How did you usually build the custom version in your product in the past? Why not redoing your custom build process with recent versions of NaviServer?
1. As Gustaf already mentioned, I always include in my distribution the tip version the Naviserver.
2. I would not have written what I wrote without having first checked that the statement printing "no zlib support built in" is indeed in the codebase.
this is just to mention that brotli executable is now (from version 4.5.1) included in my distribution.
To do this I had to remove one piece of custom code from our code to get it to startup without error.
I also had to remove the section of config that enabled SSL (module/nsssl). We normally run systems with NSSSL enabled.
If I don't do this, I get error during startup:
"Warning: modules nsssl requires a version of NaviServer built with OpenSSL"
Error: modload: c:/naviserver/bin/nsssl.dll: Ns_ModuleInit returned:
Fatal: modload: failed to load module 'c:/naviserver/bin/nsssl.dl
Is there some reason why your version of Naviserver does not have OpenSSL enabled?
Also I can't static compression to do anything even though it is turned on (turning on settings in Fastpath section to enable Brotli or gzip)
ns_param gzip_static true ;# check for static gzip; default: false
ns_param gzip_refresh true ;# refresh stale .gz files on the fly using ::ns_gzipfile
ns_param gzip_cmd "c:/naviserver/bin/gzip -9" ;# use for re-compressing
ns_param brotli_static true ;# check for static brotli files; default: false
ns_param brotli_refresh true ;# refresh stale .br files on the fly using ::ns_brotlifile
ns_param brotli_cmd "c:/naviserver/bin/brotli -f -Z" ;# use for re-compressing
Unless i am doing something wrong... none of the CSS/JS files are ever compressed on server and sent to browser.
I really don't understand why there are so many problems and things not working with your version of Naviserver. Are they all intentionally removed?
1. NSSL support is not turned on by default. You can adjust the configuration in the provided solution file to turn it on.
2. I never used the compression before. The testing is all up to you (did you verify on the first place that commands like "c:/naviserver/bin/brotli" actually work in your environment? did you verify if you have to use the *.exe extension?)
You get something for free, you want it to exactly meet your requirements and then you complain if it doesn't...
😊 😊 😊
In version 4.5.2 I believe I have activated the NSSL support, but for the compression part you're on your own...