Forum OpenACS Q&A: How to enable gzip compression

Collapse
Posted by Héctor Romojaro on
Hi to all;

I'm trying to enable the gzip compression in aolserver 4.5.0 + openacs 5.3.1 + zlib 1.2.3 with no luck.

I've added the following to config.tcl:
---------------------------------------
ns_section ns/server/${server}/adp
ns_param gzip on

ns_section ns/server/${server}
ns_param gzip on
ns_param gzipmin 4096
---------------------------------------

My client supports gzip compression, but the server is not returning gzipped pages, as shown on output headers, on developer support.

Am I forgetting something?

Thanks In Advance.
--
Héctor Romojaro

Collapse
Posted by Tom Jackson on

In AOLserver 4.5, you have to load the nszlib module:

ns_section ns/server/${server}/modules
ns_param nszlib "$Home/bin/nszlib.so"

The other params look good, but there is one additional one:

ns_section ns/server/${server}
ns_param gzip on     ; # default = on
ns_param gzipmin 4096; # default = 4*1024
ns_param gziplevel 3 ; # default = 4
Collapse
Posted by Héctor Romojaro on
Thanks Tom, the module is loading correctly now.

I've tested the example included with the nszlib module (example.tcl) and it works perfectly, but aolserver isn't returning gzipped pages yet, as I saw in the output headers and in http://www.whatsmyip.org/mod_gzip_test/.

Do i have to change anything in my openacs/dotlrn configuration to enable this?

Collapse
Posted by Tom Jackson on
You could help everyone out if you could explain how you test to see that it is or isn't working. I don't know how reliable it is to use the web page you reference to do the checking. It would be a real service to everyone to have a method to ensure that compression is working, when and where.

Maybe we need some kind of debug statement in the code to let developers know what the outgoing format is. This would include compression, encoding, caching, etc. There is a lot of information maintained in ns_conn.

Anyone have any experience with tracking what finally gets sent to the client?

Collapse
Posted by Héctor Romojaro on
Gzipped pages, as shown on rfc should have the following output header:

Content-Encoding: gzip

There is a list of the Browser and Output Headers in developer support pages, so i just expected to see that header there.
There is also a firefox extension to check the headers too.

I've tested the pages with external checks too, like the one I posted before (there are a lot of them) and checked the size of the page with the module nszlib enabled and disabled, but, to be really really sure if it is compressing or not, i did a tcpdump to the net interface of the client:

$ tcpdump -A src $URL -s 3000 > /tmp/dump

Examining the /tmp/dump file, I could see clearly the html plain code, so aolserver it's surely not compressing.

Anybody is using successfully gzip compression with aolserver 4.5 + openacs/dotlrn? Any tips?

Collapse
Posted by Tom Jackson on
Thanks for the link to the firefox plugin.

I have visited several websites, the browser is sending the gzip/deflate header, but nobody returns compressed content. Do you have any links to sites which do that?

Yes, www.mozilla.org is doing it, for example:

----------------------------------
http://www.mozilla.org/

GET / HTTP/1.1

Host: www.mozilla.org

User-Agent: Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.8.1) Gecko/20061010 Firefox/2.0

Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Accept-Language: es-es,es;q=0.8,en-us;q=0.5,en;q=0.3

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Connection: keep-alive

HTTP/1.x 200 OK

Age: 449

Date: Mon, 29 Oct 2007 19:25:01 GMT

Expires: Mon, 29 Oct 2007 19:35:01 GMT

Cache-Control: max-age=600

Connection: Keep-Alive

Via: NS-CACHE-6.1: 1

Etag: "2b5a-cc8b36c0"

Server: Apache/2.0.52 (Red Hat)

Last-Modified: Thu, 25 Oct 2007 20:08:35 GMT

Accept-Ranges: bytes

Content-Type: text/html

Content-Encoding: gzip

Content-Length: 3068

----------------------------------

That's the data from the headers given by the firefox plugin.

Hello Hector,

I would like to ask if you were successful with enabling gzip compression on your server.

I've been playing with nszlib in the past couple of days and like you I have been unable to get aolserver 4.5 to server gzip content.

However, I found that if I remove 0-acs-init.tcl and zz-postload.tcl from openacs/tcl/ aolserver ns_zlib works.

I'm guessing that it must be something in acs-init that is overriding ns_zlib.

I'm trying to trace acs-init now with the hopes of pin pointing exactly what is preventing compressed content and I was wondering if you have had any luck.

Thanks,

Hamilton

Hello Hamilton,

No, I couldn't, but i did a workaround using nginx to serve the pages in top of aolserver, as seen in malte's tutorial.

Now, nginx is doing gzip compression, ssl negotiation and serving static content (css, js, images...).

BTW, aolserver without ssl didn't restart with signal 11 errors anymore.

Greetings, Héctor

Collapse
Posted by Dafydd Crosby on
Has anyone been able to get this working with just OpenACS and AOLserver? I've been unsuccessful so far.
Collapse
Posted by Jeff Rogers on
For AOLserver specifically, 3 things are needed:
- the ns_zlib module must be included in the config

ns_section "ns/server/${server}/modules"
    ns_param nszlib nszlib.so

- gzip must be enabled for the server in the config:

ns_section "ns/server/${server}"
    ns_param gzip on

- gzip must be enabled on each connection. There is no default way to do this. An easy way is to set up a preauth filter; put this in the tcl lib directory (e.g., serverroot/modules/tcl/filter_gzip.tcl):

proc _ns_filter_gzip {args} {
    ns_conn gzip 1
    return filter_ok
}

ns_register_filter preauth * * _ns_filter_gzip

This should get gzip compression enabled on most pages (there's a further consideration with some static content, but OACS's request filter I think bypasses this).

Collapse
Posted by Gustaf Neumann on
For NaviServer, the situation is simpler. NaviServer does not require an extra module, the "configure" script determines at compile time of NaviServer the availability of zlib (which is pretty ubiquitous). In the startup script, one can enable/configure gzip compression for dynamic content (adp pages) via the following parameters:
ns_section ns/server/${server} 
   ...
   # Compress response character data: ns_return, ADP etc.
   #
   ns_param    compressenable	on	;# false, use "ns_conn compress" to override
   #ns_param    compresslevel       4	;# 4, 1-9 where 9 is high compression, high overhead
   #ns_param    compressminsize     512	;# Compress responses larger than this
So far Naviserver does not handle gzip compression for static content automatically. Does one really want to gzip a 500MB video file? For static content one would need at least some additional configuration on what files compression is wanted. It can be discussed, whether it is desirable to compress static content dynamically.

With the following snippet, one can deliver "precompressed" static files, if the client accepts gzipped content. To generate the gzipped content, one can iterate with a script over the file system, and produce the compressed content in addition to the uncompressed content.

set file graph.js
set fn [ns_info pageroot]/$file
set mime [ns_guesstype $file]

if {[ns_conn zipaccepted] && [ns_set iget [ns_conn headers] Range] eq ""} {
    if {![file readable $fn.gz] || [file mtime $fn] > [file mtime $fn.gz]} {
    exec gzip -9 <  $fn > $fn.gz
    }
    if {[file readable $fn.gz]} {
    set fn $fn.gz
    ns_set put [ns_conn outputheaders] Vary Accept-Encoding
    ns_set put [ns_conn outputheaders] Content-Encoding gzip
    }
}
ns_returnfile 200 $mime $fn 
The only NaviServer specific part is ns_conn zipaccepted which evaluates the "Accept-Encoding" rules from the client. This option was added recently and requires the head version, which is currently in quite active development (several commits every week). To get a more conservative version, download the tarfile for tag "naviserver-4.99.4" from Nov. 2012. It contains as well a sample startup script for openacs and works well with recent versions of OpenACS.

Hope this helps

Gustaf Neumann

Collapse
Posted by Dafydd Crosby on
Awesome - putting the filter in got the pages gzipped. What would be involved in getting the static content gzipped?