Forum OpenACS Development: ANNOUNCE: NaviServer 4.99.11 available

all the best
-gustaf neumann
[1] https://sourceforge.net/projects/naviserver/
[2] https://bitbucket.org/naviserver/naviserver/

=======================================
NaviServer 4.99.11, released 2016-05-01
=======================================

Changes relative to 4.99.10

 156 files changed, 8613 insertions(+), 3142 deletions(-)

New Features:

  * IPv6 support: NaviServer works now with IPv6 and IPv4
    addresses.  It is e.g. possible to use "::" as address (to denote
    the unspecified address, roughly similar to 0.0.0.0 in IPv4) to
    accept any IPv6 or IPv4 address, when the OS supports IPv4-mapped
    addresses, or one can use arbitrary addresses. Note, that one can
    as well load the driver for a server multiple times, once with an
    IPv6 and once with an IPv4 address.  Adding this feature required
    changes in the drivers, since earlier versions had IPv4 structures
    compiled in. As a consequence, also drivers for additional
    protocols have to be adjusted (for this release, the IPv6-adjusted
    modules are all modules of the naviserver core, plus nsssl, nsudp,
    nssmtpd, nsdns)

  * Added HTTP/1.1 pipelining support: Pipelining is a (not so often
    used) feature of HTTP/1.1 where multiple requests are sent on a
    persistent connection without waiting for the results of the first
    request. Recently more (esp. mobile) clients started to use
    pipelining. Previous versions of NaviServer handled just the first
    request of the pipeline and swallowed (ignored) later requests of
    the pipeline.

  * Added support for HTML5 file attribute "multiple": HTML5 allows
    for uploading multiple files under the same INPUT name. Previous
    version of NaviServer just accepted the first file with the given
    name.

  * Reduced memory consumption:
      - Improved support from upload handling of large input. When
      uploads larger than "maxupload" is spooled to a file, the
      results are available via [ns_conn contentfile] rather than
      [ns_conn content].  There might be many places in application
      programs, where this distinction might be relevant. The new
      version gets now form content via [ns_getform] from both
      sources, fixes potential encoding problems.  Similarly, the new
      function [ns_getcontent] returns the unparsed content either
      binary or encoded as file or as string. These new features makes
      it easer to use upload spooling to avoid memory bloats.
          - Furthermore, (spooled) files can be sent from disk to some http
      server via ns_http (or ns_ssl) without the need to load the
      content first to memory by using the new parameter body_file)

  * Improved crypto-support: the new version supports SHA2 hash codes
    (command ns_sha2) and keyed message authentications based on SHA2
    (command ns_hmac_sha2). The new version uses the C implementation
    of the crypto functions by Olivier Gay <olivier.gay@a3.epfl.ch>.

  * Reduced memory consumption: avoid memory bloat on uploads and in
    HTTP client operations with large contents (esp. useful for
    contents >1GB).  ns_getform an the new ns_getcontent work now on
    spooled and in-memory content in a uniform manner.
      * Improved debugging support: new severity "Debug(request)" can be
    used follow life-time of a request (useful especially for
    inspecting concurrency of long-running requests).


Bug Fixes:

 - Fixed potential lockup, when bind operation fails,
   e.g. when server is already running (old bug)

 - Keep query parameter after directory induced redirect in fastpath
   (bug report of David Osborne in the NaviServer list)
    - Don't swallow erroneous HTTP requests: previously, error replies
   generated from the driver (e.g. 400 invalid request) were
   not logged at all (neither in the access.log or in the error.log).
   This is due to the nature, how the nslog modules works. Now
   such conditions are reported in the error.log.

 - Make HTTP-request line parser more precise. Previously
   many invalid requests were treated as HTTP/0.9 requests leading
   to strange error messages.

 - Provide more (and more detailed) error messages to ease debugging.

 - Improve validity of HTML error messages generated by NaviServer

 - Resurrect handling of non-HTTP drivers (based on
   drvPtr->requestProc) such as nssmtpd.

 - apply url-encoded form parsing only when content is
   www-form-urlencoded.  Previously, it was attempted on every content
   which might lead to unexpected results.

Documentation improvements:

 - Align style and logo of default web server configuration with style
   of man pages
 - Improved validity of HTML man pages
 - Fixed documentation bug on ns_cookie
 - Improved documentation of configure parameters
  - various more man pages updated

Tcl API Changes:

  - new function: [ns_info ipv6] obtains information whether the binary
    supports IPv6 or not

  - As a consequence of "multiple file" support (see above) the
    following functions return for the specified name potentially a
    list of values
            [ns_conn fileoffset $name]
    [ns_conn filelength $name]
    [ns_conn fileheaders $name]
    
    and the ns_set returned by [ns_getform] will contain multiple
    entries for the file (and $name.tmpfile and $name.content-type).

  - new commands:
      * ns_sha2 ?-length ...? data
      * ns_hmac_sha2 ?-length ...? key message
      * ns_getcontent (obtain body of request either from memory
        of from spoolfile without memory bloat on huge files)

  - new option "-body_file" for ns_http and ns_ssl. By using this
    parameter one can send huge files without memory bloat
    (i.e. loading the full file into memory). The function is more or
    less the counterpart to the spoolfile for uploads when the content
    is larger than maxupload.

  - ns_getform processes content from [ns_conn content] or
    [ns_conn contentfile] (whatever is available).

  - interface change:
      * old interface of ns_sendmail based on positional arguments is deprecated
      * new interface of ns_sendmail is
          ns_sendmail ?-to ...?  ?-from ...? ?-subject ...? ?-body ...? \
                      ?-headers ...? ?-bcc ...? ?-cc ...?

   C API Changes:
  - New functions:
      * Ns_HttpParseHost(): parses host and port number from a string.
      Reason: IPv4 address conventions use simply a colon for
      separating host and port, but IPv6 address contain as well
      various colons. The new function supports IP-literal notation
      according to RFC 3986 section 3.2.2 (IP address between square
      brackets), which has to be used as well for IPv6.
          * Similarly, the new function Ns_HttpLocationString() to unifies
      now generation of host and location strings

    * New IP version agnostic functions for manipulating sockets,
      addresses and ports:
    Ns_LogSockaddr(), Ns_SockaddrGetPort(), Ns_SockaddrGetSockLen(),
    Ns_SockaddrMask(), Ns_SockaddrMaskBits(), Ns_SockaddrSetPort(),
    Ns_SockaddrSameIP(), ns_inet_ntop(), ns_inet_pton()
    
    * New function Ns_ConnReturnInvalidMethod() to report that the provided
      method is invalid (was returning "not found" before)

    * New function Ns_DStringAppendPrintable() to append a string which might
      contain non-printable characters in a printable fashion to a DString
      (useful for e.g. debugging)

    * Ns_Stat() is the former function FastStat() from fastpath made
      reusable in other modules.

  - Changed C-API functions (incompatible with earlier versions):
    * The following function accept now generic (version agnostic) sockaddr:
      Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
      Ns_SockBind(), Ns_SockBindUdp()
    * Ns_CtxString() renamed to Ns_HexString() to make its intention
      more clear and added a parameter to control lower and upper case



Configuration Changes:
  * New configure flag --disable-ipv6 to build NaviServer just with IPv4 support,
    even when IPv6 support is available.
  * Updated sample config files for IPv6
  * Added makefile target: "lldb-sample" lets one running nsd with
    the sample config file "nsd-config.tcl" under lldb

Command Line Changes:
  * Extended prebind options to work with IPv6 addresses

Code Changes:

  * All built-in MIME types are updated and completed with the IANA
    media type definitions. Warning: This change might return now
    different results from ns_guesstype unless the mime type is
    specified in the config file.

  * Improve reusability: The new version exports the low-level I/O functions
    of nssock to make it reusable in other driver modules. The following
    functions are now exported to a library:
      Ns_DriverSockAccept(), Ns_DriverSockRecv(), Ns_DriverSockSend(),
    Ns_DriverSockClose()

  * Code cleanup:
    - used clean approach to obtain hex string from function pointer
    - use boolean types on more occasions
    - use more often typedef-ed function definitions
    - make NS_TCL_SET_STATIC and NS_TCL_SET_DYNAMIC an enum
      (Ns_TclSetType), since the values are mutual exclusive
      * Regression tests
    - extended test (we have now 1084 test cases)
    - remove deprecated calls in regression test