Forum OpenACS Development: ANNOUNCE: Next Scripting Framework 2.1.0 is available

Request notifications

Dear Community,

We are pleased to announce the availability of the Next Scripting
Framework (NSF) 2.1.0 as a Christmas present to the community.

Over the last years, NSF is constantly being developed based
on experience in largish projects. To document these changes
we have decided it to bump the version numbers and make
a new release.

Since the release of NSF 2.0.0, there have been more than 450
commits to our code repository. For this release NSF was
tested in 192 build configurations with Tcl 8.5.19 and
Tcl 8.6.6 on Linux and Mac OS X (both using recent gcc and
clang). Common build configurations have been tested under Windows
(Windows 2012 Server: VS15 & MSVC 19, MinGW).

Diff stats since 2.0.0:
 186 files changed, 26475 insertions(+), 15671 deletions(-)

Major changes relative to 2.0.0 are:

- Improved debugging/tracing/profiling support (NSF)
- Deprecated and debug modes for NSF procs and methods (NSF, NX, XOTcl2)
- Script-level access to NSF parameter/argument parser (NSF).
- Slot-trace reform (NX, XOTcl2)
- Updated MongoDB interface to the newest mongo-c-driver (NSF, NX)
- Improved scripted shells (e.g., nxsh, xotclsh)

Below is a summary of are the changes in more details. The detailed
changelog is available at
https://next-scripting.org/xowiki/download/file/ChangeLog-2.0.0-2.1.0.log

NSF 2.1.0 (containing NX 2.1.0 and XOTcl 2.1.0) can be obtained from
https://next-scripting.org/. Please report issues and wishes by opening
a ticket at https://sourceforge.net/p/next-scripting/tickets/.

Best regards
- Gustaf Neumann
- Stefan Sobernig

========================================================================
New Features

- NSF:

  * Improved debugging/tracing/profiling support:

    + NSF procs and methods can be marked "deprecated" and/or
      "debug-enabled". For this, all NSF proc and method definitions
      in NX and XOTcl2 now accept the new flags "-debug" and
      "-deprecated". The low-level interface (nsf::method::property)
      enables introspection and selective enabling during runtime.

          nsf::proc ... ?-debug? ?-deprecated? ... /name/ /argspec/ /body/
          nsf::method::property /obj/ /method/ deprecated ?0|1?
          nsf::method::property /obj/ /method/ debug ?0|1?

      A debug-enabled NSF proc or method will be reported on entering
      and exiting the proc and method (similar to a traced proc/
      method incl. a time probe; see above):

          % nsf::proc -debug foo {} {return 123}
          % foo
          Debug: call(1) -  foo
          Debug: exit(1) -  ::nsf::procs::foo 72 usec -> 123

      The enter and exit messages can be tailored and redirected
      at the script level by redefining "::nsf::debug::call" and
      "::nsf::debug::exit", respectively.

      To actively manage API evolution, NSF procs and methods can be
      marked "deprecated". Using deprecated procs/ methods will result
      in a warning:

          % nsf::proc -deprecated foo {} {;}
          % foo
          Warning: *** proc foo is deprecated.

      The warning message can be tailored and redirected
      at the script level by redefining the proc "::nsf::deprecated".

    + NSF procs and methods can be profiled (and optionally traced)
      when profiling support is configured during compilation

          ./configure --enable-profile (default: disabled)

      This can be controlled via

          nsf::__profile_trace -enable /bool/ ?-verbose /bool/?
          nsf::__profile_get
          nsf::__profile_clear   

      When profiling is enabled, the profiling data is kept in memory
      and can be obtained via "nsf::__profile_get" in form of a list
      structure containing the wall clock time since start of
      profiling, the aggregated ms and lists containing object times
      (what time was spent in which object/classes), method times
      (what time was spent in which methods) and proc data (what time
      was used in which procs) followed by trace data (showing calls
      and exits of methods/procs/cmds). When "nsf::__profile_trace" is
      called with the "-verbose" flag, the trace is printed via
      "nsf::log".

  * Add script-level access to the NSF argument parser to conveniently
    process, e.g., non-positional parameters/arguments and value
    checkers. See
    https://next-scripting.org/xowiki/docs/nx/tutorial/index1#_parameters
    for the details.

    The command

         ::nsf::parseargs /paramspec/ /arglist/

    can be used to parse arglist based on the paramspec to set the
    parsed arguments in the local context. For example, the command

         % nsf::parseargs {{-x:integer 1} y} {123}
         % set x
         1
         % set y
         123

     will define variables x holding "1" and y holding "123" for the
     current Tcl scope.

  * Minor logging reform (NsfLog, interp): "nsf::configure debugLevel
    /severity/" will print error messages at a level equal or greater
    than the given severity (0, 1, 2, 3). For example, "nsf::configure
    debugLevel 0" will print any "nsf::log" message, and
    "nsf::configure debugLevel 3" will print just "nsf::log" level 3
    (error) messages while omitting warnings etc. This does not entail
    changed logging semantics, but helped remove some confusion at the
    NSF/C level.

  * Improved NSF/C code generator (gentclAPI.tcl): Allows for
    specifying and generating enum types.

  * Misc:

    + New flag "-notrace" for "nsf::var::set" and "nsf::var::get" to
      implement ability to use these low-level commands for accessing
      variables without firing Tcl traces.

    + New cmd "nsf::method::forward::property" for reading for writing
      introspection of forwarders. This is important for the
      serializer to map the per-object forwarder when different target
      objects are specified.

    + New option for call-stack introspection: "nsf::current
      level". It returns the stack level of the currently executing
      NSF method (like "info level"), or an empty string outside of an
      NSF context.

- NX:

  * NX method definitions ("alias", "forward", and "method") now
    accept the optional flags "-deprecated" and "-debug". See NSF
    section above for usage details.

        /cls/ public alias   ?-deprecated? ?-debug? /method/ ...
        /cls/ public forward ?-deprecated? ?-debug? /method/ ...
        /cls/ public method  ?-deprecated? ?-debug? /method/ ...
        /obj/ public object alias   ?-deprecated? ?-debug? /method/ ...
        /obj/ public object forward ?-deprecated? ?-debug? /method/ ...
        /obj/ public object method  ?-deprecated? ?-debug? /method/ ...

  * Improved object and method introspection: Newly added
    introspection sub-commands complete the set of introspection
    commands and reflect the newly introduced method options.

        /cls/ info method callprotection
        /cls/ info method debug
        /cls/ info method deprecated
        /obj/ info baseclass
        /obj/ info object method callprotection
        /obj/ info object method debug
        /obj/ info object method deprecated

  * Reform of traces on slot-managed object variables: Revised the
    interface and semantics for the interaction of variable traces and
    slots. It is now possible to specify in the definition of a
    "property" or "variable" whether the slot-accessor methods
    ("value=get", "value=set") should be fired whenever a variable is
    read/written or when it is initialized to a default value for the
    first time ("value=default").

        /obj/ object property|variable ?-trace set|get|default? ...
        /cls/ property|variable ?-trace set|get|default? ...
    
    See the nx::Object and nx::Class man pages for details.
    This supersedes the experimental interface available in
    XOTcl 1.* based on "initcmd", "valuecmd" and "valuechangedcmd".

  * Documentation (API and examples):

    + Added

      Rosetta implementations: Inheritance/ single, add object
      variable dynamically, tree traversal, and tokenizer.
    
    + Updated

      Object.man: Added missing description on "info lookup
      parameters" and "info lookup syntax"; corrected description of
      "copy"; added "info baseclass"

      {alias|forward|method}.man.inc: Added documentation of
      "-debug" and "-deprecated" switches.

      current.man: Added description of 'level' option.

      Rosetta implementations: Inheritance/multiple, polymorphic
      copy, multiple distinct objects

- XOTcl2:

  * XOTcl2 method definitions ("proc", "instproc", "forward", and
    "instforward") now accept the optional flags "-deprecated" and
    "-debug". See NSF section above for usage details.

        /cls/ instforward -deprecated|-debug /method/ ...
        /cls/ instproc  -deprecated|-debug /method/ ...

        /obj/ forward -deprecated|-debug /method/ ...
        /obj/ proc  -deprecated|-debug /method/ ...

  * New "-return" flag: XOTcl2 method definitions can now specify a
    return-value checker, similar to NX methods.

        /cls/ instproc -returns /method/ ...
        /obj/ proc -returns /method/ ...


- Shells:

  * There is now an improved and packaged shell implementation:
    nx::shell2. It is used by all four shell scripts. nx::shell2
    builds on Tcl's event loop, rather than while + update. This
    avoids blocking the Tk main window (update). In addition,
    nx::shell2's behavior is more akin to Tcl's native shells (e.g.,
    no extra lines on enter, catches EOF). The Tcl package can be
    conveniently sourced, e.g., in Tclkit main scripts.

  * The new shells accept script input now via stdin or as provided
    via "-c" in the command line.

  * They are more robust: Don't quit due to inner [return] calls,
     [exit] is handled gracefully.

  * Added tests for the shells to the regression test suite
    (shells.test).


- Traits:

  * The namespace "nx::traits::XXX" was renamed to "nx::trait::XXX" in
    order to improve naming orthogonality with the trait management
    package "nx::trait".

- MongoDB interface upgrade:

  * Upgraded MongoDB and drivers to mongodb-c-driver 1.5.1, libbson
    1.5.1 and MongoDB v3.4.0.

  * The upstream mongodb-c-driver has deprecated the old OP_QUERY syntax
    with $-modifiers and old names like $orderby and $query, which will
    be removed from the driver. The new interface is more in the mongodb
    style, many former parameters (e.g. -limit, -skip, -project, -sort)
    went to the new "-opts" argument.

          mongo::collection::query /collection/ /filter/ ?-opts /opts/?
          mongo::cursor::find /collection/ /filter/ ?-opts /opts/?

    See the begin of nsf-mongo.test file for examples for queries with
    the old and new interface. The high-level interface for nx::Object
    is unchanged.

  * Support for new binary type "decimal128"
  - Maintenance & bug fixes:

  * Tcl command resolvers and command literals:

    Between Tcl 8.6.0 and (including) Tcl 8.6.6, there was a problem
    in the interaction between Tcl 8.6 and the command resolvers of
    NSF in certain corner cases and Tcl's command literals. The issue
    and patches were reported upstream to the Tcl core team, was
    recognized and will enter an upcoming patch release (8.6.7). Our
    regression test-suite was extended accordingly to track the issue.

    See Tcl Fossil tickets d4e7780ca1 and 3418547.

  * Finalizing NSF and NSF object systems:

    In this release, the finalization procedures of NSF (on process
    exits, thread exits, and interp teardowns) has been revised. For
    example, NSF's ExitHandler is guaranteed to be called just once in
    the above scenarios (e.g., to avoid double frees of NSF
    structures).

  * Fixed a long-standing and not easy reproducible bug when deleting
    classes in multiple nested diamond inheritance structures.

  * UnsetTracedVars:

    Provide for a two-pass deletion logic during object shutdown, to
    account for unset traces possibly reviving an object variable
    under deletion. This corresponds to a recent memory leak fix to
    Tcl itself. See also Tcl Fossil ticket 4dbdd9af144dbdd9af14.

  * Serializer:

    The combined NX and XOTcl2 serializer has been extended to cover
    newly added features (esp., NSF procs, method flags) and has been
    improved based on experiences from multi-threaded NSF/Tcl
    applications (NaviServer). Improvements and fixes relate to "info"
    methods and forwarders.

  * Tcl 8.7:

    Preliminary support for a future Tcl 8.7 (as of Dec 2, 2016;
    commit 71fa1f9c91). NSF compiles under 8.7a0 and its regression
    tests execute successfully.

  * Misc (esp. NSF/C):

    + Refactoring: There is now a common infrastructure for
      hash-tables using function pointers as keys (as required for
      NSF/C command definitions and enumerations).
    + Removed all implicit type-conversions, signed/unsigned
      comparisons as flagged by the following gcc/clang flags (ensure
      cleanness on 64bit) -Wconversion -Wsign-conversion
      -Wfloat-conversion -Wsign-compare.
    + Reduced variable scopes.
    + Guarded against potential Tcl_Obj leaks, esp. when using
      "NsfMethodNamePath", as indicated by valgrind.
    + Refactored functions for topological sorting ("TopoSort") to
      avoid code redundancies.
    + Addressed more than 18 code-quality issues as indicated by
      Coverity Scan, incl. control-flow issues (risking NULL
      dereferencing), dead code branches, etc.
    + Removed uses of deprecated functions, e.g. "Tcl_AppendResult".
    + Based on the improve NSF/C code generator (see above), proper
      enum types are used in generated function signatures and the
      respective control structures (switch).

  * Extended regression test suite:
    The test suite contains (for Tcl 8.6.6) now 5852 tests.

- Packaging & distribution:

  * Updated TEA to 3.10
  * Support for MSVC 1900 (VS 2015; win/makefile.vc)
  * Debian package: https://packages.qa.debian.org/n/nsf.html
  * MacPorts port: https://trac.macports.org/browser/trunk/dports/lang/nsf/Portfile
  * Part of the KitCreator battery: https://kitcreator.rkeene.org/fossil/
  * Part of the kbskit battery: https://sourceforge.net/projects/kbskit/