View · Search · Index

Weblog

Showing 1 - 10 of 667 Postings (summary)

Handling out of memory on "exec" calls

Created by Gustaf Neumann, last modified by Gustaf Neumann 02:24 PM, Monday

On sites with high number of configured threads and high number of activated packages, the VM size of nsd might become large. One measures is to compile NaviServer and Tcl with the flag SYSTEM_MALLOC, and to run it with a memory efficient malloc library such as TCMalloc  (using in the service file LD_PRELOAD=/usr/lib/libtcmalloc.so").

However, even then memory might become short, when "exec" is performed, even via the nsproxy module. The nsproxy module was developed to reduce memory consumption by running a separate process which communicates via pipes to nsd. The nsproxy module is designed to run multiple worker processes (the number can be configured), but when this number runs out (more such worker processes are needed) a new nsproxy worker process has to be created. This happens via a fork() system call under Linux, which can result in an out-of-memory message like the following:

 Error: exec: ns_fork() failed: Cannot allocate memory
 Error: exec failed: not enough memory

What should one do in such cases? In general, the first rule is to reduce the "exec" calls as far as possible, since these are relative slow and resource intensive, and NaviServer/Tcl provide a large number of built-ins or library functions, which should be used if possible.

Secondly, it is preferable to start many nsproxy workers rather soon in the live-time of nsd (e.g. at startup, when it has a small footprint) and ensure that the nsproxy module keeps these worker process alive a long time (by setting "idletimeout" to a high value)

ns_section ns/server/${server}/module/nsproxy {
     # ns_param recvtimeout        5000
     # ns_param waittimeout        1000
     # ns_param idletimeout        300000
     ns_param idletimeout  700000000
 }

When all configured nsproxy worker processes are running all the time, there is not need to fork later, and the error above will not occur anymore. The following snippet shows, how to start all nsproxy worker processes by creatin an ns_job queue with sufficient threads, and start a simple command executed by via nsproxy asynchronously (using the "-detached" flag) in parallel.

set concurrency [ns_proxy configure ExecPool -maxslaves]
if {"q1" ni [ns_job queues]} {
    ns_job create q1 $concurrency
}
#
# Queue a sufficient number of jobs to be executed in parallel
#
time {ns_job queue -detached q1 {exec sleep 1}} [expr {$concurrency * 2}]

Additionally, versions of NaviServer beyond 4.99.20 show via

ns_proxy stats ExecPool

the number of running worker processes of the ExecPool (also included in the process page of nsstats).

 

Installing OpenACS on Windows

Created by Maurizio Martignano, last modified by Maurizio Martignano 06 Jul 2020, at 08:07 PM

Windows-OpenACS (vers. 4.7.2 - July 2020) is a  Windows 64 port of OpenACS 5.9.1 and the latest snapshot of NaviServer and is available at Spazio IT .

This port installs and runs on the following systems:

  • Windows 10,
  • Windows Server 2012 R2,
  • Windows Server 2016 and
  • Windows Server 2019.

 

XoWiki

Created by Gustaf Neumann, last modified by Gustaf Neumann 25 Jun 2020, at 07:16 PM

XoWiki is a wiki implementation for OpenACS  implemented based on xotcl-core. Instead of trying to implement the full set of wiki markup commands in systems like MediaWiki, XoWiki is based on a rich text editor and focuses more on integration with OpenACS (e.g. categories, general comments, adp-includes, ad-substitution of template variables). XoWiki combines aspects of wikis (ease of page-creation) with aspects of a content management system (revisions, reusable content, multiple languages, page templates).

XoWiki provides functionalities of enterprise wiki systems (such as combining structured with unstructured information, providing security policies, etc.). XoWiki is reused in several other packages such as XoWiki Content flow, S5 or  the Learning Content Tool.

Documentation:

 


Package Specification Summary for Package: xowiki

Summary: A xotcl-based enterprise wiki system with multiple object types
Description: <pre> XoWiki is a Wiki implementation for OpenACS in XOTcl. Instead of trying to implement the full set of Wiki markup commands of systems like MediaWiki, XoWiki is based on a rich text editor and focuses more on integration with OpenACS (e.g. categories, general comments, ADP-includes). XoWiki combines aspects of wikis (ease of page-creation) with aspects of a content management system (revisions, re-usable items, multiple languages). Furthermore, XoWiki allows one to define different types of links such one could define book-structures (where a navigation structure could be built on the fly) or glossaries with different kind of word relationships (like synonyms, etc.). XoWiki supports pages in multiple languages and is localized. Some features: - cross language links - inclusion of ADP pages - nesting of Wiki-pages - large set of includeable content (includelets) - search - tags - categories - RSS - weblog - podcasts - notifications - web 2.0 gadgets (digg, delicious, my yahoo) - audio embedding - different appearances (template_file) - book-structures - prototype pages - import/export - virtual presence - analysis of collaboration networks - forms - named/unnamed pages - various security policies </pre>
Maturity: Mature
This package depends on: xotcl-core acs-datetime acs-kernel acs-tcl acs-templating acs-subsite acs-lang richtext-ckeditor4 categories file-storage general-comments acs-automated-testing
Packages that depend on xowiki: chat content-portlet eduwiki learning-content learning-content-portlet pages s5 xolirc xowf xowiki-includelet xowiki-portlet
Package parameters:
CacheSize
Size of the xowiki_cache (default 400000, type number, scope global)
MenuBar
Activate experimental menubar stuff (default 0, type number, scope instance)
MenuBarSymLinks
Activate experimental menubar stuff (default 0, type number, scope instance)
PackagePath
Space delimited names of local URL paths pointing to xowiki instances from where this instance should inherit pages (example: /xowiki /public/wiki) (default , type string, scope instance)
PreferredCSSToolkit
Use either "yui" or "bootstrap" (default bootstrap, type string, scope global)
PreferredRichtextEditor
Preferred richtext editor, such as: xinha, ckeditor, ckeditor4 (default) or wym (default ckeditor4, type string, scope global)
extra_css
One or more additional css file for this Instance (default , type string, scope instance)
hstore_max_value_size
When hstore is activate, don't store entries larger than this size in the hkey. This makes it possible to produce smaller hkey indices and to use e.g. GIN, which has a size limit of 2K. Set to 0 to disable the size limit (default 2000, type number, scope global)
parameter_page
::xowiki::FormPage to search for parameters. The page name has to contain a language prefix and can refer to a different xowiki instance. Example: //xowiki/en:xowiki-standard-parameter (default , type string, scope instance)
security_policy
Define the valid operations for different kind of users. Currently, two policies are predefined: ::xowiki::policy1 and ::xowiki::policy2. Policy1 requires for all destructive operations (deletes, delete_revision) and programmatical operations (involving tcl code, e.g. editing the ::xotcl::Objects) package admin rights, for reindex site wide admin right). Policy 2 requires also for destructive operations site wide admin rights. (default ::xowiki::policy1, type string, scope instance)
template_file
Name of the ADP file to be used for viewing xowiki pages. The default value is 'view-default'. Alternatively, one can use 'view-book' to view pages in book style (needs page_ordering) or e.g. 'oacs-view' for providing a view with the category-tree on the left, or any other tailored view (default view-default, type string, scope instance)
top_includelet
Specify an xowiki includelet (defined in ::xowiki::includelet::*) to be included on top over every pages. In order or add e.g. on each page of this wiki instance a listing of the users currently active in this xowiki instance, set the value of this parameter e.g. to 'presence -interval "10 minutes"'. The valid parameters are defined by the xowiki includlets. (default presence -interval "10 minutes", type string, scope instance)
use_hstore
Use hstore for accessing instance attributes. Create index for existing values via ::xowiki::hstore::update_form_instance_item_index -package_id $package_id (default 0, type number, scope instance)
Description: PackageDescription
Description of the Package. This description will appear for example in the Description of the default RSS feed for this package (default , type text, scope instance)
Description: PackageTitle
Title of the Package; this title will appear for example in the default RSS feed for this package. (default , type string, scope instance)
Form: autoname
Automatically name items of this instance (default 0, type number, scope instance)
Form: display_page_order
Display page_order attribute (Section) in edit forms if possible. (default 1, type number, scope instance)
Options: include_in_google_sitemap_index
Include the package in the google sitemap index (default 1, type number, scope instance)
Options: my_yahoo_publisher
Name of the publisher, when posting URLs to my yahoo (use in connection with with_yahoo_publisher) (default , type string, scope instance)
Options: production_mode
When this parameter is set, new pages are created in a "production" state, where they are not visible to users. These page have to be released via the admin pages. Notification and syndication is decativated, while pages are under "production". (default 0, type number, scope instance)
Options: show_page_references
If enabled it shows on a page view the pages that reference it (default 1, type number, scope instance)
Options: show_per_object_categories
If enabled it shows on a page view the categories assigned to this object (default 1, type number, scope instance)
Options: with_delicious
Add a delicious button to the page (default 0, type number, scope instance)
Options: with_digg
Add a button to submit article to digg (default 0, type number, scope instance)
Options: with_general_comments
use general comments package (default 1, type number, scope instance)
Options: with_notifications
Allow the user to register notifications (default 1, type number, scope instance)
Options: with_tags
allow user to provide tags to pages (default 1, type number, scope instance)
Options: with_user_tracking
track page view usage per user (default 1, type number, scope instance)
Options: with_yahoo_publisher
When specified, a button for adding the content to my.yahoo.com is added (default 0, type number, scope instance)
Pages: index_page
name of the page to be shown when the package instance is browsed (e.g. en:index) (default , type string, scope instance)
Pages: weblog_page
name of the page to show weblog (when clicking on tags) (default en:weblog, type string, scope instance)
URL: fallback_languages
Specify space delimited two character codes for checking default languages. When this parameter is non-empty, try to get the page in the specified languages as fallback rather than offering a link for creation of a page in the requested locale. Per default this parameter is empty. (default , type number, scope instance)
URL: package_prefix
Part of the URL used before language and page name. This setting is used, when a URL is computed. Per default, the package_prefix is determined by the side not. When for example a default xowiki instance is used as start page of OpenACS, the package_prefix can be set to / (default , type string, scope instance)
URL: subst_blank_in_name
normalize names of pages in a media wiki style. Most dominantly, spaces are turned into blanks. (default 1, type number, scope instance)
URL: use_connection_locale
When this flag is set, per-user specific information is used to determine the default language. Users with different language preferences will see under the same url different content. Per default this flag is turned off, and the package or system wide locale is used. (default 0, type number, scope instance)
xinha: WidgetSpecs
Specify pairs of "pagename,fieldname" followed by a Tcl list which is used as a widget spec for ad_form. "pagename" and "fieldname" can contain wild card characters. The following rather complex widget-specs are from OpenACS.org: *,text {richtext(richtext),nospell,optional {label Content} {html {style {width: 100%}}} {options {editor xinha plugins {Stylist OacsFs} height 350px javascript { xinha_config.toolbar = [ ['popupeditor', 'bold','italic','createlink','insertimage','separator'], ['killword','removeformat'] ]; xinha_config.stylistLoadStylesheet('/resources/xowiki/examples/xinha-mc-styles.css', {'p.angabe' : 'Aufgabenstellung', 'p.loesungshinweis' : 'Lösungshinweis', 'li.correct_choice' : 'Richtige Antwort', 'li.incorrect_choice' : 'Falsche Antwort'}); }}}} (default , type text, scope instance)


Bug Tracker Summary for Package: xowiki

Open Bugs: 3
All Tracked Issues: 52
Latest Bug Opened: 2018-12-12 Unnecessary dependency between folders includelet and MenuBar
Latest Bug Fixed: 2020-02-11 name-based selection of default values for category based form fields..
Top Bug Submitters: Robert Taylor (10) Michael Aram (9) Ryan Gallimore (9) Carl Robert Blesius (6) Malte Sussdorff (3)
Top Bug Fixers: Gustaf Neumann (47) Stan Kaufman (2) Malte Sussdorff (1)


Code Metrics Summary for Package: xowiki

# Tcl Procs 29
# Tcl Lines 32791
# Tcl Blank Lines 3298
# Tcl Comment Lines 6518
# Automated Tests 8
# Stored Procedures PG: 0 ORA: 0
# SQL Lines PG: 0 (blank 1 comments 0) ORA: 0 (blank 1 comments 0)
# ADP pages 23
# ADP lines 1698
# Include pages (xowiki/lib/) 1
# Documentation pages 0
# Documentation lines 0
Browse Source API-browser

Available Includelets

The following includelets can be used in a page
  • {{activity-graph -max_edges 70 -cutoff 0.1 -max_activities:integer 100 -show_anonymous message}}

    Include an activity graph

  • {{available-formfields -flat:boolean false}}

    List the available form field types of this installation.

  • {{available-includelets}}

    List the available includelets of this installation.

  • {{book -category_id -menu_buttons edit -folder_mode false -locale "" -range "" -allow_reorder "" -with_footer false}}

    Show contents in book mode.

  • {{bookmarklet-button -siteurl "" -label ""}}

    Include bookmarklet button that makes it easy to add the current page as a bookmark in the browser of the client.

  • {{categories -tree_name "" -tree_style:boolean 1 -no_tree_name:boolean 0 -count:boolean 0 -summary:boolean 0 -locale "" -open_page "" -order_items_by title,asc -style mktree -category_ids "" -except_category_ids "" -allow_edit false -ordered_composite}}

    List the specified category tree.

  • {{categories-recent -max_entries:integer 10 -tree_name "" -locale "" -pretty_age off}}

    Display recent entries by categories.

  • {{chat -title "" -chat_id "" -mode "" -path "" -skin -login_messages_p -logout_messages_p -avatar_p -timewindow}}

    Include a chat in the current page

  • {{chat_room -chat_id -mode:optional "" -path:optional ""}}

    Include a chat room

  • {{child-resources -skin:optional yui-skin-sam -show_types ::xowiki::Page,::xowiki::File,::xowiki::Form,::xowiki::FormPage -regexp:optional -with_subtypes:boolean,optional false -orderby:token,optional last_modified,desc -publish_status:wordchar ready -view_target "" -html-content -parent . -columns objects edit object_type name last_modified mod_user duplicate delete -hide "" -menubar ""}}

    Include the content of the current folder somewhat similar to explorer.

  • {{collab-graph -max_edges 70 -cutoff 0.1 -show_anonymous message -user_id}}

    Include a collaboration graph

  • {{community-link -text "" -url ""}}

    Include a link to the community including the current page. This includelet is designed to work with dotlrn.

  • {{composite-form -edit_links:boolean false -pages "" -ordered_pages}}

    Create a form from the selection

  • {{copy-item-button -page_id -alt copy -book_mode false}}

    Button to copy a page

  • {{countdown-timer -target_time ""}}

    Countdown timer

  • {{create-item-button -page_id -alt new -book_mode false}}

    Button to create a new page based on the current one

  • {{creation-date -source "" -format %m-%d-%Y}}

    Include the creation date of the current or specified page in the provided format.

  • {{current-irc-log -date ""}}

  • {{delete-item-button -page_id -title Delete -alt delete -book_mode false}}

    Button to delete the current or a different page

  • {{delicious -description "" -tags "" -url}}

    Add a button to submit article to delicious.

  • {{digg -description "" -url}}

    Add a button to submit article to digg.

  • {{edit-item-button -page_id -title Edit -alt edit -book_mode false -link "" -target ""}}

    Button to edit the current or a different page

  • {{flowplayer -mp4:required}}

    Include an mp4 image using flowplayer

  • {{folders -show_full_tree false -context_tree_view false}}

    List the folder tree of the current instance

  • {{form-menu -form_item_id:integer -parent_id -form -buttons new answers -button_objs -return_url}}

    Include a form menu for the specified Form

  • {{form-stats -form -parent_id -property _state -orderby count,desc -renderer table}}

    Include form statistics for the specofied Form page.

  • {{form-usages -form_item_id:integer -form -parent_id -package_ids "" -orderby _raw_last_modified,desc -view_field _name -publish_status all -field_names -hidden_field_names "" -extra_form_constraints "" -inherit_from_forms "" -category_id -unless -where -csv true -voting_form -voting_form_form "" -voting_form_anon_instances t -generate -with_form_link true -with_categories -wf -bulk_actions "" -buttons edit delete -renderer "" -return_url -date_format}}

    Show usages of the specified form.

  • {{get -variable -form_variable -source ""}}

    Get an instance variable from the current or from a different page.

  • {{graph}}

  • {{gravatar -email:required -size 80}}

    Include gravatar picture for the specified email

  • {{html-file -title "" -extra_css "" -levels 0 -file:required}}

    Include the specified HTML file

  • {{iframe -title "" -url:required -width 100% -height 500px}}

    Include an iframe contining the specified URL

  • {{item-button}}

  • {{jquery-carousel}}

    Display a sequence of pages via jquery-carousel, based on book includelet.

  • {{jquery-cloud-carousel}}

    Display a sequence of pages via jquery-cloud-carousel, based on book includelet.

  • {{jquery-infinite-carousel}}

    Display a sequence of pages via jquery-infinite-carousel, based on book includelet.

  • {{jquery-spacegallery}}

    Display a sequence of pages via jquery-spacegalleryl, based on book includelet.

  • {{kibana -chart openacs-status-codes -from now-24h -to now -hash "" -width:integer 800 -height:integer 400}}

    Include a Kibana chart identified by the provided hash

  • {{last-visited -max_entries:integer 20}}

    Display last visited pages.

  • {{link-with-local-return-url -text "" -url ""}}

    Insert a link with extra return URL pointing the current object. This is particularly useful in cases, where a return URL must be created for a page that does not yet exist at time of definition (e.g. for link pointing to concrete workflow instances)

  • {{most-frequent-visitors -max_entries:integer 15}}

    List the most frequent visitors.

  • {{most-popular -max_entries:integer 10 -interval}}

    Display most popular pages of this wiki instance.

  • {{my-categories -summary 1}}

    List the categories associated with the current page.

  • {{my-general-comments}}

    List the general comments available for the current page.

  • {{my-references}}

    List the pages which are referring to the current page.

  • {{my-refers}}

    List the pages which are referred to the current page.

  • {{my-tags -summary 1}}

    List the tags associated with the current page.

  • {{my-yahoo-publisher -publisher "" -rssurl}}

    Name of the publisher, when posting URLs to my yahoo (use in connection with with_yahoo_publisher).

  • {{presence -interval 10 minutes -max_users:integer 40 -show_anonymous summary -page}}

    Show users actively in the wiki.

  • {{random-form-page -form:required -publish_status ready -expires 600}}

    Include random form page (instance of the specified form)

  • {{recent -max_entries:integer 10 -allow_edit:boolean false -allow_delete:boolean false -pretty_age off}}

    Display recent modified entries.

  • {{references-graph -folder . -page "" -link_type link -rankdir LR -fontsize 12}}

    Include a graph of the (partial) link structure in a wiki, starting either with a page or a folder. When a page is provided, the local link structure of this page is visualized (including incoming and outgoing links of the page; e.g. -page "." for the current page). Alternatively, the content of a folder can be shown.

  • {{rss-button -span 10d -name_filter -entries_of -title}}

    Include an RSS button referring to pages of the specified time span.

  • {{rss-client -url:required -max_entries:integer 15}}

    Include RSS content

  • {{selection -edit_links:boolean true -pages "" -ordered_pages "" -source -menu_buttons edit -range ""}}

    Provide a selection of pages

  • {{set-parameter}}

    Set a parameter accessible to the current page (for certain tailorings), accessible in the page via e.g. the query parameter interface.

  • {{slidy}}

    Display a sequence of pages via W3C slidy, based on book includelet

  • {{tags -limit:integer 20 -summary:boolean 0 -popular:boolean 0 -page}}

    Display specified tags.

  • {{timeline -user_id -data timeline-data -interval1 DAY -interval2 MONTH}}

    Include a timeline of changes (based on yahoo timeline API)

  • {{toc -style "" -renderer "" -open_page "" -book_mode false -folder_mode false -ajax false -expand_all false -remove_levels 0 -category_id -locale "" -source "" -range "" -allow_reorder "" -include_in_foldertree true}}

    Show table of contents of the current wiki. The "toc" includelet renders the page titles of the current files based on the value of the "page_order" attributes. Only those pages are rendered that have a non-empty "page_order" field.

  • {{unread-items -max_entries:integer 20}}

    List unread items.

  • {{unresolved-references}}

    List the pages with unresolved references in the current xowiki/xowf package. This is intended for use by admins.

  • {{user-timeline -user_id -data timeline-data -interval1 DAY -interval2 MONTH}}

    Include a timeline of changes of the current or specified user (based on yahoo timeline API)

  • {{view-item-button -page_id -title View -alt view -link "" -book_mode false}}

    Button to view the current or a different page

  • {{wf-todo -workflow "" -user_id -ical 0 -max_entries}}

  • {{yui-carousel -title "" -item_size 600x400 -image_size -num_visible 1 -play_interval 0 -auto_size 0 -folder -glob "" -form ""}}

    Include YUI carousel showing the pages of the specified or current folder.

 

OpenACS TODO List

Created by Dave Bauer, last modified by Gustaf Neumann 21 May 2020, at 03:36 PM

Release Status

See openacs-release-status

Development is taking place in the oacs-5-10 branch.

OpenACS Version 5.10.0 Agenda/wish list

  • Postponed for later releases:
    • implement subsite-singleton (in addition to the classical singleton)
    • say farewell to CVS
    • Make more parts of xotcl-core a first-class citizen of the OpenACS core (e.g. database interface)
    • Data bloat hygiene:
      • rethink package parameter and portlet parameter data models
      • parameters: include "subsite-parameters" in parameter resolution (package->subsite->global)
         
  • Functional improvements
    • additional page_contract filter: oneof(red|green|blue)
       
    • add text/markdown to the accepted text formats or rich-text widget
       
    • Get rid of bugging "login page expired" messages. The 17 years old construct was replaced by newer means to avoid caching of form values from the login form. Admins of existing sites should set the kernel parameter "LoginPageExpirationTime" to 0
       
    • new API call "category::get" to obtain category description for a category_id and locale
    •  
    • Added doc(base_href) and doc(base_target) for setting <base> element via blank-baster (see issue #3435)
       
    • visualize code dependencies and test-coverage in API browser
       
    • warn site administrators about expiring certificates
       
    • dynamic reloading reform:
      • When reloading  apm packages, watched files, etc. the NaviServer blueprint is now updated correctly. This solves the long-standing problem that changes for scheduled procedures required a restart of the server. Furthermore the old-style reloading was based on a ever-growing list of reload operations each time a new reload was requested, causing slow-downs in long running server instances, especially, when new threads are started.
      • Now changes are applied to all NaviServer threads, including threads for
        schedules procedures, ns_jobs and the like. Note that the current job has to be finished before the changes can be applied.
      • For the transition, it is possible to switch between the classical reloading style and blueprint reloading by changing a variable in acs-tcl/tcl/apm-procs.tcl
         
    • new packages:
      • cookie-consent: alerting users about the use of cookies on a website
      • boomerang: performance of your website from your end user’s point of view
         
    • Forums reform:
      • remove hard-coded dependency with registered_users group when checking forum permissions
      • don't rely so heavily on acs_permissions to model forum configuration, as this can have unexpected consequences in convoluted multi-group/multi-subsite scenarios. Prefer simpler table attributes instead
         
    • Registry for .js and .css libraries: allow besides classical URLs symbolic names for loading external resources (e.g. jquery), this makes it easier to upgrade  libraries in multiple packages (without running into problems with duplicate versions) and supports  switching between CDN and local pathsURN.
      The existing implementation is based on URNs and extends the existing template-head API to support registration for URNs. A URN provides an abstraction and a single place for e.g updating references to external resources when switching between a CDN and a locally stored resource, or when a resource should be updated. Instead of adding e.g. a CDN URL via template::head::add_script, one can add an URN and control its content from a single place. Use common namespaces for OpenACS such as urn:ad:css:* and urn:ad:js:*.
      • Register URNs:

        Example provider (e.g. in some theme):
           template::register_urn \ 
               -urn urn:ad:js:jquery \ 
               -resource /resources/xowiki/jquery/jquery.min.js
        
      • The registered URN can be used like classical URL after registration.
        Example consumer:

           template::head::add_javascript -src urn:ad:js:jquery
      • Declare composite files: Provide an interface to define that  a .js file or a .css file contains multiple other .js/.css files in order to reduce the number of requests.
           template::head::includes -container urn:js::style.js -parts {urn:ad:js:jquery ...}
    • TODO: improved support for streaming HTML: currently, all template::script or CSP contents are lost, when streaming HTML (incremental HTML) is used, since only the blank.master collects this data. Some HTML-finalize (or end_of_HTML) call is should be factored out to handle such cases in a uniform way.
       
  • non-functional changes
    • Improved regression testing
      • Goal is that all packages in oacs-5-10 pass regression test
      • Include web testing in standard regression testing
         
    • Improved scalability:
      • provided lock-free implementation of ad_page_contract_filters and ad_page_contract_filter_rules. This change improves parallel processing of requests and is primarily interesting for sites with a few mio page views per days. These locks were among the most frequent nsv locks

      • Reduced locks on util_memoize_cache my more invariants values into per-thread caching (acs_lookup_magic_object, ad_acs_version, .... ) and by avoiding specialized calls, which can be realized by already optimized ones ("apm_package_installed_p_not_cached ref-timezones" was most frequently used util_memoize_cache entry). These changes are necessary to avoid full still-stand on the unfortunate long-time locks on util_memoize_cache stemming from permission and user management with wild-card flush operations, which require to iterate over all cache entries (which might be a server hundred thousands)

      • Added new interface for cache partitioning to reduce lock latencies on high load websites

      • Added new interface for lock-free per-thread and per-request caching to avoid scattered ad-hoc implementations

      • Better reuse of DB handles (reduced expiring/reopen/etc.), faster access to handles

    • Code smell reduction: fix typos, white-space etc.
    •  
    • Improved startup time:
      • when the package acs-automated-testing is disabled, startup time is reduced by avoiding loading of support functions and tests; the size of the blueprint is reduced
      • xowf: loading of at-jobs is significantly improved.
         
    • Security improvements:
      • strengthened page contracts
      • CSP support for application packages
      • CSP fine tuing
         
    • Better exception handling based on Tcl 8.6 exception handlers ("try" and "throw", also available in Tcl 8.5)
      • a new "ad_try" implementation based on Tcl's "try" replaces now the old "ad_try", "with_catch" and "with_finally", which are marked as deprecated
      • the new "ad_try" is in essence Tcl's "try" but with predefined handling of "ad_script_abort" and should be also used instead of "catch", when the OpenACS API is used (which might use script aborts)
      • all core packages use the new "ad_try" instead of the deprecated versions
         
    • Connection close reform:
      • NaviServer/AOLserver continue after connection closing commands to execute a script. This is in many situation not desired, especially, when for the page as well a .adp file exists, which will try to deliver this on the already closed connection. This can lead to errors in the error.log file, which are sometimes hard to analyze
      • therefore, developers should use in most such cases cases "ad_script_abort"
      • connection closing commands are e.g. ad_returnredirect, ad_redirect_for_registration, cr_write_content, ad_page_contract_handle_datasource_error, ad_return_string_as_file, ad_return_complaint,  ad_return_error,
        ad_return_forbidden, ad_return_warning, ad_return_exception_page, ns_returnredirect, ns_return, ns_returnerror
      • The new version has made on most occasions explicit, when the script should abort.
         
    • SQL cleanup:
      • cleanup of obsolete non-portable SQL constructs in a way Oracle and PostgreSQL code base divergency is reduced:
        • "nvl" -> "coalesce"
        • "sysdate" / "now()" -> standard "current_date" or "current_timestamp"
        • use standard-compliant "dual" table where appropriate (required by Oracle, supported by PostgreSQL)
        • use non-dialectal cast idioms when appropriate
        • adopt CTE idioms in Oracle codebase as well (e.g. connect -> with recursive)
        • ... (reference Oracle version will be 11gr2 as is oldest version officially supported by Oracle (See here and here)
      • reduce superfluous .xql queries
        • acs-subsite: delete 21 files with un-referenced .xql queries
        • acs-tcl: delete 4 files
        • news: 3 files
        • file-storage: 1 file
        • dotlrn: 9 files
           
    • Deprecated commands:
      • move deprecated code into separate files
      • make loading of deprecated code optional (can be controlled via parameter  "WithDeprecatedCode" in section "ns_section ns/server/${server}/acs" of the config file. By default, deprecated procs are still loaded
      • When deprecated code is not loaded, the blueprint of the interpreter is smaller. The following number of lines of code can be omitted when loading without the deprecated procs:
        • acs-tcl: 3178
        • acs-templating: 450
        • xotcl-core http-client-procs: 830
        • acs-content-repository: 1717 (including .xql files)
           
    • Improved handling of server implementation-specific code
      • provide *-procs-aolserver.tcl and *-procs-naviserver.tcl similar to *.postgresql.xql and *.oracle.xql where appropriate
         
  • Require Tcl 8.6, XOTcl 2.1, PostgreSQL 9.5 (PostgreSQL 9.4 EOL: February 2020), tdom 0.9
     

OpenACS Version 5.9.1 Agenda

Refactoring of rich-text editor integration

  • Driving force: Debian packaging (e.g. js minified code is not allowed)
  • Moved out code from acs-templating, provided interfaces to add many different rich-text editors as separate packages
  • New OpenACS packages:
    • richtext-xinha
    • richtext-tinymce
    • richtext-ckeditor4 (has ability to choose between CDN and local installation via web interface)

Improving admin interface

  • New theme manager:
    • Goals:
      • Make it easier to keep track of themes with local modifications
      • Make it easier to create local modification a new themes and to update these
      • Show differences between default theme parameter (in DB) and actual settings (in subsite parameters)
      • Allow to delete unused themes
      • Give site admin hints, which theme is used at which subsite
      • Ease theme switching
    • Added a subsite::theme_changed callback to be able to handle theme changes in custom themes (was also necessary for proper integration with DotLRN theming)
    • Added support for these features under subsite admin (/admin/)
    • Improved support for themed templates via [template::themed_template]

       

  • Improved (broken) interface to define/manage groups over web interface
  • Allow to send as well mail, when membership was rejected
  • New functions [membership_rel::get_user_id], [membership_rel::get] and [membership_rel::get_group_id] to avoid code duplication
  • Added support to let user include %forgotten_password_url% in self-registration emails (e.g. in message key acs-subsite.email_body_Registration_password)

     

  • Improved subsite/www/members
    • Make it possible to manage members of arbitrary groups
    • Improved performance for large groups
    • Improved configurability: when ShowMembersListTo is set to "3", show list to members only, when this is not the whole subsite

       

  • Improved user interface for /admin/applications for large number of applications
  • Various fixes for sitewide-admin pages (under /acs-admin)
  • Update blueprint in "install from repository" (currently just working in NaviServer)

SQL

  • Further cleanup of .xql files (like what as done for acs-subsite in OpenACS 5.9.0):
    • 36 files deleted
    • Removed more than 100 obsolete named queries
    • Stripped misleading SQL statements

       

  • Marked redundant / uncalled SQL functions as deprecated
  • Replaced usages of obsolete view "all_object_party_privilege_map" by "acs_object_party_privilege_map"
     
  • Removed type discrepancy introduced in 2002:
    • acs_object_types.object_type has type varchar(1000), while
    • acs_object_types.supertype has type varchar(100)
    • ... several more data types are involved, using acs_object_types.object_type as foreign key

       

  • Simplified core SQL functions by using defaults:
    • Number of functions reduced by a factor of 2 compared to OpenACS 5.9.0 (while providing compatibility for clients using old versions),
    • Reduced code redundancy
    • Affected functions:
      • Reduced content_item__new from 12 versions to 6,
      • Reduce content_revision__new from 7 to 4
      • Similar in image__new, image__new_revision, content_item__copy, content_item__get_title, content_item__move
    • PostgreSQL 9.5 supports named parameter in the same syntax as in Oracle. Further reduction of variants will be possible, once OpenACS requires at least PostgreSQL 9.5

       

  • Reduced usage of deprecated versions of SQL functions (mostly content repository calls)
  • Reduced generation of dead tuples by combining multiple DML statements to one (reduces costs of checkpoint cleanups in PostgreSQL)

     

  • Permission queries:
    • Improved performance
    • Support PACKAGE.FUNCTION notation for PostgreSQL to allow calls permission queries exactly the same way as in Oracle (e.g. "acs_permission.permission_p()"). This helps to reduce the number of PostgreSQL specific .xql files.
  • Modernize SQL:
    • Use real Boolean types instead of character(1)
      (done for new-portal, forums, faq, attachments, categories, dotlrn, dotlrn-forums, evaluation)
    • Use real enumeration types rather than check constraints (done for storage_type text/file/lob)

CR hygienics (reduce cr bloat)

  • Provided means to avoid insert/update/delete operations in the search queue:

    OpenACS adds for every new revision often multiple entries to the search_queue, without providing any means to prevent this. This requires for busy sites very short intervals between queue sweeps (otherwise too many entries pile up). Another consequence is that this behavior keeps the PostgreSQL auto-vacuum daemons permanently active. Many of these operations are useless in cases where the content repository is used for content that should not be provided via search. The changed behavior should honors a publish-date set to the future, since it will not add any content with future publish dates to the search-queue.

     

  • Reduced number of insert cr_child_rels operations, just when needed:

    cr_child_rels provide only little benefit (allow to use roles in a child-rel), but the common operation is a well available in cr_items via the parent_id. cr_child_rels do not help for recursive queries either. One option would be to add an additional argument for content_item__new to omit child-rel creation (default is old behavior) and adapt the other cases.

Security improvements

  • Added support against CSRF (cross site request forgery)
    • OpenACS maintains a per-request CSRF token that ensures that form replies are coming just from sites that received the form
    • CSRF support is optional for packages where CSRF is less dangerous, and such requests are wanted (e.g. search and api-browser)
  • Added Support for W3C "Upgrade-Insecure-Headers" (see https://www.w3.org/TR/upgrade-insecure-requests/):
    For standard compliant upgrade for requests from HTTP to HTTPS

  • Added support for W3C "Subresource Integrity" (SRI; see https://www.w3.org/TR/SRI/)

  • Added support for W3C "Content Security Policy" (CSP; see https://www.w3.org/TR/CSP/)

    • Removed "javascript:*" links (all such urls are removed from the 90 packages in oacs-5-9, excluding js libraries (ajaxhelper) and richtext code)
    • Removed "onclick", "onfocus", "onblur", "onchange" handlers from all .adp and .tcl files in the 90 packages in oacs-5-9 (excluding js libraries (ajaxhelper) and richtext code)
    • Added optional nonces to all <script> elements with literal JavaScript content

       

  • Removed "generic downloader", which allowed to download arbitrary content items, when item_id was known (bug-fix)
  • Improved protection against XSS and SQL-injection (strengthen page contracts, add validators, added page_contract_filter "localurl", improve HTML escaping, and URI encoding)
  • Fixed for potential traversal attack (acs-api-documentation-procs)

Improvements for "host-node mapped" subsites

  • Fixed links from host-node mapped subsite pages to swa-functions (must be always on main subsite)
  • Made "util_current_directory" aware of host-node-mapped subsites
  • Added ability to pass "-cookie_domain" to make it possible to use the same cookie for different domains
  • Fixed result of affected commands "util_current_location", "ad_return_url", "ad_get_login_url" and "ad_get_logout_url" for HTTP and HTTPS, when UseHostnameDomainforReg is 0 or 1.
  • Improved UI for host-node maps when a large number of site nodes exists

Reform of acs-rels

  • Made acs-rels configurable to give the developer the option to specify, whether these are composable or not (default fully backward compatible). This is required to control transitivity in rel-segments
  • The code changes are based on a patch provided by Michael Steigman.
    For details, see:
    • https://openacs.org/forums/message-view?message_id=4031049
    • https://openacs.org/forums/message-view?message_id=5330734

Improved status code handlers for AJAX scenarios

  • Don't report data source errors with status code 200 (use 422 instead)
  • Let "permission::require_permission" return forbidden (403) in AJAX calls (determined via [ad_conn ajaxp])

Improved Internationalization

  • Extended language catalogs for
    • Russian (thanks to v v)
    • Italian (thanks to Antonio Pisano)
    • Spanish (thanks to Hector Romojaro)
    • German (thanks to Markus Moser)

       

  • Added (missing) message keys
  • Improved wording of entries
  • Added message keys for member_state changes, provide API via group::get_member_state_pretty

Improved online documentation (/doc)

  • Fixed many broken links
  • Removed fully obsolete sections
  • Improved markup (modernize HTML)
  • Updated various sections

Misc code improvements:

  • 18 issues from the OpenACS-bug-tracker fixed
  • Made code more robust against invalid/incorrect input (page_contracts, validators, values obtained from header fields such as Accept-Language)
  • Fixed quoting of message keys on many places
  • Improved exception handling (often, a "catch" swallows to much, e.g. script_aborts), introducing "ad_exception".
  • Generalized handling of leading zeros:

    • Fixed cases where leading zeros could lead to unwanted octal interpretations
    • Switch to use of " util::trim_leading_zeros" instead of "template::util::leadingTrim", "dt_trim_leading_zeros" and "template::util::leadingTrim", marked the latter as deprecated
  • URL encoding

    • "ad_urlencode_folder_path": new function to perform an urlencode operation on the segments of the provided folder path
    • "export_vars": encode path always correctly, except -no_base_encode is specified
    • Fixed encoding of the URL path in "ad_returnredirect"
  • Improvements for "ad_conn":

    • Added [ad_conn behind_proxy_p] to check, whether the request is coming from behind a proxy server
    • Added [ad_conn behind_secure_proxy_p] to check, whether the request is coming from behind a secure proxy server
    • Added [ad_conn ajax_p] to check, whether the request is an AJAX requests (assumption: AJAX request sets header-field Requested-With: XMLHttpRequest")
    • Added [ad_conn vhost_url] to obtain the url of host-node-mapped subsites

       

  • Added various missing upgrade scripts (missing since many years) of changes that were implemented for new installs to reduce differences between "new"-and "old" (upgraded) installations
  • Templating

    • Get rid of various pesky "MISSING FORMWIDGET: ...formbutton:ok" messages
    • Improved support for javascript event handlers in template::head
    • New functions "template::add_event_listener" and "template::add_confirm_handler"
    • Fix handling, when "page_size_variable_p" is set (was broken since ages)
  • Improved location and URL handling:

    • Refactored and commented "util_current_location" to address security issues, handle IPv6 addresses, IP literal notation, multiple drivers, "
    • Improved "security::get_secure_location" (align with documentation)

       

    • New functions:
      • "util::configured_location"
      • "util::join_location", "util::split_location"
      for working on HTTP locations to reduce scattered regexps handling URL components
    • Improved IPv6 support
    • Use native "ns_parseurl" when available, provide backward compatible version for AOLserver
  • MIME types:

    • Added more Open XML formats for MS-Office to allowed content types
    • Modernized entries to IANA recommendations
    • New function "cr_check_mime_type" centralizing the retrieval of the mime_type from uploaded content
  • Finalized cleanup of permissions (started in OpenACS 5.9.0):

    • Get rid of "acs_object_context_index " (and therefore on "acs_object_party_privilege_map " as well) on PostgreSQL.
      Reasons:
      • huge table,
      • expensive maintenance, used only in a few places,
  • Misc new functions:

    • "lang::util::message_key_regexp": factor out scattered regexp for detecting message keys
    • "ns_md5" and "ns_parseurl": improve compatibility between AOLserver and NaviServer
    • "ad_dom_sanitize_html": allow to specify different sets of tags, attributes and protocols and "ad_dom_fix_html", which is a light weight tidy variant.

       

  • Improved HTML rendering (acs-api-browser), provide width and height to speed up rendering
  • Improved ADP files (e.g. missing doc(title))
  • Added usage of "ad_include_contract" on more occasions
  • Modernize Tcl and HTML coding
  • Reduced dependency on external programs (use Tcl functions instead)
  • Improved robustness of "file delete" operations all over the code
  • Improved documentation, fix demo pages
  • Aligned usages of log notification levels (distinction between "error", "warning" and "notice") with coding-standards

     

  • Cleaned up deprecated calls:
    • Removed usage of deprecated API functions (e.g. "cc_lookup_email_user", "cc_email_from_party", "util_unlist", ...)
    • Moved more deprecated procs to acs-outdated
    • Marked remaining (and unused) "cc_*" functions as well as deprecated.

       

  • Improved Oracle and windows support
  • Fixed common spelling errors and standardize spelling of product names all over the code (comments, documentation, ...)
  • Many more small bug fixes

Version numbers:

  • require PG 9.2 (End Of Life of PostgreSQL 9.0 was Oct 2015)
  • require XOTcl 2.0 (presented at the Tcl conference in 2011).

 

Packages:

 

  • New Package Parameters

    • acs-kernel:

      • MaxUrlLength: remove hard-coded constant in request processor for max accepted url paths
      • SecureSessionCookie: Let site admin determine, whether or not to use secured session cookies (useful, when not all requests are over HTTPS)
      • CSPEnabledP: activate/deactivate CSP
    • acs-kernel (recommended to be set via config file in section "ns/server/${server}/>acs"

      • NsShutdownWithNonZeroExitCode: tell NaviServer to return with a non-zero return code to cause restart (important under windows)
      • LogIncludeUserId: include user_id in access log
    • acs-api-browser:

      • ValidateCSRFP: make checking of CSRF optional (default 1)
    • acs-content-repository:

      • AllowMimeTypeCreationP: Decides whether we allow unknown mime types to be automatically registered (default: 0}
    • news-portlet:

      • display_item_lead_p: Should we display news leads in the portlet? (default 0)
    • search:

      • ValidateCSRFP: make checking of CSRF optional (default 1)
    • xotcl-request-monitor:

      • do_track_activity: turn activity monitoring on or off (default 0)
         
  • New OpenACS packages:

    • richtext-xinha
    • richtext-tinymce
    • richtext-ckeditor4 (has ability to choose between CDN and local installation via GUI)
    • openacs-bootstrap3-theme (as used on openacs.org)
    • dotlrn-bootstrap3-theme
  • xotcl-core:

    • Improved XOTcl 2.0 and NX support (e.g. api-browser)
    • Added "-debug", "-deprecated" to ad_* defined methods (such as e.g. "ad_instproc")
    • Make use of explicit "create" statements when creating XOTcl/NX objects (makes it easier to grab intentions and to detect typos)
    • Added parameter to "get_instance_from_db" to specify, whether the loaded objects should be initialized
    • Added support for PostgreSQL prepared statements of SQL interface in ::xo::dc (nsdb driver)
  • xowiki:

    • Named all web-callable methods www-NAME (to make it clear, what is called, what has to be checked especially carefully)
    • Moved templates from www into xowiki/resources to avoid naming conflicts
    • Improved ckeditor support
    • Added usage of prepared statements for common queries
    • Improved error handling
    • Better value checking for query parameter, error reporting via ad_return_complaint
    • Added option "-path_encode" to methods "pretty_link" and "folder_path" to allow to control, whether the result should be encoded or not (default true)

       

    • Form fields:
      • Improved repeatable form fields (esp. composite cases), don't require preallocation (can be costly in composite cases)
      • Added signing of form-fields
      • Added HTML5 attributes such as "multiple" (for "file") or "autocomplete"
      • Fixed generation of "orderby" attribute based on form-field names
      • richtext: allow to specify "extraAllowedContent" via options
      • Improved layout of horizontal check boxes

         

    • Menu bar:
      • Added dropzone (requires bootstrap): drag and drop file upload
      • Added mode toggle (requires bootstrap)
      • Extended default policies for handling e.g. dropzone (file-upload method)
      • Distinguish between "startpage" (menu.Package.Startpage) and "table of contents" (menu.Package.Toc)

         

    • Notifications:
      • Added support for better tailorable notifications: introduced method "notification_render" (similar to "search_render")
      • Added support for tailorable subject lines (method "notification_subject")

         

    • Improved bootstrap support, use "bootstrap" as PreferredCSSToolkit
    • Switched to ckeditor4 as PreferredRichtextEditor
    • Improved handling of script-abort from within the payload of ::xowiki::Object payloads
    • Added parameter to "get_all_children" to specify, whether the child objects should be initialized
  • xowf:

    • Added property "payload" to "WorkflowConstruct" in order to simplify customized workflow "allocate" actions
    • Internationalized more menu buttons
  • xotcl-request-monitor

    • Added class "BanUser" (use. e.g. ip address to disallow requests from a user via request monitor)
    • Added support for optional user tracking in database
    • Added support for monitoring response-time for certain urls via munin
    • Increased usage of XOTcl 2.0 variable resolver (potentially speed improvement 4x)
    • Performed some refactoring of response-time handling to allow site-admin to make e.g. use of NaviServer's dynamic connection pool management (not included in CVS)
    • Added support for partial times in long-calls.tcl to ease interpretation of unexpected slow calls
    • last100.tcl: Don't report hrefs to URLs, except to SWAs
  • chat:

    • Introduced new options to set chat rooms so login and/or logout message are not issued every time a user enters/exits a chat-room (important for chats with huge number of participants)
    • Parameterized viewing of chat-logs
    • Fixed cases of over-/under-quoting
    • Fixed JavaScript for IE, where innerHTML can cause problems
  • file-storage:

    • Don't show action keys, when user has no permissions
    • Added support for copying of same-named files into a folder (adding suffix)
    • Fixed old bugs in connection with "views" package

OpenACS Version 5.9.0 Agenda

  • Slimming pg SQL core:
     
    • Part 1: improve performance of object deletion
      • remove manual delete operations from acs_object__delete()
         
    • Part 2: content-repository - manual referential integrity management
      • handle referential integrity via pg's integrity constraints rather by functions cr_revision_del_ri_tr, cr_revision_ins_ri_tr, cr_revision_up_ri_tr, cr_revision_del_rev_ri_tr, and cr_revision_del_rev_ri_tr
      • fix broken/missing upgrade scripts from earlier updates
         
    • Part 3: content-repository - manual deletions and nulling
      • Removed manual nulling of live_revision and latest_revision
      • Removed manual deletion of old_revision and new_revision in cr_item_publish_audit
      • Removed manual deletion of item_id in cr_item_publish_audit, cr_release_periods, cr_item_template_map, and cr_item_keyword_map
      • Removed manual deletion of direct permissions
      • Added missing index for child_id to cr_child_rels.
         
    • Part 4: get rid of tree_sortkey in acs-objects
      • Check/fix dependencies in oacs-5-8 packages
      • Get rid of broken/uncalled functions using the column
      • Check/fix dependencies in other packages
      • Remove tree_sortkey and max_child_sortkey
  • Web interface:
    • Improve client performance
      • moving core.js from head to body
      • provide kernel parameter to control expiration date for /resources/
    • Protect against more XSS attacks
    • Improved HTML validity (see oacs-5-9-html-validity for the checklist)
    • Add lightweight support for ckeditor4 for templating::richtext widget (configurable via package parameter "RichTextEditor" of acs-templating. ckeditor4 supports mobile devices (such as iPad, ...).
    • New kernel parameter ResourcesExpireInterval to control expiration dates of resources
       
  • Templating:
    • Improve theme-ability
      • Move more information into theme packages in order to create responsive designs
      • Reduce hard-coding of paths, HTML etc.
    • Dimensional slider reform (ad_dimensional):
      • Remove hard-coded table layout from dimensional slider
      • Add backwards compatible templates
      • Move hard-coded styles into theme styling
      • Remove obsolete comments from ad_dimensional
    • Complete template variable controls (adding noi18n, addressing bug #2692):
      • @foo@: perform html quoting and internationalization
      • @foo;noquote@: perform internationalization
      • @foo;noi18n@: perform html quoting
      • @foo;literal@: perform neither html quoting nor internationalization
    • Improved Russian nationalization
    • Support of expiration dates and passwords for signed variables
       
  • Documentation:
    • Use ACS templating for the (static) OpenACS documentation to provide a more consistent layout and user experience.
    • Make pretty-naming of acs-core packages more consistent.
       
  • Misc improvements:
    • Mark unused functions of acs-tcl/tcl/table-display-procs.tcl as deprecated
    • Reduce number of muxtex locks by pre-request and per-thread caching
    • Improved development und debugging aids:
      • use "ad_log error|warning  .... " instead of "ns_log" to include information of request and callstack in error.log
      • ability to display ns_log entries caused by a request in ds-footer
      • ability to save delivered web pages in file-system for testing HTML validity (especially for admin pages, which are unaccessible for external validity testers)
    • More bug fixes
       
  • Version numbers:
    * require PG 9.0 (End Of Life of PostgreSQL 8.4 was July 2014)
    * require XOTcl 2.0 (presented at the Tcl conference in 2011).
     

OpenACS Version 5.8 Agenda

  • PostgreSQL 9.2+:
    • Get rid of nonstandard backslash escapes in function definitions
    • Change quote syntax in sql files (single quotes around the functions) to recommended PostgreSQL quoting using (recommended since pg8.0, jan 2005). li>Drop aliases in favor of named function arguments (recommended since pg8.0)
    • Fix wrong function_args, add missing function_args, align default semantics with the defaults in pg (providing "null" as default means the argument is optional)
    • Make OpenACS loadable without any tweaks in the pg config files
  • Use recursive queries for e.g. permission lookup to avoid performance problems in pg 8.4 and newer)
  • ADP: Use byte-compiled function wherever possible in compiled adp-code, support "@var;literal@" when neither quotes nor localization is needed in compiled adp-code
  • Improve support of NaviServer
  • Switch to Tcl 8.5 (TIP #143)
  • Improve scalability: Reduce mutex-stress on util-memoize cache and for cache maintenance in general
  • Code cleanup:
    • Get rid of calls to deprecated code (e.g. ad_tables, ad_parameter, ... in acs-core and main packages)
    • Improve awareness of usage of deprecated code (complain to error.log)
    • Use Tcl 8.5 idioms
    • cleanup of various http-client approaches and introduce a common implementation util::http::get and util::http::post; get rid of other usages, mark these as deprecated
    • page-contracts: Perform checking of all ids in acs-core and main packages to improve error messages and to improve security
  • OpenACS 5.8.1 should be released with main packages

OpenACS Version 5.7 Agenda

  • Support for object management in core 
  • Postgresql 9.0
  • TinyMCE update (fix for random JS injection issue, affecting Safari)
  • Fix for "remember me" issue
  • WCAG2-AA

OpenACS Version 5.6 Agenda

  • global parameters
  • package "embeds" 
  • fix search by package_id
  • core works on Postgresql 8.4

OpenACS Version 5.5 Agenda

  • DONE: Postgresql 8.3 support: especially regarding tsearch2
  • DONE: acs-authentication:
    • fix upgrade, add conditional logic into site wide tcl library so that you can login to perform the rest of the upgrade
  • DONE: tinymce:
    • upgrade to 3.1.1 + language packs
    • HTML Strict cleanup
    • create appropriate parameters for its config in acs-templating
  • acs-mail-lite:
    • DONE: cleanup duplicated procs (bounce)
    • review the parsing of bouncing messages (case user_id 0)
    • DONE: rollout support
  • Documentation improvements as discussed at the Guatemala conference:
    • Make current openacs.org/test-doc source for static files included in the release and provide ease means to achieve this for the release manager
      • DONE (CVS HEAD): Provide in XoWiki an alternative table of contents by nested UL/LI (without JavaScript) for static output
      • DONE (CVS HEAD): Provide in XoWiki a prototype page similar to "book" without edit-buttons etc., using the new table of contents
    • Update openacs.org/test-doc where necessary (incomplete list):
      • DONE: Fix the page ordering for the higher chapters (the original document  had no 3rd. level numbering)
      • update pages in /test-doc which are more recent in openacs/xowiki
      • bump version numbers of OpenACS, where appropriate (some places talk about openacs-5-0, others about openacs-5-1, oacs-5-2-3rc1 or 5-3) 
      • some version numbers of the required components are quite a mess. e.g. some parts say that Postgres 7.3 is required,  some examples talks about postgres 7.4.7 and 8.2.4 in the same listing.
      • also the dotlrn version numbers are old dotrln-2.0
      • Tcl version numbers should be 8.4.19
      • The install section for XOTcl is missing in II.3.4
      • remove ChangeLog from documentation
      • find some other prominent place for the ChangeLog
      • Fix indenting in examples  (e.g. in Rocael's robust web    development framework)
      • overthink Win2000 guidelines.  There are the native compiled packages from Maurizio, including everything from postgres, xotcl ....
    • It is desired to find a single person responsible for overworking the documentation, however, funding is unclear.

OpenACS Version 5.4 Agenda

  • DONE: HTML Strict (openacs core)
  • DONE: finish template::head (daveb)
  • DONE: test acs-mail-lite (complex send)
  • DONE: test notifications (complex send)
  • DONE: new XinHA release, get rid of RTE & HTMLarea, test on Safari
  • DONE: Form builder: add the ID attribute  to the form tag
  • DONE: acs-lang - keepLocalTranslationP to be removed
  • DONE search and intermedia-driver: move intermedia specific stuff to its package
  • DONE: acs-mail-lite - patch for mime::qp_encode bug

Future

  • Split Xinha and TinyMCE into separate packages see: https://openacs.org/forums/message-view?message_id=2750958
  • Usability ("my account" page)
  • XHTML ?
  • Testing and documentation for recording automated tests using the firefox plugin and the upload feature for it new in automated testing. Probably needs some polishing and should be talked to with Quest who are getting into this.
  • Parameter Scope Patch https://openacs.org/bugtracker/openacs/patch?patch%5fnumber=845 
  • Remove obsolete master template stuff (default and site master template in openacs-4/www, acs-subsite's group-master, and related CSS and images).  Probably in the version which follows 5.5 (probably 5.6).  Also remove the compat master stuff at the same time.

Things to merge into this page

Old 5.0 Roadmap  discussion 

Roadmap discussion 1 

Ideas for Boston 2006 Future of OpenACS discussion

A .LRN Roadmap 

Another .LRN Roadmap discussion 

 

What's on this page?

This page should include work that is planned on and has someone committed to working on it.

Site Wide Image Upload Widget

Created by Dave Bauer, last modified by Gustaf Neumann 01 May 2020, at 04:50 PM

MGH and Solution Grove are collaborating on a new image upload widget that will hopefully be easier to use and more easily integrated into any Xinha text element. 

Here is how it will work. The user will be editing the content of an object using a rich text editor (XINHA). If they decide an image or reference to a file is needed in the content, they can click the attach-file or attach-image icon.  A popup window appears allowing the user to choose a file to upload. The file is uploaded and stored in the content repository. If it is an image a thumbnail will be generated and linked to the full size image. When the content of the object is saved, the content is scanned for references to files or images, and a link is stored in the database that the particular object is "using" a file or image. This is the simplest case. Some more complex cases include allowing the user to search or browse images they have previously uploaded in a popup window to choose and image to insert.
  Insert_Image.jpg

The use case we are trying to solve:

  • In assessment, right now an admin can associate one file or image with a question. The admin can already enter HTML for the question description. We need to allow image upload from Xinha so the admin can add an arbitraty number of images to one question description. We need this functionality in any richtext widget within the assessment creation process. 

 Requirements:

 

  • upload image without thinking about where it will go, just entry optional title, description and choose a file from your Desktop
  • image will have parent_id=package_id where its created.   if an object is used in multiple pages, we will keep track of that and let the owner see where it is used. We'll need to write a parser to extract /image/ url links. (DONE uses application_data_link feature from acs-tcl)
  • image will inherit permissions depedning on the setting of the user. In general it'll inherit from the package_id or the subsite where its created.
    • To avoid namespace conflicts we can do someting like this for the cr_items.name
        • cr_items.name = '{$item_id} {$filename}' or automatically rename the item filename(1) (2)...  (DONE, we strip the item id on display)
    • Image may be "private" which means it does not inherit permissions from any other object. This means the only person who can see it is the owner. If the image is uploaded within the context of another object, ie Xowiki page, a link is created between the page and the image. If someone has permission to view he page, they can also see the image. We make this work by creating a special URL /image/${image_id}/private/${xowiki_page_id}/${filename} This generated a unique URL that includes the "viewing context". When this URL is accessed 1) the system is checked to make sure a data link exists, that is, that the image is used "in" the viewing context object, and that the viewing user has permission to see the viewing context object.
    •  
  • User uploading image will be granted direct "admin" permission over the image  DONE
  • images will have a centralized URL for delivery, ie: an acs-subsite/www/images.vuh similar to o.vuh so urls will appear to be /images/${item_id}/image-filename.jpg (for example)  (Thanks to gustaf to notice the obvious namespace conflict with just using filename)  DONE
  • images are stored as image type in the content repository (stored with "file" storage type) DONE
  • thumbnails will be generated (propose to use the image-magick package, I have some code to contribute that will process an image upload and generate a thumbnail (this will be disabled if image-magick package is not installed) 
    • A new image thumbnail API is DONE

Once we have this in place, we have a simple, site-wide solution to upload image attachments to any object. This can be used in places like forums etc, and solves the problem of figuring out how to configure each Xinha widget in every package to find a place to store images.  By attaching the image directly to the object we solve the issue of finding a place to store images.

 

Future Ideas

 

These ideas are not planned for an initial implementation, but definitely are on our TODO list

  • Add a UI to browse images you uploaded
  • Add a UI to allow users to browse any images they can read?
  • Add a way to search metadata on images when choosing an image so when you choose the add image button in Xinha, it will allow you to search existing images you have uploaded
  • Add a recently viewed/uploaded feature to the image picker, so you can choose an image you recently viewed or uploaded without searching for it
    • Here is an example of what that could look like (actual working code exists for this now)
  • Add a clipboard feature to the image picker, so you can use the optional clipboard package to mark an image in your clipboard while viewing it, then choose it from the clipboard to insert a link into another object. 

 

 



 

2006 November 3rd and November 4th (Training and Hacking Days)

Created by Carl Robert Blesius, last modified by Gustaf Neumann 01 May 2020, at 04:49 PM

Training and Hacking Days

The training and hacking days are part of the Fall_Conference_2006 and will be held in parallel at the Boston Museum of Science (details to follow).

Newbie Training for OpenACS/.LRN

This will be held in a beautiful classroom at the Musuem of Science with about 20 computers and taught by Tracy Adams.  This is an overview course designed to give you cover just enough of the right topics to lay the foundation for more advanced work.  The objective of the day will be to

1) Get hand-on introduction to the very basics
2) Get a high-level introduction to the more advanced topics and know where to go for more.

Agenda

9:30-10 - Preparation time, download and explore your virtual server image 

10-11 - The Basics

  • TCL Basics - set, list
  • ADP Basics - tags
  • Templating Basics
  • Hands on exercise

11-12 - Package Management, Site Map, Request Processor

  • Package Manager, Installation Example
  • Site Map Tour - Mounting
  • Hands on Exercise - Modifying Site Map

12-1 - Basic Apis

  • Procedure Documentation - api_doc
  • URL Parameters - ad_page_contract
  • Request Parameters - ad_conn
  • Protecting pages to members
  • Logging  ns_log 
  • Hands on example

1-2 - Lunch at Museum of Science - There is no food or drink in the Museum of Science classrooms.

2-3 - Creating Packages

  • Package Info File
  • Directory Structure
  • Creating Data Model
  • Installing Package on System
  • Creating Upgrade Script
  • Upgrading
  • Adding to Site Map
  • Hands on Exercise

3-4 Procedures, Forms, Mail

  • Forms - ad_form and components
  • Mail - acs_mail_lite
  • Procedures and self documenting
  • Hands on Exercise

4-5 Database API, Displaying Multirow Database Results

  • Database API
  • Multirow/Multiple
  • Hands on Exercise

5-6 Advance Topics

  • ACS Objects
  • Permission System

 

Hacking Days (a.k.a. Bug Bash)

 

more soon

OpenACS Core Packages (oacs-core)

Created by Benjamin Brink, last modified by Gustaf Neumann 01 May 2020, at 04:48 PM

Core packages are defined in CVS. See Aliases_at_CVS

Chart showing core package dependencies according to package specification files for oacs-5-9:

chart showing openacs core packages according to package definition

Chart is available in 2 versions:

2006 OpenACS/.LRN Fall Conference

Created by Carl Robert Blesius, last modified by Gustaf Neumann 01 May 2020, at 11:39 AM

Location 

Boston, Massachusetts, USA

Harvard Conference Center 

Map (includes location, close hotels, and public transportation stops) 

Dates and Schedule

Registration and Fees

  • Registration is open. Please let us know if you need any documents beforehand for visa applications or reimbursement
  • Fees: Main conference fee is $95 (if you can not afford the cost of the conference please let us know - we will have a discounted rate and financial assistance may be available). Please see the registration page for additional details.

Travel

Weather

Average from Weather Underground

Local Airports

Logan International Airport is the airport in Boston. It is the closest airport to the conference center. Logan's airport code is BOS. International flights can land at Logan.

The Manchester Airport in Manchester, New Hampshire, and T.F. Green in Providence, Rhode Island, are airports within about an hour's drive of Boston and Cambridge. Many inexpensive airlines serve Providence and Manchester. It is possible to get from downtown Providence to Boston via commuter rail or bus. Hanscom Field in Bedford, Massachusetts, about 25 minutes away from Boston, has flights to and from Trenton, New Jersey.

Local Train Stations

Amtrak, the major American rail company, serves several stations in the Boston area. South Station is one of the area's major transit hubs. It is also on the Red Line (subway line). Amtrak also arrives at North Station, which is on the Green Line, and Back Bay station on the Orange Line.

Bus Lines and Subway

Lodging

Hotels

Best Western Inn at Longwood (SOLDOUT)
342 Longwood Ave, Boston, MA 02115
(617) 731-4700 or 1-(800)-GOT BEST
$159/night - "PARTNERS HEALTHCARE" rate
(try to reserve before October 16th to ensure availability)
0.5 miles from conference center

Longwood Guest Suites
63 Parker Hill Avenue, Boston, MA 02120
1-800-246-8357
$132/night - 1 person occupancy
1 mile from conference center

Longwood Inn
123 Longwood Ave, Brookline, MA 02446
1-800-754-6835
$119/night standard rate
0.8 miles from conference center

 

 

Looking for Roommates

(please add your name here if you are looking for someone to share a hotel room with) 

Crash Space

 (please add to these lists with a link to your user profile)

 

Crash Space Available:

Crash Space Needed:

Who needs a visa to enter the US?

If you are not a resident of the United States, you may need a visa to enter. If you already have a visa and need to enter the US, make sure it will not expire before you use it to apply for admission (entry) at the port. Citizens of the following 27 countries do not need a visa to enter the US for tourism or business for stays of 90 days or less [1]:

Andorra Iceland Norway
Australia Ireland Portugal
Austria Italy San Marino
Belgium Japan Singapore
Brunei Liechtenstein Slovenia
Denmark Luxembourg Spain
Finland Monaco Sweden
France the Netherlands Switzerland
Germany New Zealand United Kingdom

[1] Providing you meet the visa waiver criteria:

a) You must have a machine-readable passport (this means it should have a line of chevrons (e.g., <<< ) valid for six months past your expected stay in the U.S.

b) Any passport issued after October 26, 2005 must include a digital photo.

c) You must have a ticket going out of the US. I.e. only a one-way ticket into the US is not sufficient.

Information for Canadians

Canadian citizens generally do not need either visa or passport to enter the US. All travelers should bring evidence of their identity (e.g., a government photo-id) and citizenship (e.g., a passport; or a birth or citizenship certificate). Fiancees of US citizens, or spouses of US permanent residents, may need a visa even if they are Canadian citizens — especially if they are coming to the US to await final immigration status. .

Getting a visa

To get a visa, contact the nearest US Embassy, Consulate, or other authorized institute to find out how long the process will take (generally anywhere from 3 to 45 days).

Some guidelines for painless visa processing:

  • Ask us for a letter of invitation.
  • Start applying for a visa early. The basic visa process will not take this long, but you will want time to resubmit the application if necessary, and to buy your tickets after the visa has been granted.
  • Prepare for your visa interview/application. You should have
    1. Your entire travel itinerary, from when you leave your country to when you return

      Note that your travel plans depend upon early approval of the visa application

    2. Your invitation, and printed information about the conference

      If you are getting financial support to attend the conference, make sure you have printed documentation of this as well.

    3. Proof of association with OpenACS and/or .LRN (information about you as a researcher, developer, .LRN user, etc.)
    4. Evidence that you will return home -- that is, of "binding or sufficient" ties to your home country (normally your country of residence). Useful examples include:
      • evidence of family ties in your home country
      • evidence of property ownership
      • evidence/statements of bank accounts
      • an employment contract or letter from an employer demonstrating you have employment beyond the end of your trip
      • evidence of attending school, or a letter from a school official demonstrating you will be a student there beyond the end of your trip

Getting visa help

A letter of invitation alone does not guarantee you will be issued a visa. If you have followed the above steps, and your visa application is rejected, let us know immediately. Immigration officials in Massachusetts will contact your embassy to try to overturn the rejection.

If you cannot afford the cost of visas or related fees, let us know. Financial assistance may be available.

Interest in Attending

Enter your name, and any topics you are interested in here Fall Conference 2006 Interest in Attending

Social Program

Dinner  on November 1st will take place at

Jasper White's Summer Shack
Starts at 8:00PM
50 Dalton Street in the Back Bay, Boston, MA
across from the Sheraton Hotel entrance and the Hynes Auditorium. Upstairs from Kings Bowling Alley.
Closest Subway T Station: Green Line - Hynes Auditorium.

The social program on subsequent days will be informal and will be announced at the event

Sponsors

 

 

 MGH LCS ACSPropel  Solution Grove 
     
     

 

File Storage

Created by Robert Taylor, last modified by Gustaf Neumann 01 May 2020, at 11:37 AM

I added views support to file storage folder-chunk as can be seen here: 

Furthermore I added the ability to download checked files (instead of having to download the whole folder). THe name of the zipfile will be the name of the folder you are looking at (so the same as if you download the whole folder).

As the file is zipped, this effectively lets you download a compressed ZIP file, so you don't have to download that much data.

just playing around

Created by Robert Taylor, last modified by Gustaf Neumann 01 May 2020, at 11:36 AM




One stormy night I was bored ... and this was born:

 

1.  Some links to other design work ideas:

a) http://www.thedesignexperience.org/openacs/oacs-design-hack.png 

b) http://www.thedesignexperience.org/openacs/?=

c)  http://www.thedesignexperience.org/openacs/image-library

 

2.  I was playing around with a screenshot of openacs.org the other night and I wanted to see what the site would look like  with the following criteria:

a) REUSE all standard OACS components.  No funky layouts, no funky font work, just standard OACS out of the box layout and components.

b) SIMPLIFY the design.  Don't change anything, just remove the unnecessary design components already present.  We should NOT get caught up in a multimonth redesign sessions that takes forever to implement.  Modify what we have and make it look unified.

c)  Consider the UTILITY factor of the website.  I don't want to think about the website as a 'special use case' of our toolkit.  I would like to re-use our own tools as much as possible.  Out of the box the site would default to XoWiki for the content management, we would have the bt, cal and forums tabs as normal.  I was playing around with having a PM (pm needs some cleanup) and SEARCH tabs for advanced search and for project managing oacs based activities by those of us that want to use that tool.

 

3.  The oacs logo was stripped of the alex graphic.  It was kinda goofy and I'm going to try to work on something and see what I come up with.  Maybe we can holda contest for the logo ... the joomla team ended up with a pretty cool logo as a result.

 

4.  I will create a sample site layout here and make it available online live for evaluations by everyone.

 

5.  Here is the sample i was playing around with:

Next Page
previous July 2020
Sun Mon Tue Wed Thu Fri Sat
28 29 30 1 2 3 4
5 (1) 6 7 8 9 10 11
12 (1) 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31 1

Popular tags

17 , 5.10.0 , 5.9.0 , 5.9.1 , ad_form , ADP , ajax , aolserver , asynchronous , bgdelivery , bootstrap , bugtracker , CentOS , COMET , compatibility , CSP , CSRF , cvs , debian , emacs , engineering-standards , fedora , FreeBSD , guidelines , host-node-map , hstore , includelets , install , installation , installers
No registered users in community xowiki
in last 30 minutes
Contributors

OpenACS.org