Weblog

Installing OpenACS on Windows

Created by Maurizio Martignano, last modified by Maurizio Martignano 01 Mar 2023, at 09:09 PM

Windows-OpenACS (vers. 5.14.0 - March 2023) is a  Windows 64 port of OpenACS 5.10.0 and the latest snapshot of NaviServer and is available at Spazio IT.

This port installs and runs on the following systems:

• Windows 10,
• Windows 11,
• Windows Server 2012 R2,
• Windows Server 2016,
• Windows Server 2019 and 
• Windows Server 2022.

 

E-Mail: Outgoing E-Mail

Created by Malte Sussdorff, last modified by Gustaf Neumann 06 Feb 2023, at 04:48 PM

Current documentation for ACS Mail Lite, the standard API for inbound and outbound email is at https://openacs.org/doc/acs-mail-lite/.

Recommendation

The current recommendation for sending outgoing mail is via the NaviServer module nssmtpd, since this handles well large outgoing mails and works as well with >1000 file descriptors open. In OpenACS 5.10, set the package parameter "EmailDeliveryMode" in the acs-mail-lite package to "nssmtpd".

Older Discussion

Outgoing E-Mail at the moment in acs-mail-lite is split in a multitude of procedures, some doubling the efforts of the other. To clean this up, I propose the following (in general):

• Replace acs-mail-lite::send with acs-mail-lite::complex_send, making it a wrapper for complex_send or the other way round (rename complex_send to send and so on).
• Only support sending of e-mails via SMTP. Use smtp::sendmessage from TCLLIB for it (as does complex_send).
• Daveb suggested splitting up complex_send to make it easier to test parts of it. Here are some ideas:
  • Sub procedure to generate "to/cc/bcc/_lists" which are used in the respective TO/CC/BCC header. This will also clean up the sending of individual emails.
  • Have only one call to smtp::sendmessage and one hook for the complex_send callback.
  • Sub-procedure to append file tokens. Not sure if this is useful as we need to do upvar for the tokens and could not do automated tests on them anyway. So I'd not do it.
• Have only one sweeper (the complex_sweeper) with support for multiple mail sending servers (so you can have multiple mail senders in the cluster instead of only one).
• Split of incoming email handing into a separate file
• Delete acs-mail-lite-procs.tcl :-). Just kidding, but deprecate most of the procedures.

Sadly, it is not a straightforward approach, as there is a catch. Using smtp::sendmessage forces us to figure out a new way for setting the bounce-email address header, as the old approach will not work any more (using the SMTP command "FROM" set to the bounce address). Another option is to use the non-standard "Errors-To" Header instead.

Double Click Handling

Created by Gustaf Neumann, last modified by Gustaf Neumann 02 Nov 2022, at 06:06 PM

Many users tend to click twice on links/and forms, causing therefore multiple identical requests, that can kill the performance of a server, especially when these are expensive requests. On one site, we saw users firing up to several hundred identical requests, probably via a “rapid fire” function available for some mice, coming from the gaming world.

There are essentially two generic ways to handle double clicks in OpenACS:

1. Server-side double click handling: The xotcl-request-monitor has since a long time the feature to block (optionally) multiple identical requests from the same client. The user will receive for subsequent identical requests the message “Repeated Operation, please resubmit”, which as some “educative” value. It has several heuristics built in to allow certain double requests (e.g., for embedded resources).
    
2. Client-side double click handling: Via HTML markup, one can disable the same requests for a certain time on the client side (i.e., in the browser). For this time, double-clicks are blocked (the button/link is disabled), such that the true double-click, or “rapid fire” functions are mitigated. The request is only one time executed. Potential problems are potential confusion, when a user wants to open the same page in rapid sequence in multiple tabs, windows. Furthermore, repeated requests from bots are not mitigated. This is a new feature which will part of the OpenACS 5.10.1 release.

The client-side double click handling is activated by a content developer for a link prone to this concern by adding the CSS class “prevent-double-click” to the HTML anchor or button. The double click prevention deactivates a button or an anchor element after clicking for a short time (per default for 2s) and ignores in this time window further clicks. The default time window can be specified via the package parameter DefaultPreventDoubleClickTimeoutMs of acs-templating. The time window can be specified for every occurrence via the data element “oacs-timeout”. When a double click occurs, the duplicate click is determined at the client side (in the browser) such that the first request will continue to run and will render its result when finished. Without double click prevention (or with the double click-prevention on the server side), the results of the first request are lost for the client, although these are still computed at the server side.

Usage Example:

    <p>text with a <a class="prevent-double-click" href="/slow.tcl?t=10s">link</a>
    <p>
    <form action="/slow.tcl">
    <input class="btn prevent-double-click" data-oacs-timeout="4000" type="submit" value="Submit">
    ...
    </form>

OpenACS TODO List

Created by Dave Bauer, last modified by Gustaf Neumann 01 Nov 2022, at 10:51 AM

Release Status

See openacs-release-status

Development is taking place in the oacs-5-10 branch.

OpenACS Version 5.10.1 Agenda/wish list

Version numbers:

• Postponed for later releases:
  • Implement subsite-singleton (in addition to the classical singleton)
  • Say farewell to CVS
  • Make more parts of xotcl-core a first-class citizen of the OpenACS core (e.g. database interface)
  • Data bloat hygiene:
    • Rethink package parameter and portlet parameter data models
    • Parameters: include subsite-parameters in parameter resolution (package->subsite->global)
       
• Security:
  • Stronger password hashes for OpenACS ("scram-sha-256" hash in addition to the classical "salted-sha1")
    See https://openacs.org/forums/message-view?message_id=5537869
     
  • Added optional CSP rules based on MIME types. This is important for user-contributed content. When users upload e.g. SVG-files to the file storage, and the content is served from there, it poses a potential security hole. One can now define an additional parameter called "StaticCSP" in the section "ns/server/$server/acs" of the OpenACS configuration file to deactivate execution of script files from static content.
    
    ns_param StaticCSP {
    image/svg+xml "script-src 'none'"
}
     
  • Cookie-Namespace: When multiple OpenACS instances are served from the same domain name, the same cookies (e.g. ad_session_id, ad_login, ...) are set to all servers. For sensible cases, a cookie-namespace can be used, which can be used as a replacement of the traditional "ad_" prefix. This can be as well set in the section "ns/server/$server/acs" of the OpenACS configuration file
    
    # Provide optionally a different cookie namespace
# (used for prefixing OpenACS cookies)
ns_param CookieNamespace "ad_"
     
• Improved templating:
  • Client-side double click prevention
  • Support for generic icon names, which can be mapped differently depending on the installed packages and themes: The generic names are supported via <adp:icon name="NAME" title=...>. By using this feature, one can use font-based icons (like e.g. glyphicons of Bootstrap5, bootstrap-icons, fa-icons, ...) instead of the old-style .gif and .png images. This makes the appearance more uniform, has better resizing behavior, and works more efficiently (fewer requests for embedded resources). Most of the occurrences of the old-style images in standard core and non-core packages in oacs-5-10 are already replaced.
  • Support for listing registered URNs
     
• Require NaviServer (i.e. drop AOLserver support).
  Rationale: AOLserver cannot be compiled with the required modules with recent Tcl versions. Trying to backport NaviServer compatibility functions seems to be an overkill for the OpenACS project.
   
• Further reduce divergence between Oracle and Postgres SQL. Target version of Oracle could be 12.*, as Extended support ends in 2022 (see https://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf)
  • limit / rownum -> fetch first
  • use Postgres schemas for stored procedures so that they can be invoked with the same Oracle idiom
     
• Bootstrap 3 reached EOL in 2019, Bootstrap 4 had EOL 2022, so we should migrate to Bootstrap 5 (details: https://github.com/twbs/release)
   
• New Packages:
  • openacs-bootstrap5
  • bootstrap-icons
  • fa-icons
  • highcharts
     
• Support for fresh installations on Oracle 19c (for details, see: oacs-5-10-on-oracle-19c)
• Require Tcl 8.6, XOTcl 2.1, PostgreSQL 11 (PostgreSQL 10 EOL: November 22), tdom 0.9

OpenACS Version 5.10.0 Agenda

• Functional improvements
  • Features:
    • Added additional page_contract filter: oneof(red|green|blue)
    • template::add_event_listener and template::add_confirm_handler now can target elements by CSS selector
    • Added text/markdown to the accepted text formats or rich-text widget
    • Support for range types in .xql files:

      PostgreSQL supports range types since 9.5. When using range types, square braces have to be used in SQL statements. Since OpenACS uses always Tcl substitution in .xql files, and OpenACS does NOT allow backslash substitution in these files, square brackets could not be escaped and therefore not be used in .xql files so far. This change allows now a developer to deactivate the substitution by passing e.g. -subst none to the db_* command using the .xql file. Valid values for -subst are all, none, vars, and commands, default is all which is exactly the behavior of previous releases. Therefore, this change is fully backward compatible.

    • Improved API browser: Visualization for code dependencies (which procs calls what, from where is a proc being called) and test-coverage
    • Warn site administrators about expiring certificates
    • Additional input types (and widgets) for ad_form:
      • checkbox_text
      • color
      • email
      • tel
      • url
      • number
      • file (multiple)
      • h5date and h5time: date and time fields using native HTML5 visualization and input normalization
    • Registry for .js and .css libraries: allow besides classical URLs symbolic names for loading external resources (e.g. jquery), this makes it easier to upgrade libraries in multiple packages (without running into problems with duplicate versions) and supports switching between CDN and local pathsURN. The existing implementation is based on URNs and extends the existing template-head API to support registration for URNs. A URN provides an abstraction and a single place for e.g. updating references to external resources when switching between a CDN and a locally stored resource, or when a resource should be updated. Instead of adding e.g. a CDN URL via template::head::add_script, one can add an URN and control its content from a single place. Use common namespaces for OpenACS such as urn:ad:css:* and urn:ad:js:*.

      • Register URNs:

        Example provider (e.g. in some theme):

           template::register_urn \ 
               -urn urn:ad:js:jquery \ 
               -resource /resources/xowiki/jquery/jquery.min.js
        

      • The registered URN can be used like classical URL after registration.

        Example consumer:

           template::head::add_javascript -src urn:ad:js:jquery

      • Declare composite files: Provide an interface to define that a .js file or a .css file contains multiple other .js/.css files in order to reduce the number of requests.

           template::head::includes -container urn:js::style.js -parts {urn:ad:js:jquery ...}

    • Improved support for streaming HTML: The new API function template::collect_body_scripts can be used to get the content of template::script or CSP calls (template::add_body_script, template::add_event_listener, template::add_body_handler, template::add_script) when streaming HTML (incremental HTML) is used. Before, these call could bot be used for streaming HTML.

  • Reforms:
    • Dynamic blueprint reloading:
      • When reloading apm packages, watched files, etc. the NaviServer blueprint is now updated correctly. This solves the long-standing problem that changes for scheduled procedures required a restart of the server. Furthermore, the old-style reloading was based on a ever-growing list of reload operations each time a new reload was requested, causing slow-downs in long running server instances, especially, when new threads are started.
      • Now changes are applied to all NaviServer threads, including threads for schedules procedures, ns_jobs and the like. Note that the current job has to be finished before the changes can be applied.
      • For the transition, it is possible to switch between the classical reloading style and blueprint reloading by changing a variable in acs-tcl/tcl/apm-procs.tcl
    • Login:
      • Get rid of bugging "login page expired" messages. The 17 years old construct was replaced by newer means to avoid caching of form values from the login form. Admins of existing sites should set the kernel parameter LoginPageExpirationTime to 0
    • Forums:
      • Removed hard-coded dependency with registered_users group when checking forum permissions
      • Don't rely so heavily on acs_permissions to model forum configuration, as this can have unexpected consequences in convoluted multi-group/multi-subsite scenarios. Prefer simpler table attributes instead
      • New style of attachments to the forums, allowing multiple attachments to a single message directly from the message post page, using the multiple file input widget. Retain compatibility with old style attachments, using the new 'AttachmentStyle' package instance parameter. Currently, this supports two values: 'simple' (new behavior) and 'complex' previous behavior.
    • Chat:
      • Revamping of the GUI
      • Responsiveness
      • Full screen mode
      • Skins support (minimal, classic and bubbles, included): Skins are located in the new /packages/xowiki/www/resources/chat-skins/ directory. New skins can be created by just adding the css and js files in the skins directory, and naming them accordingly (chat-$SKIN_NAME.{js|css}).
      • Avatars (can be enabled per room)
      • Number of active users in chat
      • Tab notifications of new messages
      • Web Notifications:
        • https://www.w3.org/TR/notifications/
        • https://developer.mozilla.org/en-US/docs/Web/API/Notifications_API/Using_the_Notifications_API
    • acs-lang:
      • admin pages:
        • Added the option to unregister (delete permanently the message key from all locales) a message key that has been already marked as deleted. Useful for cleaning up old message keys.
        • Added the option to undelete, using the new ::message::undelete proc.
        • Made number and category (untranslated/deleted/...) of messages coherent in all pages.
        • Added the columns 'total' and 'deleted' to the index page.
      • object_id reference: it is now possible to associate a message key to an object_id in a way that e.g. when the object is deleted, so is the message key. This addresses cases such as the message keys generated by group creation or by the new XoWiki localized fields
    • Notifications:
      • Improved scalability for notifications: One of the most expensive operations in large site is the cleanup for notification_requests in situations, where the user has lost permissions on an object, on which the user wanted to receive notifications. This check was performed previously in notification::sweep::cleanup_notifications via a permission check over all notification requests, which can be very costly on large sites. This change moves this cleanup into the actual notification sending, where the permissions have to be sent anyhow.
      • When sending a notification on behalf of a person, if the system is not configured to process replies to notification, do not set the reply-to address to anything different than the sender
      • Notifications: proper cleanup of acs_objects resulting from the deletion of dynamic notification requests
    • User/Person/Party API: rework and rationalize caching of all party, person and user API, create separate caches for each of these types, make the API and return dicts. acs_user::get will not fail anymore with non-existing user.
    • User Portrait: created API to retrieve and create, store and delete the user's portrait. Also address leftover child relationships from the past and delete them properly.
• Non-functional Changes
  • Improved automated regression test infrastructure and test coverage
    • All packages in the oacs-5-10 branch pass regression test
    • Web testing was separated from non-maintained tcltest and was built on the standard OpenACS infrastructure
    • Include web testing per default in standard regression testing
    • Introduced new test authentication authority, allowing to run many user administration tests outside the context of a "real authority": in cases where the real authority depends on external services to proof identity, (e.g. Kerberos), those tests would just fail.
    • Introduce the display of warnings in the UI
    • Added test coverage information in the automated testing pages, using the new proc-coverage API and providing test coverage information for packages and system wide.
    • Increased overall coverage of public API
    • New tests checking various data-model properties and smells
       
  • Improved scalability:

    • Provided lock-free implementation of ad_page_contract_filters and ad_page_contract_filter_rules. This change improves parallel processing of requests and is primarily interesting for sites with a few mio page views per days. These locks were among the most frequent nsv locks

    • Reduced locks on util_memoize_cache my more invariants values into per-thread caching (acs_lookup_magic_object, ad_acs_version, .... ) and by avoiding specialized calls, which can be realized by already optimized ones (apm_package_installed_p_not_cached ref-timezones was most frequently used util_memoize_cache entry). These changes are necessary to avoid full still-stand on the unfortunate long-time locks on util_memoize_cache stemming from permission and user management with wild-card flush operations, which require to iterate over all cache entries (which might be on a busy server several hundred thousands)

    • Added new interface for cache partitioning to reduce lock latencies on high load websites

    • Added new interface for lock-free per-thread and per-request caching to avoid scattered ad-hoc implementations

    • Better reuse of DB handles (reduced expiring/reopen/etc.), faster access to handles

  • Improved startup time:
    • When the package acs-automated-testing is disabled, startup time is reduced by avoiding loading of support functions and tests; the size of the blueprint is reduced
    • xowf: loading of at-jobs is significantly improved.
       
  • Security improvements:
    • Strengthened page contracts
    • CSP support for application packages
    • CSP fine tuning
       
  • Better exception handling based on Tcl 8.6 exception handlers (try and throw, also available in Tcl 8.5)
    • Provided a new ad_try implementation based on Tcl's try replaces now the old ad_try, with_catch and with_finally, which are marked as deprecated
    • The new ad_try is in essence Tcl's try but with predefined handling of ad_script_abort and should be also used instead of catch, when the OpenACS API is used (which might use script aborts)
    • All core packages use the new ad_try instead of the deprecated versions.
       
  • Connection close reform:
    • NaviServer/AOLserver continue after connection closing commands to execute a script. This is in many situations not desired, especially, when for the page as well a .adp file exists, which will try to deliver this on the already closed connection. This can lead to errors in the error.log file, which are sometimes hard to analyze
    • Due to this cleanup, developers should use in most such cases cases ad_script_abort
    • Connection closing commands are e.g. ad_returnredirect, ad_redirect_for_registration, cr_write_content, ad_page_contract_handle_datasource_error, ad_return_string_as_file, ad_return_complaint, ad_return_error, ad_return_forbidden, ad_return_warning, ad_return_exception_page, ns_returnredirect, ns_return, ns_returnerror
    • The new version has made on most occasions explicit, when the script should abort.
       
  • API changes (new and extended API calls):
    • New API call category::get to obtain category description for a category_id and locale
    • New utility ad_pad emulating both lpad and rpad typically available in DBMSs
    • New proc lc_content_size_pretty, prettify data size given in bytes. It supports three different standards (SI base-10, IEC base-2 and the old JEDEC base-2), default is SI base-10.
    • New flag -export for ad_form: this flag uses export_vars under the hood and supports all of this API's features (e.g. :multiple, :sign, :array). This addresses a long standing TODO
    • util::pdfinfo: simple poppler-utils wrapper to extract pdf information
    • util::http: leverage new ns_http features such as request file spooling. Native implementation will now be used only on NaviServer >= 4.99.15.
    • Database API:
      • db_foreach: queries executed inside of a db_foreach will not be issued using a different handle and will therefore be safe to use in a transaction
      • db_list_of_lists: new -with_headers flag, which will make the first element of the returned list to be the column names as defined in the query
    • Groups API:
      • Logics to delete a group type have now been included in the API
      • Allow to filter group members by member_state in the API
         
  • Deprecated commands:
    • Many deprecated API calls were included in the code (and sometimes still in use) sometimes more than 10 years after these calls have been deprecated. In case a site modification still uses deprecated code, the user is warned about this. The OpenACS 5.10 code base does not depend on deprecated code.
    • Move deprecated code into separate files
    • Made loading of deprecated code optional (can be controlled via parameter "WithDeprecatedCode" in section "ns_section ns/server/${server}/acs" of the config file. By default, deprecated procs are still loaded
    • When deprecated code is not loaded, the blueprint of the interpreter is smaller. The following number of lines of code can be omitted when loading without the deprecated procs:
      • acs-tcl: 3178
      • acs-templating: 450
      • xotcl-core http-client-procs: 830
      • acs-content-repository: 1717 (including .xql files)
         
  • Bugfix and Code Maintenance:
    • Made sure all party emails are stored as lowercase through the API
    • Fixed long standing regression in template::list: we were looping through the list "elements", rather than the "display_elements". This prevents specifying different sets of columns to be returned depending on the -formats and -selected_format options in template::list::create.
    • acs-content-repository: New HEIC and HEIF mimetypes
    • acs-mail-lite: handle to_addr specified as "DisplayName <email>" without errors
    • Fixed invalidating of all existing user logins, (aka) "Logout from everywhere" feature, useful e.g. to make sure no device still holds a valid login when we change our password on a device
    • Don't lose the return URL when one tries to join a subsite before being logged in
    • Added doc(base_href) and doc(base_target) for setting <base> element via blank-baster (see issue #3435)
    • Groups:
      • When a new group is created, flush all the group::get_id caches with the same name so that the new group can be fetched correctly in case it replaces a previously deleted one
      • Cleanup message keys coming from groups in acs-translations when a group is deleted
    • acs-lang:
      • lang::util::convert_to_i18n: do not always register a en_US translation, which would be always overridden. Instead, let lang::message::register make sure that a en_US message exists and create one only as a fallback.
      • lc_time_fmt: leverage Tcl clock to address shortcomings such as handling of dates in Julian/Gregorian calendar and impossible dates such as 1999-02-29, implement missing formats, support previously undocumented formats explicitly
    • search: make sure objects in the search indexer queue still exist by the time they are swept by the indexer (e.g. items deleted before the indexer could sweep them)
    • attribute::delete: fix proc so it leverages stored procedure capability of dropping the database table as well
    • util::http: fix UTF-8 encoding issues for some cornercases
    • Localization: Complete Italian and Spanish localization for the whole .LRN set of packages (including themes). Message keys for new and previously localized packages have also been updated
       
  • General cleanup/maintenance
    • Improved handling of server implementation-specific code: server-specific code can be optionally loaded via specifying the server family in the filename. Provided *-procs-aolserver.tcl and *-procs-naviserver.tcl similar to *.postgresql.xql and *.oracle.xql where appropriate
    • Modernization of Tcl idioms.
    • Compliance of files, proc names, ... to the naming conventions.
    • White space cleanup, indentation changes.
    • Improvement of public API documentation
    • Adjustment of proc protection levels (public, private)
    • Adjustment of log severity
    • Cleanup of obsolete files
    • Replacement of handcrafted forms by ad_form
    • Typo fixing
    • Editor hints
    • Replacement of deprecated calls
    • Addition of missing contracts
    • ...
       
  • SQL cleanup:
    • Cleanup of obsolete nonportable SQL constructs in a way Oracle and PostgreSQL code base divergency is reduced:
      • "nvl" -> "coalesce"
      • "sysdate" / "now()" -> standard "current_date" or "current_timestamp"
      • Use standard-compliant "dual" table where appropriate (required by Oracle, supported by PostgreSQL)
      • Use non-dialectal cast idioms when appropriate
      • Adopt CTE idioms in Oracle codebase as well (e.g. connect -> with recursive)
      • ... (reference Oracle version will be 11gr2 as is oldest version officially supported by Oracle (See here and here)
    • Reduced superfluous .xql queries
      • acs-subsite: delete 21 files with un-referenced .xql queries
      • acs-tcl: delete 4 files
      • news: 3 files
      • file-storage: 1 file
      • dotlrn: 9 files
         
  • New Packages:
    • cookie-consent: alerting users about the use of cookies on a website
    • boomerang: performance of your website from your end user’s point of view
    • xooauth: OAuth implementation, including LTI (Learning Tools Interoperability)
    • dotlrn-bootstrap3-theme: Bootstrap 3 theme for DotLRN
    • xowf-monaco-plugin: Integration of Monaco editor with for code exercise types in xowf
    • proctoring-support: utilities and user interfaces to implement proctoring of the user session, mainly intended in the context of distance education and online exams. The main proctoring feature relies only on web technologies and does not require any plugin or additional software. Optional support for the Safe Exam Browser has also been introduced. The package is currently at the core of WU Online Exam infrastructure and is integrated in the inclass exam implementation for xowf.
       
• Require Tcl 8.6, XOTcl 2.1, PostgreSQL 9.6 (PostgreSQL 9.5 EOL: February 2021), tdom 0.9
   

OpenACS Version 5.9.1 Agenda

Refactoring of rich-text editor integration

• Driving force: Debian packaging (e.g. js minified code is not allowed)
• Moved out code from acs-templating, provided interfaces to add many different rich-text editors as separate packages
• New OpenACS packages:
  • richtext-xinha
  • richtext-tinymce
  • richtext-ckeditor4 (has ability to choose between CDN and local installation via web interface)

Improving admin interface

• New theme manager:
  • Goals:
    • Make it easier to keep track of themes with local modifications
    • Make it easier to create local modifications of existing themes and to update these
    • Show differences between default theme parameter (in DB) and actual settings (in subsite parameters)
    • Allow to delete unused themes
    • Give site admin hints, which theme is used at which subsite
    • Ease theme switching
  • Added a subsite::theme_changed callback to be able to handle theme changes in custom themes (was also necessary for proper integration with DotLRN theming)
  • Added support for these features under subsite admin (/admin/)
  • Improved support for themed templates via [template::themed_template]

     

• Improved (broken) interface to define/manage groups over web interface
• Allow to send as well mail, when membership was rejected
• New functions [membership_rel::get_user_id], [membership_rel::get] and [membership_rel::get_group_id] to avoid code duplication
• Added support to let user include %forgotten_password_url% in self-registration emails (e.g. in message key acs-subsite.email_body_Registration_password)

   

• Improved subsite/www/members
  • Make it possible to manage members of arbitrary groups
  • Improved performance for large groups
  • Improved configurability: when ShowMembersListTo is set to "3", show list to members only, when this is not the whole subsite

     

• Improved user interface for /admin/applications for large number of applications
• Various fixes for sitewide-admin pages (under /acs-admin)
• Update blueprint in "install from repository" (currently just working in NaviServer)

SQL

• Further cleanup of .xql files (like what as done for acs-subsite in OpenACS 5.9.0):
  • 36 files deleted
  • Removed more than 100 obsolete named queries
  • Stripped misleading SQL statements

     

• Marked redundant / uncalled SQL functions as deprecated
• Replaced usages of obsolete view "all_object_party_privilege_map" by "acs_object_party_privilege_map"
   
• Removed type discrepancy introduced in 2002:
  • acs_object_types.object_type has type varchar(1000), while
  • acs_object_types.supertype has type varchar(100)
  • ... several more data types are involved, using acs_object_types.object_type as foreign key

     

• Simplified core SQL functions by using defaults:
  • Number of functions reduced by a factor of 2 compared to OpenACS 5.9.0 (while providing compatibility for clients using old versions),
  • Reduced code redundancy
  • Affected functions:
    • Reduced content_item__new from 12 versions to 6,
    • Reduce content_revision__new from 7 to 4
    • Similar in image__new, image__new_revision, content_item__copy, content_item__get_title, content_item__move
  • PostgreSQL 9.5 supports named parameter in the same syntax as in Oracle. Further reduction of variants will be possible, once OpenACS requires at least PostgreSQL 9.5

     

• Reduced usage of deprecated versions of SQL functions (mostly content repository calls)
• Reduced generation of dead tuples by combining multiple DML statements to one (reduces costs of checkpoint cleanups in PostgreSQL)

   

• Permission queries:
  • Improved performance
  • Support PACKAGE.FUNCTION notation for PostgreSQL to allow calls permission queries exactly the same way as in Oracle (e.g. "acs_permission.permission_p()"). This helps to reduce the number of PostgreSQL specific .xql files.
• Modernized SQL:
  • Use real Boolean types instead of character(1)
    (done for new-portal, forums, faq, attachments, categories, dotlrn, dotlrn-forums, evaluation)
  • Use real enumeration types rather than check constraints (done for storage_type text/file/lob)

CR hygienics (reduce cr bloat)

• Provided means to avoid insert/update/delete operations in the search queue:

  OpenACS adds for every new revision often multiple entries to the search_queue, without providing any means to prevent this. This requires for busy sites very short intervals between queue sweeps (otherwise too many entries pile up). Another consequence is that this behavior keeps the PostgreSQL auto-vacuum daemons permanently active. Many of these operations are useless in cases where the content repository is used for content that should not be provided via search. The changed behavior should honors a publish-date set to the future, since it will not add any content with future publish dates to the search-queue.

   

• Reduced number of insert cr_child_rels operations, just when needed:

  cr_child_rels provide only little benefit (allow one to use roles in a child-rel), but the common operation is a well available in cr_items via the parent_id. cr_child_rels do not help for recursive queries either. One option would be to add an additional argument for content_item__new to omit child-rel creation (default is old behavior) and adapt the other cases.

Security improvements

• Added support against CSRF (cross site request forgery)
  • OpenACS maintains a per-request CSRF token that ensures that form replies are coming just from sites that received the form
  • CSRF support is optional for packages where CSRF is less dangerous, and such requests are wanted (e.g. search and API-browser)

• Added Support for W3C "Upgrade-Insecure-Headers" (see https://www.w3.org/TR/upgrade-insecure-requests/):
  For standard compliant upgrade for requests from HTTP to HTTPS

• Added support for W3C "Subresource Integrity" (SRI; see https://www.w3.org/TR/SRI/)

• Added support for W3C "Content Security Policy" (CSP; see https://www.w3.org/TR/CSP/)

  • Removed "javascript:*" links (all such urls are removed from the 90 packages in oacs-5-9, excluding js libraries (ajaxhelper) and richtext code)
  • Removed "onclick", "onfocus", "onblur", "onchange" handlers from all .adp and .tcl files in the 90 packages in oacs-5-9 (excluding js libraries (ajaxhelper) and richtext code)
  • Added optional nonces to all <script> elements with literal JavaScript content

     

• Removed "generic downloader", which allowed to download arbitrary content items, when item_id was known (bug-fix)
• Improved protection against XSS and SQL-injection (strengthen page contracts, add validators, added page_contract_filter "localurl", improve HTML escaping, and URI encoding)
• Fixed for potential traversal attack (acs-api-documentation-procs)

Improvements for "host-node mapped" subsites

• Fixed links from host-node mapped subsite pages to swa-functions (must be always on main subsite)
• Made "util_current_directory" aware of host-node-mapped subsites
• Added ability to pass "-cookie_domain" to make it possible to use the same cookie for different domains
• Fixed result of affected commands "util_current_location", "ad_return_url", "ad_get_login_url" and "ad_get_logout_url" for HTTP and HTTPS, when UseHostnameDomainforReg is 0 or 1.
• Improved UI for host-node maps when a large number of site nodes exists

Reform of acs-rels

• Made acs-rels configurable to give the developer the option to specify, whether these are composable or not (default fully backward compatible). This is required to control transitivity in rel-segments
• The code changes are based on a patch provided by Michael Steigman.
  For details, see:
  • https://openacs.org/forums/message-view?message_id=4031049
  • https://openacs.org/forums/message-view?message_id=5330734

Improved status code handlers for AJAX scenarios

• Don't report data source errors with status code 200 (use 422 instead)
• Let "permission::require_permission" return forbidden (403) in AJAX calls (determined via [ad_conn ajaxp])

Improved Internationalization

• Extended language catalogs for
  • Russian (thanks to v v)
  • Italian (thanks to Antonio Pisano)
  • Spanish (thanks to Hector Romojaro)
  • German (thanks to Markus Moser)

     

• Added (missing) message keys
• Improved wording of entries
• Added message keys for member_state changes, provide API via group::get_member_state_pretty

Improved online documentation (/doc)

• Fixed many broken links
• Removed fully obsolete sections
• Improved markup (modernize HTML)
• Updated various sections

Misc code improvements:

• 18 issues from the OpenACS-bug-tracker fixed
• Made code more robust against invalid/incorrect input (page_contracts, validators, values obtained from header fields such as Accept-Language)
• Fixed quoting of message keys on many places
• Improved exception handling (often, a "catch" swallows one to much, e.g. script_aborts), introducing "ad_exception".

• Generalized handling of leading zeros:

  • Fixed cases where leading zeros could lead to unwanted octal interpretations
  • Switch to use of " util::trim_leading_zeros" instead of "template::util::leadingTrim", "dt_trim_leading_zeros" and "template::util::leadingTrim", marked the latter as deprecated

• URL encoding

  • "ad_urlencode_folder_path": new function to perform an urlencode operation on the segments of the provided folder path
  • "export_vars": encode path always correctly, except -no_base_encode is specified
  • Fixed encoding of the URL path in "ad_returnredirect"

• Improvements for "ad_conn":

  • Added [ad_conn behind_proxy_p] to check, whether the request is coming from behind a proxy server
  • Added [ad_conn behind_secure_proxy_p] to check, whether the request is coming from behind a secure proxy server
  • Added [ad_conn ajax_p] to check, whether the request is an AJAX requests (assumption: AJAX request sets header-field Requested-With: XMLHttpRequest")
  • Added [ad_conn vhost_url] to obtain the url of host-node-mapped subsites

     

• Added various missing upgrade scripts (missing since many years) of changes that were implemented for new installs to reduce differences between "new"-and "old" (upgraded) installations

• Templating

  • Get rid of various pesky "MISSING FORMWIDGET: ...formbutton:ok" messages
  • Improved support for javascript event handlers in template::head
  • New functions "template::add_event_listener" and "template::add_confirm_handler"
  • Fix handling, when "page_size_variable_p" is set (was broken since ages)

• Improved location and URL handling:

  • Refactored and commented "util_current_location" to address security issues, handle IPv6 addresses, IP literal notation, multiple drivers, "
  • Improved "security::get_secure_location" (align with documentation)

     

  • New functions:
    • "util::configured_location"
    • "util::join_location", "util::split_location"
    for working on HTTP locations to reduce scattered regexps handling URL components
  • Improved IPv6 support
  • Use native "ns_parseurl" when available, provide backward compatible version for AOLserver

• MIME types:

  • Added more Open XML formats for MS-Office to allowed content types
  • Modernized entries to IANA recommendations
  • New function "cr_check_mime_type" centralizing the retrieval of the mime_type from uploaded content

• Finalized cleanup of permissions (started in OpenACS 5.9.0):

  • Get rid of "acs_object_context_index " (and therefore on "acs_object_party_privilege_map " as well) on PostgreSQL.
    Reasons:
    • huge table,
    • expensive maintenance, used only in a few places,

• Misc new functions:

  • "lang::util::message_key_regexp": factor out scattered regexp for detecting message keys
  • "ns_md5" and "ns_parseurl": improve compatibility between AOLserver and NaviServer
  • "ad_dom_sanitize_html": allow one to specify different sets of tags, attributes and protocols and "ad_dom_fix_html", which is a light weight tidy variant.

     

• Improved HTML rendering (acs-api-browser), provide width and height to speed up rendering
• Improved ADP files (e.g. missing doc(title))
• Added usage of "ad_include_contract" on more occasions
• Modernize Tcl and HTML coding
• Reduced dependency on external programs (use Tcl functions instead)
• Improved robustness of "file delete" operations all over the code
• Improved documentation, fix demo pages
• Aligned usages of log notification levels (distinction between "error", "warning" and "notice") with coding-standards

   

• Cleaned up deprecated calls:
  • Removed usage of deprecated API functions (e.g. "cc_lookup_email_user", "cc_email_from_party", "util_unlist", ...)
  • Moved more deprecated procs to acs-outdated
  • Marked remaining (and unused) "cc_*" functions as well as deprecated.

     

• Improved Oracle and windows support
• Fixed common spelling errors and standardize spelling of product names all over the code (comments, documentation, ...)
• Many more small bug fixes

Version numbers:

• require PG 9.2 (End Of Life of PostgreSQL 9.0 was Oct 2015)
• require XOTcl 2.0 (presented at the Tcl conference in 2011).

 

Packages:

 

• New Package Parameters

  • acs-kernel:

    • MaxUrlLength: remove hard-coded constant in request processor for max accepted url paths
    • SecureSessionCookie: Let site admin determine, whether or not to use secured session cookies (useful, when not all requests are over HTTPS)
    • CSPEnabledP: activate/deactivate CSP

  • acs-kernel (recommended to be set via config file in section "ns/server/${server}/>acs"

    • NsShutdownWithNonZeroExitCode: tell NaviServer to return with a nonzero return code to cause restart (important under windows)
    • LogIncludeUserId: include user_id in access log

  • acs-api-browser:

    • ValidateCSRFP: make checking of CSRF optional (default 1)

  • acs-content-repository:

    • AllowMimeTypeCreationP: Decides whether we allow unknown mime types to be automatically registered (default: 0}

  • news-portlet:

    • display_item_lead_p: Should we display news leads in the portlet? (default 0)

  • search:

    • ValidateCSRFP: make checking of CSRF optional (default 1)

  • xotcl-request-monitor:

    • do_track_activity: turn activity monitoring on or off (default 0)
       

• New OpenACS packages:

  • richtext-xinha
  • richtext-tinymce
  • richtext-ckeditor4 (has ability to choose between CDN and local installation via GUI)
  • openacs-bootstrap3-theme (as used on openacs.org)
  • dotlrn-bootstrap3-theme

• xotcl-core:

  • Improved XOTcl 2.0 and NX support (e.g. api-browser)
  • Added "-debug", "-deprecated" to ad_* defined methods (such as e.g. "ad_instproc")
  • Make use of explicit "create" statements when creating XOTcl/NX objects (makes it easier to grab intentions and to detect typos)
  • Added parameter to "get_instance_from_db" to specify, whether the loaded objects should be initialized
  • Added support for PostgreSQL prepared statements of SQL interface in ::xo::dc (nsdb driver)

• xowiki:

  • Named all web-callable methods www-NAME (to make it clear, what is called, what has to be checked especially carefully)
  • Moved templates from www into xowiki/resources to avoid naming conflicts
  • Improved ckeditor support
  • Added usage of prepared statements for common queries
  • Improved error handling
  • Better value checking for query parameter, error reporting via ad_return_complaint
  • Added option "-path_encode" to methods "pretty_link" and "folder_path" to allow one to control, whether the result should be encoded or not (default true)

     

  • Form fields:
    • Improved repeatable form fields (esp. composite cases), don't require preallocation (can be costly in composite cases)
    • Added signing of form-fields
    • Added HTML5 attributes such as "multiple" (for "file") or "autocomplete"
    • Fixed generation of "orderby" attribute based on form-field names
    • richtext: allow one to specify "extraAllowedContent" via options
    • Improved layout of horizontal check boxes

       

  • Menu bar:
    • Added dropzone (requires bootstrap): drag and drop file upload
    • Added mode toggle (requires bootstrap)
    • Extended default policies for handling e.g. dropzone (file-upload method)
    • Distinguish between "startpage" (menu.Package.Startpage) and "table of contents" (menu.Package.Toc)

       

  • Notifications:
    • Added support for better tailorable notifications: introduced method "notification_render" (similar to "search_render")
    • Added support for tailorable subject lines (method "notification_subject")

       

  • Improved bootstrap support, use "bootstrap" as PreferredCSSToolkit
  • Switched to ckeditor4 as PreferredRichtextEditor
  • Improved handling of script-abort from within the payload of ::xowiki::Object payloads
  • Added parameter to "get_all_children" to specify, whether the child objects should be initialized

• xowf:

  • Added property "payload" to "WorkflowConstruct" in order to simplify customized workflow "allocate" actions
  • Internationalized more menu buttons

• xotcl-request-monitor

  • Added class "BanUser" (use. e.g. IP address to disallow requests from a user via request monitor)
  • Added support for optional user tracking in database
  • Added support for monitoring response-time for certain urls via munin
  • Increased usage of XOTcl 2.0 variable resolver (potentially speed improvement 4x)
  • Performed some refactoring of response-time handling to allow site-admin to make e.g. use of NaviServer's dynamic connection pool management (not included in CVS)
  • Added support for partial times in long-calls.tcl to ease interpretation of unexpected slow calls
  • last100.tcl: Don't report hrefs to URLs, except to SWAs

• chat:

  • Introduced new options to set chat rooms so login and/or logout message are not issued every time a user enters/exits a chat-room (important for chats with huge number of participants)
  • Parameterized viewing of chat-logs
  • Fixed cases of over-/under-quoting
  • Fixed JavaScript for IE, where innerHTML can cause problems

• file-storage:

  • Don't show action keys, when user has no permissions
  • Added support for copying of same-named files into a folder (adding suffix)
  • Fixed old bugs in connection with "views" package

OpenACS Version 5.9.0 Agenda

• Slimming pg SQL core:
   
  • Part 1: improve performance of object deletion
    • remove manual delete operations from acs_object__delete()
       
  • Part 2: content-repository - manual referential integrity management
    • handle referential integrity via pg's integrity constraints rather by functions cr_revision_del_ri_tr, cr_revision_ins_ri_tr, cr_revision_up_ri_tr, cr_revision_del_rev_ri_tr, and cr_revision_del_rev_ri_tr
    • fix broken/missing upgrade scripts from earlier updates
       
  • Part 3: content-repository - manual deletions and nulling
    • Removed manual nulling of live_revision and latest_revision
    • Removed manual deletion of old_revision and new_revision in cr_item_publish_audit
    • Removed manual deletion of item_id in cr_item_publish_audit, cr_release_periods, cr_item_template_map, and cr_item_keyword_map
    • Removed manual deletion of direct permissions
    • Added missing index for child_id to cr_child_rels.
       
  • Part 4: get rid of tree_sortkey in acs-objects
    • Check/fix dependencies in oacs-5-8 packages
    • Get rid of broken/uncalled functions using the column
    • Check/fix dependencies in other packages
    • Remove tree_sortkey and max_child_sortkey
• Web interface:
  • Improve client performance
    • moving core.js from head to body
    • provide kernel parameter to control expiration date for /resources/
  • Protect against more XSS attacks
  • Improved HTML validity (see oacs-5-9-html-validity for the checklist)
  • Add lightweight support for ckeditor4 for templating::richtext widget (configurable via package parameter "RichTextEditor" of acs-templating. ckeditor4 supports mobile devices (such as iPad, ...).
  • New kernel parameter ResourcesExpireInterval to control expiration dates of resources
     
• Templating:
  • Improve theme-ability
    • Move more information into theme packages in order to create responsive designs
    • Reduce hard-coding of paths, HTML etc.
  • Dimensional slider reform (ad_dimensional):
    • Remove hard-coded table layout from dimensional slider
    • Add backwards compatible templates
    • Move hard-coded styles into theme styling
    • Remove obsolete comments from ad_dimensional
  • Complete template variable controls (adding noi18n, addressing bug #2692):
    • @foo@: perform html quoting and internationalization
    • @foo;noquote@: perform internationalization
    • @foo;noi18n@: perform html quoting
    • @foo;literal@: perform neither html quoting nor internationalization
  • Improved Russian nationalization
  • Support of expiration dates and passwords for signed variables
     
• Documentation:
  • Use ACS templating for the (static) OpenACS documentation to provide a more consistent layout and user experience.
  • Make pretty-naming of acs-core packages more consistent.
     
• Misc improvements:
  • Mark unused functions of acs-tcl/tcl/table-display-procs.tcl as deprecated
  • Reduce number of muxtex locks by pre-request and per-thread caching
  • Improved development und debugging aids:
    • use "ad_log error|warning  .... " instead of "ns_log" to include information of request and callstack in error.log
    • ability to display ns_log entries caused by a request in ds-footer
    • ability to save delivered web pages in file-system for testing HTML validity (especially for admin pages, which are unaccessible for external validity testers)
  • More bug fixes
     
• Version numbers:
  * require PG 9.0 (End Of Life of PostgreSQL 8.4 was July 2014)
  * require XOTcl 2.0 (presented at the Tcl conference in 2011).
   

OpenACS Version 5.8 Agenda

• PostgreSQL 9.2+:
  • Get rid of nonstandard backslash escapes in function definitions
  • Change quote syntax in SQL files (single quotes around the functions) to recommended PostgreSQL quoting using (recommended since pg8.0, jan 2005). li>Drop aliases in favor of named function arguments (recommended since pg8.0)
  • Fix wrong function_args, add missing function_args, align default semantics with the defaults in pg (providing "null" as default means the argument is optional)
  • Make OpenACS loadable without any tweaks in the pg config files
• Use recursive queries for e.g. permission lookup to avoid performance problems in pg 8.4 and newer)
• ADP: Use byte-compiled function wherever possible in compiled adp-code, support "@var;literal@" when neither quotes nor localization is needed in compiled adp-code
• Improve support of NaviServer
• Switch to Tcl 8.5 (TIP #143)
• Improve scalability: Reduce mutex-stress on util-memoize cache and for cache maintenance in general
• Code cleanup:
  • Get rid of calls to deprecated code (e.g. ad_tables, ad_parameter, ... in acs-core and main packages)
  • Improve awareness of usage of deprecated code (complain to error.log)
  • Use Tcl 8.5 idioms
  • cleanup of various http-client approaches and introduce a common implementation util::http::get and util::http::post; get rid of other usages, mark these as deprecated
  • page-contracts: Perform checking of all ids in acs-core and main packages to improve error messages and to improve security
• OpenACS 5.8.1 should be released with main packages

OpenACS Version 5.7 Agenda

• Support for object management in core 
• Postgresql 9.0
• TinyMCE update (fix for random JS injection issue, affecting Safari)
• Fix for "remember me" issue
• WCAG2-AA

OpenACS Version 5.6 Agenda

• global parameters
• package "embeds" 
• fix search by package_id
• core works on Postgresql 8.4

OpenACS Version 5.5 Agenda

• DONE: Postgresql 8.3 support: especially regarding tsearch2
• DONE: acs-authentication:
  • fix upgrade, add conditional logic into site wide tcl library so that you can login to perform the rest of the upgrade
• DONE: tinymce:
  • upgrade to 3.1.1 + language packs
  • HTML Strict cleanup
  • create appropriate parameters for its config in acs-templating
• acs-mail-lite:
  • DONE: cleanup duplicated procs (bounce)
  • review the parsing of bouncing messages (case user_id 0)
  • DONE: rollout support
• Documentation improvements as discussed at the Guatemala conference:
  • Make current openacs.org/test-doc source for static files included in the release and provide ease means to achieve this for the release manager
    • DONE (CVS HEAD): Provide in XoWiki an alternative table of contents by nested UL/LI (without JavaScript) for static output
    • DONE (CVS HEAD): Provide in XoWiki a prototype page similar to "book" without edit-buttons etc., using the new table of contents
  • Update openacs.org/test-doc where necessary (incomplete list):
    • DONE: Fix the page ordering for the higher chapters (the original document  had no 3rd. level numbering)
    • update pages in /test-doc which are more recent in openacs/xowiki
    • bump version numbers of OpenACS, where appropriate (some places talk about openacs-5-0, others about openacs-5-1, oacs-5-2-3rc1 or 5-3) 
    • some version numbers of the required components are quite a mess. e.g. some parts say that Postgres 7.3 is required,  some examples talks about postgres 7.4.7 and 8.2.4 in the same listing.
    • also the dotlrn version numbers are old dotrln-2.0
    • Tcl version numbers should be 8.4.19
    • The install section for XOTcl is missing in II.3.4
    • remove ChangeLog from documentation
    • find some other prominent place for the ChangeLog
    • Fix indenting in examples  (e.g. in Rocael's robust web    development framework)
    • overthink Win2000 guidelines.  There are the native compiled packages from Maurizio, including everything from postgres, xotcl ....
  • It is desired to find a single person responsible for overworking the documentation, however, funding is unclear.

OpenACS Version 5.4 Agenda

• DONE: HTML Strict (openacs core)
• DONE: finish template::head (daveb)
• DONE: test acs-mail-lite (complex send)
• DONE: test notifications (complex send)
• DONE: new XinHA release, get rid of RTE & HTMLarea, test on Safari
• DONE: Form builder: add the ID attribute  to the form tag
• DONE: acs-lang - keepLocalTranslationP to be removed
• DONE search and intermedia-driver: move intermedia specific stuff to its package
• DONE: acs-mail-lite - patch for mime::qp_encode bug

Future

• Split Xinha and TinyMCE into separate packages see: https://openacs.org/forums/message-view?message_id=2750958
• Usability ("my account" page)
• XHTML ?
• Testing and documentation for recording automated tests using the firefox plugin and the upload feature for it new in automated testing. Probably needs some polishing and should be talked to with Quest who are getting into this.
• Parameter Scope Patch https://openacs.org/bugtracker/openacs/patch?patch%5fnumber=845
• Remove obsolete master template stuff (default and site master template in openacs-4/www, acs-subsite's group-master, and related CSS and images).  Probably in the version which follows 5.5 (probably 5.6).  Also remove the compat master stuff at the same time.

Things to merge into this page

Old 5.0 Roadmap discussion

Roadmap discussion 1

Ideas for Boston 2006 Future of OpenACS discussion

A .LRN Roadmap

Another .LRN Roadmap discussion

 

What's on this page?

This page should include work that is planned on and has someone committed to working on it.

Release Status

See openacs-release-status

Development is taking place in the oacs-5-10 branch.

OpenACS Version 5.10.0 Agenda/wish list

• Postponed for later releases:
  • Implement subsite-singleton (in addition to the classical singleton)
  • Say farewell to CVS
  • Make more parts of xotcl-core a first-class citizen of the OpenACS core (e.g. database interface)
  • Data bloat hygiene:
    • Rethink package parameter and portlet parameter data models
    • Parameters: include "subsite-parameters" in parameter resolution (package->subsite->global)
• Functional improvements
  • Features:
    • Added additional page_contract filter: oneof(red|green|blue)
    • template::add_event_listener and template::add_confirm_handler now can target elements by CSS selector
    • Added text/markdown to the accepted text formats or rich-text widget
    • Support for range types in .xql files:
      PostgreSQL supports range types since 9.5. When using range types, square braces have to be used in SQL statements. Since OpenACS uses always Tcl substitution in .xql files, and OpenACS does NOT allow backslash substitution in these files, square brackets could not be escaped and therefore not be used in .xql files so far. This change allows now a developer to deactivate the substitution by passing e.g. "-subst none" to the db_* command using the .xql file. Valid values for "-subst" are "all", "none", "vars", and "commands", default is "all" which is exactly the behavior of before. Therefore, this change is fully backward compatible.
    • New API call "category::get" to obtain category description for a category_id and locale
    • Added doc(base_href) and doc(base_target) for setting <base> element via blank-baster (see issue #3435)
    • Improved API browser: Visualization for code dependencies (which procs calls what, from where is a proc being called) and test-coverage
    • Warn site administrators about expiring certificates
    • Content repository: New HEIC and HEIF mimetypes
    • New utility ad_pad emulating both lpad and rpad typically available in DBMSs
    • New proc lc_content_size_pretty, prettify data size given in bytes. It supports three different standards (SI base-10, IEC base-2 and the old JEDEC base-2), default is SI base-10.
    • Database API:
      • db_foreach: queries executed inside of a db_foreach will not be issued using a different handle and will therefore be safe to use in a transaction
      • db_list_of_lists: new -with_headers flag, which will make the first element of the returned list to be the column names as defined in the query
    • Groups:
      • Logics to delete a group type have now been included in the API
      • Allow to filter group members by member_state in the API
    • ad_form:
      • new input types:
        • checkbox_text
        • color
        • email
        • tel
        • url
        • number
        • file (multiple)
        • h5date and h5time: date and time fields using native HTML5 visualization and input normalization
      • -export flag: this flag now uses export_vars under the hood and therefore support all of this API's features (e.g. :multiple, :sign, :array). This addresses a long standing TODO
    • Registry for .js and .css libraries: allow besides classical URLs symbolic names for loading external resources (e.g. jquery), this makes it easier to upgrade  libraries in multiple packages (without running into problems with duplicate versions) and supports  switching between CDN and local pathsURN.
      The existing implementation is based on URNs and extends the existing template-head API to support registration for URNs. A URN provides an abstraction and a single place for e.g. updating references to external resources when switching between a CDN and a locally stored resource, or when a resource should be updated. Instead of adding e.g. a CDN URL via template::head::add_script, one can add an URN and control its content from a single place. Use common namespaces for OpenACS such as urn:ad:css:* and urn:ad:js:*.

      • Register URNs:

        Example provider (e.g. in some theme):

           template::register_urn \ 
               -urn urn:ad:js:jquery \ 
               -resource /resources/xowiki/jquery/jquery.min.js
        

      • The registered URN can be used like classical URL after registration.
        Example consumer:

           template::head::add_javascript -src urn:ad:js:jquery

      • Declare composite files: Provide an interface to define that  a .js file or a .css file contains multiple other .js/.css files in order to reduce the number of requests.

           template::head::includes -container urn:js::style.js -parts {urn:ad:js:jquery ...}

    • Improved support for streaming HTML: The new API function template::collect_body_scripts can be used to get the content of template::script or CSP calls (template::add_body_script, template::add_event_listener, template::add_body_handler, template::add_script) when streaming HTML (incremental HTML) is used. Before, these call could bot be used for streaming HTML
    • util::pdfinfo: simple poppler-utils wrapper to extract pdf information
  • Reforms:
    • Dynamic blueprint reloading:
      • When reloading  apm packages, watched files, etc. the NaviServer blueprint is now updated correctly. This solves the long-standing problem that changes for scheduled procedures required a restart of the server. Furthermore, the old-style reloading was based on a ever-growing list of reload operations each time a new reload was requested, causing slow-downs in long running server instances, especially, when new threads are started.
      • Now changes are applied to all NaviServer threads, including threads for
        schedules procedures, ns_jobs and the like. Note that the current job has to be finished before the changes can be applied.
      • For the transition, it is possible to switch between the classical reloading style and blueprint reloading by changing a variable in acs-tcl/tcl/apm-procs.tcl
    • Login: 
      • Get rid of bugging "login page expired" messages. The 17 years old construct was replaced by newer means to avoid caching of form values from the login form. Admins of existing sites should set the kernel parameter "LoginPageExpirationTime" to 0
    • Forums:
      • Removed hard-coded dependency with registered_users group when checking forum permissions
      • Don't rely so heavily on acs_permissions to model forum configuration, as this can have unexpected consequences in convoluted multi-group/multi-subsite scenarios. Prefer simpler table attributes instead
      • New style of attachments to the forums, allowing multiple attachments to a single message directly from the message post page, using the multiple file input widget. Retain compatibility with old style attachments, using the new 'AttachmentStyle' package instance parameter. Currently, this supports two values: 'simple' (new behavior) and 'complex' previous behavior.
    • Chat:
      • Revamping of the GUI
      • Responsiveness
      • Full screen mode
      • Skins support (minimal, classic and bubbles, included): Skins are located in the new /packages/xowiki/www/resources/chat-skins/ directory. New skins can be created by just adding the css and js files in the skins directory, and naming them accordingly (chat-$SKIN_NAME.{js|css}).
      • Avatars (can be enabled per room)
      • Number of active users in chat
      • Tab notifications of new messages
      • Web Notifications:
        • https://www.w3.org/TR/notifications/
        • https://developer.mozilla.org/en-US/docs/Web/API/Notifications_API/Using_the_Notifications_API
    • acs-lang:
      • admin pages:
        • Added the option to unregister (delete permanently the message key from all locales) a message key that has been already marked as deleted. Useful for cleaning up old message keys.
        • Added the option to undelete, using the new ::message::undelete proc.
        • Made number and category (untranslated/deleted/...) of messages coherent in all pages.
        • Added the columns 'total' and 'deleted' to the index page.
      • object_id reference: it is now possible to associate a message key to an object_id in a way that e.g. when the object is deleted, so is the message key. This addresses cases such as the message keys generated by group creation or by the new XoWiki localized fields
    • Notifications:
      • When sending a notification on behalf of a person, if the system is not configured to process replies to notification, do not set the reply-to address to anything different than the sender
    • User/Person/Party API: rework and rationalize caching of all party, person and user API, create separate caches for each of these types, make the API and return dicts. acs_user::get will not fail anymore with non-existing user.
    • User Portrait: created API to retrieve and create, store and delete the user's portrait. Also address leftover child relationships from the past and delete them properly.
    • util::http: leverage new ns_http features such as request file spooling. Native implementation will now be used only on Naviserver >= 4.99.15.
       
  • Bugfixes:
    • Made sure all party emails are stored as lowercase through the API
    • Fixed long standing regression in template::list: we were looping through the list "elements", rather than the "display_elements". This prevents specifying different sets of columns to be returned depending on the "-formats" and "-selected_format" options in template::list::create.
    • acs-mail-lite: handle to_addr specified as "DisplayName <email>" without errors
    • Fixed invalidating of all existing user logins, (aka) "Logout from everywhere" feature, useful e.g. to make sure no device still holds a valid login when we change our password on a device
    • Don't lose the return URL when one tries to join a subsite before being logged in
    • Notifications: proper cleanup of acs_objects resulting from the deletion of dynamic notification requests
    • Groups:
      • When a new group is created, flush all the group::get_id caches with the same name so that the new group can be fetched correctly in case it replaces a previously deleted one
      • Cleanup message keys coming from groups in acs-translations when a group is deleted
    • acs-lang:
      • lang::util::convert_to_i18n: do not always register a en_US translation, which would be always overridden. Instead, let lang::message::register make sure that a en_US message exists and create one only as a fallback.
      • lc_time_fmt: laverage tcl clock to address shortcomings such as handling of dates in Julian/Gregorian calendar and impossible dates such as 1999-02-29, implement missing formats, support previously undocumented formats explicitly
    • search: make sure objects in the search indexer queue still exist by the time they are swept by the indexer (e.g. items deleted before the indexer could sweep them)
    • attribute::delete: fix proc so it leverages stored procedure capability of dropping the database table as well
    • util::http: fix UTF-8 encoding issues for some cornercases
  • New Packages:
    • cookie-consent: alerting users about the use of cookies on a website
    • boomerang: performance of your website from your end user’s point of view
    • xooauth: OAuth implementation, including LTI
    • dotlrn-bootstrap3-theme: Bootstrap 3 theme for DotLRN
    • xowf-monaco-plugin: Integration of Monaco editor with for code exercise types in xowf
    • proctoring-support: utilities and user interfaces to implement proctoring of the user session, mainly intended in the context of distance education and online exams. The main proctoring feature is relies only on web technologies and does not require any plugin or additional software. Optional support for the Safe Exam Browser has also been introduced. The package is currently at the core of WU Online Exam infrastructure and is integrated in the inclass exam implementation for XoWf
       
• Non-functional Changes
  • Improved regression testing
    • Goal is that all packages in oacs-5-10 pass regression test
    • Include web testing in standard regression testing
    • Introduced new test authentication authority, allowing to run many user administration tests outside the context of a "real authority": in cases where the real authority depends on external services to proof identity, (e.g. Kerberos), those tests would just fail.
    • Introduce the display of warnings in the UI
    • Added test coverage information in the automated testing pages, using the new proc-coverage API and providing test coverage information for packages and system wide.
    • Increased overall coverage of public API
    • New tests checking various data-model properties and smells
       
  • Improved scalability:

    • Provided lock-free implementation of ad_page_contract_filters and ad_page_contract_filter_rules. This change improves parallel processing of requests and is primarily interesting for sites with a few mio page views per days. These locks were among the most frequent nsv locks

    • Reduced locks on util_memoize_cache my more invariants values into per-thread caching (acs_lookup_magic_object, ad_acs_version, .... ) and by avoiding specialized calls, which can be realized by already optimized ones ("apm_package_installed_p_not_cached ref-timezones" was most frequently used util_memoize_cache entry). These changes are necessary to avoid full still-stand on the unfortunate long-time locks on util_memoize_cache stemming from permission and user management with wild-card flush operations, which require to iterate over all cache entries (which might be on a busy server several hundred thousands)

    • Added new interface for cache partitioning to reduce lock latencies on high load websites

    • Added new interface for lock-free per-thread and per-request caching to avoid scattered ad-hoc implementations

    • Better reuse of DB handles (reduced expiring/reopen/etc.), faster access to handles

  • Improved startup time:
    • When the package acs-automated-testing is disabled, startup time is reduced by avoiding loading of support functions and tests; the size of the blueprint is reduced
    • xowf: loading of at-jobs is significantly improved.
       
  • Security improvements:
    • Strengthened page contracts
    • CSP support for application packages
    • CSP fine tuing
       
  • Better exception handling based on Tcl 8.6 exception handlers ("try" and "throw", also available in Tcl 8.5)
    • Provided a new "ad_try" implementation based on Tcl's "try" replaces now the old "ad_try", "with_catch" and "with_finally", which are marked as deprecated
    • The new "ad_try" is in essence Tcl's "try" but with predefined handling of "ad_script_abort" and should be also used instead of "catch", when the OpenACS API is used (which might use script aborts)
    • All core packages use the new "ad_try" instead of the deprecated versions
       
  • Connection close reform:
    • NaviServer/AOLserver continue after connection closing commands to execute a script. This is in many situations not desired, especially, when for the page as well a .adp file exists, which will try to deliver this on the already closed connection. This can lead to errors in the error.log file, which are sometimes hard to analyze
    • Due to this cleanup, developers should use in most such cases cases "ad_script_abort"
    • Connection closing commands are e.g. ad_returnredirect, ad_redirect_for_registration, cr_write_content, ad_page_contract_handle_datasource_error, ad_return_string_as_file, ad_return_complaint,  ad_return_error,
      ad_return_forbidden, ad_return_warning, ad_return_exception_page, ns_returnredirect, ns_return, ns_returnerror
    • The new version has made on most occasions explicit, when the script should abort.
       
  • SQL cleanup:
    • Cleanup of obsolete nonportable SQL constructs in a way Oracle and PostgreSQL code base divergency is reduced:
      • "nvl" -> "coalesce"
      • "sysdate" / "now()" -> standard "current_date" or "current_timestamp"
      • Use standard-compliant "dual" table where appropriate (required by Oracle, supported by PostgreSQL)
      • Use non-dialectal cast idioms when appropriate
      • Adopt CTE idioms in Oracle codebase as well (e.g. connect -> with recursive)
      • ... (reference Oracle version will be 11gr2 as is oldest version officially supported by Oracle (See here and here)
    • Reduced superfluous .xql queries
      • acs-subsite: delete 21 files with un-referenced .xql queries
      • acs-tcl: delete 4 files
      • news: 3 files
      • file-storage: 1 file
      • dotlrn: 9 files
         
  • Deprecated commands:
    • Move deprecated code into separate files
    • Made loading of deprecated code optional (can be controlled via parameter  "WithDeprecatedCode" in section "ns_section ns/server/${server}/acs" of the config file. By default, deprecated procs are still loaded
    • When deprecated code is not loaded, the blueprint of the interpreter is smaller. The following number of lines of code can be omitted when loading without the deprecated procs:
      • acs-tcl: 3178
      • acs-templating: 450
      • xotcl-core http-client-procs: 830
      • acs-content-repository: 1717 (including .xql files)
         
  • Improved handling of server implementation-specific code
    • Provided *-procs-aolserver.tcl and *-procs-naviserver.tcl similar to *.postgresql.xql and *.oracle.xql where appropriate
       
  • General cleanup/mainteneance
    • Modernization of Tcl idioms.
    • Compliance of files, proc names, ... to the naming conventions.
    • White space cleanup, indentation changes.
    • Improvement of public API documentation
    • Adjustment of proc protection levels (public, private)
    • Adjustment of log severity
    • Cleanup of obsolete files
    • Replacement of handcrafted forms by ad_form
    • Typo fixing
    • Editor hints
    • Replacement of deprecated calls
    • Addition of missing contracts
    • ...
       
• Require Tcl 8.6, XOTcl 2.1, PostgreSQL 9.6 (PostgreSQL 9.5 EOL: February 2021), tdom 0.9
   

OpenACS Version 5.9.1 Agenda

Refactoring of rich-text editor integration

• Driving force: Debian packaging (e.g. js minified code is not allowed)
• Moved out code from acs-templating, provided interfaces to add many different rich-text editors as separate packages
• New OpenACS packages:
  • richtext-xinha
  • richtext-tinymce
  • richtext-ckeditor4 (has ability to choose between CDN and local installation via web interface)

Improving admin interface

• New theme manager:
  • Goals:
    • Make it easier to keep track of themes with local modifications
    • Make it easier to create local modifications of existing themes and to update these
    • Show differences between default theme parameter (in DB) and actual settings (in subsite parameters)
    • Allow to delete unused themes
    • Give site admin hints, which theme is used at which subsite
    • Ease theme switching
  • Added a subsite::theme_changed callback to be able to handle theme changes in custom themes (was also necessary for proper integration with DotLRN theming)
  • Added support for these features under subsite admin (/admin/)
  • Improved support for themed templates via [template::themed_template]

     

• Improved (broken) interface to define/manage groups over web interface
• Allow to send as well mail, when membership was rejected
• New functions [membership_rel::get_user_id], [membership_rel::get] and [membership_rel::get_group_id] to avoid code duplication
• Added support to let user include %forgotten_password_url% in self-registration emails (e.g. in message key acs-subsite.email_body_Registration_password)

   

• Improved subsite/www/members
  • Make it possible to manage members of arbitrary groups
  • Improved performance for large groups
  • Improved configurability: when ShowMembersListTo is set to "3", show list to members only, when this is not the whole subsite

     

• Improved user interface for /admin/applications for large number of applications
• Various fixes for sitewide-admin pages (under /acs-admin)
• Update blueprint in "install from repository" (currently just working in NaviServer)

SQL

• Further cleanup of .xql files (like what as done for acs-subsite in OpenACS 5.9.0):
  • 36 files deleted
  • Removed more than 100 obsolete named queries
  • Stripped misleading SQL statements

     

• Marked redundant / uncalled SQL functions as deprecated
• Replaced usages of obsolete view "all_object_party_privilege_map" by "acs_object_party_privilege_map"
   
• Removed type discrepancy introduced in 2002:
  • acs_object_types.object_type has type varchar(1000), while
  • acs_object_types.supertype has type varchar(100)
  • ... several more data types are involved, using acs_object_types.object_type as foreign key

     

• Simplified core SQL functions by using defaults:
  • Number of functions reduced by a factor of 2 compared to OpenACS 5.9.0 (while providing compatibility for clients using old versions),
  • Reduced code redundancy
  • Affected functions:
    • Reduced content_item__new from 12 versions to 6,
    • Reduce content_revision__new from 7 to 4
    • Similar in image__new, image__new_revision, content_item__copy, content_item__get_title, content_item__move
  • PostgreSQL 9.5 supports named parameter in the same syntax as in Oracle. Further reduction of variants will be possible, once OpenACS requires at least PostgreSQL 9.5

     

• Reduced usage of deprecated versions of SQL functions (mostly content repository calls)
• Reduced generation of dead tuples by combining multiple DML statements to one (reduces costs of checkpoint cleanups in PostgreSQL)

   

• Permission queries:
  • Improved performance
  • Support PACKAGE.FUNCTION notation for PostgreSQL to allow calls permission queries exactly the same way as in Oracle (e.g. "acs_permission.permission_p()"). This helps to reduce the number of PostgreSQL specific .xql files.
• Modernized SQL:
  • Use real Boolean types instead of character(1)
    (done for new-portal, forums, faq, attachments, categories, dotlrn, dotlrn-forums, evaluation)
  • Use real enumeration types rather than check constraints (done for storage_type text/file/lob)

CR hygienics (reduce cr bloat)

• Provided means to avoid insert/update/delete operations in the search queue:

  OpenACS adds for every new revision often multiple entries to the search_queue, without providing any means to prevent this. This requires for busy sites very short intervals between queue sweeps (otherwise too many entries pile up). Another consequence is that this behavior keeps the PostgreSQL auto-vacuum daemons permanently active. Many of these operations are useless in cases where the content repository is used for content that should not be provided via search. The changed behavior should honors a publish-date set to the future, since it will not add any content with future publish dates to the search-queue.

   

• Reduced number of insert cr_child_rels operations, just when needed:

  cr_child_rels provide only little benefit (allow one to use roles in a child-rel), but the common operation is a well available in cr_items via the parent_id. cr_child_rels do not help for recursive queries either. One option would be to add an additional argument for content_item__new to omit child-rel creation (default is old behavior) and adapt the other cases.

Security improvements

• Added support against CSRF (cross site request forgery)
  • OpenACS maintains a per-request CSRF token that ensures that form replies are coming just from sites that received the form
  • CSRF support is optional for packages where CSRF is less dangerous, and such requests are wanted (e.g. search and API-browser)

• Added Support for W3C "Upgrade-Insecure-Headers" (see https://www.w3.org/TR/upgrade-insecure-requests/):
  For standard compliant upgrade for requests from HTTP to HTTPS

• Added support for W3C "Subresource Integrity" (SRI; see https://www.w3.org/TR/SRI/)

• Added support for W3C "Content Security Policy" (CSP; see https://www.w3.org/TR/CSP/)

  • Removed "javascript:*" links (all such urls are removed from the 90 packages in oacs-5-9, excluding js libraries (ajaxhelper) and richtext code)
  • Removed "onclick", "onfocus", "onblur", "onchange" handlers from all .adp and .tcl files in the 90 packages in oacs-5-9 (excluding js libraries (ajaxhelper) and richtext code)
  • Added optional nonces to all <script> elements with literal JavaScript content

     

• Removed "generic downloader", which allowed to download arbitrary content items, when item_id was known (bug-fix)
• Improved protection against XSS and SQL-injection (strengthen page contracts, add validators, added page_contract_filter "localurl", improve HTML escaping, and URI encoding)
• Fixed for potential traversal attack (acs-api-documentation-procs)

Improvements for "host-node mapped" subsites

• Fixed links from host-node mapped subsite pages to swa-functions (must be always on main subsite)
• Made "util_current_directory" aware of host-node-mapped subsites
• Added ability to pass "-cookie_domain" to make it possible to use the same cookie for different domains
• Fixed result of affected commands "util_current_location", "ad_return_url", "ad_get_login_url" and "ad_get_logout_url" for HTTP and HTTPS, when UseHostnameDomainforReg is 0 or 1.
• Improved UI for host-node maps when a large number of site nodes exists

Reform of acs-rels

• Made acs-rels configurable to give the developer the option to specify, whether these are composable or not (default fully backward compatible). This is required to control transitivity in rel-segments
• The code changes are based on a patch provided by Michael Steigman.
  For details, see:
  • https://openacs.org/forums/message-view?message_id=4031049
  • https://openacs.org/forums/message-view?message_id=5330734

Improved status code handlers for AJAX scenarios

• Don't report data source errors with status code 200 (use 422 instead)
• Let "permission::require_permission" return forbidden (403) in AJAX calls (determined via [ad_conn ajaxp])

Improved Internationalization

• Extended language catalogs for
  • Russian (thanks to v v)
  • Italian (thanks to Antonio Pisano)
  • Spanish (thanks to Hector Romojaro)
  • German (thanks to Markus Moser)

     

• Added (missing) message keys
• Improved wording of entries
• Added message keys for member_state changes, provide API via group::get_member_state_pretty

Improved online documentation (/doc)

• Fixed many broken links
• Removed fully obsolete sections
• Improved markup (modernize HTML)
• Updated various sections

Misc code improvements:

• 18 issues from the OpenACS-bug-tracker fixed
• Made code more robust against invalid/incorrect input (page_contracts, validators, values obtained from header fields such as Accept-Language)
• Fixed quoting of message keys on many places
• Improved exception handling (often, a "catch" swallows one to much, e.g. script_aborts), introducing "ad_exception".

• Generalized handling of leading zeros:

  • Fixed cases where leading zeros could lead to unwanted octal interpretations
  • Switch to use of " util::trim_leading_zeros" instead of "template::util::leadingTrim", "dt_trim_leading_zeros" and "template::util::leadingTrim", marked the latter as deprecated

• URL encoding

  • "ad_urlencode_folder_path": new function to perform an urlencode operation on the segments of the provided folder path
  • "export_vars": encode path always correctly, except -no_base_encode is specified
  • Fixed encoding of the URL path in "ad_returnredirect"

• Improvements for "ad_conn":

  • Added [ad_conn behind_proxy_p] to check, whether the request is coming from behind a proxy server
  • Added [ad_conn behind_secure_proxy_p] to check, whether the request is coming from behind a secure proxy server
  • Added [ad_conn ajax_p] to check, whether the request is an AJAX requests (assumption: AJAX request sets header-field Requested-With: XMLHttpRequest")
  • Added [ad_conn vhost_url] to obtain the url of host-node-mapped subsites

     

• Added various missing upgrade scripts (missing since many years) of changes that were implemented for new installs to reduce differences between "new"-and "old" (upgraded) installations

• Templating

  • Get rid of various pesky "MISSING FORMWIDGET: ...formbutton:ok" messages
  • Improved support for javascript event handlers in template::head
  • New functions "template::add_event_listener" and "template::add_confirm_handler"
  • Fix handling, when "page_size_variable_p" is set (was broken since ages)

• Improved location and URL handling:

  • Refactored and commented "util_current_location" to address security issues, handle IPv6 addresses, IP literal notation, multiple drivers, "
  • Improved "security::get_secure_location" (align with documentation)

     

  • New functions:
    • "util::configured_location"
    • "util::join_location", "util::split_location"
    for working on HTTP locations to reduce scattered regexps handling URL components
  • Improved IPv6 support
  • Use native "ns_parseurl" when available, provide backward compatible version for AOLserver

• MIME types:

  • Added more Open XML formats for MS-Office to allowed content types
  • Modernized entries to IANA recommendations
  • New function "cr_check_mime_type" centralizing the retrieval of the mime_type from uploaded content

• Finalized cleanup of permissions (started in OpenACS 5.9.0):

  • Get rid of "acs_object_context_index " (and therefore on "acs_object_party_privilege_map " as well) on PostgreSQL.
    Reasons:
    • huge table,
    • expensive maintenance, used only in a few places,

• Misc new functions:

  • "lang::util::message_key_regexp": factor out scattered regexp for detecting message keys
  • "ns_md5" and "ns_parseurl": improve compatibility between AOLserver and NaviServer
  • "ad_dom_sanitize_html": allow one to specify different sets of tags, attributes and protocols and "ad_dom_fix_html", which is a light weight tidy variant.

     

• Improved HTML rendering (acs-api-browser), provide width and height to speed up rendering
• Improved ADP files (e.g. missing doc(title))
• Added usage of "ad_include_contract" on more occasions
• Modernize Tcl and HTML coding
• Reduced dependency on external programs (use Tcl functions instead)
• Improved robustness of "file delete" operations all over the code
• Improved documentation, fix demo pages
• Aligned usages of log notification levels (distinction between "error", "warning" and "notice") with coding-standards

   

• Cleaned up deprecated calls:
  • Removed usage of deprecated API functions (e.g. "cc_lookup_email_user", "cc_email_from_party", "util_unlist", ...)
  • Moved more deprecated procs to acs-outdated
  • Marked remaining (and unused) "cc_*" functions as well as deprecated.

     

• Improved Oracle and windows support
• Fixed common spelling errors and standardize spelling of product names all over the code (comments, documentation, ...)
• Many more small bug fixes

Version numbers:

• require PG 9.2 (End Of Life of PostgreSQL 9.0 was Oct 2015)
• require XOTcl 2.0 (presented at the Tcl conference in 2011).

 

Packages:

 

• New Package Parameters

  • acs-kernel:

    • MaxUrlLength: remove hard-coded constant in request processor for max accepted url paths
    • SecureSessionCookie: Let site admin determine, whether or not to use secured session cookies (useful, when not all requests are over HTTPS)
    • CSPEnabledP: activate/deactivate CSP

  • acs-kernel (recommended to be set via config file in section "ns/server/${server}/>acs"

    • NsShutdownWithNonZeroExitCode: tell NaviServer to return with a nonzero return code to cause restart (important under windows)
    • LogIncludeUserId: include user_id in access log

  • acs-api-browser:

    • ValidateCSRFP: make checking of CSRF optional (default 1)

  • acs-content-repository:

    • AllowMimeTypeCreationP: Decides whether we allow unknown mime types to be automatically registered (default: 0}

  • news-portlet:

    • display_item_lead_p: Should we display news leads in the portlet? (default 0)

  • search:

    • ValidateCSRFP: make checking of CSRF optional (default 1)

  • xotcl-request-monitor:

    • do_track_activity: turn activity monitoring on or off (default 0)
       

• New OpenACS packages:

  • richtext-xinha
  • richtext-tinymce
  • richtext-ckeditor4 (has ability to choose between CDN and local installation via GUI)
  • openacs-bootstrap3-theme (as used on openacs.org)
  • dotlrn-bootstrap3-theme

• xotcl-core:

  • Improved XOTcl 2.0 and NX support (e.g. api-browser)
  • Added "-debug", "-deprecated" to ad_* defined methods (such as e.g. "ad_instproc")
  • Make use of explicit "create" statements when creating XOTcl/NX objects (makes it easier to grab intentions and to detect typos)
  • Added parameter to "get_instance_from_db" to specify, whether the loaded objects should be initialized
  • Added support for PostgreSQL prepared statements of SQL interface in ::xo::dc (nsdb driver)

• xowiki:

  • Named all web-callable methods www-NAME (to make it clear, what is called, what has to be checked especially carefully)
  • Moved templates from www into xowiki/resources to avoid naming conflicts
  • Improved ckeditor support
  • Added usage of prepared statements for common queries
  • Improved error handling
  • Better value checking for query parameter, error reporting via ad_return_complaint
  • Added option "-path_encode" to methods "pretty_link" and "folder_path" to allow one to control, whether the result should be encoded or not (default true)

     

  • Form fields:
    • Improved repeatable form fields (esp. composite cases), don't require preallocation (can be costly in composite cases)
    • Added signing of form-fields
    • Added HTML5 attributes such as "multiple" (for "file") or "autocomplete"
    • Fixed generation of "orderby" attribute based on form-field names
    • richtext: allow one to specify "extraAllowedContent" via options
    • Improved layout of horizontal check boxes

       

  • Menu bar:
    • Added dropzone (requires bootstrap): drag and drop file upload
    • Added mode toggle (requires bootstrap)
    • Extended default policies for handling e.g. dropzone (file-upload method)
    • Distinguish between "startpage" (menu.Package.Startpage) and "table of contents" (menu.Package.Toc)

       

  • Notifications:
    • Added support for better tailorable notifications: introduced method "notification_render" (similar to "search_render")
    • Added support for tailorable subject lines (method "notification_subject")

       

  • Improved bootstrap support, use "bootstrap" as PreferredCSSToolkit
  • Switched to ckeditor4 as PreferredRichtextEditor
  • Improved handling of script-abort from within the payload of ::xowiki::Object payloads
  • Added parameter to "get_all_children" to specify, whether the child objects should be initialized

• xowf:

  • Added property "payload" to "WorkflowConstruct" in order to simplify customized workflow "allocate" actions
  • Internationalized more menu buttons

• xotcl-request-monitor

  • Added class "BanUser" (use. e.g. IP address to disallow requests from a user via request monitor)
  • Added support for optional user tracking in database
  • Added support for monitoring response-time for certain urls via munin
  • Increased usage of XOTcl 2.0 variable resolver (potentially speed improvement 4x)
  • Performed some refactoring of response-time handling to allow site-admin to make e.g. use of NaviServer's dynamic connection pool management (not included in CVS)
  • Added support for partial times in long-calls.tcl to ease interpretation of unexpected slow calls
  • last100.tcl: Don't report hrefs to URLs, except to SWAs

• chat:

  • Introduced new options to set chat rooms so login and/or logout message are not issued every time a user enters/exits a chat-room (important for chats with huge number of participants)
  • Parameterized viewing of chat-logs
  • Fixed cases of over-/under-quoting
  • Fixed JavaScript for IE, where innerHTML can cause problems

• file-storage:

  • Don't show action keys, when user has no permissions
  • Added support for copying of same-named files into a folder (adding suffix)
  • Fixed old bugs in connection with "views" package

OpenACS Version 5.9.0 Agenda

• Slimming pg SQL core:
   
  • Part 1: improve performance of object deletion
    • remove manual delete operations from acs_object__delete()
       
  • Part 2: content-repository - manual referential integrity management
    • handle referential integrity via pg's integrity constraints rather by functions cr_revision_del_ri_tr, cr_revision_ins_ri_tr, cr_revision_up_ri_tr, cr_revision_del_rev_ri_tr, and cr_revision_del_rev_ri_tr
    • fix broken/missing upgrade scripts from earlier updates
       
  • Part 3: content-repository - manual deletions and nulling
    • Removed manual nulling of live_revision and latest_revision
    • Removed manual deletion of old_revision and new_revision in cr_item_publish_audit
    • Removed manual deletion of item_id in cr_item_publish_audit, cr_release_periods, cr_item_template_map, and cr_item_keyword_map
    • Removed manual deletion of direct permissions
    • Added missing index for child_id to cr_child_rels.
       
  • Part 4: get rid of tree_sortkey in acs-objects
    • Check/fix dependencies in oacs-5-8 packages
    • Get rid of broken/uncalled functions using the column
    • Check/fix dependencies in other packages
    • Remove tree_sortkey and max_child_sortkey
• Web interface:
  • Improve client performance
    • moving core.js from head to body
    • provide kernel parameter to control expiration date for /resources/
  • Protect against more XSS attacks
  • Improved HTML validity (see oacs-5-9-html-validity for the checklist)
  • Add lightweight support for ckeditor4 for templating::richtext widget (configurable via package parameter "RichTextEditor" of acs-templating. ckeditor4 supports mobile devices (such as iPad, ...).
  • New kernel parameter ResourcesExpireInterval to control expiration dates of resources
     
• Templating:
  • Improve theme-ability
    • Move more information into theme packages in order to create responsive designs
    • Reduce hard-coding of paths, HTML etc.
  • Dimensional slider reform (ad_dimensional):
    • Remove hard-coded table layout from dimensional slider
    • Add backwards compatible templates
    • Move hard-coded styles into theme styling
    • Remove obsolete comments from ad_dimensional
  • Complete template variable controls (adding noi18n, addressing bug #2692):
    • @foo@: perform html quoting and internationalization
    • @foo;noquote@: perform internationalization
    • @foo;noi18n@: perform html quoting
    • @foo;literal@: perform neither html quoting nor internationalization
  • Improved Russian nationalization
  • Support of expiration dates and passwords for signed variables
     
• Documentation:
  • Use ACS templating for the (static) OpenACS documentation to provide a more consistent layout and user experience.
  • Make pretty-naming of acs-core packages more consistent.
     
• Misc improvements:
  • Mark unused functions of acs-tcl/tcl/table-display-procs.tcl as deprecated
  • Reduce number of muxtex locks by pre-request and per-thread caching
  • Improved development und debugging aids:
    • use "ad_log error|warning  .... " instead of "ns_log" to include information of request and callstack in error.log
    • ability to display ns_log entries caused by a request in ds-footer
    • ability to save delivered web pages in file-system for testing HTML validity (especially for admin pages, which are unaccessible for external validity testers)
  • More bug fixes
     
• Version numbers:
  * require PG 9.0 (End Of Life of PostgreSQL 8.4 was July 2014)
  * require XOTcl 2.0 (presented at the Tcl conference in 2011).
   

OpenACS Version 5.8 Agenda

• PostgreSQL 9.2+:
  • Get rid of nonstandard backslash escapes in function definitions
  • Change quote syntax in SQL files (single quotes around the functions) to recommended PostgreSQL quoting using (recommended since pg8.0, jan 2005). li>Drop aliases in favor of named function arguments (recommended since pg8.0)
  • Fix wrong function_args, add missing function_args, align default semantics with the defaults in pg (providing "null" as default means the argument is optional)
  • Make OpenACS loadable without any tweaks in the pg config files
• Use recursive queries for e.g. permission lookup to avoid performance problems in pg 8.4 and newer)
• ADP: Use byte-compiled function wherever possible in compiled adp-code, support "@var;literal@" when neither quotes nor localization is needed in compiled adp-code
• Improve support of NaviServer
• Switch to Tcl 8.5 (TIP #143)
• Improve scalability: Reduce mutex-stress on util-memoize cache and for cache maintenance in general
• Code cleanup:
  • Get rid of calls to deprecated code (e.g. ad_tables, ad_parameter, ... in acs-core and main packages)
  • Improve awareness of usage of deprecated code (complain to error.log)
  • Use Tcl 8.5 idioms
  • cleanup of various http-client approaches and introduce a common implementation util::http::get and util::http::post; get rid of other usages, mark these as deprecated
  • page-contracts: Perform checking of all ids in acs-core and main packages to improve error messages and to improve security
• OpenACS 5.8.1 should be released with main packages

OpenACS Version 5.7 Agenda

• Support for object management in core 
• Postgresql 9.0
• TinyMCE update (fix for random JS injection issue, affecting Safari)
• Fix for "remember me" issue
• WCAG2-AA

OpenACS Version 5.6 Agenda

• global parameters
• package "embeds" 
• fix search by package_id
• core works on Postgresql 8.4

OpenACS Version 5.5 Agenda

• DONE: Postgresql 8.3 support: especially regarding tsearch2
• DONE: acs-authentication:
  • fix upgrade, add conditional logic into site wide tcl library so that you can login to perform the rest of the upgrade
• DONE: tinymce:
  • upgrade to 3.1.1 + language packs
  • HTML Strict cleanup
  • create appropriate parameters for its config in acs-templating
• acs-mail-lite:
  • DONE: cleanup duplicated procs (bounce)
  • review the parsing of bouncing messages (case user_id 0)
  • DONE: rollout support
• Documentation improvements as discussed at the Guatemala conference:
  • Make current openacs.org/test-doc source for static files included in the release and provide ease means to achieve this for the release manager
    • DONE (CVS HEAD): Provide in XoWiki an alternative table of contents by nested UL/LI (without JavaScript) for static output
    • DONE (CVS HEAD): Provide in XoWiki a prototype page similar to "book" without edit-buttons etc., using the new table of contents
  • Update openacs.org/test-doc where necessary (incomplete list):
    • DONE: Fix the page ordering for the higher chapters (the original document  had no 3rd. level numbering)
    • update pages in /test-doc which are more recent in openacs/xowiki
    • bump version numbers of OpenACS, where appropriate (some places talk about openacs-5-0, others about openacs-5-1, oacs-5-2-3rc1 or 5-3) 
    • some version numbers of the required components are quite a mess. e.g. some parts say that Postgres 7.3 is required,  some examples talks about postgres 7.4.7 and 8.2.4 in the same listing.
    • also the dotlrn version numbers are old dotrln-2.0
    • Tcl version numbers should be 8.4.19
    • The install section for XOTcl is missing in II.3.4
    • remove ChangeLog from documentation
    • find some other prominent place for the ChangeLog
    • Fix indenting in examples  (e.g. in Rocael's robust web    development framework)
    • overthink Win2000 guidelines.  There are the native compiled packages from Maurizio, including everything from postgres, xotcl ....
  • It is desired to find a single person responsible for overworking the documentation, however, funding is unclear.

OpenACS Version 5.4 Agenda

• DONE: HTML Strict (openacs core)
• DONE: finish template::head (daveb)
• DONE: test acs-mail-lite (complex send)
• DONE: test notifications (complex send)
• DONE: new XinHA release, get rid of RTE & HTMLarea, test on Safari
• DONE: Form builder: add the ID attribute  to the form tag
• DONE: acs-lang - keepLocalTranslationP to be removed
• DONE search and intermedia-driver: move intermedia specific stuff to its package
• DONE: acs-mail-lite - patch for mime::qp_encode bug

Future

• Split Xinha and TinyMCE into separate packages see: https://openacs.org/forums/message-view?message_id=2750958
• Usability ("my account" page)
• XHTML ?
• Testing and documentation for recording automated tests using the firefox plugin and the upload feature for it new in automated testing. Probably needs some polishing and should be talked to with Quest who are getting into this.
• Parameter Scope Patch https://openacs.org/bugtracker/openacs/patch?patch%5fnumber=845
• Remove obsolete master template stuff (default and site master template in openacs-4/www, acs-subsite's group-master, and related CSS and images).  Probably in the version which follows 5.5 (probably 5.6).  Also remove the compat master stuff at the same time.

Things to merge into this page

Old 5.0 Roadmap discussion

Roadmap discussion 1

Ideas for Boston 2006 Future of OpenACS discussion

A .LRN Roadmap

Another .LRN Roadmap discussion

 

What's on this page?

This page should include work that is planned on and has someone committed to working on it.

Get the OpenACS Source Code

Created by roc@, last modified by Gustaf Neumann 27 Oct 2022, at 10:32 AM

These instructions describe, how to obtain OpenACS, either as a released distribution (a .tar.gz file) or from CVS.

Obtain a released version of OpenACS via .tar file:

Download from OpenACS.org: //projects/openacs/download/?versions=all

Unpack the OpenACS tarball. Usually something like this works:

tar zxvf openacs-5.10.0.tgz

Obtain OpenACS from CVS (a certain release with potential patches, or the HEAD version):

If you want to track fresh code developments between releases, or you are an OpenACS core developer, you may want to install from CVS. This is identical to downloading a distribution, except that you get the files from CVS instead of the tarball. The following commands are used to obtain the newest version of the OpenACS 5.10 branch from CVS:

cvs -d:pserver:anonymous@cvs.openacs.org:/cvsroot login
# press enter for password
cvs -d:pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r oacs-5-10 acs-core

The command above checks out the core packages of OpenACS in a directory named openacs-4. For  the entire OpenACS version 5.10 branch, you can use the following commands (adjust as required going forward):

cvs -d:pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r oacs-5-10 openacs-4

If the branch name (like oacs-5-10) is omitted, the leading edge developer version (the HEAD release) is obtained

cvs -d:pserver:anonymous@cvs.openacs.org:/cvsroot checkout openacs-4

In order to check out a single package (e.g. the package cronjob) from e.g. the leading edge developer version (HEAD), use

cvs -d:pserver:anonymous@cvs.openacs.org:/cvsroot checkout openacs-4/packages/cronjob

For most OpenACS packages, CVS aliases are defined. In order to check out e.g. the forums package from OpenACS 5.10, just use:

cvs -d:pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r oacs-5-10 forums

Depending, from which directory you are performing the checkout, you might have to move the checked-out package directory to the main "packages" directory of your installation.

More info here: https://openacs.org/test-doc/using-cvs-with-openacs

Looking for README instructions or installers? View the OpenACS Installation instructions: en:openacs-system-install, otherwise continue by setting up the OpenACS distribution:

Set up the file system for one or more OpenACS sites

According to the Linux Filesystem Hierarchy Standard is currently not strict, where the files of web servers should be stored. Common places are /var/www/ or /srv/www/. Sticking to common names eases the maintenance of systems. This name is called SERVERROOT belwo. In case, multiple OpenACS installations are on a single system, it is recommended to use e.g. domain name of the system as a path, like e.g. /var/www/openacs.org/. We use here the conventions to name the installation based on the version number /var/www/oacs-5-10/. This folder is called OPENACS_SERVICE_NAME. All the files in each OpenACS site are stored in a subdirectory (see also: en:openacs-reference-platform). The first time you install an OpenACS site on a server, you must create the parent directory and set its permissions:

While logged in as root:

mkdir -p /var/www/oacs-5-10
chgrp web /var/www/oacs-5-10
chmod 770 /var/www/oacs-5-10

Move the uncompressed code to SERVERROOT and rename the directory to $OPENACS_SERVICE_NAME:

mv openacs-4 /var/www/oacs-5-10/$OPENACS_SERVICE_NAME

Tcl Procs

Created by Rocael Hernández Rizzardini, last modified by Gustaf Neumann 28 Sep 2022, at 11:32 AM

• Use namespace

  Define your procs with a namespace like mypackage::foo_proc. Here is a discussion about [this]. Check many examples in the code, example:

  namespace eval auth {} 
  
  ad_proc -public auth::require_login { 
       {-level ok} 
       {-account_status ok} 
    } { 
       doc...  
       @return something 
       @see ad_script_abort 
    } { 
    ... proc body 
  }
  

• Use procs safely and their safer variations to help keep code robust and avoid security issues.

  Particularly in cases, where user_input is processed, be sure to avoid executing unwanted code. Use the Tcl expand operator {*} instead of eval. Use
      {*}$cmd
  instead of
      eval $cmd
  For legacy code, you might use  util::safe_eval instead of eval in such cases; subst_safe precedes meta characters with backslashes.

• Use named parameters whenever possible 

  Define named parameters on your procs such that parameters will not be mixed up if somebody makes a mistake on passing the order of parameters. Also, this makes the proc easier to add additional parameters in the future if needed.

  Use:

     ad_proc proc_name { {-parent_id pid} {-child_id cid} } ...

  and not

     ad_proc proc_name {pid cid} ...

  This way, developers will need to call proc stating explicitly which parameter are passed. This is especially useful when some parameters are optional.

  Also, when calling a proc in your Tcl script, it is recommended to write one parameter per line like this:

     set my_var [proc_name  \ 
                      -parent_id $pid \ 
                      -child_id $cid]

  Rather than:

     set my_var [proc_name -parent_id $pid -child_id cid]

  Again, this helps to make the code more clean and readable.



• Use ad_proc to define your Tcl procs

  Make use of ad_proc. And make use of the self documentation facility of ad_proc.

  	ad_proc foo {}
  	   Use this area to document
  	} 
  	   # .... your implementation of proc foo
  	}
  

  This way, the API browser will pick up your documentation automatically. Is encouraged to use automatic api-doc documentation that ad_provides, such as: @author, @return, @see

• Use "ad_proc -private ..." always when a proc is used only in one package
  
  This reduces the size of the public API and improves the flexibility of the package maintainers.

• Use "ad_proc -deprecate ..." when removing definitions from the public API
  
  When deprecated code is called, the error.log of the site will show its usage. This way, a site maintainer can update with code with the new replacement call.
  
  Don't move deprecated calls immediately to the long-range backward compatibility procs (tcl/deprecated-procs.tc). When OpenACS is configured to omit loading of long deprecated code (WithDeprecatedCode set to 0) these files are not loaded to reduce the every growing blueprint bloat. Therefore, these files should only contain code, which was deprecated at LEAST ONE RELEASE EARLIER, such that site admins have one release time to fix calls to deprecated code.

• Avoid using "upvar"

  Try to avoid using "upvar". If needed, pass in a parameter that specifies the "upvar" name. This way, the one using your proc has the option to name his/her variable. Example:

      ad_proc upvaring {-upvar_name:required} {
          upvar $upvar_name local_var
      }
  

• Use modern Tcl idioms

  Do not use "==" in comparing strings. Using "if {$string == "foo"}" tries to make a numeric comparison first. Instead, make use of "if {"foo" eq $string}" or if you need the negation "if {"foo" ne $string}".
  
  Do not use "if {[lsearch -exact $list $element] > -1}", but use "if {$element in $list}" instead, or "if {$element ni $list}" in case a "not in" test is required.

• Always "return" at the end of your proc

  And if you have to return more than one variable, use associative arrays, which can be extended by additional fields without breaking code

  So instead of this:

     ad_proc ... {
        ..... 
        return [list $creation_status $creation_message ...]
     } 

  use key/value pairs or Tcl arrays to group related information:

     ad_proc ... {     
        array set creation_info {
                   creation_status {}
                   creation_message {}
                   element_messages {}
                   account_status {}
                   account_message {}              
        } 
        .....     
        return [array get creation_info] 
     } 

• ... or even better: use Tcl dicts

     ad_proc proc ... {} {
  	  set creation_info [dict create  \
  	               creation_status {}   \
  	               creation_message {}  \
  	               element_messages {}  \
  	               account_status {}    \
  	               account_message {}   ]
  	  ....     
        return $creation_info 
    }
  
  

• Read the Tcl Style guide

  This is the Tcl style guide (PDF), try to apply relevant guidelines. In particular chapter 4,5 and 7

Setup with docker-s6

Created by Gustaf Neumann Vlad V, last modified by Vlad V 22 Sep 2022, at 11:20 PM

Basic installation

A quick way to install and test-drive OpenACS is via docker. The s6 docker images were developed by Vlad and use the docker overlay structures as supported by docker-compose, which is a tool for defining and running multi-container Docker applications.

The minimal setup for testing is to get the repository

git clone https://github.com/oupfiz5/openacs-s6
cd openacs-s6/src

and to run it there, e.g.

docker compose up

This command installs OpenACS (oacs-5-10) from GitHub, PostgreSQL (14.1) and NaviServer 4.99.23 and starts the server on port 8080.  We are planning to merge and probably move this repository to the OpenACS project on GitHub.

More advanced installations

In some cases, a developer might need more control over versions used, running multiple different OpenACS versions and instances in parallel, etc. Docker allows composing newly configured setups more or less on the fly via specifying more command line options, or via .env files. Such a setup can be provided by creating the docker-compose setup on a common place on the machine and by creating multiple .env files.

git clone https://github.com/oupfiz5/openacs-s6 /var/www/openacs/docker-s6
cd /var/www/openacs/docker-s6/src

Create in this directory e.g. a file named oacs1.env with the following content:

# * Define port and project name
COMPOSE_PROJECT_NAME="${COMPOSE_PROJECT_NAME:-oacs-8071}"

# * Using potentially different compose files
COMPOSE_FILE="${COMPOSE_FILE:-docker-compose.yaml}"

# * Common
TZ="${TZ:-UTC}"

# * Postgres
PGDATABASE="${PGDATABASE:-openacs}"
PGHOST="${PGHOST:-postgres}"
PGPASSWORD="${PGPASSWORD:-testing}"
PGPORT="${PGPORT:-5432}"
PGUSER="${PGUSER:-openacs}"
POSTGRES_DB="${PGDATABASE}"
POSTGRES_PASSWORD="${PGPASSWORD}"
POSTGRES_REPOSITORY="postgres"
POSTGRES_TAG="14.1-alpine"
POSTGRES_USER="${PGUSER}"

# * OpenACS
NS_CONF="${NS_CONF:-/usr/local/ns/conf/openacs-config.tcl}"
OACS_REPOSITORY="${OACS_REPOSITORY:-oupfiz5}"
OACS_IMAGE_NAME="${OACS_IMAGE_NAME:-openacs-s6}"
OACS_IMAGE_TAG="${OACS_IMAGE_TAG:-v3-oacs-5-10}"
OACS_LISTEN_PORT="${OACS_LISTEN_PORT:-8071}"
oacs_httpport='8000'
oacs_db_user="${PGUSER}"
oacs_db_passwod="${PGPASSWORD}"
oacs_db_name="${PGDATABASE}"
oacs_db_host="${PGHOST}"
oacs_db_port="${PGPORT}"
oacs_ipaddress='0.0.0.0'

Important variables are the listen port (OACS_LISTEN_PORT), the project name (COMPOSE_PROJECT_NAME) for running multiple instances at the same time without conflicting. In order to install other versions than the default (v3-oacs-5-10), the variable OACS_TAG can be used. See for more details the README file on GitHub (openacs-s6). For multiple instances, multiple such .env-files can be created.

To use an .env file oacs1.env one can run it as follows

(cd /var/www/openacs/docker-s6/src; docker compose up --env-file oacs1.env)

or run it in daemon mode by adding  a “-d” after up.

Preliminaries

To use this setup, recent versions of docker and docker-compose are needed. There is a constant development, especially on docker-compose. The setup is tested with docker-compose 1.27 (as provided by macPorts on macOS) and with docker compose v2.10.2 under Linux. For docker compose under Linux, it is usually sufficient to get the binary (single file) from the docker download pages.

https://docs.docker.com/
https://docs.docker.com/compose/

Install OpenACS with NaviServer from Scratch

Created by Gustaf Neumann, last modified by Gustaf Neumann 30 Aug 2022, at 02:43 PM

This page describes how to install OpenACS with NaviServer on Unix-like systems (e.g. Linux, macOS, Solaris, OmniOS) by compiling all but PostgreSQL from scratch, guided by script that collects the components from various sources, compiles it, etc.

The installation is done in two steps:

• install-ns.sh: Install NaviServer and its components for a PostgreSQL installation from scratch by obtaining the relevant sources and compiling it. The script assumes PostgreSQL to be already installed (or obtainable via package managers), and installs all other components by obtaining it from the source repositories and compiling it from scratch (e.g. Tcl, tcllib, tDOM, libthread, nsf/XOTcl 2).
   
• install-oacs.sh: Install OpenACS from CVS/git. This script configures a (pre-installed) PostgreSQL installation for
  OpenACS, adds hstore, installs OpenACS core, basic OpenACS packages, xowiki, xowf and optionally dotlrn from CVS/git and generates a config file and startup files (for Ubuntu and Fedora Core). The script assumes a pre-existing NaviServer installation, installed e.g. via install-ns.sh

 These install scripts are frequently updated when new components are released or problems are detected (commit log).

If you open the links above, use save-as in the browser to save the files. Alternatively, download the files as .zip file or clone the repository via GitHub.

   cd /usr/local/src
   git clone https://github.com/gustafn/install-ns
   cd install-ns

The scripts work under a typical Linux installation (e.g. Ubuntu, Fedora Core) as well as on Mac OS X or on OmniOS,  OpenBSD 6.1, 6.3, 6.6, 6.8, 6.9, FreeBSD 12.2, 13.0, Ubuntu 12.04, 13.04, 14.04, 16.04, 18.04, 20.04, Raspbian GNU/Linux 9.4 (stretch), Fedora Core 18, 20, 32, 35, CentOS 7, Roxy Linux 8.4, ArchLinux.. The scripts are tested (over the years) with PostgreSQL 9.1 to 14.5.

On a fresh Ubuntu installation, you should be able to download the two scripts from this page and install OpenACS with NaviServer in the following steps:

   sudo bash

   bash install-ns.sh
   bash install-ns.sh build

   bash install-oacs.sh
   bash install-oacs.sh build

When running "install-ns.sh", you will see an output like the following (shortened here).

SETTINGS   build_dir              (Build directory)                 /usr/local/src
           ns_install_dir         (Installation directory)          /usr/local/ns
           version_ns             (Version of NaviServer)           4.99.24
           git_branch_ns          (Branch for git checkout of ns)   main
           version_modules        (Version of NaviServer Modules)   4.99.24
           version_tcllib         (Version of Tcllib)               1.20
           version_thread         (Version Tcl thread library)      
           version_xotcl          (Version of NSF/NX/XOTcl)         2.4.0
           version_tcl            (Version of Tcl)                  8.6.12
           version_tdom           (Version of tDOM)                 0.9.1
           ns_user                (NaviServer user)                 nsadmin
           ns_group               (NaviServer group)                nsadmin
                                  (Make command)                    make
                                  (Type command)                    type -a
           ns_modules             (NaviServer Modules)              nsdbpg
           with_mongo             (Add MongoDB client and server)   0
           with_postgres          (Install PostgreSQL DB server)    1
           with_postgres_driver   (Add PostgreSQL driver support)   1
           with_system_malloc     (Tcl compiled with system malloc) 0
           with_ns_doc            (NaviServer documentation)        1

The values in the first column can be used to tailor the system for your needs by setting same-named shell variables. One can use e.g.

   version_tdom=0.9.3 bash install-ns.sh 

to configure the compilation to use tDOM in version 0.9.3 instead of 0.9.1. The default settings should be OK for standard setups.

After running both scripts in the default configuration, you will see e.g.

   Congratulations, you have installed OpenACS with NaviServer on your machine.
   You might start the server manually with
       sudo /usr/local/ns/bin/nsd -t /usr/local/ns/config-oacs-5-10-0.tcl -u nsadmin -g nsadmin

One can start the server manually with the mentioned command.

Using systemd

On Fedora/CentOS or on Ubuntu installations starting with 15.04, systemd is used. The generated startup file for RedHat/Fedora is in /lib/systemd/system/oacs-5-10-0.service. The startup commands for systemd are

    sudo systemctl status oacs-5-10-0
    sudo systemctl start oacs-5-10-0
    sudo systemctl stop oacs-5-10-0

Remember, when a new systemd service is installed, systemd requires the following command to re-scan its service files:

    sudo systemctl daemon-reload

To start OpenACS automatically on every new start of the machine, issue the following command:

    sudo systemctl enable oacs-5-10-0

Using upstart

Alternatively, some older Ubuntu versions (up to 15.04) use upstart. For upstart, the the generated startup file is in /etc/init/oacs-5-9-1.conf. The service can be started/managed with the following commands

    sudo initctl status oacs-5-10-0
    sudo initctl start oacs-5-10-0
    sudo initctl stop oacs-5-10-0

Configuration

When the service is running, one can use OpenACS by browsing to http://localhost:8000/ (when the browser and server is running on the same host). The relevant files are stored under the following locations:

Configuration file	/usr/local/ns/config-oacs-5-10-0.tcl
access.log	/var/www/oacs-5-10-0/log/
error.log	/var/www/oacs-5-10-0/log/
Source tree	/var/www/openacs-5-10-0/packages/

 

OpenACS conferences

Created by Gustaf Neumann, last modified by Gustaf Neumann 06 Jul 2022, at 07:39 PM

1. Copenhagen, April 11, 2003
2. Heidelberg, April 23-28, 2004
3. Madrid, May 9-11, 2005
4. Boston, November 1-3, 2006
5. Vienna, April 25-28, 2007
6. Guatemala City, February 12-16, 2008
7. Valencia, November 18-19, 2008
8. Costa Rica, November 3-6, 2009
9. Vienna, June 30 and July 1, 2022

Install Oracle

Created by OpenACS community, last modified by Gustaf Neumann 26 Jun 2022, at 04:43 PM

Installing Oracle is optional, if you are installing PostgreSQL.

If you are using new versions of the oacs-5-10 branch of OpenACS 5.10.1, read oacs-5-10-on-oracle-19c for the installation of Oracle 19c and the improved support in the oacs-core packages. The description below is for earlier versions.

An excellent guide for installing Oracle 10g on Linux can be found at www.puschitz.com.

OpenACS 5.2.x will install with Oracle 9i but has not been extensively tested so may still have bugs or tuning issues.

This installation guide attempts to present all of the information necessary to complete an OpenACS installation. We try hard to make all of the steps possible in one pass, rather than having a step which amounts to "go away and develop a profound understanding of software X and then come back and, in 99% of all cases, type these two lines." The exception to our rule is Oracle production systems. This page describes a set of steps to get a working Oracle development server, but it is unsuitable for production systems. If you will be using OpenACS on Oracle in a production environment, you will experience many problems unless you develop a basic understanding of Oracle which is outside the scope of this document.

This document assumes that you are installing Oracle on the same box as AOLserver. For more details on a remote Oracle installation, see Daryl Biberdorf's document.

Defaults

We use the following defaults while installing Oracle.

Variable	Value	Reason
ORACLE_HOME	/ora8/m01/app/oracle/product/8.1.7	This is the default Oracle installation directory.
ORACLE_SERVICE	ora8	The service name is a domain-qualified identifier for your Oracle server.
ORACLE_SID	ora8	This is an identifier for your Oracle server.
ORACLE_OWNER	oracle	The user who owns all of the oracle files.
ORACLE_GROUP	dba	The special oracle group. Users in the dba group are authorized to do a connect internal within svrmgrl to gain full system access to the Oracle system.

Get Oracle

You can register and download Oracle (for free) from Oracle Downloads. You need this if you want to use an Oracle database.

Production Oracle systems should run on certified platforms. Follow the metalink note 223718.1to find certified platforms. If you don't have metalink access, take a look at the Oracle on Linux FAQ: http://www.oracle.com/us/technologies/027617.pdf. In summary, free and inexpensive Linux distributions are not certified.

If you don't have an account at OTN get one: you can download the Oracle software from the Oracle Downloads page.

Each Oracle release comes with extensive and usually quite well-written documentation. Your first step should be to thoroughly read the release notes for your operating system and your Oracle version. Find the docs here:

• http://www.oracle.com/technetwork/documentation/oracle8i-085806.html

• http://www.oracle.com/pls/db92/homepage

• https://docs.oracle.com/cd/B19306_01/server.102/b14214/chapter2.htm#g62359

It is generally useful to run a particular Oracle version with its latest patch-set. At the time of writing these were 8.1.7.4 and 9.2.0.5, both of which are considered to be very stable.

To be able to download a patch-set, you need an account on Metalink (not free). You may find the appropriate patch-set by following Andrew's suggestion.

Things to Keep in Mind

Oracle is very well-documented software, the online documentation comes with printable PDFs and full-text search. Altogether there is more than 20.000 pages of documentation, so do not expect to understand Oracle within in a few hours. The best starting pointing into Oracle is the Concepts book. Here's the 8i version and the 9.2 version.

To give you an idea of how configurable Oracle is and how much thought you may need to put into buying the proper hardware and creating a sane setup, you should thoroughly read Cary Millsap's Configuring Oracle Server for VLDB and the Optimal Flexible Architecture standard.

Throughout these instructions, we will refer to a number of configurable settings and advise certain defaults. With the exception of passwords, we advise you to follow these defaults unless you know what you are doing. Subsequent documents will expect that you used the defaults, so a change made here will necessitate further changes later. For a guide to the defaults, please see the section called “Defaults”.

In order for OpenACS to work properly you need to set the environment appropriately.

export ORACLE_BASE=/ora8/m01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/8.1.7
export PATH=$PATH:$ORACLE_HOME/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export ORACLE_SID=ora8
export ORACLE_TERM=vt100
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data

umask 022

open_cursors = 500

nls_date_format = "YYYY-MM-DD"

For additional resources/documentation, please see this thread and Andrew Piskorski's mini-guide.

Pre-Installation Tasks

Though Oracle 8.1.7 has an automated installer, we still need to perform several manual, administrative tasks before we can launch it. You must perform all of these steps as the root user. We recommend entering the X window system as a normal user and then doing a su -. This command gives you full root access.

• Login as a non-root user and start X by typing startx

  [joeuser ~]$ startx
  

• Open a terminal window type and login as root

  [joeuser ~]$ su -
  Password: ***********
  [root ~]#
  

• Create and setup the oracle group and oracle account

  We need to create a user oracle, which is used to install the product, as well as starting and stopping the database.

  [root ~]# groupadd dba
  [root ~]# groupadd oinstall
  [root ~]# groupadd oracle
  [root ~]# useradd -g dba -G oinstall,oracle -m oracle
  [root ~]# passwd oracle
  

  You will be prompted for the New Password and Confirmation of that password.

• Setup the installation location for Oracle. While Oracle can reside in a variety of places in the file system, OpenACS has adopted /ora8 as the base directory.

  Note: the Oracle install needs about 1 GB free on /ora8 to install successfully.

  [root ~]# mkdir /ora8
  root:/ora8# cd /ora8
  root:/ora8# mkdir -p m01 m02 m03/oradata/ora8
  root:/ora8# chown -R oracle.dba /ora8
  root:/ora8# exit
  

• Set up the oracle user's environment

  • Log in as the user oracle by typing the following:

    [joeuser ~]$ su - oracle
    Password: ********
    

  • Use a text editor to edit the .bash_profile file in the oracle account home directory.

    [oracle ~]$ emacs .bash_profile
    

    You may get this error trying to start emacs:

    Xlib: connection to ":0.0" refused by server
    Xlib: Client is not authorized to connect to Server
    emacs: Cannot connect to X server :0.
    Check the DISPLAY environment variable or use `-d'.
    Also use the `xhost' program to verify that it is set to permit
    connections from your machine.
    

    If so, open a new terminal window and do the following:

    [joeuser ~]$ xhost +localhost
    

    Now, back in the oracle terminal:

    [oracle ~]$ export DISPLAY=localhost:0.0
    [oracle ~]$ emacs .bash_profile
    

    Try this procedure anytime you get an Xlib connection refused error.

  • Add the following lines (substituting your Oracle version number as needed) to .bash_profile:

    export ORACLE_BASE=/ora8/m01/app/oracle
    export ORACLE_HOME=$ORACLE_BASE/product/8.1.7
    export PATH=$PATH:$ORACLE_HOME/bin
    export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
    export ORACLE_SID=ora8
    export ORACLE_TERM=vt100
    export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
    
    umask 022
    

    Save the file by typing CTRL-X CTRL-S and then exit by typing CTRL-X CTRL-C. Alternatively, use the menus.

  Make sure that you do not add any lines like the following

  # NLS_LANG=american
  # export NLS_LANG
  

  These lines will change the Oracle date settings and will break OpenACS since OpenACS depends on the ANSI date format, YYYY-MM-DD dates.

• Log out as oracle

  [oracle ~]$ exit
  

• Log back in as oracle and double check that your environment variables are as intended. The env command lists all of the variables that are set in your environment, and grep shows you just the lines you want (those with ORA in it).

  [joeuser ~]$ su - oracle
  [oracle ~]$ env | grep ORA
  

  If it worked, you should see:

  ORACLE_SID=ora8
  ORACLE_BASE=/ora8/m01/app/oracle
  ORACLE_TERM=vt100
  ORACLE_HOME=/ora8/m01/app/oracle/product/8.1.7
  ORA_NLS33=/ora8/m01/app/oracle/product/8.1.7/ocommon/nls/admin/data
  

  If not, try adding the files to ~/.bashrc instead of .bash_profile. Then logout and log back in again. Also, be certain you are doing su - oracle and not just su oracle. The - means that .bashrc and .bash_profile will be evaluated.

  Make sure that /bin, /usr/bin, and /usr/local/bin are in your path by typing:

  [oracle ~]$ echo $PATH
  /bin:/usr/bin:/usr/local/bin:/usr/bin/X11:/usr/X11R6/bin:/home/oracle/bin:/ora8/m01/app/oracle/product/8.1.7/bin
  

  If they are not, then add them to the .bash_profile by changing the PATH statement above to PATH=$PATH:/usr/local/bin:$ORACLE_HOME/bin

Installing Oracle 8.1.7 Server

• Log in as oracle and start X if not already running. Start a new terminal:

  [joeuser ~]$ xhost +localhost
  [joeuser ~]$ su - oracle
  Password: **********
  [oracle ~]$ export DISPLAY=localhost:0.0
  

• Find the runInstaller script

  • If you are installing Oracle from a CD-ROM, it is located in the install/linux path from the cd-rom mount point

    [oracle ~]$ su - root
    [root ~]# mount -t iso9660 /dev/cdrom /mnt/cdrom
    [root ~]# exit
    [oracle ~]$ cd /mnt/cdrom
    

  • If you are installing from the tarball, the install script is located in the Oracle8iR2 directory that was created when you expanded the archive.

    [oracle ~]$ cd /where/oracle/Disk1
    

  Check to make sure the file is there.

  oracle:/where/oracle/Disk1$ ls
  doc  index.htm  install  runInstaller  stage  starterdb
  

  If you don't see runInstaller, you are in the wrong directory.

• Run the installer

  oracle:/where/oracle/Disk1$ ./runInstaller
  

  A window will open that welcomes you to the 'Oracle Universal Installer' (OUI). Click on "Next"

  Note

  Some people have had trouble with this step on RedHat 7.3 and 8.0. If so, try the following steps before calling ./runInstaller:

  1. Execute the following command: /usr/i386-glibc21-linux/bin/i386-glibc21-linux-env.sh

  2. Type export LD_ASSUME_KERNEL=2.2.5

• The "File Locations" screen in the OUI:

  • "Source" path should have been prefilled with "(wherever you mounted the CDROM)/stage/products.jar"

  • "destination" path says "/ora8/m01/app/oracle/product/8.1.7"

    If the destination is not correct it is because your environment variables are not set properly. Make sure you logged on as oracle using su - oracle. If so, edit the ~/.bash_profile as you did in the section called “Pre-Installation Tasks”

  • Click "Next" (a pop up window will display Loading Product information).

• The "Unix Group Name" screen in the OUI:

  • The Unix Group name needs to be set to 'oinstall' ( we made this Unix group earlier ).

  • Click "Next"

  • A popup window appears instantly, requesting you to run a script as root:

  • • Debian users need to link /bin/awk to /usr/bin/awk before running the script below

      [joueser ~]$ su -
      [root ~]# ln -s /usr/bin/awk /bin/awk
      

  • Open a new terminal window, then type:

    [joeuser ~]$ su -
    [root ~]# cd /ora8/m01/app/oracle/product/8.1.7
    [root ~]# ./orainstRoot.sh  
    ; You should see:
    Creating Oracle Inventory pointer file (/etc/oraInst.loc)
    Changing groupname of /ora8/m01/app/oracle/oraInventory to oinstall.
    [root ~]# mkdir -p /usr/local/java
    [root ~]# exit
    [joeuser ~]$ exit
    

  • Click "Retry"

• The "Available Products" screen in the OUI:

  • Select "Oracle 8i Enterprise Edition 8.1.7.1.0"

  • Click "Next"

• The "Installation Types" screen

  • Select the "Custom" installation type.

  • Click "Next"

• The "Available Product Components" screen

  • In addition to the defaults, make sure that "Oracle SQLJ 8.1.7.0," "Oracle Protocol Support 8.1.7.0.0," and "Linux Documentation 8.1.7.0.0" are also checked.

  • Click "Next"

  • A progress bar will appear for about 1 minute.

• The "Component Locations" screen in the OUI

  • Click on the "Java Runtime Environment 1.1.8" It should have the path "/ora8/m01/app/oracle/jre/1.1.8"

  • Click "Next"

  • A progress bar will appear for about 1 minute.

• The "Privileged Operation System Groups" screen in the OUI

  • Enter "dba" for "Database Administrator (OSDBA) Group"

  • Enter "dba" for the "Database Operator (OSOPER) Group"

  • Click "Next"

  • A progress bar will appear for about 1 minute.

• The "Authentication Methods" screen

  • Click "Next"

• The next screen is "Choose JDK home directory"

  • Keep the default path: /usr/local/java

  • Click "Next"

• The "Create a Database" screen in the OUI

  • Select "No" as we will do this later, after some important configuration changes.

  • Click "Next"

• The next screen is "Oracle Product Support"

  • TCP should be checked with "Status" listed as Required

  • Click "Next"

• The "Summary" screen in the OUI

  • Check the "Space Requirements" section to verify you have enough disk space for the install.

  • Check that "(144 products)" is in the "New Installations" section title.

  • Click "Install"

  • A progress bar will appear for about 20 - 30 minutes. Now is a good time to take a break.

  • A "Setup Privileges" window will popup towards the end of the installation asking you to run a script as root

  • Run the script. Switch to the oracle user first to set the environment appropriately and then do su to get root privileges, while keeping the oracle user's environment.

    [joeuser ~]$ su - oracle
    Password: *********
    [oracle ~]$ su
    Password: *********
    [root ~]# /ora8/m01/app/oracle/product/8.1.7/root.sh
    ; You should see the following.   
    
    Creating Oracle Inventory pointer file (/etc/oraInst.loc)
    Changing groupname of /ora8/m01/app/oracle/oraInventory to oinstall.
    # /ora8/m01/app/oracle/product/8.1.7/root.sh
    Running Oracle8 root.sh script...
    The following environment variables are set as:
        ORACLE_OWNER= oracle
        ORACLE_HOME=  /ora8/m01/app/oracle/product/8.1.7
        ORACLE_SID=   ora8
    
    Enter the full pathname of the local bin directory: [/usr/local/bin]: 
    
    Press ENTER here to accept default of /usr/local/bin
          
    
    Creating /etc/oratab file...
    Entry will be added to the /etc/oratab file by
    Database Configuration Assistants when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    IMPORTANT NOTE: Please delete any log and trace files previously
                    created by the Oracle Enterprise Manager Intelligent
                    Agent. These files may be found in the directories
                    you use for storing other Net8 log and trace files.
                    If such files exist, the OEM IA may not restart.
    

  • Do not follow the instructions on deleting trace and log files, it is not necessary.

  [root ~]# exit
  [joeuser ~]$ exit
  

• Go back to the pop-up window and click "OK"

• The "Configuration Tools" screen in the OUI

  • This window displays the config tools that will automatically be launched.

• The "Welcome" screen in the "net 8 Configuration Assistant"

  • Make sure the "Perform Typical installation" is not selected.

  • Click "Next"

  • The "Directory Service Access" screen in the "Net 8 Configuration Assistant"

  • Select "No"

  • Click "Next"

• The "Listener Configuration, Listener Name" screen in the "Net 8 Configuration Assistant"

  • Accept the default listener name of "LISTENER"

  • Click "Next"

• The "Listener Configuration, Select Protocols" screen in the "Net 8 Configuration Assistant"

  • The only choice in "Select protocols:" should be "TCP/IP"

  • Click "Next"

• The "Listener Configuration TCP/IP Protocol" screen in the "Net 8 Configuration Assistant"

  • Default Port should be 1521 and selected.

  • Click "Next"

• The "Listener Configuration, More Listeners" screen in the "Net 8 Configuration Assistant"

  • Select "No"

  • Click "Next"

• The "Listener Configuration Done" screen in the "Net 8 Configuration Assistant"

  • Click "Next"

• The "Naming Methods Configuration" screen in the "Net 8 Configuration Assistant"

  • Select "No"

  • Click "Next"

• The "Done" screen in the "Net 8 Configuration Assistant"

  • Click "Finish"

• The "End of Installation" screen in the OUI

  • Click "Exit"

  • Click "Yes" on the confirmation pop up window.

  • The Oracle Universal Installer window should have disappeared!

Congratulations, you have just installed Oracle 8.1.7 Server! However, you still need to create a database which can take about an hour of non-interactive time, so don't quit yet.

Creating the First Database

This step will take you through the steps of creating a customized database. Be warned that this process takes about an hour on a Pentium II with 128 MB of RAM.

Note

RedHat 7.3 and 8.0 users: Before running dbassist, do the following.

1. Download the glibc patch from Oracle Technet into /var/tmp.

2. cd $ORACLE_HOME

3. tar xzf /var/tmp/glibc2.1.3-stubs.tgz

4. ./setup_stubs

• Make sure you are running X. Open up a terminal and su to oracle and then run the dbassist program.

  [joeuser ~]$ xhost +localhost
  [joeuser ~]$ su - oracle
  Password: *********
  [oracle ~]$ export DISPLAY=localhost:0.0
  [oracle ~]$ dbassist
  

• The "Welcome" screen in the Oracle Database Configuration Agent (ODCA)

  • Select "Create a database"

  • Click "Next"

• The "Select database type" screen in the ODCA

  • Select "Custom"

  • Click "Next"

• The "Primary Database Type" window in ODCA

  • Select "Multipurpose"

  • Click "Next"

• The "concurrent users" screen of the ODCA

  • Select "60" concurrent users.

  • Click "Next"

• Select "Dedicated Server Mode", click "Next"

• Accept all of the options, and click Next Oracle Visual Information Retrieval may be grayed out. If so, you can ignore it; just make sure that everything else is checked.

• For "Global Database Name", enter "ora8"; for "SID", also enter "ora8" (it should do this automatically). Click "Change Character Set and select UTF8. Click "Next".

• Accept the defaults for the next screen (control file location). Click "Next"

• Go to the "temporary" and "rollback" tabs, and change the Size (upper-right text box) to 150MB. Click "Next"

• Increase the redo log sizes to 10000K each. Click "Next"

• Use the default checkpoint interval & timeout. Click "Next"

• Increase "Processes" to 100; "Block Size" to 4096 (better for small Linux boxes; use 8192 for a big Solaris machine).

• Accept the defaults for the Trace File Directory. Click "Next"

• Finally, select "Save information to a shell script" and click "Finish" (We're going to examine the contents of this file before creating our database.)

• Click the "Save" button. Oracle will automatically save it to the correct directory and with the correct file name. This will likely be /ora8/m01/app/oracle/product/8.1.7/assistants/dbca/jlib/sqlora8.sh

• It will alert you that the script has been saved successfully.

• Now we need to customize the database configuration a bit. While still logged on as oracle, edit the database initialization script (run when the db loads). The scripts are kept in $ORACLE_HOME/dbs and the name of the script is usually initSID.ora where SID is the SID of your database. Assuming your $ORACLE_HOME matches our default of /ora8/m01/app/oracle/product/8.1.7, the following will open the file for editing.

  [oracle ~]$ emacs /ora8/m01/app/oracle/product/8.1.7/dbs/initora8.ora
  

• Add the following line to the end:

  nls_date_format = "YYYY-MM-DD"
  

• Now find the open_cursors line in the file. If you're using emacs scroll up to the top of the buffer and do CTRL-S and type open_cursors to find the line. The default is 100. Change it to 500.

  open_cursors = 500
  

• Save the file. In emacs, do CTRL-X CTRL-S to save followed by CTRL-X CTRL-C to exit or use the menu.

• At this point, you are ready to initiate database creation. We recommend shutting down X to free up some RAM unless you have 256 MB of RAM or more. You can do this quickly by doing a CRTL-ALT-BACKSPACE, but make sure you have saved any files you were editing. You should now be returned to a text shell prompt. If you get sent to a graphical login screen instead, switch to a virtual console by doing CRTL-ALT-F1. Then login as oracle.

• Change to the directory where the database creation script is and run it:

  [oracle ~]$ cd /ora8/m01/app/oracle/product/8.1.7/assistants/dbca/jlib
  oracle:/ora8/m01/app/oracle/product/8.1.7/assistants/dbca/jlib$ ./sqlora8.sh
  

  In some instances, Oracle will save the file to /ora8/m01/app/oracle/product/8.1.7/assistants/dbca Try running the script there if your first attempt does not succeed.

• Your database will now be built. It will take > 1 hour - no fooling. You will see lots of errors scroll by (like: "ORA-01432: public synonym to be dropped does not exist") Fear not, this is normal.

  Eventually, you'll be returned to your shell prompt. In the meantime, relax, you've earned it.

Acceptance Test

For this step, open up a terminal and su to oracle as usual. You should be running X and Netscape (or other web browser) for this phase.

• You need to download the "Oracle Acceptance Test" file. It's available here and at http://philip.greenspun.com/wtr/oracle/acceptance-sql.txt. Save the file to /var/tmp

• In the oracle shell, copy the file.

  [oracle ~]$ cp /var/tmp/acceptance-sql.txt /var/tmp/acceptance.sql
  

• Once you've got the acceptance test file all set, stay in your term and type the following:

  [oracle ~]$ sqlplus system/manager
  

  SQL*Plus should startup. If you get an ORA-01034: Oracle not Available error, it is because your Oracle instance is not running. You can manually start it as the oracle user.

  [oracle ~]$ svrmgrl
  SVRMGR> connect internal
  SVRMGR> startup
  

• Now that you're into SQL*Plus, change the default passwords for system, sys, and ctxsys to "alexisahunk" (or to something you'll remember):

  SQL> alter user system identified by alexisahunk;
  SQL> alter user sys identified by alexisahunk;
  SQL> alter user ctxsys identified by alexisahunk;
  

• Verify that your date settings are correct.

  SQL> select sysdate from dual;
  

  If you don't see a date that fits the format YYYY-MM-DD, please read the section called “Troubleshooting Oracle Dates”.

• At this point we are going to hammer your database with an intense acceptance test. This usually takes around 30 minutes.

  SQL> @ /var/tmp/acceptance.sql
  
  ; A bunch of lines will scroll by.  You'll know if the test worked if
  ; you see this at the end:
  
  SYSDATE
  ----------
  2000-06-10
  
  SQL>
  

  Many people encounter an error regarding maximum key length:

  ERROR at line 1:
  ORA-01450: maximum key length (758) exceeded
  

  This error occurs if your database block size is wrong and is usually suffered by people trying to load OpenACS into a pre-existing database. Unfortunately, the only solution is to create a new database with a block size of at least 4096. For instructions on how to do this, see the section called “Creating the First Database” above. You can set the parameter using the dbassist program or by setting the DB_BLOCK_SIZE parameter in your database's creation script.

  If there were no errors, then consider yourself fortunate. Your Oracle installation is working.

Automating Startup & Shutdown

You will want to automate the database startup and shutdown process. It's probably best to have Oracle spring to life when you boot up your machine.

• Oracle includes a script called dbstart that can be used to automatically start the database. Unfortunately, the script shipped in the Linux distribution does not work out of the box. The fix is simple. Follow these directions to apply it. First, save dbstart to /var/tmp. Then, as oracle, do the following:

  [oracle ~]$ cp /var/tmp/dbstart.txt /ora8/m01/app/oracle/product/8.1.7/bin/dbstart 
  [oracle ~]$ chmod 755 /ora8/m01/app/oracle/product/8.1.7/bin/dbstart
  

• While you're logged in as oracle, you should configure the oratab file to load your database at start. Edit the file /etc/oratab:

  • You will see this line.

    ora8:/ora8/m01/app/oracle/product/8.1.7:N
    

    By the way, if you changed the service name or have multiple databases, the format of this file is:

    service_name:$ORACLE_HOME:Y || N (for autoload)

  • Change the last letter from "N" to "Y". This tells Oracle that you want the database to start when the machine boots. It should look like this.

    ora8:/ora8/m01/app/oracle/product/8.1.7:Y
    

  • Save the file & quit the terminal.

• You need a script to automate startup and shutdown. Save oracle8i.txt in /var/tmp. Then login as root and install the script. (Debian users: substitute /etc/init.d for /etc/rc.d/init.d throughout this section)

  [oracle ~]$ su -
  [root ~]# cp /var/tmp/oracle8i.txt /etc/rc.d/init.d/oracle8i
  [root ~]# chown root.root /etc/rc.d/init.d/oracle8i
  [root ~]# chmod 755 /etc/rc.d/init.d/oracle8i
  

• Test the script by typing the following commands and checking the output. (Debian Users: as root, do mkdir /var/lock/subsys first)

  [root ~]# /etc/rc.d/init.d/oracle8i stop
  Oracle 8i auto start/stop
  Shutting Oracle8i:
  Oracle Server Manager Release 3.1.7.0.0 - Production
  
  Copyright (c) 1997, 1999, Oracle Corporation.  All
  Rights Reserved.
  
  Oracle8i Enterprise Edition Release 8.1.7.0.1 -
  Production
  With the Partitioning option
  JServer Release 8.1.7.0.1 - Production
  
  SVRMGR> Connected.
  SVRMGR> Database closed.
  Database dismounted.
  ORACLE instance shut down.
  SVRMGR>
  Server Manager complete.
  Database "ora8" shut down.
        
  [root ~]# /etc/rc.d/init.d/oracle8i start
  Oracle 8i auto start/stop
  Starting Oracle8i: 
  SQL*Plus: Release 8.1.7.0.0 - Production on Wed Mar 6 17:56:02 2002
  
  (c) Copyright 2000 Oracle Corporation.  All rights reserved.
  
  SQL> Connected to an idle instance.
  SQL> ORACLE instance started.
  
  Total System Global Area   84713632 bytes
  Fixed Size                    73888 bytes
  Variable Size              76079104 bytes
  Database Buffers            8388608 bytes
  Redo Buffers                 172032 bytes
  Database mounted.
  Database opened.
  SQL> Disconnected
  
  Database "ora8" warm started.
  
  Database "ora8" warm started.
  

• If it worked, then run these commands to make the startup and shutdown automatic.

  • Red Hat users:

    [root ~]# cd /etc/rc.d/init.d/                      
    [root ~]# chkconfig --add oracle8i
    [root ~]# chkconfig --list oracle8i
    ; You should see:
    oracle8i        0:off   1:off   2:off   3:on    4:on    5:on    6:off
    

  • Debian users:

    [root ~]# update-rc.d oracle8i defaults
     Adding system startup for /etc/init.d/oracle8i ...
       /etc/rc0.d/K20oracle8i -> ../init.d/oracle8i
       /etc/rc1.d/K20oracle8i -> ../init.d/oracle8i
       /etc/rc6.d/K20oracle8i -> ../init.d/oracle8i
       /etc/rc2.d/S20oracle8i -> ../init.d/oracle8i
       /etc/rc3.d/S20oracle8i -> ../init.d/oracle8i
       /etc/rc4.d/S20oracle8i -> ../init.d/oracle8i
       /etc/rc5.d/S20oracle8i -> ../init.d/oracle8i
    

  • SuSE users:

    [root ~]# cd /etc/rc.d/init.d
    root:/etc/rc.d/init.d# ln -s /etc/rc.d/init.d/oracle8i K20oracle8i
    root:/etc/rc.d/init.d# ln -s /etc/rc.d/init.d/oracle8i S20oracle8i
    root:/etc/rc.d/init.d# cp K20oracle8i rc0.d
    root:/etc/rc.d/init.d# cp S20oracle8i rc0.d
    root:/etc/rc.d/init.d# cp K20oracle8i rc1.d
    root:/etc/rc.d/init.d# cp S20oracle8i rc1.d 
    root:/etc/rc.d/init.d# cp K20oracle8i rc6.d
    root:/etc/rc.d/init.d# cp S20oracle8i rc6.d
    root:/etc/rc.d/init.d# cp K20oracle8i rc2.d
    root:/etc/rc.d/init.d# cp S20oracle8i rc2.d
    root:/etc/rc.d/init.d# cp K20oracle8i rc3.d
    root:/etc/rc.d/init.d# cp S20oracle8i rc3.d 
    root:/etc/rc.d/init.d# cp K20oracle8i rc4.d  
    root:/etc/rc.d/init.d# cp S20oracle8i rc4.d  
    root:/etc/rc.d/init.d# cp K20oracle8i rc5.d
    root:/etc/rc.d/init.d# cp S20oracle8i rc5.d
    root:/etc/rc.d/init.d# rm K20oracle8i
    root:/etc/rc.d/init.d# rm S20oracle8i
    root:/etc/rc.d/init.d# cd
    [root ~]# SuSEconfig
    Started the SuSE-Configuration Tool.
    Running in full featured mode.
    Reading /etc/rc.config and updating the system...
    Executing /sbin/conf.d/SuSEconfig.gdm...   
    Executing /sbin/conf.d/SuSEconfig.gnprint...
    Executing /sbin/conf.d/SuSEconfig.groff...   
    Executing /sbin/conf.d/SuSEconfig.java...    
    Executing /sbin/conf.d/SuSEconfig.kdm...   
    Executing /sbin/conf.d/SuSEconfig.pcmcia...
    Executing /sbin/conf.d/SuSEconfig.perl...
    Executing /sbin/conf.d/SuSEconfig.postfix...
    Executing /sbin/conf.d/SuSEconfig.sendmail...
    Executing /sbin/conf.d/SuSEconfig.susehilf...
    Executing /sbin/conf.d/SuSEconfig.susehilf.add...
    Executing /sbin/conf.d/SuSEconfig.susewm...
    Executing /sbin/conf.d/SuSEconfig.tetex...
    Executing /sbin/conf.d/SuSEconfig.ypclient...
    Processing index files of all manpages...
    Finished.
    

• You also need some scripts to automate startup and shutdown of the Oracle8i listener. The listener is a name server that allows your Oracle programs to talk to local and remote databases using a standard naming convention. It is required for Intermedia Text and full site search.

  Download these three scripts into /var/tmp

  • startlsnr.txt

  • stoplsnr.txt

  • listener8i.txt

  Now issue the following commands (still as root).

  [root ~]# su - oracle
  [oracle ~]$ cp /var/tmp/startlsnr.txt /ora8/m01/app/oracle/product/8.1.7/bin/startlsnr
  [oracle ~]$ cp /var/tmp/stoplsnr.txt /ora8/m01/app/oracle/product/8.1.7/bin/stoplsnr    
  [oracle ~]$ chmod 755 /ora8/m01/app/oracle/product/8.1.7/bin/startlsnr
  [oracle ~]$ chmod 755 /ora8/m01/app/oracle/product/8.1.7/bin/stoplsnr
  [oracle ~]$ exit
  [root ~]# cp /var/tmp/listener8i.txt /etc/rc.d/init.d/listener8i
  [root ~]# cd /etc/rc.d/init.d
  root:/etc/rc.d/init.d# chmod 755 listener8i
  

  Test the listener automation by running the following commands and checking the output.

  root:/etc/rc.d/init.d# ./listener8i stop
  Oracle 8i listener start/stop
  Shutting down Listener for 8i: 
  LSNRCTL for Linux: Version 8.1.7.0.0 - Production on 06-MAR-2002 18:28:49
  
  (c) Copyright 1998, Oracle Corporation.  All rights reserved.
  
  Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost.localdomain)(PORT=1521)))
  The command completed successfully
  
      
  root:/etc/rc.d/init.d# ./listener8i start
  Oracle 8i listener start/stop
  Starting the Listener for 8i: 
  LSNRCTL for Linux: Version 8.1.7.0.0 - Production on 06-MAR-2002 18:28:52
  
  (c) Copyright 1998, Oracle Corporation.  All rights reserved.
  
  Starting /ora8/m01/app/oracle/product/8.1.7/bin/tnslsnr: please wait...
  
  TNSLSNR for Linux: Version 8.1.7.0.0 - Production
  System parameter file is /ora8/m01/app/oracle/product/8.1.7/network/admin/listener.ora
  Log messages written to /ora8/m01/app/oracle/product/8.1.7/network/log/listener.log
  Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost.localdomain)(PORT=1521)))
  Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC)))
  
  Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost.localdomain)(PORT=1521)))
  STATUS of the LISTENER
  ------------------------
  Alias                     LISTENER
  Version                   TNSLSNR for Linux: Version 8.1.7.0.0 - Production
  Start Date                06-MAR-2002 18:28:53
  Uptime                    0 days 0 hr. 0 min. 0 sec
  Trace Level               off
  Security                  OFF
  SNMP                      OFF
  Listener Parameter File   /ora8/m01/app/oracle/product/8.1.7/network/admin/listener.ora
  Listener Log File         /ora8/m01/app/oracle/product/8.1.7/network/log/listener.log
  Services Summary...
    PLSExtProc        has 1 service handler(s)
    ora8      has 1 service handler(s)
  The command completed successfully
  

  This test will verify that the listener is operating normally. Login into the database using the listener naming convention.

  sqlplus username/password/@SID

  [root ~]# su - oracle
  [oracle ~]$ sqlplus system/alexisahunk@ora8
  
  SQL> select sysdate from dual;
  
  SYSDATE
  ----------
  2002-02-22
  
  SQL> exit
  [oracle ~]$ exit
  [root ~]#
  

  • RedHat users:

    Now run chkconfig on the listener8i script.

    [root ~]# cd /etc/rc.d/init.d/
    root:/etc/rc.d/init.d# chkconfig --add listener8i
    root:/etc/rc.d/init.d# chkconfig --list listener8i
    listener8i      0:off   1:off   2:off   3:on    4:on    5:on    6:off
    

  • Debian users:

    Now run update-rc.d on the listener8i script.

    [root ~]# update-rc.d listener8i defaults 21 19
     Adding system startup for /etc/init.d/listener8i ...
       /etc/rc0.d/K19listener8i -> ../init.d/listener8i
       /etc/rc1.d/K19listener8i -> ../init.d/listener8i
       /etc/rc6.d/K19listener8i -> ../init.d/listener8i
       /etc/rc2.d/S21listener8i -> ../init.d/listener8i
       /etc/rc3.d/S21listener8i -> ../init.d/listener8i
       /etc/rc4.d/S21listener8i -> ../init.d/listener8i
       /etc/rc5.d/S21listener8i -> ../init.d/listener8i
    

• Test the automation

  As a final test, reboot your computer and make sure Oracle comes up. You can do this by typing

  [root ~]# /sbin/shutdown -r -t 0 now
  

  Log back in and ensure that Oracle started automatically.

  [joeuser ~]$ su - oracle
  [oracle ~]$ sqlplus system/alexisahunk@ora8
  
  SQL> exit
  

Congratulations, your installation of Oracle 8.1.7 is complete.

ref: https://openacs.org/doc/current/oracle.html