Weblog

Upgrade from PostgreSQL 9.6 to 13.1

Created by Gustaf Neumann, last modified by Gustaf Neumann 08 Oct 2021, at 09:57 PM

Here is a summary of steps to upgrade a legacy OpenACS installation (recent OpenACS, but old PostgreSQL version). In general it is possible to restore a "pg_dump" of some older version in newer versions of PostgreSQL. It is not necessary, to upgrade in small steps, version by version, but one can also skip major versions, when keeping certain things in mind. The upgrade on openacs.org was from PostgreSQL 9.6.4 to 13.1.

OIDs

Newer versions of PostgreSQL do not support OIDs anymore. OpenACS does not need OIDs in its queries, but one has to create a pg_dump without OIDs first. This can be achieved by the following commands.

 #
 # Settings of old PostgreSQL installation
 #
 PGBIN=/usr/local/pg964/bin
 PGPORT=5496

 #
 # Database from where OIDs should be removed
 #
 DB=openacs.org

 for tbl in `$PGBIN/psql -p $PGPORT -U postgres -qAt -c "select schemaname || '.' || tablename from pg_tables WHERE schemaname <> 'pg_catalog' AND schemaname <> 'information_schema';" $DB`
 do
    $PGBIN/psql -U postgres -c "alter table $tbl SET WITHOUT OIDS" $DB
 done

tsearch2

tsearch2 was an extension in PostgreSQL versions before version 10, but was dropped as an extension at this time, being now integral part of newer PostgreSQL versions. Users who have been continuously upgrading since a long time (from before 8.3) have to manually modify their databases by stripping leftovers from the extension. Otherwise, loading the dump will raise errors doe to missing tsearch2 functions in new versions (e.g. "ERROR: function "dex_init(internal)" does not exist").

By excluding the "pg_ts_*" table, the dump can be imported (although it will complain about the missing tsearch2 module). Nevertheless, the imported dump will be fully functional. 

Dump from PostgreSQL 9.6

Here is, what i've used on openacs.org for dumping. For your local installation, you have to replace the database names in the for loop by the names that you use.

  # 
  # Use variable DATE to disambiguate multiple dumps in the same directory 
  #
  DATE=`date +"%Y-%m-%d"`

  for DB in 5-9-1 dotlrn-test fisheye oacs-5-8 openacs.org openacs.org-test translate wi1.org wi1.org-test
  do
    echo "dumping to ~/$DB.dump.$DATE.gz"
    $PGBIN/pg_dump -p $PGPORT -U postgres --exclude-table=public.pg_ts_\* $DB | gzip > ~/$DB.dump.$DATE.gz
  done

Restore in PostgreSQL 13

After this, restoring went smooth:

  #
  # Adjust settings to your pg13 installation
  #
  export PG13=pg13
  export PGBIN13=/usr/local/${PG13}/bin
  export PGPORT13=5413

  for DB in 5-9-1 dotlrn-test oacs-5-8 openacs.org openacs.org-test translate wi1.org  wi1.org-test
  do
    echo "reloading from ~/$DB.dump.$DATE.gz"
    $PGBIN13/dropdb -p $PGPORT13 -U nsadmin $DB
    $PGBIN13/createdb -p $PGPORT13 -U openacs -T template0 $DB
    $PGBIN13/psql -p $PGPORT13 -U openacs $DB -tAc "create extension ltree"
    gunzip < ~/$DB.dump.$DATE.gz | $PGBIN13/psql -p $PGPORT13 -U nsadmin $DB > ~/$DB.log
  done

You might not need the line with the ltree extension in your installation.

Upgrade from PostgreSQL 13 to PostgreSQL 14

Just dump and restore.

Upgrade to OpenACS 5.10

Created by Gustaf Neumann, last modified by Gustaf Neumann 07 Oct 2021, at 01:57 PM

Upgrades from earlier version than OpenACS 5.8 should read upgrade-oacs-5-8 and upgrade-oacs-5-9.

OpenACS 5.10 requires PostgreSQL 9.6 or newer, Tcl 8.6.2 or newer and XOTcl 2.1 or newer (part of the nsf package). XOTcl 2.0 can be installed e.g. via install-ns, or from Debian sid https://packages.debian.org/sid/, or from sources https://next-scripting.org/. In order to check the versions of these packages already installed in your OpenACS installation, check the output of /xotcl/version-numbers in your installation.

Before upgrading to OpenACS 5.10, upgrade all packages in your current installation to recent versions in your OpenACS 5.9 installation, which OpenACA 5.9.1(i.e. in the oacs-5-9 channel).

In general it is always recommended to backup your current installation before overwriting it. Make e.g. a tar archive of all files of the OpenACS tree and a database dump.

After upgrading the packages in the oacs-5-9 channel,  the following steps are recommended:

• Restart the server with the newest OpenACS 5.9
• Install newest version of the source files (e.g. from the OpenACS 5.10 tar distributions, or install/upgrade from the oacs-5-10 branch from cvs),
• Upgrade the acs-core packages via package manager, restart server
• Upgrade/install application packages via package manager as needed (from file-system or from repository)

Note: when one installs now from the openacs-5.9.1 tar ball, there will be a problem with a PostgreSQL 11, since the tar ball is older than this PostgreSQL release. However, the version of OpenACS 5.9.1 in the repositories (CVS or github) can be installed as well with newer PostgreSQL versions.

Step by step upgrade to OpenACS 5.10.0

The following steps assume, you have a plain, not modified version of OpenACS running.

1. Upgrade to the latest version of OpenACS 5.9.1, e.g. via
      cvs -r oacs-5-9 openacs-4
   upgrade the packages, restart the server and make sure, you have really upgraded all packages;
   under /acs-admin/apm/ you should see a Kernel like 5.9.1* running.
    
2. Either fet release tar file of openacs-5.10.0, unpack it (e.g. under /usr/local), make sure, your configuration file points to the right source tree, and restart server, or upgrade via cvs
      cvs -r oacs-5-10 -Pd openacs-4
    
3. Upgrade packages:
   Browse to /acs-admin/install click on "upgrade from local", select all packages, restart server - done.

In case, you have a locally modified version of OpenACS, but you have not modified the kernel packages, then make sure to get the tar file of the core packages in step (2), proceed as described, upgrade the acs-core packages first and then the application packages of your installation.

In case you have in your installation as well changes in the acs-core packages (which is not recommended, but it happens), then get the diffs (you local changes) between the acs-core packages of oacs-5-10 and your installation, proceed as with the unmodified acs-core packages (paragraph above) and apply/merge your local changes after upgrading.

These steps are working also with recent versions of PostgreSQL such as 13.3.

xowf (XoWiki Workflow)

Created by Gustaf Neumann, last modified by Gustaf Neumann 02 Oct 2021, at 10:41 AM

XoWiki Content Flow - an XoWiki based workflow system implementing state-based behaviour of wiki pages and forms (short intro).

Package Specification Summary for Package: xowf

Bug Tracker Summary for Package: xowf

There is no package with the name "xowf" known to bug-tracker.

Code Metrics Summary for Package: xowf

xowf depends on xowiki.

Workflow Examples:

• A two-chambers legislative process, loosely based on the Italian one. A law proposal is submitted to the first chamber, then passed to the second one. The text keeps being passed between the two chambers until no amendments (changes) are made. Once this condition is met, the law is approved. Workflow example of a legislative process

OpenACS Release Status

Created by Dave Bauer, last modified by Gustaf Neumann 27 Sep 2021, at 02:15 PM

Current Stable Release

5.10.0 Released 2021-09-15

Download OpenACS 5.10.0 Core

Download OpenACS 5.10.0 Full

Previous Releases

5.9.1 Released 2017-08-08

Download OpenACS 5.9.1 Core

Download OpenACS 5.9.1 Full

5.9.0 Released 2015-12-01

Download OpenACS 5.9.0

5.8.1 Released 2014-10-25

Download OpenACS 5.8.1

5.8.0 Released 2013-08-30 

Download OpenACS 5.8.0

5.7.0 Released 2013-07-17 

Download OpenACS 5.7.0

5.6.0 Released 2010-09-22 

Download OpenACS 5.6.0

5.5.0 Released 2009-06-22 

Download OpenACS 5.5.0

5.4.3 Released  2008-11-26 (No more releases on the 5.4 branch)

Download OpenACS 5.4.3

5.4.2 Released  2008-06-08

Download OpenACS 5.4.2

5.4.1 Released  2008-04-07

Download OpenACS 5.4.1

5.4.0 Released  2008-02-07

Download OpenACS 5.4.0

5.3.2 Released  2007-07-18 (No more releases on the 5.3 branch)

Download OpenACS 5.3.2

Next Bugfix Release 

undecided

Next Major Release

undecided

TODO list for next release

Streaming HTML

Created by Gustaf Neumann, last modified by Gustaf Neumann 11 Sep 2021, at 09:58 AM

Streaming HTML can be used to output HTML content to a user incrementally. This is in particular useful for pages for pages with longer response time, to inform during processing about the progress of the tasks.

Newer OpenACS versions come with templates for the standard themes for streaming pages. The template for streaming pages can be retrieved for the current theme via the API call "template::streaming_template". An application developer should structure streaming HTML pages as follows:

Output top of page:

set context ...
set title ...
ad_return_top_of_page [ad_parse_template \
    -params [list context title] \
    [template::streaming_template]]

Output HTML incrementally:

# Assume, we are collecting HTML in the Tcl variable HTML.
# Send this HTML chunk incrementally to the user
ns_write [lang::util::localize $HTML]

Flush Output from body scripts:
(e.g. template::add_body_script, template::add_event_listener, template::add_body_handler, template::add_script)

ns_write [template::collect_body_scripts]

End of Page:

# Optionally
ns_write [lang::util::localize [template::get_footer_html]]
ns_write [template::collect_body_scripts]

Full sample script:
Putting everything together

set title "Sample HTML streaming page"
set context [list $title]
ad_return_top_of_page [ad_parse_template \
                           -params [list context title] \
                           [template::streaming_template]]
ns_write "<ul>\n"
foreach i {1 2 3 4} {
    set HTML "<li>finish $i: ...</li>\n"
    ns_write [lang::util::localize $HTML]
    ns_write [template::collect_body_scripts]
    ns_sleep 1s
}
ns_write "</ul>\n"
ns_write "<p>Done.</p>"

ns_write [lang::util::localize [template::get_footer_html]]
ns_write [template::collect_body_scripts]

See this sample script in action: https://openacs.org/streaming

Caveat: Windows PCs having the current (Sept 2020) version of Bitdefender (antivirus software ) installed with HTTP traffic scanning activated don't show the incremental rendering of the page, but just the full page after it has finished. This might be seen as a problem of Bitdefender and affects streaming HTML from all sites.

Available OpenACS Packages

Created by Gustaf Neumann, last modified by Gustaf Neumann 03 Sep 2021, at 12:00 PM

Wiki pages for the packages available in the OpenACS code repository:

List of the actively maintained packages

In the oacs-5-10 channel, currently 93 packages are actively maintained. Additionally, the packages is use on openacs.org are maintained as well, but not necessarily recommended for new installations. The packages in the main branch are in different states, some of these work perfecty, some of these migh use outdated calls an can be seen as starting points for new projects.

Packages in the oacs-5-9 channel.
Packages in the oacs-5-10 channel.

Packages available in the oacs-5-9 channel

Created by Gustaf Neumann, last modified by Gustaf Neumann 03 Sep 2021, at 11:59 AM

acs-admin
acs-api-browser
acs-authentication
acs-automated-testing
acs-bootstrap-installer
acs-content-repository
acs-core-docs
acs-datetime
acs-developer-support
acs-events
acs-kernel
acs-lang
acs-mail-lite
acs-messaging
acs-outdated
acs-reference
acs-service-contract
acs-subsite
acs-tcl
acs-templating
acs-translations
ajaxhelper
assessment
assessment-portlet
attachments
bm-portlet
bulk-mail
calendar
calendar-portlet
categories
chat
chat-portlet
dotlrn
dotlrn-assessment
dotlrn-bm
dotlrn-calendar
dotlrn-chat
dotlrn-dotlrn
dotlrn-evaluation
dotlrn-faq
dotlrn-forums
dotlrn-fs
dotlrn-homework
dotlrn-news
dotlrn-portlet
dotlrn-static
dotlrn-xowiki
evaluation
evaluation-portlet
faq
faq-portlet
file-storage
forums
forums-portlet
fs-portlet
general-comments
intermedia-driver
new-portal
news
news-portlet
notifications
oacs-dav
openacs-bootstrap3-theme
openacs-default-theme
profile-provider
ref-countries
ref-language
ref-timezones
richtext-ckeditor4
richtext-tinymce
richtext-xinha
rss-support
search
static-portlet
survey
theme-zen
tsearch2-driver
user-profile
views
xotcl-core
xotcl-request-monitor
xowf
xowiki
xowiki-portlet

Packages available in the oacs-5-10 channel

Created by Gustaf Neumann, last modified by Gustaf Neumann 03 Sep 2021, at 11:48 AM

acs-admin
acs-api-browser
acs-authentication
acs-automated-testing
acs-bootstrap-installer
acs-content-repository
acs-core-docs
acs-datetime
acs-developer-support
acs-events
acs-kernel
acs-lang
acs-mail-lite
acs-messaging
acs-outdated
acs-reference
acs-service-contract
acs-subsite
acs-tcl
acs-templating
acs-translations
ajaxhelper
assessment
assessment-portlet
attachments
bm-portlet
boomerang
bulk-mail
calendar
calendar-portlet
categories
chat
chat-portlet
contacts-lite
cookie-consent
dotlrn
dotlrn-assessment
dotlrn-bm
dotlrn-bootstrap3-theme
dotlrn-calendar
dotlrn-chat
dotlrn-dotlrn
dotlrn-evaluation
dotlrn-faq
dotlrn-forums
dotlrn-fs
dotlrn-homework
dotlrn-news
dotlrn-portlet
dotlrn-static
dotlrn-xowiki
evaluation
evaluation-portlet
faq
faq-portlet
file-storage
forums
forums-portlet
fs-portlet
general-comments
intermedia-driver
new-portal
news
news-portlet
notifications
oacs-dav
openacs-bootstrap3-theme
openacs-default-theme
proctoring-support
profile-provider
ref-countries
ref-language
ref-timezones
ref-us-states
richtext-ckeditor4
richtext-tinymce
richtext-xinha
rss-support
search
static-portlet
survey
theme-zen
tsearch2-driver
user-profile
views
xml-rpc
xolti
xooauth
xotcl-core
xotcl-request-monitor
xowf
xowf-monaco-plugin
xowiki
xowiki-portlet

When to use URLencode

Created by Gustaf Neumann, last modified by Gustaf Neumann 09 Aug 2021, at 12:31 PM

In general, HTTP requires that URLs are properly encoded. This is not a big issue, when just plain ASCII characters without special characters are used in URL paths and query variables. However, when the framework allows the end-user to define also URLs (such as in xowiki and derivatives), one has to be careful when extending the framework. The section below refers to the behavior in OpenACS 5.10, earlier version might differ in some details.

URLs in HTML

When URLs are embedded in HTML (href, src, ...) the URL must be both first urlencoded and then HTML quoted to be on the safe side from the
point of view of HTML.  Although most characters problematic for HTML are encoded by ns_urlencode, but the single quote is not (this is not a bug but according to the specs, RFC 3986).

ns_urlencode {<a> 'b' "c" &}
# returns:  %3ca%3e+'b'+%22c%22+%26

Only, when it can be guaranteed the the URL contains no "funny characters" the URLencoding can be omitted. Note that double encoding with ns_urlencode leads to over-quoting in the same way as double encoding with ns_quotehtml.

Return_urls

In general, return_urls have to be proper URL encoded according to the HTTP specs. Setting these URLs is more complex, since one has to be aware whether or not an URL as encoded before or not before passing it as a return URL.

Here is a short guideline:

1) Query functions return URLs always URLdecoded

• ns_conn url
• ad_conn url
• xo::cc url

2) Output functions return URLs per default URLencoded

• export_vars (input parameter "-base" has to be unencoded)
• ad_return_url
• :pretty_link

3) Redirect operations have to receive encoded input

• ad_returnredirect
• ns_returnredirect
• :returnredirect

4) Query-variables

When setting query variables with URLs, these should be already URL encoded

   #
   # Setting a query variable
   # 
   set return_url [ad_returnurl]
   set url [export_vars -base . {return_url}]

   #
   # Using query-variable as default value in xo* package
   #
   ad_returnredirect [:query_parameter return_url:localurl [ad_return_url]]

   #
   # Usage in classical OpenACS
   #
   ...
   # get return_url e.g. via page_contract
   ...
   if {[info exists return_url] && $return_url ne ""} {
        ad_returnredirect $return_url
    }

OpenACS/dotLRN windows installer how to

Created by Byron Linares, last modified by Gustaf Neumann 26 May 2021, at 12:32 PM

This page is obsolete and kept as a reference.

The how-to for a well maintained Windows installation is here.

OpenACS/dotLRN windows installer how to
Required software:

• Inno Setup 5 or higher

Download

Installer Sources:
The installer source consists in a main Inno Setup script “installer.iss” and could be found in cvs or obtained with the installer.

Other files:
There are some necessary file numbers in order to build the installer.

•  tcl\bintallerw.tcl: the AOLserver configuration script
•  Install.xml: this XML file control the packages installed by the OpenAcs installer
•  tcl\Windows-procs.tcl: This file includes various hacks to make OpenACS work     on Windows.Replaces the TCL exec command on Windows, allowing exec to be called with unix-y arguments.Index.tcl : file used for automatic installation.
• Index-org.tcl: file used for manual installation.
• Installer.tcl: file used in automatic installation.
• Installer-org.tcl: file used in manual installation.
• Stara.bat, stop.bat: batch files for starting and stopping all services.
• Tools/setx.exe, tools/reg.exe ,/tools/uzip.exe, /tools/reg.exe Windows command line tools necessary for the installation process.
•  AOLserver_4-0-beta-10_2003-08-04.zip: AOLServer
•  cygwin.zip:  CygWin Unix environment for Windows. Cygwin.zip also contains the PostgreSQL database
•  cygwin.bat: file to launch cygwin unix environment console.
•  license_en.txt: license text file.
•  important.txt: extra info text file.
•  Note.txt : especial notes text file.

Download Resources

Building the installer:

1. In order to build this installer we need to place all the resources in a folder.
2. Create tools and tcl folders with the files before mentioned.
3. Download OpenACS, dotLRN and all desired package and place in a folder and named “oacs-dotlrn”
4. Open the file installer.iss and generate the installer.

The installer.iss has to important parts:
    [Files] section:
        In this section are listed all the sources for the installer, in this part you
        Can change or add any source that you want.
    [Run] section:
        In this section is where all commands are executed, like the cygwin installation, postgreSQL installation and AOLserver.
    [Code] section:
        Section for define procedures or functions to work around the installation.

These are the main sections in the “installer.iss” script. And the sections that have to be changed form personalize the installer.

developed at the Galileo University (www.galileo.edu) by Byron Haroldo Linares Roman bhlr@galileo.edu as part of the E-LANE project (www.e-lane.org)
 

Instructions on using the installer are at:

• openacs-system-install-windows-server