View · Index

Weblog Page

Showing 51 - 60 of 693 Postings (summary)

Contacts Lite

Created by Gustaf Neumann, last modified by Gustaf Neumann 23 Jan 2021, at 01:07 AM

Package Specification Summary for Package: contacts-lite

Summary:
Description: This package is unknown (not available at this site)
Maturity: 0
This package depends on: None
Packages that depend on contacts-lite: None
Package parameters: None

Bug Tracker Summary for Package: contacts-lite

Open Bugs: 0
All Tracked Issues: 2
Latest Bug Opened: This package has no open bugs.
Latest Bug Fixed: 2021-01-23 Expressions like {value {<i>$contact_info</i>}}} do not work.
Top Bug Submitters: Maurizio Martignano (2)
Top Bug Fixers: Gustaf Neumann (2)

Code Metrics Summary for Package: contacts-lite

# Tcl Procs 0
# Tcl Lines 0
# Tcl Blank Lines 1
# Tcl Comment Lines 0
# Automated Tests 0
# Stored Procedures PG: 0 ORA: 0
# SQL Lines PG: 0 (blank 1 comments 0) ORA: 0 (blank 1 comments 0)
# ADP pages 0
# ADP lines 0
# Include pages (contacts-lite/lib/) 0
# Documentation pages 0
# Documentation lines 0
Browse Source Not installed
Github Repository: https://github.com/openacs/contacts-lite/tree/oacs-5-10

 

 

SQL - XQL

Created by Malte Sussdorff, last modified by Poor Yorick 27 Oct 2020, at 01:35 PM

By convention, OpenACS stores all its SQL code in .xql files, which facilitates support for OpenACS on multiple databases.  Though this adds an extra burden to the development, please adhere to this convention.

  • Keep SQL scripts out of .tcl or .adp files, unless they conform to SQL99.  See details below.
  • Assign a name to each db_statement.  see Naming Conventions.
  • Database-specific SQL statements, i.e. statements not conforming to SQL99, belongs in a -database.xql file:
    • select content_item__new() => content-item-postgresql.xql
    • select item_id from content_items => content-item.xql
  • Do not spread pieces of an SQL statement into multiple places, e.g. inline in Tcl, generic .xql, database-specific .xql.
  • If in doubt, be conservative and add SQL scripts into the database-specific file of the database you are working with.
  • Format each SQL script properly.  E.g. break each statement into multiple lines as appropriate to make it more readable.  See code-formatting

To make changes to a .xql file take effect, reload the file.

As an exception to the rule about keeping SQL out of Tcl scripts, pieces of an SQL script that is dynamically generated may be placed in a Tcl script if they conform to the SQL 99 standard.  This is often the case for "where_clause" of listbuilder.

 

Creation of tables in PostgreSQL:

  • Do not use the "SERIAL" datatype.  SERIAL is an ugly hack meant to make it easier to port Sybase scripts to PostgreSQL.  We have a standard way of defining integer primary keys and giving values to them: Integer with a sequence.
  • When creating a sequence use the CREATE VIEW hack so sequence queries can be shared between PG and Oracle without any extra work.  Otherwise, do the work to make the sequence work both in -oracle.xql as well as -postgresql.xql

How to manage/upgrade CKEditor versions

Created by Gustaf Neumann, last modified by Gustaf Neumann 27 Aug 2020, at 07:46 PM

As of this writing (oacs-5-10), OpenACS supports CKEditor 4, which is kept as a separate package.

Per default, the CKEditor is used via CDN. If you prefer to install CKEditor locally, then go to /acs-admin/, click on the "Richtext CKeditor4" package on the link under site-wide admin. The pages show, how CKEditor is currently used, and offer a link for downloading and installing (gzipped, brotli if available).

Since upgrading to a newer version of CKEditor means often a different user experience.So, many site are reluctant to upgrade automatically to different versions. In these cases, one can configure the version of CKEditor in the NaviServer configuration file.

    ns_section ns/server/${server}/acs/richtext-ckeditor
           ns_param CKEditorVersion   4.14.1
           ns_param CKEditorPackage   standard
           ns_param CKFinderURL       /acs-content-repository/ckfinder
           ns_param StandardPlugins   uploadimage

A version specified in the configuration file override version numbers in the richtext package. When there is no such entry, then the version from the package is used.

 

SQL: How to log (slow) queries in the system log

Created by Gustaf Neumann, last modified by Gustaf Neumann 27 Aug 2020, at 11:47 AM

SQL logging is usually controlled via the configuration file of NaviServer. However, it can be as well activated at runtime via ds/shell by using the following commands:

  • make sure to turn sql-debugging level on
  • make sure, the logminduration is small enough (NaviServer allows to log only entries a threshold)
ns_logctl severity "Debug(sql)" on
foreach pool [ns_db pools] {ns_db logminduration $pool 0}

If you are interested e.g. in queries taking longer than 0.5 seconds, you can use

foreach pool [ns_db pools] {ns_db logminduration $pool 0.5}

Creating adp box tags for consistent html/css

Created by openacs community, last modified by Gustaf Neumann 26 Aug 2020, at 02:01 PM

Box tag code:

# The box tag is intended to make the markup around a "box"
# standard sitewide so that you can use the same css everywhere to   
# style the site.

  

template_tag box { chunk params } {
    set class [ns_set iget $params class]
    if {[template::util::is_nil class]} {
        set class box
    }
    set id [ns_set iget $params id]
    if {![template::util::is_nil id]} {
        set id " id=\\\"$id\\\""
    }

    template::adp_append_code "append __adp_output \"<div class=\\\"$class\\\"$id><div class=\\\"boxTop\\\"></div><div class=\\\"boxContent\\\">\""
    template::adp_compile_chunk $chunk
    template::adp_append_code "append __adp_output {</div><div class=\"boxBottom\"></div></div>}"
} 

E-Mail: Incoming E-Mail

Created by Malte Sussdorff, last modified by Gustaf Neumann 24 Aug 2020, at 01:43 PM

Incoming E-Mail in OpenACS works with the latest version of acs-mail-lite in a general fashion using callbacks.

The original version of this documentation is found via archive.org at: http://www.cognovis.de/developer/en/incoming_email

We will take a look on what needs to be done to get incoming e-mail working and then continue on to see how packages can benefit.

Project notes:  ACS Mail Lite sends via SMTP which permits the use of an external server to handle email. For scalability, consider expanding the incoming E-mail paradigm to likewise use Tcllib's imap4 or NaviServer's nsimap so that most all email can be handled on separate servers. 

Install incoming E-Mail

First, one must have an understanding of postfix basics. See http://www.postfix.org/BASIC_CONFIGURATION_README.html.

These instructions use the following example values:

  • hostname: www.yourserver.com
  • oacs user: service0
  • OS: Linux
  • email user: service0
  • email's home dir: /home/service0
  • email user's mail dir: /home/service0/mail

Important: The email user service0 does not have a ".forward" file. This user is only used for running the OpenACS website. Follow careful use of email rules by following strict guidelines to avoid email looping back unchecked.

For postfix, the email user and oacs user do not have to be the same. Furthermore, postfix makes distinctions between virtual users and user aliases.  Future versions of this documentation should use examples with different names to help distinguish between standard configuration examples and the requirements of ACS Mail Lite package.

Postfix configuration parameters:

myhostname=www.yourserver.com

myorigin=$myhostname

inet_interfaces=$myhostname, localhost

mynetworks_style=host

virtual_alias_domains = www.yourserver.com

virtual_maps=regexp:/etc/postfix/virtual

home_mailbox=mail/

Here is the sequence to follow if installing email service on system for first time. If your system already has email service, adapt these steps accordingly:

  1. Install postfix
  2. Install smtp (for postfix)
  3. Install metamail (for acs-mail-lite)
  4. Edit /etc/postfix/main.cf
  5. Edit /etc/postfix/virtual  Add a regular expression to filter relevant incoming emails for processing by OpenACS. 
    @www.yourserver.com service0
  6. Edit /etc/postfix/master.cf - uncomment this line so postfix listens to emails from internet
    smtp inet n - n - - smtpd
  7. Create a mail directory as service0
    mkdir /home/service0/mail
  8. Configure ACS Mail Lite parameters
    BounceDomain: www.yourserver.com
    BounceMailDir: /home/service0/mail
    EnvelopePrefix: bounce

    The EnvelopePrefix is for bounce e-mails only.

    NOTE: Parameters should be renamed: 
    BounceDomain to IncomingDomain
    BounceMailDir to IncomingMaildir
    EnvelopePrefix to BouncePrefix
    ..to reflect that acs-mail-lite is capable of dealing with other types of incoming e-mail.

    Furthermore, setting IncomingMaildir parameter clarifies that incoming email handling is setup. This is useful for other packages to determine if they can rely on incoming e-mail working (e.g. to set the reply-to email to an  e-mail address which actually works through a callback if the IncomingMaildir parameter is enabled).
  9. Configure Notifications parameters
    EmailReplyAddressPrefix: notification
    EmailQmailQueueScanP: 0

    We want acs-mail-lite incoming handle the Email Scanning, not each package separately.
    Configure other packages likewise
     
  10. Invoke postmap in OS shell to recompile virtual db:
    postmap /etc/postfix/virtual
  11. Restart Postfix. 
    /etc/init.d/postfix restart
  12. Restart OpenACS

 

Processing incoming e-mail

 

A sweeper procedure like acs_mail_lite::load_mails should:

  1. scan the e-mails which are in the IncomingMaildir directory on a regular basis.
  2. check if any email came from an auto mailer.
  3. Parse new ones, and
  4. process them by firing off callbacks.

Vinod has made a check for auto mailers by using procmail as follows. Maybe we could get this dragged into Tcl code (using regexp or a Procmail recipe parser) instead, thereby removing the need for setting up procmail in the first place.

Revised procmail filters:

:0 w * ^subject:.*Out of Office AutoReply /dev/null 
:0 w * ^subject:.*Out of Office /dev/null :0 w * ^subject:.*out of the office /dev/null 
:0 w * ^subject:.*NDN /dev/null :0 w * ^subject:.*[QuickML] Error: /dev/null 
:0 w * ^subject:.*autoreply /dev/null :0 w * ^from.*mailer.*daemon /dev/null

To make things granular a separate parsing procedure should deal with loading the e-mail into the Tcl interpreter and setting variables in an array for further processing.

ad_proc parse_email { 
    -file:required
    -array:required
} { 
   ...
}

An email is split into several parts: headers, bodies and files.

The headers consists of a list with header names as keys and their corresponding values. All keys are lower case.

The bodies consists of a list with two elements: content-type and content.

The files consists of a list with three elements: content-type, filename and content.

An array with all the above data is upvarred to the caller environment.

Processing an email should result in an array like this:

HEADERS

  • message_id
  • subject
  • from
  • to
  • date
  • received
  • references
  • in-reply-to
  • return-path
  • .....

X-Headers:

  • X-Mozilla-Status
  • X-Virus Scanned
  • .....

We do not know which headers are going to be available in the e-mail. We set all headers found in the array. The callback implementation then checks if a certain header is present or not.

        #get all available headers
        set keys [mime::getheader $mime -names]
 
        set headers [list]

        # create both the headers array and all headers directly for the email array
        foreach header $keys {
            set value [mime::getheader $mime $header]
            set email([string tolower $header]) $value
            lappend headers [list $header $value]
        }
        set email(headers) $headers

Bodies 

An e-mail usually consists of one or more bodies. With the advent of complex_send, OpenACS supports sending of multi-part e-mails which are needed if you want to send out and e-mail in text/html and text/plain (for old mail readers).

switch [mime::getproperty $part content] {
     "text/plain" {
          lappend bodies [list "text/plain" [mime::getbody $part]]
     }
     "text/html" {
          lappend bodies [list "text/html" [mime::getbody $part]]
     }
}

Files

OpenACS supports tcllib mime functions. Getting incoming files to work is a matter of looking for a part where there exists a "Content-disposition" part. All these parts are file parts. Together with scanning for email bodies, code looks something like this:

        set bodies [list]
        set files [list]
 
        #now extract all parts (bodies/files) and fill the email array
        foreach part $all_parts {

            # Attachments have a "Content-disposition" part
            # Therefore we filter out if it is an attachment here
            if {[catch {mime::getheader $part Content-disposition}]} {
                switch [mime::getproperty $part content] {
                    "text/plain" {
                        lappend bodies [list "text/plain" [mime::getbody $part]]
                    }
                    "text/html" {
                        lappend bodies [list "text/html" [mime::getbody $part]]
                    }
                }
            } else {
                set encoding [mime::getproperty $part encoding]
                set body [mime::getbody $part -decode]
                set content  $body
                set params [mime::getproperty $part params]
                if {[lindex $params 0] == "name"} {
                    set filename [lindex $params 1]
                } else {
                    set filename ""
                }

                # Determine the content_type
                set content_type [mime::getproperty $part content]
                if {$content_type eq "application/octet-stream"} {
                    set content_type [ns_guesstype $filename]
                }

                lappend files [list $content_type $encoding $filename $content]
            }
        }
        set email(bodies) $bodies
        set email(files) $files

Note that the files ie attachments are actually stored in the /tmp directory from where they can be processed further. It is up to the callback to decide if to import the file into OpenACS or not. Once all callbacks have been fired files in /tmp will have to be deleted again though.

Firing off callbacks 

Now that we have the e-mail parsed and have an array with all the information, we can fire off the callbacks. The firing should happen in two stages.

The first stage is where we support a syntax like "object_id@yoursite.com".

Second, incoming e-mail could look up the object_type, and then call the callback implementation specific to this object_type. If object_type = 'content_item', use content_type instead. 

ad_proc -public -callback acs_mail_lite::incoming_object_email { -array:required -object_id:required } { }

callback acs_mail_lite::incoming_object_email -impl $object_type -array email -object_id $object_id

 

ad_proc -public -callback acs_mail_lite::incoming_object_email -impl user {

    -array:required

    -object_id:required

} {

    Implementation of mail through support for incoming emails

} {

    # get a reference to the email array

    upvar $array email

 # make the bodies an array

        template::util::list_of_lists_to_array $email(bodies) email_body

        if {[exists_and_not_null email_body(text/html)]} {

            set body $email_body(text/html)

        } else {

            set body $email_body(text/plain)

        }

        set reply_to_addr "[party::get_by_email $email(from)]@[ad_url]"

        acs_mail_lite::complex_send \

            -from_addr $from_addr \

            -reply_to $reply_to_addr \

            -to_addr $to_addr \

            -subject $email(subject) \

            -body $body \

            -single_email \

            -send_immediately

}

Object id based implementations are useful for automatically generating "reply-to" addresses. With ProjectManager and Contacts object_id is also handy, because Project / TaskID is prominently placed on the website. If you are working on a task and you get an e-mail by your client that is related to the task, just forward the email to "$task_id@server.com" and it will be stored along with the task. Highly useful :).

Obviously you could have implementations for:

  • forums_forum_id: Start a new topic

  • forums_message_id: Reply to an existing topic

  • group_id: Send an e-mail to all group members

  • pm_project_id: add a comment to a project

  • pm_task_id: add a comment to a task and store the files in the projects folder (done)

 

Once the e-mail is dealt with in an object oriented approach we are either done with the message (an object_id was found in the to address) or we need to process it further.

ad_proc -public -callback acs_mail_lite::incoming_email {
    -array:required
    -package_id
} {
}
array set email {}
            
parse_email -file $msg -array email
set email(to) [parse_email_address -email $email(to)]
set email(from) [parse_email_address -email $email(from)]

# We execute all callbacks now
callback acs_mail_lite::incoming_email -array email

For this a general callback should exist which can deal with every leftover e-mail and each implementation will check if it wants to deal with this e-mail. How is this check going to happen? As an example, a package could have a prefix, as is the case with bounce e-mails as handled in acs_mail_lite::parse_bounce_address (see below):

ad_proc -public -callback acs_mail_lite::incoming_email -impl acs-mail-lite {
    -array:required
    -package_id:required
} {
    @param array        An array with all headers, files and bodies. To access the array you need to use upvar.
    @param package_id   The package instance that registered the prefix
    @return             nothing
    @error
} {
    upvar $array email

    set to [acs_mail_lite::parse_email_address -email $email(to)]
    ns_log Debug "acs_mail_lite::incoming_email -impl acs-mail-lite called. Recepient $to"

    util_unlist [acs_mail_lite::parse_bounce_address -bounce_address $to] user_id package_id signature
    
    # If no user_id found or signature invalid, ignore message
    # Here we decide not to deal with the message anymore



    if {[empty_string_p $user_id]} {
        if {[empty_string_p $user_id]} {
            ns_log Debug "acs_mail_lite::incoming_email impl acs-mail-lite: No equivalent user found for $to"
        } else {
            ns_log Debug "acs_mail_lite::incoming_email impl acs-mail-lite: Invalid mail signature $signature"
        }
    } else {
        ns_log Debug "acs_mail_lite::incoming_email impl acs-mail-lite: Bounce checking $to, $user_id"
        
        if { ![acs_mail_lite::bouncing_user_p -user_id $user_id] } {
            ns_log Debug "acs_mail_lite::incoming_email impl acs-mail-lite: Bouncing email from user $user_id"
            # record the bounce in the database
            db_dml record_bounce {}
            
            if {![db_resultrows]} {
                db_dml insert_bounce {}
            }
        }
    }
}
 

Alternatively we could just check the whole to address for other things, e.g. if the to address belongs to a group (party)

ad_proc -public -callback acs_mail_lite::incoming_email -impl contacts_group_mail {
    -array:required
    {-package_id ""}
} {
    Implementation of group support for incoming emails
    
    If the to address matches an address stored with a group then send out the email to all group members

     @author Malte Sussdorff (malte.sussdorff@cognovis.de)
     @creation-date 2005-12-18

     @param array        An array with all headers, files and bodies. To access the array you need to use upvar.
     @return             nothing
     @error
} {

    # get a reference to the email array
    upvar $array email

    # Now run the simplest mailing list of all
    set to_party_id [party::get_by_email -email $email(to)]
    
    if {[db_string group_p "select 1 from groups where group_id = :to_party_id" -default 0]} {
        # make the bodies an array
        template::util::list_of_lists_to_array $email(bodies) email_body
        
        if {[exists_and_not_null email_body(text/html)]} {
            set body $email_body(text/html)
        } else {
            set body $email_body(text/plain)
        }
        
        acs_mail_lite::complex_send \
            -from_addr [lindex $email(from) 0] \
            -to_party_ids [group::get_members -group_id $to_party_id] \
            -subject $email(subject) \
            -body $body \
            -single_email \
            -send_immediately

    }
} 

Or check if the to address follows a certain format.

ad_proc -public -callback acs_mail_lite::incoming_email -impl contacts_mail_through {
    -array:required
    {-package_id ""}
} {
    Implementation of mail through support for incoming emails
 
    You can send an e-amil through the system by sending it to user#target.com@yoursite.com
    The email will be send from your system and if mail tracking is installed the e-mail will be tracked.

    This allows you to go in direct communication with a customer using you standard e-mail program instead of having to go to the website.

    @author Malte Sussdorff (malte.sussdorff@cognovis.de)
    @creation-date 2005-12-18
 
    @param array        An array with all headers, files and bodies. To access the array you need to use upvar.
    @return             nothing
    @error
} {
    # get a reference to the email array
    upvar $array email

    # Take a look if the email contains an email with a "#"
    set pot_email [lindex [split $email(to) "@"] 0]
    if {[string last "#" $pot_email] > -1} {
       ....
   }
}

 Alternatives to this are:

  • ${component_name}-bugs@openacs.org (where component_name could be openacs or dotlrn or contacts or whatever), to store a new bug in bug-tracker
  • username@openacs.org (to do mail-through using the user name, which allows you to hide the actual e-mail of the user whom you are contacting).

Cleanup

Once all callbacks have been fired off,  e-mails need to be deleted from the Maildir directory and files which have been extracted need to be deleted as well from the /tmp directory. 

E-Mail: Event Handling

Created by Dave Bauer, last modified by Gustaf Neumann 24 Aug 2020, at 01:43 PM

Sending email on certain events in OpenACS/.LRN is done very haphazardly. This needs to be rewritten so there is a simple way to figure out when an email will be sent, and allow proper handling of user preferences, administrative parameters, and customization.

There are cases where the system (OpenACS or .LRN) needs to send out email, for example, when a new user joins, requests a password reset, or is added to a subsite or .LRN community.

Right now there isn't any system-wide way to mange this email. In some cases, the administrator is notified an email will be sent and is given the option to edit the email before it is sent, but there is no one way this is done.  There are several pages that call  ns_sendmail explicitly, or acs_mail_lite::send explicitly. There are more places this happens in .LRN. Unfortunately there is also a "magic" place where email is sent that is totally unexpected. Inside the dotlrn_community::membership_approve procedure, there is a call to dotlrn_community::send_member_email, which will send an email to the user when the membership is approved, if 1) a parameter is set and 2) an administrator has created and enabled an email message to be sent.

 In addition there is a email sent using the "spam" package in dotlrn/www/admin/users-add-to-community where a dotlrn sitewide administrator can add users to a community, and the users are automatically emailed. In this case the administrator is not notified that an email is sent, or given an opportunity to customize or suppress the email. This causes problems when an administrator attempts to fix a problem by adding a user to a community automatically, and the user is sent a confusing message.


The dotlrn package allows for a custom email to get written by the community admin for each community, but the admin is not allowed to choose if the email is sent when an individual member is added. It is either on or off, always sent, or never sent.

dotlrn-ecommerce extends this by adding several more events for application submission, approval, rejection, etc. And admin can edit these emails on a sitewide or per community basis. In most cases the email is automatically sent, in one o r two cases the admin can edit the email, but not suppress it.

This leads to unwanted email. Often an admin must manually add or remove someone from a subsite or community or otherwise handle a problem. This can lead to welcome emails being sent at the wrong time, confusing the users. 

 A system wide solution would allow packages to create events where email is sent by the system (besides subscribed notifications). This solution would provide an includable interface for creating and editing a default email message for the events. It would also provide an interface to notify an admin that an email will be sent, giving options to suppress the email or edit the content of the email before sending it.

 The beginnings of this feature exist in the dotlrn_member_emails table, dotlrn_community::send_member_email procedure, and the dotlrn-ecommerce package which has a few pages that replicate this interface, but probably needs more generalization to allow working with subsites as well as dotlrn communities. There is also the reusable include for editing the default emails under dotlrn/lib/member-email

This proposal would provide a comprehensive solution for handling system level email events, allowing admins to know when an email is sent, and provide a consistent user interface to manage the emails.

 

MS:

You could achieve this in a general way by using the acs-lang interface at least for the subject and body. For each object_id you would create a new message key, e.g. acs-translations.welcome_email_subject_${object_id} and acs-translations.welcome_email_body_${object_id}. If you have multiple emails per community / subsite, you would rename them to acs-translations.confirmation_email_subject_${object_id} aso. A general interface would then be provided to look for all language key combinations of acs-translations.email_xxx, allowing you to edit the messages for all communities and subsites. If you want to edit them for only one, then you can look for all who have the same object_id. 

A default message would be given with acs-translations.email_subject_welcome, which the mail sending could default to in case no specific language key exists for the community / object_id. Furthermore, acs-translations.welcome_email_help message key is present describing what this email is about. A package like dotlrn would register the three default e-mail keys to start off with this and then the email-handling package can do the rest (e.g. with email-handler::send -to_party_ids -from_addr -email_type "welcome" -object_id).

This approach has the major advantage that you have internationalization by default.

Alternatively you could mimic the message handling done by contacts which allows you to have multiple message types, e.g. email, which you can fill in with default values, which has it's own I18N by having a locale stored in the DB table. But if you ask me, the acs-translations idea sounds better to me :). Though, you can obviously do this with your own tables as well, but you would loose on the nice features acs-lang has to offer.

 

 Files that currently could trigger email

add_user calls add_user_to_community

add_user_to_community calls membership_approve (if applicable). membership_approve calls send_member_email unconditionaly

so any call to add_user or add_user_to_community could result in a call to send_member_email

dotlrn_community::add_user
dotlrn/www/admin/add-instructor-3.tcl
                 users-add-to-community.tcl
                 member-add-3.tcl
                 members-chunk-table.tcl
                 members.tcl
                 register.tcl

dotlrn-ecommerce/www/admin/gwu-section-new.tcl
                     ecommerce/shopping-cart-add.tcl.backup
                     register/index.tcl                 
                tcl/implementation-procs.tcl
                
dotlrn_community::add_user_to_community
dotlrn/tcl/class-procs.tcl
           club-procs.tcl
           community-procs.tcl
           dotlrn-callback-procs.tcl
           
dotlrn_community::membership_approve
dotlrn/www/approve.tcl
dotlrn/www/admin/commmunity-members-add-to-community.tcl
                 users-add-to-community-email.tcl
dotlrn-ecommerce/www/admin/application-approve.tcl

 

 

Security: Cross Site Request Forgery (CSRF)

Created by Gustaf Neumann, last modified by Gustaf Neumann 24 Aug 2020, at 01:41 PM

Starting with OpenACS 5.9.1, OpenACS offers support for protecting against Cross Site Request Forgery (CSRF). In essence, this attack can cause a user’s web browser to perform an unwanted action on a trusted site for which the user is currently authenticated. The user gets a page presented, which looks harmless, but contains links or images that perform actions with the users credentials without the users consent. Note that the CSP does not protect a user against clicks on a malicious link.

CSRF protection works by ensuring that values for an action (e.g. by from a HTML form) are only accepted from a user that has received the form before. OpenACS generates by its security-procs a secure CSRF token value and provides it to a developer it in a global variable ::__csrf_token. When requests secured with the CSRF token are received, it can be validated on the server side. Note, that this mechanism is similar to "signing" values in OpenACS.

CSRF protection concerns of two parts: add the CSRF token to the form (POST requests) or to the href, and checking the received in the queries expecting input from CSRF protected resources. The first part works technically quite similar as securing CSP via nonces. Add code to the Tcl or ADP page that outputs the global variable (the test for the token is mostly for backwards compatibility)

    <form ...>
        ...
        <if @::__csrf_token@ defined>
           <input type="hidden" name="__csrf_token" value="@::__csrf_token;literal@">
        </if>
        ...
    </form>

Secondly, the page contract on the receiving side has to validate the csrf token. This can be achieved by adding a call to csrf::validate to the validation part of a page contract.

ad_page_contract {
    @author ...
    @creation-date ...
} -query {
  ...
} -validate {
   ...
   csrf { csrf::validate }
}

In the code base of OpenACS, CSRF protection was added on several places (e.g. public pages, the list template, etc.) such the checks of OpenACS sites on vulnerability scanners improve. Technically, it would be desirable to secure more places against CSRF attacks in the future. However, it depends on the requirements of a site whether or not e.g. the API browser or search should be CSRF protected. With protection turned on, one cannot share e.g. a link to a search with some other user (or a search engine). A site admin has to decide, how protected/public such links should be.

Security: Content Security Policy (CSP)

Created by Gustaf Neumann, last modified by Gustaf Neumann 24 Aug 2020, at 01:40 PM

Starting with version 5.9.1, OpenACS supports Content Security Policies (CSP), which is a means to secure websites against a range of Cross Side Scripting (XSS) attacks. In short, a CSP allows a developer to deactivate unneeded features in the browser of the client to provide there a sandbox with the minimum required capabilities. The CSP can allow e.g. just to retrieve .js files just form certain sites, or it can disallow script tags within the page, which might be injected by an attacker (for a more detailed introduction and tutorial, see CSP Reference, Google Developer Guide for CSP).

In general, a CSP defines the rules what should be allowed in a page. This could be done static for the whole page, but this means that the CSP rules must allow everything which is needed on a page with the highest requirements (e.g. a page with a richtext editor needs probably a script-src 'unsafe-eval' directive). This could render CSP pretty useless.

Therefore, OpenACS supports a CSP generator, which generates a CSP rule-set for every page dynamically based on the requirements of the page. A web developer can specify the requirements of a page/proc with the command security::csp::require. For example, the current OpenACS theme uses in its plain-master the following directives.

security::csp::require img-src ipv6-test.com

security::csp::require style-src maxcdn.bootstrapcdn.com
security::csp::require script-src maxcdn.bootstrapcdn.com

security::csp::require font-src 'self'
security::csp::require font-src maxcdn.bootstrapcdn.com

Based on the directives of the pages and the directives of the master templates, the security policy of the pages is built (typically in the blank-master). For example, the content security policy of the start page of OpenACS is

default-src 'self';
font-src 'self' maxcdn.bootstrapcdn.com data:;
img-src ipv6-test.com 'self';
report-uri /SYSTEM/csp-collector.tcl;
script-src maxcdn.bootstrapcdn.com 'self' 'nonce-49DBB4A924EA648C3025F7DD8C2553DC0EC700D1';
style-src maxcdn.bootstrapcdn.com 'self' 'unsafe-inline';

With this CSP, openacs.org gets an A+ rating from securityheaders.io.

Deactivating CSP for a Site

Per default, the content security policies are turned on. All packages of the oacs-5-9 branch can be used with the enabled content security policies. However, when a website contains legacy code using JavaScript, for which no content security policies are defined, this will result into non-functioning pages. Therefore, a website administrator can set the package parameter CSPEnabledP (in the package parameters of ACS Kernel in "security" section) to "0" to deactivate the CSP.

For Developers

In order to make old packages (not included in the oacs-5-9 branch) or newly developed packages CSP compliant, one should be aware that all inline code is considered harmful. This includes <script> elements, but also "javascript:" URIs or on* event handlers.

<script> Elements

The CSP guidelines recommend to replace the such elements in favor of JavaScript files obtained from the same source as the page itself. However, this is not always practical, especially, when JavaScript is generated dynamically. In such cases, two approaches are possible to make the script tag acceptable (without allowing all scripts on the page). CSP 2 offers the ability to add nonces or cryptographic hashes to secure this elements. OpenACS supports the first approach.

A nonce value is essentially a one-time value which can't be predicted by an attacker. OpenACS generates by its security-procs such as value and saves it in a global variable ::__csp_nonce. This can be used in the Tcl code or in an ADP page like in the following example:

<script language="JavaScript" 
   type="text/javascript"
   <if @::__csp_nonce@ not nil> nonce="@::__csp_nonce;literal@"</if>
>
...
</script>

Event handlers and "javascript:" URI

Most work are probably changes concerning event handlers (e.g. onclick, onblur, ...) and "javascript:" URIs (having "javascript" in the protocol part of the URI). In general, such code pieces must be refactored (see e.g. 1 or 2 for examples).

OpenACS 5.9.1 offers to ease this process the function template::add_event_listener, which can be used to register event handlers in a compliant fashion either per HTML ID or per CSS class (see cal-item-new.tcl or in forums/lib/message/row2.tcl for examples, how add_event_listener can be used).

 

 

 

OpenACS Community Wiki

Created by gustaf neumann, last modified by Gustaf Neumann 21 Aug 2020, at 04:54 PM

This is the OpenACS Wiki system, built with the xowiki package. This wiki contains user documentation, how-tos, and tips and tricks related to OpenACS. It also serves as a collaboration area for OpenACS contributors.

Quick Links:

Recent Wiki Page Edits:

Cookbook

Subsystems Documentation

External Resources

OpenACS Projects

Package Documentation

Events

OpenACS conferences (5 months ago)

Tutorials for Administrators

OpenACS Monitoring (1 year ago)
rl_json (1 year ago)

Non-Core Packages

Chat (1 year ago)
XoWiki (1 year ago)
XOTcl Core (1 year ago)
xooauth (1 year ago)
xowf (XoWiki Workflow) (3 years ago)

Documentation

HOME

for beginning developers (1 year ago)
Privacy Policy (1 year ago)

Tutorials

for beginning developers (1 year ago)
Theming in OpenACS (3 years ago)

Core Documentation

Installing OpenACS (1 year ago)

Coding Standards

Security Guidelines (1 year ago)
Coding Standards - Index (1 year ago)
Tcl Procs (2 years ago)

Packages

   

Next Page
previous December 2024
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31 1 2 3 4

Popular tags

17 , 5.10 , 5.10.0 , 5.10.1 , 5.9.0 , 5.9.1 , ad_form , ADP , ajax , aolserver , asynchronous , bgdelivery , bootstrap , bugtracker , CentOS , COMET , compatibility , CSP , CSRF , cvs , debian , docker , docker-compose , emacs , engineering-standards , exec , fedora , FreeBSD , guidelines , host-node-map
No registered users in community xowiki
in last 30 minutes
Contributors

OpenACS.org