Forum OpenACS Development: Win32-OpenACS Version 1.8

Collapse
Posted by Maurizio Martignano on
Dear all,
I've just compiled version 1.8 (19th of April 2008) of Win32-OpenACS: an actual native Win32 port (no Cygwin, no VMware) of OpenACS.

This version of Win32-OpenACS consists of the following components:
1. aolserver-4.5.0 - including
1.1 nscache 1.4
1.2 nssha1 1.1.1.1
1.3 nspostgres 4.1
1.4 nsoracle 2.7
2. tcl-8.5.1
3. tk-8.5.1
4. tDom-0.8.2
5. xotcl-1.6.0
6. thread-2.6.5
7. tcllib-1.10
8. postgresql-8.3.0-2
9. openacs-5-4.1
10. xowiki-0.8.3 - from cvs head at the 18th of March 2008
11. dotlrn-2.3.1

The port can be obtained at the following URL:
http://www.friendlybits.com/en/inf_tec_en/win32openacs_en/

NOTES:
1. The Aolserver executable (nsd.exe) has been improved with a patch suggested by Titi Alailima. Thanks to this patch now the executable can run as a well behaved Windos Service (i.e. it ends properly).

2. This distribution supports Oracle 10g and 11g. Anyhow only OpenACS 5.4.1 core works properly with Oracle, the other extensions (e.g. xowiki) do not.

Hope it helps,
Maurizio

Collapse
Posted by Emmanuelle Raffenne on
Hi Maurizio,

This is great!

OpenACS and .LRN are not yet compatible with postgresql 8.3, mostly because of tsearch2 that is now part of pg since 8.3 but the function names have changed. This means that search won't work on pg 8.3. Is there a chance that you substitute the 8.3 version for a 8.2 one?

Collapse
Posted by Maurizio Martignano on
Dear Emannuelle, thanks for your remark.
I've included Potsgres 8.2.7-1 instead of 8.3.0-2.

Maybe it's a good time to start making sure that OpenACS/.LRN work also with the new versions of Postgres.

I've to admit that another point that I found somehow frustrating is that there are some packages that work with Postgres and don't with Oracle.

Once again many thanks,
Maurizio

Collapse
Posted by Emmanuelle Raffenne on
Hi Maurizio,

Thanks a lot for addressing this and moreover for the windows installer :).

Postgresql 8.3 will be supported in the next versions of OpenACS and .LRN (5.5, 2.5).

It would be nice if all the packages included in .LRN has support for Oracle, indeed. Most of us use Postgresql and there's very few people using Oracle who contribute to the community, I guess that's the reason.

Although, the most important packages support Oracle (core and dotlrn-all, see http://www.openacs.org/xowiki/Aliases_at_CVS for details). dotln-extras packages like views, survey and chat provide support for Oracle too (from the top of my head).

Collapse
Posted by Maurizio Martignano on
Dear Emanuelle, thanks for your message.
Honestly for the windows installer at the moment I only have a plan in my mind.... no time yet to work on that.
When I progress a bit on this topic, I'll keep you posted.

Cheers and thanks,
Maurizio

Collapse
Posted by Gustaf Neumann on
What is the problem with xowiki 0.8.3 on Oracle?
Last time i checked, it worked fine.
Collapse
Posted by Maurizio Martignano on
Dear Gustav,
I'll send you the log file I get with Oracle.
Perhaps you can understand what's happening.

Cheers and thanks,
Maurizio

Collapse
Posted by Gustaf Neumann on
Dear Maurizio,
actually, there were four different errors in the log file:

a) ORACLE_HOME not set

[21/Apr/2008:21:07:50][5000.4828][-conn:0-] Error: Error sourcing c:/aolserver/servers/openacs/packages/acs-bootstrap-installer/installer/install.tcl:

    (read trace on "env(env)")
    invoked from within
"file join $env(ORACLE_HOME) bin sqlplus"

i can't say how serious this is, since surprisingly, most stuff is loaded correctly. With this problem, "db_source_sql_file" is be broken.

Do you have the environment variable ORACLE_HOME set?

b) incorrect path:

[21/Apr/2008:21:12:34][2460.5568][-conn:0-] Notice: Loading data file c:/aolserver/servers/openacs/packages/ref-timezones/sql/oracl
e/00-timezones.ctl...
[21/Apr/2008:21:12:35][2460.5568][-conn:0-] Notice: SQL*Loader: Release 10.2.0.3.0 - Production on Mon Apr 21 21:12:35 2008

[21/Apr/2008:21:12:35][2460.5568][-conn:0-] Notice: Copyright (c) 1982, 2005, Oracle. All rights reserved. [21/Apr/2008:21:12:35][2460.5568][-conn:0-] Error: apm_package_install: Error installing Reference Data - Timezone version 5.4.1: SQL*Loader-500: Unable to open file (s2cs.) SQL*Loader-553: file not found SQL*Loader-509: System error: The system cannot find the file specified. SQL*Loader-500: Unable to open file (s2cs.) SQL*Loader-553: file not found SQL*Loader-509: System error: The system cannot find the file specified. while executing "close $fd" (procedure "db_load_sql_data" line 32) invoked from within "db_load_sql_data -callback $callback $path/$file_path" (procedure "apm_package_install_data_model" line 52) invoked from within "apm_package_install_data_model -callback $callback -data_model_files $data_model_files $spec_file_path" invoked from within

Here "db_load_sql_data" fails, which loads CSV formatted data. In the concrete case, it fails to load Timezone data, which has no immediate problem on loading the packages, but will cause troubles later. I would suggest that you add an "ns_log notice ..." statement to output the "file" which it tries to open, which the code assumes that it exists.

Most likely, you should see the same error under postgres.

(c) tsearch 2

21/Apr/2008:21:18:48][4860.3732][-conn:4-] Notice:  ********** STARTING BEFORE-INSTALL CALLBACK ****************
[21/Apr/2008:21:18:48][4860.3732][-conn:4-] Error: nsoracle.c:3905:<unknown>: error in `OCIStmtExecute ()': ORA-00942: table or view do
es not exist

SQL: select typname from !>>>!pg_type where typname='tsvector' [21/Apr/2008:21:18:48][4860.3732][-conn:4-] Error: apm_package_install: Error installing Tsearch2 Driver version 5.4.1: nsoracle.c:3905 :<unknown>: error in `OCIStmtExecute ()': ORA-00942: table or view does not exist

Don't install tsearch2 under Oracle, this is postgres only. For Oracle, there is the "intermedia" driver for full text search. Last time i tried to install that under Oracle, it did not work out of the box, but worked after some more tweaks, but i was not sure if that was a local installation problem. I think that people at UNED work on that issue for the forthcoming release, so i would suggest that you wait for this.

d) categories:

[21/Apr/2008:21:19:29][4860.3732][-conn:4-] Notice: --try en:categories-portlet -> 0, ::818 ::xowiki::Package->resolve_request (395ms, 0ms)
[21/Apr/2008:21:19:29][4860.3732][-conn:4-] Error: nsoracle.c:3905:<unknown>: error in `OCIStmtExecute ()': ORA-00904: "WIDGET": invalid identifier

SQL: select tree_id, subtree_category_id, assign_single_p, require_category_p, !>>>!widget from category_tree_map where object_id = :object_id

This is the error, you saw in XoWiki, but this is a problem with categories. We had some discussing already about the column "widget" that was introduced lately https://openacs.org/forums/message-view?message_id=1470107

It seems to me, as if the installed data model does NOT contain the column "widget" in the category_tree_map, but the query does. Maybe an incomplete rollback, since it seems to work under postgres?

What version of categories do you use?

The good news is that XoWiki seems to work well.
-gustaf

Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
thanks for your help. My answers/comments here below.

a) ORACLE_HOME not set

[MM] True. In the first run this variable wasn't set. But then I fixed this problem and I restarted the server.

b) incorrect path:

[MM] This a real problem. It seams that Timezone package requires some changes. Is this an Oracle or a Windows vrs. Linux problem?

(c) tsearch 2

[MM] Correct, my mistake. No need to install this.

d) categories:

[MM] This is another real problem. Categories seem to be working with Postgresql while they don't with Oracle.

The end result of points b) and d) is that XoWiki shows some error messages (though not directly generated by XoWiki itself).

The problem I see is that not all the packages/extensions work in the same way when running on Oracle or on Postgresql.

Cheers and once again many thanks,
Maurizio

Collapse
Posted by Gustaf Neumann on
Dear Maurizio,

Concerning (b): please add the log statement as indicated in my last posting to see which files are searched where.

concerning (d): please tell me the version of categories you are using.

btw, xotcl has only problems with (d), when categories are used. But this is just a nit, we want to get a full working system.

Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
about point (b) the function seems to attempt the proper file (i.e. the file name is correct).
What I noticed is that there's a difference between the Postgresql and Oracle caes about how the file names are built.... may be this is the cause of the problem.....

please have a look at the following code:

d_proc -public db_load_sql_data {{
-dbn ""
-callback apm_ns_write_callback
} file } {

Loads a CSV formatted file into a table using PostgreSQL's COPY command or
Oracle's SQL*Loader utility. The file name format consists of a sequence
number used to control the order in which tables are loaded, and the table
name with "-" replacing "_". This is a bit of a kludge but greatly speeds
the loading of large amounts of data, such as is done when various "ref-*"
packages are installed.

@param dbn The database name to use. If empty_string, uses the default database.
@file Filename in the format dd-table-name.ctl where 'dd' is a sequence number
used to control the order in which data is loaded. This file is an
RDBMS-specific data loader control file.

} {

switch [db_driverkey $dbn] {

oracle {
global env

set user_pass [db_get_sql_user -dbn $dbn]
set tmpnam [ns_tmpnam]

set fd [open $file r]
set file_contents [read $fd]
close $fd

set file_contents [subst $file_contents]

set fd1 [open "${tmpnam}.ctl" w]
puts $fd1 $file_contents
close $fd1

cd [file dirname $file]

set fd [open "|[file join $env(ORACLE_HOME) bin sqlldr] userid=$user_pass control=$tmpnam" "r"]

while { [gets $fd line] >= 0 } {
# Don't bother writing out lines which are purely whitespace.
if { ![string is space $line] } {
apm_callback_and_log $callback "[ad_quotehtml $line]\n"
}
}
close $fd
}

postgresql {
global tcl_platform

set pguser [db_get_username]
if { ![string equal $pguser ""] } {
set pguser "-U $pguser"
}

set pgport [db_get_port]
if { ![string equal $pgport ""] } {
set pgport "-p $pgport"
}

set pgpass [db_get_password]
if { ![string equal $pgpass ""] } {
set pgpass "<<$pgpass"
}

if { [string equal [db_get_dbhost] "localhost"] || [string equal [db_get_dbhost] ""] } {
set pghost ""
} else {
set pghost "-h [db_get_dbhost]"
}

set fd [open $file r]
set copy_command [subst -nobackslashes [read $fd]]
close $fd
set copy_file [ns_mktemp /tmp/psql-copyfile-XXXXXX]
set fd [open $copy_file "CREAT EXCL WRONLY" 0600]
puts $fd $copy_command
close $fd

if { $tcl_platform(platform) == "windows" } {
set fp [open "|[file join [db_get_pgbin] psql] -f $copy_file $pghost $pgport $pguser [db_get_database]" "r"]
} else {
set fp [open "|[file join [db_get_pgbin] psql] -f $copy_file $pghost $pgport $pguser [db_get_database] $pgpass" "r"]
}

while { [gets $fp line] >= 0 } {
# Don't bother writing out lines which are purely whitespace.
if { ![string is space $line] } {
apm_callback_and_log $callback "[ad_quotehtml $line]\n"
}
}

# PSQL dumps errors and notice information on stderr, and has no option to turn
# this off. So we have to chug through the "error" lines looking for those that
# really signal an error.

set errno [ catch {
close $fp
} error]

# remove the copy file.
file delete -force $copy_file

if { $errno == 2 } {
return $error
}

# Just filter out the "NOTICE" lines, so we get the stack dump along with real
# ERRORs. This could be done with a couple of opaque-looking regexps...

set error_found 0
foreach line [split $error "\n"] {
if { [string first NOTICE $line] == -1 } {
append error_lines "$line\n"
set error_found [expr { $error_found || [string first ERROR $line] != -1 || \
[string first FATAL $line] != -1 } ]
}
}

if { $error_found } {
global errorCode
return -code error -errorinfo $error_lines -errorcode $errorCode $error_lines
}

}

nsodbc {
error "db_load_sql_data is not supported for this database."
}
default {
error "db_load_sql_data is not supported for this database."
}
}
}

About point (d) the version of Categories I'm using is 1.1.2d3 .

Hope it helps,
Maurizio

Collapse
Posted by Gustaf Neumann on
concerning db_load_sql_data:

Seems as if the following sequence leads to the problem

 set fd [open "|[file join $env(ORACLE_HOME) bin sqlldr] userid=$user_pass control=$tmpnam" "r"]

... close $fd

Do you have the sqlldr under [file join $env(ORACLE_HOME) bin sqlldr]?
Are the provded parameters ok (e.g. the $tmpnam)?

Concerning (d): You are using the head version of categories, where it seems, that the column "widget" was not added in the Oracle version. Although i hate add something to CVS, which i cannot test, i added the column in the Oracle creation procs. Please get the head version from CVS and test it with Oracle. If the error is gone, i will apply the same fix to the oacs-5-4 branch, which has apparently the same problem.

Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
success!!!

1. The first problem was due to $tmpnam (which was wrong: in Vista you can't write files wherever you want).
To fix it I had to introduce the following change:

# set tmpnam [ns_tmpnam]
set tmpnam [ns_mktemp /tmp/ctl-XXXXXX]

2. Your correction to the categories Oracle creation procs worked.

Thanks a lot for your help!

Cheers,
Maurizio

Collapse
Posted by Gustaf Neumann on
Dear Maurizio,

i count current 77 occurrences of ns_tmpnam in OpenACS and its packages. These places will have the same problem, so the fix is not sufficient in the general case. Since ns_tmpnam is implemented in C, it is most probably the best solution to address the problem in the aolserver implementation. Again, as i do not have a WIN32 environment, please test the following replacement of NsTclTmpNamObjCmd() in aolserver/nsd/tclfile.c. With this fix, set tmpnam [ns_tmpnam] should be ok again.

best
-gustaf neumann

int
NsTclTmpNamObjCmd(ClientData arg, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[])
{
#ifdef WIN32
    /*
      The WIN32 implmentation of tmpnam() ignores the environment
      variable TMP and generates filenames for the root
      directory. Unfortunately, new WIN versions (Vista) don't allow
      this. The suggested replacement is _tempnam().

      The first argument of _tempnam() is the default directory, in case
      the environment variable TMP is not set or points to a directory
      that does not exist.
    */
    char *buf = _tempnam("/tmp", NULL);

    if (buf == NULL) {
	Tcl_SetResult(interp, "could not generate temporary filename.", TCL_STATIC);
        return TCL_ERROR;
    }
    /*
       The documentation says that _tempnam() allocates memory via
       malloc(); to be sure, that the "right" free() is used, we do
       not use TCL_DYNAMIC but the TCL_VOLATILE followed by the manual
       free().
    */
    Tcl_SetResult(interp, buf, TCL_VOLATILE);
    free(buf);
#else
    char buf[L_tmpnam];

    if (tmpnam(buf) == NULL) {
	Tcl_SetResult(interp, "could not generate temporary filename.", TCL_STATIC);
        return TCL_ERROR;
    }
    Tcl_SetResult(interp, buf, TCL_VOLATILE);
#endif
    return TCL_OK;
}
Collapse
Posted by Enrique Catalan on
Hi,

This could work for you as a workaround. We've fixed that overloading that function adding this code to one of our *-init.tcl procs:

if { [ns_info platform] == "win32"} {
rename ns_tmpnam aol_tmpnam
proc ns_tmpnam {} {
set tmp [aol_tmpnam]
set dir [parameter::get_from_package_key -package_key acs-subsite -parameter TmpDir]
return "${dir}${tmp}"
}
}

You just need to make sure that the MainSite parameter TmpDir in the site-map has the right path (i.e. C:\temp ).

In Windows, according to their documentation, the C function tmpnam() is deprecated and is not safe. They recommend to use tempnam(dir,filename) instead where you can specify the path and the filename.

Hope that helps!

Collapse
Posted by Gustaf Neumann on
Yes, this will work, but is awful, since it means that every aolserver application under windows using ns_tmpnam will have to do this. i was not daring to suggest this. it is better to fix aolserver.
Collapse
Posted by Maurizio Martignano on
Dear Gustaf, I absolutely agree with you.

The changes you proposed in your last post are allmost perfect. I had to change them as follows:

int i;
char *buf = _tempnam("/tmp", "nsd");

if (buf == NULL) {
Tcl_SetResult(interp, "could not generate temporary filename.", TCL_STATIC);
return TCL_ERROR;
}
/*
Change back slash characters into slash characters
*/
for (i = 0; i < strlen(buf); i++) {
if (buf[i] == '\\') buf[i] = '/';
}
/*
The documentation says that _tempnam() allocates memory via
malloc(); to be sure, that the "right" free() is used, we do
not use TCL_DYNAMIC but the TCL_VOLATILE followed by the manual
free().
*/
Tcl_SetResult(interp, buf, TCL_VOLATILE);
free(buf);

The problem is that _tempname generates file names in the Windows style (i.e. with back slash characters) and SqlLdr doesn't like that style for the "control" parameter (it requires normal slash characters).

Once again many thanks,
Maurizio

Collapse
Posted by Gustaf Neumann on
Dear Maurizio, great news, many thanks. let me know when other problems pop up. i'll try to channel the change into the aolserver sources. It's a pleasure to work with you

-gustaf

Collapse
Posted by Maurizio Martignano on
Dear Guastaf, the pleasure is mine.

Perhaps when you suggest the changes to the Aolserver community you could use the code here below, where the use of the function strlen has been optimized.

int i, str_len;
char *buf = _tempnam("/tmp", "nsd");

if (buf == NULL) {
Tcl_SetResult(interp, "could not generate temporary filename.", TCL_STATIC);
return TCL_ERROR;
}
/*
Change back slash characters into slash characters
*/
str_len = strlen(buf);
for (i = 0; i < str_len; i++) {
if (buf[i] == '\\') buf[i] = '/';
}
/*
The documentation says that _tempnam() allocates memory via
malloc(); to be sure, that the "right" free() is used, we do
not use TCL_DYNAMIC but the TCL_VOLATILE followed by the manual
free().
*/
Tcl_SetResult(interp, buf, TCL_VOLATILE);
free(buf);

Once again many thanks,
Maurizio