Forum OpenACS Q&A: Re: Help with openacs installing

Collapse
Posted by Bjoern Kiesbye on
Hello,

I'm not sure if this is the right place for the post but i hope it will help someone. I just tried to install Aolserver from source (cvs checkout) on Ubuntu 14.04. and 12.04 and ran into the same Problem. Adding -Wl,--no-as-needed to the modules Makefile was just part of the solution. If it still does not fix the Problem, check the output of make, if the Flag '-fvisibility=hidden' is passed to gcc. It will hide all Symbols, unless explicitly specified as public within the source code. The option is "just" an optimization and can be removed from the ns.mak file without greater harm (on Suse it is not set in first place), this is the first solution.
The second and better solution is to patch the source code, and add required attributes to the Function Ns_ModuleInit and the Variable Ns_ModuleVersion (a makro to do that already exists in ns.h), which advise gcc to export this symbols and make them access able from outside the module (public), but this has to be done for each Module. Below you find links to patches for the most common Aolserver modules including a description on how to apply them.

First Solution:
The Flag is set in the File
/usr/local/aolserver-inst-dir/include/ns.mak

The Line
CFLAGS_EXTRA = -fPIC -pipe -fvisibility=hidden
can be changed to
CFLAGS_EXTRA = -fPIC -pipe

Second Solution:
First download the patches to the /tmp directory. Then apply the patches by changing into the modules source directory and run the patch command.

root#> cd /tmp
root#> wget http://www.clever-devel.com/file/35054/nssha1-visibility-0.1.1.patch
root#> cd /usr/local/src/aolserver-src-dir/nssha1/
root#> patch -p0 /tmp/nssha1-visibility-0.1.1.patch

Now compile the Module as usual.
The procedure for the patches below is the same, just the patch name and the source directory needs to be changed for the current module.

http://www.clever-devel.com/file/35054/nssha1-visibility-0.1.1.patch
http://www.clever-devel.com/file/35057/nscache-visibility-0.1.1.patch
http://www.clever-devel.com/file/35048/nsopenssl-visibility-0.1.1.patch

Collapse
Posted by Gustaf Neumann on
Dear Bjoern,

many thanks for the patches! I've applied just now quite similar ones (using NS_EXPORT instead of DllExport) to the version of aolserver on sourceforge, such that other people might have an easier life.

Out of curiosity: Do you have any technical reason for not using NaviServer? It is easier to install from scratch via [1]

all the best
-gustaf neumamn

[1] https://openacs.org/xowiki/naviserver-openacs

Collapse
Posted by Bjoern Kiesbye on
Dear Gustaf,

thanks for applying the patches, I was not sure which makro is the preferred one as both where offered in ns.h.

Until now Aolserver was sufficient, and as there was no real reason, I have not tried NaviServer yet. A while a go I did notice that the nsmemcache module was developed for NaviServer first, and now i stumbled over some recent posts on the Aolserver mailing list, which said that the naviserver-dev mailing list is much more active than the aolserver mailing lists. So I think I will try NaviServer.

If I recall right there was a Licensing issue with Aolserver which was the initial reason for the NaviServer Project, is this it, or is there more? I checked the NaviServer Pages on sourceforge.net and wiki.tcl.tk and their doesn't seem to be a great difference between Aolserver and NaviServer.

Collapse
Posted by Gustaf Neumann on
Dear Bjoern,

The preferred macros start with NS_* (but the one you used work as well, but these are for compatibility).

One reason for the split was a question in "philosophy", since the aolserver fraction was very conservative on changes and the NaviServer needed changes for integration with their products (one case was that aolserver was webserver only, while naviserver went towards multi-protocol server; see [1], thread "Support for non-HTTP protocols")

There is no exhaustive list of differences between aolserver and NaviServer, so i think the best place the check is the changelog summary[2]. The biggest hurdle for a quick check is probably the different config file, but you will find in the NaviServer sources several sample config files, including one modeled for OpenACS.

Most of the differences between aolserver and NaviServer are about extending functionality. Alex Hisen pointed out not long ago in the aolserver mailing list three (actually deprecated) features of aolserver, that might require replacements in legacy code, when one switches from aolserver to NaviServer:

1) Support for optional $conn argument in ns_return*, etc commands
2) ns_share
3) ns_set -persist

The first one is about using ignored arguments, the second and third can be handled via nsv, for (2) Stephen Deasey wrote additionally an extra module.

Recent versions of OpenACS is free of these features since several years. OpenACS.org runs on NaviServer since about one year, the performance gain was significant (see [3]).

All the best
-gustaf neumann

[1] http://news.gmane.org/group/gmane.comp.web.aolserver/last=0/force_load=very/?page=56
[2] https://bitbucket.org/naviserver/naviserver/src/default/NEWS
[3] https://openacs.org/forums/message-view?message_id=4074774

Collapse
Posted by Bjoern Kiesbye on
Dear Gustaf,

thank you for all the information, I did take a look at the config.tcl Description for NaviServer at http://wiki.tcl.tk/22673 and there does not seem to be that many differences in the configuration. The Aolserver features not available in NAviServer 1)-3) wont affect my own code, and as it seems to be that development efforts go more and more into NaviServer, i think i will give it a try.

I have another patch for nsopenssl concerning the Error

tclcmds.c:338:31: error: 'Tcl_Interp' has no member named 'result'

which occurs when compiled for tcl8.6, where the interp object fields cant be accessed directly anymore,

http://www.clever-devel.com/file/35043/nsopenssl-interp-result-0.1.1.patch

Collapse
Posted by Jim Lynch on
Hi...

I would like to tell about my exp getting 8.6tcl to work with some of the aolserver modules...

Before 8.6, the tcl result was available as a string, with some funcs to set and append to that string.

At 8.6, you actually have a choice, you can rebuild it so it works as before (so result is exposed) or you can alter code which accesses the result, as the patch suggests.

What I found out was, if you want numeric result values, you can have tcl set them as ints or maybe longs (or other object types). This is what I preferred to do when I was trying to get 8.6 going.

You want to be very cautious with sprintf, as this func doesn't limit the writing based on length. So, you either want to use things that append to the string result object, or you want to set as cast to a different type, for example int. 8.6 provides calls for doing either.

Almost every time I've seen uses of sprintf (in -anything-), I knew there was the typical "buffer overrun" possibility, and in the cases where I could, I was able to replace each with calls to snprintf (or to use a different alternative that was also safe).

As an aside, many years ago, when looking at aolserver source for my first time, I happened to look at the call Ns_DStringPrintf, which then used sprintf. I wrote an equivalent using snprintf, and aolserver devs eventually took a whole printf func from some stdio lib source, and adapted it so that it "printf"ed to a DString in a safe manner. To Gustaf, if you hadn't done so already, would you be willing to look at it and at naviserv source to see if it's been incorporated already?

If you're going to use sprintf, get the length of the buffer and use snprintf instead, as not doing so could result in a crash if sprintf overwrites a dynamically allocated buffer.

This may sound like hyperbole, but it's my very stong belief that sprintf is never safe, and should always be replaced with snprintf.

I may still have the sources of other aolserver or naviserv (don't remember which at this moment) modules, where I had to rewrite the pieces that set or get the tcl result.

On request, I'll put something together in the hope it will assist.

-Jim

Collapse
Posted by Gustaf Neumann on
Dear Björn,

The supported ssl driver for NaviServer is nsssl [1] part of the 43 NaviServer modules [2]. Nsssl is up to date in terms of the security features of OpenSSL (e.g. forward secrecy). When using this driver, one gets an A+ rating from SSL labs by following the configuration instructions in the README file. Furthermore, the driver nsssl provides the "ns_ssl" command for https client requests, compatible with "ns_http", which use the same c-level infrastructure.

The compilation problem you faced with nsopenssl are due to the fact, that this driver was not updated for Tcl 8.6. Your fix is partly correct, but setting the interp result has to go through the appropriate tcl-api to set the result (e.g. Tcl_SetResult). I am not eager for fixing this, since we don't use the driver at all.

Actually, we are using on all production environments still Tcl 8.5, also OpenACS recommends still Tcl 8.5. There are no known issues with Tcl 8.6, just a thorough testing is missing.

all the best
-gustaf neumann

[1] https://bitbucket.org/naviserver/nsssl
[2] http://sourceforge.net/projects/naviserver/files/naviserver/4.99.6/naviserver-4.99.6-modules.tar.gz/download

Collapse
Posted by Bjoern Kiesbye on
Hi Jim,

I think the tcl8.6 API (http://www.tcl.tk/man/tcl8.6/TclLib/SetResult.htm). should be used to manipulate the interp result, too. Even so I used the solution from nspostgres where a reference to the interp's result string is requested, using the API, and assigned to a local variable, which is then passed to sprintf instead of passing interp->result directly. I think the way it is currently done, partly defeats the purpose of the API, because the interp objects internal state is manipulated from outside code. I did use it because it was a 'known to work' solution, which i needed, and was not certain that there isn't a good reason why it is done this way.

The use of sprintf is original code from nsopenssl, I just changed the variable which is passed to it. Besides the manipulation of interp->result (assuming they will be replaced by calls to the API), sprintf is used twice more to create the channelName, and once to write to buf.

Collapse
Posted by Bjoern Kiesbye on
Dear Gustaf,

thank you for the information, I was just needing nsopenssl for a Development Server and thought the fix might be help full for others, even so it's not a perfect solution, it's a working one. On live systems i use nginx in front of Aolserver, besides other reasons to handle ssl client requests.

Collapse
Posted by Jim Lynch on
"Use of sprintf is original code" Doesn't make it safe. Yes, as I mentioned, I found that too, and without limiting how many bytes it writes, it has the potential of overwriting the buffer it uses. Assume, for example, that someone changes what's written by some sprintf, without knowing the size of the buffer, this has the potential to overwrite its buffer as well, especially if the sprintf arguments cause it to write more. It doesn't make for good code, it makes for the continued ignorance of an existing bug which should be fixed. I'm just trying to make sure that bugs that could cause security issues are removed and fixed, not kept, and being original code is not a reason to keep a problem.

In the case of writing integers to the tcl result, you don't have to concern yourself with either sprintf or snprintf, you can use other tcl api calls to set the result to the integer directly, and this is what is recommended.

I'll help you in any way I can to make sure these bugs no longer plague us. Would you like me to show the calls that involve writing integers to the tcl result?

Collapse
Posted by Bjoern Kiesbye on
Hello Jim,

you are right, snprintf is the preferred way, i did use that too when i was writing a simple custom gzip_http_post method. With original code I just meant, it's know to work code,and not new code introduced by the patch, which may lead to problems in case the patch is applied.
As well i was uncertain if fixing this is going to be appreciated at all, because development efforts seem to go more into NaviServer rather then Aolserver. Hearing that there is still interest i can make the changes, and would appreciated some guidance.

First Question is there a preferred way to set the interp result, as there are 2 ways Tcl_SetResult/Tcl_SetObjResult and Tcl_AppendResult. Currently the result is set to a single string, to replace it by the new API I would go with.

Example Integer:

sprintf(interp->result, "%d", nread);

is replaced by

Tcl_SetObjResult(interp, Tcl_NewIntObj(nread));




Example String(1):

interp->result = "could not register callback";

is replaced by

Tcl_SetResult(interp, "could not register callback", TCL_STATIC);



Example String(2):

sprintf(interp->result, "%s", Tcl_GetChannelName(chan));

is replaced by

Tcl_SetResult(interp, Tcl_GetChannelName(chan), TCL_VOLATILE);

Thanks Bjoern

Collapse
Posted by Jim Lynch on
Yes, -and- these API calls should work in tcl-8.5 as well.

-Jim

Collapse
Posted by Bjoern Kiesbye on
Hi Jim,

I updated the patch to use the new tcl API, at one point I had to use the Function Tcl_ObjPrintf, which is available in tcl8.5/.6 only, As the function is not available in tcl8.4 I wrapped it in a #if which checks the tcl version, and in case a the tcl version is 8.4 or lower snprintf is used instead.

Example:

+#if (TCL_MAJOR_VERSION >= 8) && (TCL_MINOR_VERSION > 4 )
+ Tcl_SetObjResult(interp, Tcl_ObjPrintf("%lu", peercert == NULL ? 0 : X509_get_version(peercert) + 1));
+#else
+ snprintf(&buf, sizeof(buf), "%lu", peercert == NULL ? 0 : X509_get_version(peercert) + 1);
+ Tcl_SetResult(interp, buf, TCL_VOLATILE);
+#endif

http://www.clever-devel.com/file/35043/nsopenssl-interp-result-0.1.1.patch

-bjoern