Forum OpenACS Development: Iteractive console for XOTcl, any takers?

It would be interesting to have an interactive console (similar to TkCon in Tcl/Tk world) for OpenACS development. I know there is the Developer Support/OpenACS Shell, but it is very cumbersome to use for xotcl or xowiki testing. I would like to have interactive introspection at CR-stored classes, compute contets for fragments, etc.

Do you have ideas on how to code such a beast?

Collapse
Posted by Gustaf Neumann on
Dear rildo,

implementing something like TkCon for OpenACS means essentially to re-implement TkCon with an Ajax front end. To get a stable state at the backend, one needs a long-running connection thread keeping the connection open (this should be possible with a streaming ajax interface) or to use nscp. This is not an easy task and requires good networking and ajax + javascript skills.

However, there are easier things to achieve better interaction than ds/shell with little work. I use requently nscp with rlwap

rlwrap telnet localhost 9999
where rlwarp provided command line history.
http://freshmeat.net/projects/rlwrap/

Another option is to use telnet localhost 9999 in emacs from the shell mode (in emacs, type in the command buffer shell). Emacs provides as well command line history, all interactions with the aolserver are in an edit-buffer... maybe, one can extend the command completion in emacs with not much effort.

Hope this helps
-gustaf neumann

Collapse
Posted by Rildo Pragana on
Dear Gustaf,

Thank you for your prompt answer.

Anyway, I found a way to access an openacs thread without any changes to tkcon. If I open an openacs shell and input the code below, it will open a server socket which may be attached to a tkcon (even in a remote machine, of course). The code is mostly adapted from tkconclient. It will rest connected for indefinite time, althout the web client (firefox, for instance) will report a time out. Doesn't matter. Tkcon will keep connected to aolserver thread and will be able to introspect everything and interact with the interpreter. That's exactly what I wanted!

There is just one thing I have to do. Integrate this code in Developer Support, so I don't have to cut-n-paste in the openacs shell everytime... I have to learn a little yet on how to do this.

Here is the code (paste into openacs shell, press "ok", run tkcon and attach it to socket <your_server>,port 8765):

 namespace eval ::tkconclient {
    variable script ""
    variable server ""
    variable socket ""
    variable wait ""
	namespace export start stop
    proc start {port {myaddr localhost}} {
	variable socket
	variable server
	if {$socket ne "" || $server ne ""} stop
	set server [socket -server [namespace current]::accept \
			-myaddr $myaddr $port]
    }
    proc stop {} {
	variable server
	if {$server ne ""} {
	    closesocket
	    close $server
	    set server ""
	}
    }
    proc closesocket {} {
	variable socket
	catch {close $socket}
	set socket ""
	# Restore [puts]
	rename ::puts ""
	rename [namespace current]::puts ::puts
    }
    proc accept {sock host port} {
	variable socket
	fconfigure $sock -blocking 0 -buffering none
	if {$socket ne ""} {
	    puts $sock "Only one connection at a time, please!"
	    close $sock
	} else {
	    set socket $sock
	    fileevent $sock readable [namespace current]::handle
	    # Redirect [puts]
	    rename ::puts [namespace current]::puts
	    interp alias {} ::puts {} [namespace current]::_puts
	}
    }
    proc handle {} {
	variable script
	variable socket
	if {[eof $socket]} {
	    closesocket
	    return
	}
	if {![catch {read $socket} chunk]} {
	    if {$chunk eq "bye\n"} {
		puts $socket "Bye!"
		closesocket
		return
	    }
	    append script $chunk
	    if {[info complete $script]} {
		catch {uplevel "#0" $script} result
		if {$result ne ""} {
		    puts $socket $result
		}
		set script ""
	    }
	} else {
	    closesocket
	}
    }
    ## This procedure is partially borrowed from tkcon
    proc _puts args {
	variable socket
	set len [llength $args]
	foreach {arg1 arg2 arg3} $args { break }

	switch $len {
	    1 {
		puts $socket $arg1
	    }
	    2 {
		switch -- $arg1 {
		    -nonewline - stdout - stderr {
			puts $socket $arg2
		    }
		    default {
			set len 0
		    }
		}
	    }
	    3 {
		if {$arg1 eq "-nonewline" &&
		    ($arg2 eq "stdout" || $arg2 eq "stderr")} {
		    puts $socket $arg3
		} elseif {($arg1 eq "stdout" || $arg1 eq "stderr") \
			      && $arg3 eq "-nonewline"} {
		    puts $socket $arg2
		} else {
		    set len 0
		}
	    }
	    default {
		set len 0
	    }
	}
	## $len == 0 means it wasn't handled above.
	if {$len == 0} {
	    global errorCode errorInfo
	    if {[catch [linsert $args 0 puts] msg]} {
		regsub tkcon_tcl_puts $msg puts msg
		regsub -all tkcon_tcl_puts $errorInfo puts errorInfo
		return -code error $msg
	    }
	    return $msg
	}
    }
 }

::tkconclient::start 8765 [ns_info address]
vwait ::tkconclient::wait
::tkconclient::stop
ReturnHeaders
ns_write "Remote Tkcon connection server stopped!"

Collapse
Posted by Gustaf Neumann on
This is pretty cool!

The only problem i see is the vwait, which should not used in the aolserver connection threads. If there is more traffic, the server easily becomes deaf and "hangs"; the problem might have to do with multiple concurrent vwaits, so maybe running one tkcon-client this way is ok).

maybe i have on the weekend some time to sketch a version without vwait.... keep us informed, what you are doing!

-gustaf neumann

Collapse
Posted by Rildo Pragana on
Hi Gustaf,

Ok, I agree with you that "vwait" will keep the thread alive. I think this is not a good thing for serving requests (as there are a lot of requests coming, which would keep many threads alive...), but in a single thread I can't see a problem (we can't have 2 tkcons at the same time, anyway).
There is a (potentially serious) problem yet with that code., though. If we do not answer the web browser, it will keep trying to connect again and again, so we'll have many threads starting. There are several ways for bypassing that. Please wait a little more and I'll get the best 😊

For now, a simple way to stop this problem is to press the "stop" button in the browser (firefox), so it will not wait for the answer.
I have measured the performance (despite vwait keeping that thread alive) and didn't notice any stall or anything else. In fact I can even see the sockets, "llength [nsv_names]", etc., at the tkcon. Please, explain me better why the vwait may be troublesome.

Well, time to work. Let me polish this crude idea, so we get a nice debugging tool (also suitable for teaching), because everyone likes interactivity!

Thank you for your motivation! I'll keep you informed on this and other results I get. I'm starting to implement a kind of e-commerce application with oacs & friends, but I'm still learning how to program in xotcl/xowiki.

BTW, many thanks also for those nice libraries. I'm really amazed with them, and XoWiki alone convinced me to try OpenACS.

greetings from Recife, brazilian's Venice,
Rildo Pragana

Collapse
Posted by Gustaf Neumann on
Dear Rildo,

The problem with vwait is not, that one connection thread "hangs", but - as i learned the hard way - the whole server can hang and does not accept any kind of new requests. This happens only with a certain load and seems to be a problem with event handling in the aolserver. This problem caused me to rewrite the http client code from xotcl to be usable in aolserver.

Below is a version of the adapter, which does not use vwait at all. One can create multiple threads, so several developers can connect to the same server from their tkcons (using different server ports).

http://alice.wu-wien.ac.at:8000/xowiki/tkcon

all the best
-gustaf neumann

Collapse
Posted by Rildo Pragana on
Dear Gustaf,

Ok, I got it. Please, tell me how to start the service. I have put it in &lt;server&gt;/www/tkcon-gustaf.tcl, but when I start it from the browser, it throws an error:

is not under the path root (/var/lib/aolserver/service0)
while executing
"error "$path is not under the path root ([acs_root_dir])""
(procedure "ad_make_relative_path" line 6)
invoked from within
"ad_make_relative_path [info script]"
(procedure "ad_library" line 3)
invoked from within
"ad_library {
...

Please, help me as I'm still a newbie in oacs and xotcl/xowiki (though, I have worked with tcl/tk for some years).

Another question: does this approach keep the connection with the webbrowser live? (many procs don't work as expected if there is no connection)

I have made some tests with another tkcon-remote interface (see it at http://pragana.net/tkcon-remote.tcl). If I put this in /my_server_path/www/tkcon-remote.tcl and open the url in Firefox, it will start, avoid double starting and keep the connection (with the browser) active at the server side. This was achieved by a simple javascript "trick" on the client.

Then I may type the following in a tkcon console:

::xowiki::Package initialize -parameter {{-m view} {-folder_id:integer 0}}
foreach {v x} {url /xowiki object "" package_key xowiki package_url /xowiki} { ::$package_id set $v $x }
::$package_id invoke -method view

It will return the html page for the main xowiki instance.

best regards,
Rildo Pragana

Collapse
Posted by Gustaf Neumann on
Put the file http://alice.wu-wien.ac.at:8000/xowiki/tkcon e.g. in packages/xotcl-core/tcl/tkcon-procs.tcl and start the first console by typing in e.g. tkcon start via ds/shell. The best would be to create an own package (tkcon), define package parameters for the ports and hosts allowed to connect, and make a manage-page in (e.g. in tkcon/www/index.tcl), which shows, which consoles are currently used (from which host) and which are free, and allows a user to grab one....

"many procs don't work as expected without connection": yes, this is a classical problem of openacs + aolserver, since many scripts expect to be able to obtain connection specific information (user_id, url, query-parms, ...) from a connection thread. If these commands are issued from a background thread, many subcommands of ns_conn terminate with a tcl-error, which causes the current eval to terminate. xotcl-core + xowiki try to be more robust against these problems than plain openacs. However, it becomes also for xowiki a problem, when common openacs code is used, e.g. the master templates.

I did today some more work in this regard; please update xotcl-core and xowiki from cvs head to get the code, which is more robust in this respect. Get as well the updated version of tkcon-procs.tcl from http://alice.wu-wien.ac.at:8000/xowiki/tkcon). With the new version one can run e.g.

::xowiki::Package initialize -url /xowiki -actual_query "" -user_id 0
$package_id invoke -method view

from tkcon. The basic idea of the Package.initialize is to be able to pass connection specific information to the connection context.

It is true, by running via the approach via ds/shell + vwait, has one problem less, since in ds/shell there is a connection, and aolserver does not barf on connection related commands. However, one gets wrong information from ns_conn (e.g. the url is ds/shell), so this is not good either.

Also the new version with the separate thread has limitations, when e.g. form-variables are used. These could be set via the associative array form_parameter in ::xo::cc, but i have not tested that...