Forum OpenACS Q&A: tcllib nntp

1: tcllib nntp

Posted by James Thornton on 07/02/02 05:51 PM

I am going to start developing a 4.5-based Usenet app, but I need some help sorting out how to use the tcllib nntp object in the AOLsever/OpenACS namespace. For reference, I posted the tcllib nntp code to: http://jamesthornton.com/try/nntp.tcl.txt -- I placed this in my shared Tcl library. If I place the following in a standard Tcl page, I can query the newsgroup, buy only once.

set NH [::nntp::nntp "" "" news_handle]
$NH authinfo "username" "password"

set list [$NH listgroup "comp.lang.tcl"]

foreach msgid $list {

    append articles [$NH article $msgid]
    

}

Immediately reloading the page causes an error (it works again if you wait a while)....

[01/Jul/2002:10:28:59][1294.103429][-conn4-] Error: command
"news_handle" already exists, unable to create nntp connection
command "news_handle" already exists, unable to create nntp connection
    while executing
"error "command "$name" already exists, unable to create nntp
connection""
    (procedure "::nntp::nntp" line 20)
    invoked from within
"::nntp::nntp "" "" news_handle"
    invoked from within
"set NH [::nntp::nntp "" "" news_handle]"
    (file "/web/james/www/try/nntp.tcl" line 10)
    invoked from within
"source $ad_conn(file)"
    (procedure "ad_handle_abstract_url" line 67)
    invoked from within
"ad_handle_abstract_url cns29 {}"

Putting the creation of the nntp object in a shared proc (as a nsv_set), causes this error when you load the page...

[01/Jul/2002:09:16:55][1193.4101][-conn0-] Error: can not find channel
named "sock13"
can not find channel named "sock13"
    while executing
"puts $sock "$cmd""
    (procedure "::nntp::cmd" line 9)
    invoked from within
"::nntp::cmd news_handle {AUTHINFO USER username}"
    ("eval" body line 1)
    invoked from within
"eval [list ::nntp::cmd $name] $args"
    (procedure "::nntp::command" line 2)
    invoked from within
"::nntp::command $name "AUTHINFO USER $user""
    (procedure "::nntp::_authinfo" line 5)
    invoked from within
"::nntp::_authinfo news_handle username password"
    ("eval" body line 1)
    invoked from within
"eval [list ::nntp::_$cmd $name] $args"
    (procedure "::nntp::NntpProc" line 20)
    invoked from within
"$NH authinfo "username" "password""
    (file "/web/james/www/try/nntp.tcl" line 11)
    invoked from within
"source $ad_conn(file)"
    (procedure "ad_handle_abstract_url" line 67)
    invoked from within
"ad_handle_abstract_url cns1 {}"

2: Response to tcllib nntp (response to 1)

Posted by Dan Wickstrom on 07/02/02 08:25 PM

The news package appears to be using a pseudo object-oriented style, so while it appears that you're creating a news handle, what you're really doing is creating a fixed command called news_handle which is used to call the news package commands. If this were a true oo implementation, it wouldn't be a problem to create multiple instances, but since all you're really doing is creating a news_handle proc, the second attempt causes an error. This case could be made to work by adding a test to see if the news handle already exists:

	if ![string length [info commmands news_handle]] {
		set NH [::nntp::nntp "" "" news_handle]
	}
	news_handle authinfo "username" "password"

For the second case, the act of storing the news handle in an nsv_set causes the news handle to be released when the underlying Tcl_Obj type is converted to a string for storage in the nsv_set. nsv_sets only store strings, so there isn't really anything that you can do to make this work.

3: Response to tcllib nntp (response to 1)

Posted by James Thornton on 07/03/02 07:44 AM

Thanks Dan. I have learned that if you don't specify a name for the connection, it will create a new 'unique' name in the form of nntp0, nntp1, nntp3, etc. This allows for reloading & serving up of multiple pages because if nntp0 is taken, it will create nntp1. There is also a "quit" command that "quits the nntp session, closes the socket, and deletes the command that was created for the connection."

Placing this code in a standard Tcl page works reasonably well:

# open a socket connection to th NNTP server and 
# create a new nntp object with an associated global Tcl command 
set NH [::nntp::nntp]

# send authentication information to the server 
$NH authinfo "username" "password"

# query the server for the server's current date
set value [$NH date] 

# gracefully close the connection 
# after sending a NNTP QUIT command down the socket
$NH quit

However, it's not perfect -- it you keep hitting reload fast enough, something happens to cause the next connection attempts to timeout (if you wait a while, it will start working again):

Error: couldn't open socket: connection timed out
couldn't open socket: connection timed out
    while executing
"socket $data(host) $data(port)"
    (procedure "::nntp::nntp" line 62)
    invoked from within
"::nntp::nntp $news"
    invoked from within
"set NH [::nntp::nntp $news]"
    (file "/web/james/www/try/nntp.tcl" line 13)
    invoked from within
"source $ad_conn(file)"
    (procedure "ad_handle_abstract_url" line 67)
    invoked from within
"ad_handle_abstract_url cns206 {}"

4: Response to tcllib nntp (response to 1)

Posted by Dan Wickstrom on 07/03/02 03:43 PM

You're opening a new socket connection to the news server each time you get a new news handle, so even though you're calling quit in the news handle, each of the sockets is in a TIME_WAIT state. If you use something like netstat or lsof, you should be able to see that you have a bunch of socket connections open to the news server when you get the error. Once the TIME_WAIT state expires, you're then able to reopen new connections.
<p>
To fix this, you need to reuse the news handle when you source the page again.
<p>
Try something like the following:
<p>
<pre>
    global NH
    if ![info exists NH] {
        set NH [::nntp::nntp]
    }
    $NH authinfo "username" "password"
</pre>
<p>
Using this approach, the news handle should stick around until the connection thread is cleaned up and the interpreter is deallocated, so if the server is not idle, the same news handle will be used for multiple connections.

5: Response to tcllib nntp (response to 1)

Posted by James Thornton on 07/04/02 01:33 AM

To fix this, you need to reuse the news handle when you source the page again.

Dan - With the following code in a standard Tcl page, I can see that NH is reset with every page load -- its value will keep incrementing as nntp1...nntp17, nntp18, etc. Aren't global variables are only global inside the context of a page, thus losing their values after the page loads?

global NH

if {![info exists NH]} {
    set NH [::nntp::nntp]
    set handle_html "Reseting handle to: $NH
"
}

set date [$NH date]

ns_write "
$handle_html
Date: $date
"

6: Response to tcllib nntp (response to 1)

Posted by Dan Wickstrom on 07/04/02 02:34 PM

Yes my mistake. Lately I've been doing some work on nsjava where I use the errorCode to store an exception object when an error occurs. It appears that the errorCode is not cleaned up until the thread exits and deallocates the interpreter. However for global variables in general, the interpreter cleans those up when the connection is closed. In addition, all channels (sockets, files, etc) are also closed when the connection is closed, so it doesn't appear that there is a simple fix for your problem.

It seems that you need a separate thread, indpendent of the connections that you can use to service your news connection, so I would try putting your news service in its own thread using "ns_thread begindetached". You could then use nsv_sets and mutexs to send data to/from the detached thread and synchronize access. This approach is a bit more complicated, but having the news service in its own thread should allow you to have a single news handle and reuse it by proxy from the various connection threads.