Forum OpenACS Q&A: Tcl client-side cookie support?

Request notifications

Collapse
Posted by Andrew Piskorski on
Anyone have Tcl code to let a client-side Tcl script properly accept cookies from the server, and send them back?

This is for web scraping. Some sites check for or set various cookies and refuse to show you a page (even an otherwise public page) if cookies are not supported. So in many cases simply being able to return statically defined cookies would be adequate, but ideally the Tcl scraping script should be able to accept arbitrary cookies from the server as well.

I thought this might be a standard part of tcllib or some other package, but I haven't found anything on the net. (cgi.tcl might have something related to this, but I couldn't really tell for sure, and I suspect not. It's intended for server-side CGIs, after all.) I could roll my own, but it seems unlikely that someone somewhere hasn't already done this...

This comp.lang.tcl thread does mention somebody doing it, but not providng any code, and there's a feature request open for this.

Collapse
Posted by Tilmann Singer on
Try http://tclwebtest.sourceforge.net - it should deal with cookies transparently.
Collapse
Posted by Andrew Piskorski on
Thanks Tilmann, that looks like it will do the trick. Btw, the regexp in proc set_cookie is bad. I have not tested it against any other cookies but the one that was giving me trouble, but this should be an improvement:

    # Cookies spec. is here:
    #   http://wp.netscape.com/newsref/std/cookie_spec.html
    # Some examples of actual cookies seen on the web follow:
    # --atp@piskorski.com, 2003/05/09 00:22 EDT
    #
    # 'cookietest=1; expires=Mon, 09-May-2033 04:18:36 GMT; path=/'

    if { [regexp -nocase {expires\s*=\s*([^;]*)} $set_cookie_string match expires] } {

Also, I noticed you don't have any tag in CVS for your tclwebtest 0.2 release (nor for 0.1d for that matter). (Which always peeves me.) If you get a chance, I recommend going back and tagging CVS so it's clear what versions of files went into your 0.2 release...

Collapse
Posted by Tilmann Singer on
Thanks for the hint, I finally committed it now.

The 0.2 version is not tagged in cvs, but at least the recently released tclwebtest 0.3 is (and future versions will be).