http-client-procs.tcl

util::get_http_status (public)

 util::get_http_status [ -url url ] [ -use_get_p use_get_p ] \
    [ -timeout timeout ]

Switches:

-url (optional)
-use_get_p (optional, defaults to "1")
-timeout (optional, defaults to "30")

Returns:
the HTTP status code, e.g., 200 for a normal response or 500 for an error, of a URL. By default this uses the GET method instead of HEAD since not all servers will respond properly to a HEAD request even when the URL is perfectly valid. Note that this means that the server may be sucking down a lot of bits that it doesn't need.

Partial Call Graph (max 5 caller/called nodes):

Testcases:
No testcase defined.

util::http::available (public)

 util::http::available [ -preference preference ] [ args... ]

Return the preferred HTTP API among those available based on preference and OpenACS installation capabilities.

Switches:

-preference (optional, defaults to "native curl")
decides which available implementation prefer in respective order. Choice is between 'native', based on ns_http api, available for NaviServer only and giving the best performances and 'curl', which wraps the command line utility (available on every system with curl installed).

Partial Call Graph (max 5 caller/called nodes):

Testcases:
No testcase defined.

util::http::basic_auth (public)

 util::http::basic_auth [ -headers headers ] -username username \
    -password password

Builds BASIC authentication header for an HTTP request

Switches:

-headers (optional)
ns_set of request headers that will be populated with auth header. If not specified, a new ns_set will be created. Existing header for BASIC authentication will be overwtitten.
-username (required)
Username for authentication
-password (required)
Password for authentication

Returns:
ns_set of headers containing authentication data

Partial Call Graph (max 5 caller/called nodes):

Testcases:
No testcase defined.

util::http::cookie_auth (public)

 util::http::cookie_auth [ -headers headers ] [ -auth_vars auth_vars ] \
    [ -auth_url auth_url ] [ -auth_form auth_form ] \
    [ -auth_cookies auth_cookies ] [ -preference preference ]

This proc implements the generic pattern for cookie-based authentication: user logs in a webpage providing username, password and optionally other information in a form, page replies generating one or more authentication cookies by which user will be recognized on subsequent interaction with the server. By this method was possible, for example, to authenticate on a remote OpenACS installation providing 'email' and 'password' as credentials to the /register/ page, and using 'ad_session_id' and 'ad_user_login' as 'auth_cookies'. This proc is a bit hacky and is nowadays not clear if it makes sense anymore... This proc takes care to submit to the login form also every other formfield on the login page. This is important because this (often hidden) formfields can contain tokens necessary for the authentication process.

Switches:

-headers (optional)
ns_set of request headers that will be populated with auth headers. If not specified, a new ns_set will be created. Existing cookies will be overwritten.
-auth_vars (optional)
Variables issued to the login page in 'export_vars -url' form.
-auth_url (optional)
Login url
-auth_form (optional)
Form to put our data into. If not specified, there must be only one form on the login page, otherwise proc will throw an error.
-auth_cookies (optional)
Cookies we should look for in the response from the login page to obtain authentication data. If not specified, this will refer to every cookie received into 'set-cookie' response headers.
-preference (optional, defaults to "native curl")

Returns:
ns_set of headers containing authentication data

Partial Call Graph (max 5 caller/called nodes):

Testcases:
No testcase defined.

util::http::get (public)

 util::http::get [ -url url ] [ -headers headers ] [ -timeout timeout ] \
    [ -max_depth max_depth ] [ -force_ssl ] [ -gzip_response ] \
    [ -spool ] [ -preference preference ]

Issue an HTTP GET request to 'url'.

Switches:

-url (optional)
-headers (optional)
specifies an ns_set of extra headers to send to the server when doing the request. Some options exist that allow one to avoid the need to specify headers manually, but headers will always take precedence over options.
-timeout (optional, defaults to "30")
Timeout in seconds. The value can be an integer, a floating point number or an ns_time value.
-max_depth (optional, defaults to "10")
-force_ssl (optional, boolean)
specifies whether we want to use SSL despite the url being in http:// form. Default behavior is to use SSL on https:// URLs only.
-gzip_response (optional, boolean)
informs the server that we are capable of receiving gzipped responses. If server complies to our indication, the result will be automatically decompressed.
-spool (optional, boolean)
enables file spooling of the request on the file specified. It is useful when we expect large responses from the server. The result is spooled to a temporary file, the name is returned in the file component of the result.
-preference (optional, defaults to "native curl")
decides which available implementation prefer in respective order. Choice is between 'native', based on ns_ api, available for NaviServer only and giving the best performances and 'curl', which wraps the command line utility (available on every system with curl installed).

Returns:
the data as dict with elements 'headers', 'page', 'file', 'status', 'time' (elapsed request time in ns_time format), and 'modified'.

Partial Call Graph (max 5 caller/called nodes):

Testcases:
util_http_json_encoding, postman_echo

util::http::post (public)

 util::http::post [ -url url ] [ -files files ] [ -base64 ] \
    [ -formvars formvars ] [ -formvars_list formvars_list ] \
    [ -body body ] [ -max_body_size max_body_size ] \
    [ -headers headers ] [ -timeout timeout ] [ -max_depth max_depth ] \
    [ -force_ssl ] [ -multipart ] [ -gzip_request ] [ -gzip_response ] \
    [ -post_redirect ] [ -spool ] [ -preference preference ]

Implement client-side HTTP POST request.

Switches:

-url (optional)
-files (optional)
File upload can be specified using actual files on the filesystem or binary strings of data using the '-files' parameter. '-files' must be a dict (flat list of key value pairs). Keys of '-files' parameter are: - data: binary data to be sent. If set, has precedence on 'file' key - file: path for the actual file on filesystem - filename: name the form will receive for this file - fieldname: name the field this file will be sent as - mime_type: mime_type the form will receive for this file If 'filename' is missing and an actual file is being sent, it will be set as the same name as the file. If 'mime_type' is missing, it will be guessed from 'filename'. If result is */* or an empty mime_type, 'application/octet-stream' will be used If '-base64' flag is set, files will be base64 encoded (useful for some kind of form).
-base64 (optional, boolean)
-formvars (optional)
These are additional form variables already in URLencoded format, for instance, by using 'export_vars -url'. They will be translated for the proper type of form (URLencoded or multipart) depending on the presence of 'files' or the 'multipart' flag. Variables specified this way will be appended to those supplied via the 'formvars_list' parameter.
-formvars_list (optional)
These are additional form variables in list format. They will be translated for the proper type of form (URLencoded or multipart) depending on the presence of files or the multipart flag. The payload will be made by the sum of data coming from 'formvars', 'formvars_list' and 'files' arguments. Default behavior is to build payload as an 'application/x-www-form-urlencoded' payload if no files are specified, and 'multipart/form-data' otherwise. If '-multipart' flag is set, format will be forced to multipart.
-body (optional)
is the payload for the request and will be passed as is (useful for many purposes, such as webDav). A convenient way to specify form variables through this argument is passing a string obtained by 'export_vars -url'.
-max_body_size (optional, defaults to "25000000")
this value in number of characters will tell how big can the whole body payload get before we start spooling its content to a file. This is important in case of big file uploads, when keeping the entire request in memory is just not feasible. The handling of the spooling is taken care of in the API. This value takes into account also the encoding required by the content type, so its value could not reflect the exact length of body's string representation.
-headers (optional)
specifies an ns_set of extra headers to send to the server when doing the request. Some options exist that allow one to avoid the need to specify headers manually, but headers will always take precedence over options.
-timeout (optional, defaults to "30")
Timeout in seconds. The value can be an integer, a floating point number or an ns_time value.
-max_depth (optional, defaults to "10")
is the maximum number of redirects the proc is allowed to follow. A value of 0 disables redirection. When max depth for redirection has been reached, proc will return response from the last page we were redirected to. This is important if redirection response contains data such as cookies we need to obtain anyway. Be aware that when following redirects, unless it is a code 303 redirect, url and POST urlencoded variables will be sent again to the redirected host. Multipart variables won't be sent again. Sending to the redirected host can be dangerous, if such host is not trusted or uses a lower level of security.
-force_ssl (optional, boolean)
specifies whether we want to use SSL despite the url being in http:// form. Default behavior is to use SSL on https:// URLs only.
-multipart (optional, boolean)
-gzip_request (optional, boolean)
informs the server that we are sending data in gzip format. Data will be automatically compressed. Notice that not all servers can treat gzipped requests properly, and in such cases response will likely be an error.
-gzip_response (optional, boolean)
informs the server that we are capable of receiving gzipped responses. If server complies to our indication, the result will be automatically decompressed.
-post_redirect (optional, boolean)
decides what happens when we are POSTing and server replies with 301, 302 or 303 redirects. RFC 2616/10.3.2 states that method should not change when 301 or 302 are returned, and that GET should be used on a 303 response, but most HTTP clients fail in respecting this and switch to a GET request independently. This option forces this kinds of redirect to conserve their original method.
-spool (optional, boolean)
enables file spooling of the request on the file specified. It is useful when we expect large responses from the server. The result is spooled to a temporary file, the name is returned in the file component of the result.
-preference (optional, defaults to "native curl")
decides which available implementation prefer in respective order. Choice is between 'native', based on ns_ api, available for NaviServer only and giving the best performances and 'curl', which wraps the command line utility (available on every system with curl installed).

Returns:
the data as dict with elements 'headers', 'page', 'file', 'status', 'time' (elapsed request time in ns_time format), and 'modified'.

Partial Call Graph (max 5 caller/called nodes):

Testcases:
util_http_json_encoding, postman_echo, util_http_post_vars

util::http::post_payload (public)

 util::http::post_payload [ -url url ] [ -files files ] [ -base64 ] \
    [ -formvars formvars ] [ -formvars_list formvars_list ] \
    [ -body body ] [ -max_body_size max_body_size ] \
    [ -headers headers ] [ -multipart ]

Build the payload for a POST request

Switches:

-url (optional)
does not affect the payload directly, but is used to check that variables specified via the URL do not conflict with those coming from other parameters. In such case, an error is returned.
-files (optional)
File upload can be specified using actual files on the filesystem or binary strings of data using the '-files' parameter. '-files' must be a dict (flat list of key value pairs). Keys of '-files' parameter are: - data: binary data to be sent. If set, has precedence on 'file' key - file: path for the actual file on filesystem - filename: name the form will receive for this file - fieldname: name the field this file will be sent as - mime_type: mime_type the form will receive for this file If 'filename' is missing and an actual file is being sent, it will be set as the same name as the file. If 'mime_type' is missing, it will be guessed from 'filename'. If result is */* or an empty mime_type, 'application/octet-stream' will be used If '-base64' flag is set, files will be base64 encoded (useful for some kind of form).
-base64 (optional, boolean)
-formvars (optional)
These are additional form variables already in URLencoded format, for instance, by using 'export_vars -url'. They will be translated for the proper type of form (URLencoded or multipart) depending on the presence of 'files' or the 'multipart' flag. Variables specified this way will be appended to those supplied via the 'formvars_list' parameter.
-formvars_list (optional)
These are additional form variables in list format. They will be translated for the proper type of form (URLencoded or multipart) depending on the presence of files or the multipart flag. The payload will be made by the sum of data coming from 'formvars', 'formvars_list' and 'files' arguments. Default behavior is to build payload as an 'application/x-www-form-urlencoded' payload if no files are specified, and 'multipart/form-data' otherwise. If '-multipart' flag is set, format will be forced to multipart.
-body (optional)
is the payload for the request and will be passed as is (useful for many purposes, such as webDav). A convenient way to specify form variables through this argument is passing a string obtained by 'export_vars -url'.
-max_body_size (optional, defaults to "25000000")
this value in number of characters will tell how big can the whole body payload get before we start spooling its content to a file. This is important in case of big file uploads, when keeping the entire request in memory is just not feasible. The handling of the spooling is taken care of in the API. This value takes into account also the encoding required by the content type, so its value could not reflect the exact length of body's string representation.
-headers (optional)
Processing the payload might set some request headers. Provide yours to either override the default behavior, or to merge your headers with those from the payload. The resulting headers will be returned in the dict.
-multipart (optional, boolean)

Returns:
a dict with fields 'payload', 'payload_file' and 'headers'

Partial Call Graph (max 5 caller/called nodes):

Testcases:
util_http_json_encoding, postman_echo, util_http_post_vars, template_widget_file

util::http::set_cookies (public)

 util::http::set_cookies -resp_headers resp_headers \
    [ -headers headers ] [ -cookie_names cookie_names ] \
    [ -pattern pattern ]

Extracts cookies from response headers. This is done reading every 'set-cookie' header and populating an ns_set of request headers suitable for issuing 'util::http' requests.

Switches:

-resp_headers (required)
Response headers, in a list form as returned by 'util::http' API.
-headers (optional)
ns_set of request headers that will be populated with extracted cookies. If not specified, a new ns_set will be created. Existing cookies will be overwritten.
-cookie_names (optional)
Cookie names we want to retrieve. Other cookies will be ignored. If omitted together with '-pattern' proc will include every cookie.
-pattern (optional)
Cookies which name respects this pattern as in 'string match' will be included. If omitted together with '-cookie_names' proc will include every cookie.

Returns:
ns_set of headers containing received cookies

Partial Call Graph (max 5 caller/called nodes):

Testcases:
No testcase defined.

util::link_responding_p (public)

 util::link_responding_p [ -url url ] \
    [ -list_of_bad_codes list_of_bad_codes ]

Switches:

-url (optional)
-list_of_bad_codes (optional, defaults to "404")

Returns:
1 if the URL is responding (generally we think that anything other than 404 (not found) is okay).

See Also:

util::get_http_status

Partial Call Graph (max 5 caller/called nodes):

Testcases:
No testcase defined.

http-client-procs.tcl

Procedures in this file

Detailed information

util::get_http_status (public)

util::http::available (public)

util::http::basic_auth (public)

util::http::cookie_auth (public)

util::http::get (public)

util::http::post (public)

util::http::post_payload (public)

util::http::set_cookies (public)

util::link_responding_p (public)