Forum OpenACS Development: Attempt to use a nonstandard variable name

Collapse
1: Attempt to use a nonstandard variable name
Posted by Iuri Sampaio on 01/28/20 10:37 PM

Hi there,

I'm trying to send a POST request within a JSON structure in the body. In a Postman client, I've set the method as POST, format as "raw" and added the following chunk to the body

{
  "sequence_id": "",
  "name": "SQ001",
  "description": "Cliente/Estado/Material"
}

URL's https://backend.tangotechapp.com/REST/addSequenceRequest

In the server side, I get the following warning. What would be the proper way to receive that request in JSON format?

Best wishes, I

Warning: ad_page_contract: attempt to use a nonstandard variable name in form.  {
    "sequence_id": "",
    "name": "SQ001",
    "description": "Cliente/Estado/Material"
}  
    called from template::adp_parse /var/www/tangotech//packages/tt-rest/www/addSequenceRequest {}
    called from adp_parse_ad_conn_file
    called from rp_serve_concrete_file /var/www/tangotech/packages/tt-rest/www/addSequenceRequest.tcl
    called from rp_serve_abstract_file 0 0 .* /var/www/tangotech/packages/tt-rest/www/addSequenceRequest
    called from rp_handler
        POST https://backend.tangotechapp.com/REST/addSequenceRequest? referred by '' peer 172.31.40.33 user_id 0
        post-data: {
    "sequence_id": "",
    "name": "SQ001",
    "description": "Cliente/Estado/Material"
}

[28/Jan/2020:19:06:27][17364.7f9665fd9700][-conn:tangotech:0:281-] Notice: CONN t0 
 *****
[28/Jan/2020:19:06:27][17364.7f9665fd9700][-conn:tangotech:0:281-] Notice: QUERY 
    {
    "sequence_id": "",
    "name": "SQ001",
    "description": "Cliente/Estado/Material"
} = (null)
Collapse
2: Re: Attempt to use a nonstandard variable name (response to 1)
Posted by Iuri Sampaio on 01/28/20 10:51 PM
Request works fine to the format application/x-www-form-urlencoded

However, when we switch content-type to application/json, body seems to get lost.

ns_conn headers and ns_conn query show empty values.

Collapse
3: Re: Attempt to use a nonstandard variable name (response to 2)
Posted by Iuri Sampaio on 01/29/20 02:52 AM
... and why, in OpenACS, do I always have the feeling that I don't even know how to ask about a problem?
Best wishes,
I
Collapse
4: Re: Attempt to use a nonstandard (response to 3)
Posted by Gustaf Neumann on 01/29/20 08:28 AM
A couple of answers:
  • when using ad_page_contracts, the parameters have to be either query parameter (in the URL) or form parameter (application/x-www-form-urlencoded). In your example, ad_page_contract tries to interpret the JSON data as form parameters. Probably ad_package_contract should be extended to provide a better error messages when receiving POST requests with unexpected content types.
  • in theory ad_page_contract could be extended to handle also JSON data, but that would require a JSON parser, it would be necessary to handle tree-structured data (like query or form parameters), and many possible error condition have to handled, etc.

Therefore, you have to handle the incoming JSON data via custom pages (without page contracts).
The proper way to handle this is to check in your page
- is this a POST request
- if yes, get the body with ns_getcontent [1] and
- parse the content with a JSON parse
- provide custom error messages in case JSON parsing fails.

[ns_conn headers] can be used to obtain the set of the HTTP request header fields, which you can use to obtain the content type of the post request. [ns_conn query] can be used, to obtain the query parameters of the HTTP request. You should not expect the body of an HTTP request returned by these commands.

hope, this helps
-gn

[1] https://naviserver.sourceforge.io/n/naviserver/files/ns_getcontent.html

Collapse
5: Re: Attempt to use a nonstandard variable name (response to 3)
Posted by Brian Fenton on 01/29/20 11:37 AM
Don't worry Iuri, you're not alone! It's not easy being an OpenACS programmer these days 😊

Brian

Collapse
6: Re: Attempt to use a nonstandard (response to 5)
Posted by Gustaf Neumann on 01/29/20 01:34 PM

i beg to differ: it was never as simple as now to process JSON data in OpenACS. For a curl-request like:

curl --header "Content-Type: application/json" --data '{"username":"xyz","password":"xyz"}' http://localhost:8100/json-receiver

the implementation below does all the "magic" i have described above. This works for compress/uncompressed transmission, for huge data (several GB) or little data without bloating the server footprint, etc.

-g

json-receiver.tcl:

if {[ns_conn method] eq "POST"} {
    package req json

    set dict [json::json2dict [ns_getcontent -as_file false]]
    # 
    # Do something with the dict
    # 
    ns_return 200 text/plain $dict\n
    
} else {
    ad_return_complaint 1 "unsupported HTTP method: [ns_conn method]"
}
Collapse
7: Re: Attempt to use a nonstandard variable name (response to 6)
Posted by Brian Fenton on 01/29/20 01:42 PM
My apologies, Gustaf, I wasn't making a technical point, rather a social point. I was referring to Iuri's comment (no 3) about not knowing how to ask about a problem. Many of us who have been here since the early days miss the time when the community was larger.

Brian

Collapse
8: Re: Attempt to use a nonstandard variable name (response to 6)
Posted by Iuri Sampaio on 01/29/20 04:22 PM
Thanks Gustaf,
I'll amend one of the Tcl's scripts with that chunk json-receiver.tcl
Then I'm going to be able to understand how it works, and actually to measure/evaluate the amount of work that will be in order to parse those JSONs.
As I'm following json.org always, I believe it will work like a charm.

However, not using ad_page_contract freaks me out. In the past, I was told to never forget ad_page_contract in my .tcl scripts. It was taken for granted and now I follow it as a premise.

Thanks Brian!

Best wishes,
I