Forum OpenACS Development: Re: XML-RPC String limit

10: Re: XML-RPC String limit (response to 1)

Posted by Iuri Sampaio on 04/14/17 07:31 PM

[14/Apr/2017:17:29:18][3231.7fef5a0da700][-conn:1contact:1-] Notice: Running TCL script test_ws.tcl
[14/Apr/2017:17:29:18][3231.7fef5a8db700][-conn:1contact:0-] Notice: BEFORE DEBUG
[14/Apr/2017:17:29:18][3231.7fef5a0da700][-conn:1contact:1-] Warning: ns_httpopen POST http://1c.1contact.ch/RPC2/ d4 30 {<?xml version="1.0"?><methodCall><methodName>is_robot.parseJson</methodName><params><param><value><string>gtregivrwjvoiejviowvwgvwegvwgre</string></value></param><param><value><string>XXXXXXXXXX</string></value></param></params></methodCall>} is deprecated. Use 'ns_http' instead!
[14/Apr/2017:17:29:18][3231.7fef5a8db700][-conn:1contact:0-] Notice: BEFORE DEBUG
[14/Apr/2017:17:29:18][3231.7fef5a8db700][-conn:1contact:0-] Notice: Running ad_proc ix_robot.parseJson ...
[14/Apr/2017:17:29:18][3231.7fef5a8db700][-conn:1contact:0-] Notice: gtregivrwjvoiejviowvwgvwegvwgre
[14/Apr/2017:17:29:18][3231.7fef5a8db700][-conn:1contact:0-] Notice: JSON 
 XXXXXXXXXX
[14/Apr/2017:17:29:18][3231.7fef5a0da700][-conn:1contact:1-] Notice:

11: Re: XML-RPC String limit (response to 10)

Posted by Gustaf Neumann on 04/14/17 08:03 PM

Post the result for a non-working case. For the debug line

ns_log notice "xmlrpc::invoke last chars: '[string range $xml end-30 end]'"

i would expect to see "invoke last chars:" in the log file.

12: Re: XML-RPC String limit (response to 11)

Posted by Iuri Sampaio on 04/15/17 09:27 PM

I've post it before.

The logs pointed to JavaScript syntax errors within a Tcl file which is well written. I removed that JS chunk and the syntax errors still appears but in a different part of the same file.

Notice: error "Entity parsing error" at position 2006
" + '?locale='+l+'&return_url='+return_url;

Furthermore, as mentioned before if the string has less than 95 thousand chars the requests work just fine with no sintax errors.

####
...
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</string></value></param></params></methodCall>} is deprecated. Use 'ns_http' instead!
[15/Apr/2017:19:19:40][3231.7fef5a0da700][-conn:1contact:1-] Notice: BEFORE DEBUG string range
[15/Apr/2017:19:19:40][3231.7fef5a0da700][-conn:1contact:1-] Notice: xmlrpc::invoke last chars: ''
[15/Apr/2017:19:19:40][3231.7fef5a0da700][-conn:1contact:1-] Error: xmlrpc::invoke: error parsing request: error "Unexpected end" at position 0
""
[15/Apr/2017:19:19:40][3231.7fef5a0da700][-conn:1contact:1-] Notice: checking entry 127.0.0.1 from host_node_map -
[15/Apr/2017:19:19:40][3231.7fef5a0da700][-conn:1contact:1-] Warning: ignore untrusted host header field: '127.0.0.1:8000'
[15/Apr/2017:19:19:40][3231.7fef5a0da700][-conn:1contact:1-] Notice: ignore non-existing or untrusted host header, fall back to 1c.1contact.ch
"xml_doc_free $doc"
(procedure "xmlrpc::invoke" line 40)
invoked from within
"xmlrpc::invoke $content"
("uplevel" body line 15)
invoked from within
"uplevel {
# /packages/xml-rpc/www/index.tcl
ad_page_contract {
Accept XML-RPC POST requests and processes them. GET requests are shown
lin..."
(procedure "code::tcl::/var/www/1contact/packages/xml-rpc/www/index" line 2)
invoked from within
"code::tcl::$__adp_stub"
("uplevel" body line 12)
invoked from within
"uplevel {

if { [file exists $__adp_stub.tcl] } {

# ensure that data source preparation procedure exists and is up-to-date
..."
(procedure "adp_prepare" line 2)
invoked from within
"adp_prepare"
invoked from within
"template::adp_parse [file rootname [ad_conn file]] {}"
(procedure "adp_parse_ad_conn_file" line 6)
invoked from within
"$handler"
("uplevel" body line 2)
invoked from within
"uplevel $code"
invoked from within
"ad_try {
$handler
} ad_script_abort val {
# do nothing
}"
invoked from within
"rp_serve_concrete_file [ad_conn file]"
(procedure "::nsf::procs::rp_serve_abstract_file" line 60)
invoked from within
"rp_serve_abstract_file "$root/$extra_url""
("uplevel" body line 2)
invoked from within
"uplevel $code"
invoked from within
"ad_try {
rp_serve_abstract_file "$root/$extra_url"
set ::tcl_url2file([ad_conn url]) [ad_conn file]
se..."
called from rp_handler
POST http://1c.1contact.ch/RPC2/? referred by '' peer 127.0.0.1 user_id 0
post-data:

X-Forwarded-For: 192.199.241.130
Host: 127.0.0.1:8000
Connection: close
Content-Length: 1000234
Accept: */*
User-Agent: NaviServer-Tcl/4.99
Content-type: text/xml

[15/Apr/2017:19:19:40][3231.7fef5a8db700][-conn:1contact:0-] Notice: error "Entity parsing error" at position 2006
" + '?locale='+l+'&return_url='+return_url;

13: Re: XML-RPC String limit (response to 12)

Posted by Gustaf Neumann on 04/16/17 12:20 PM

We see now that in the error case, the XML text passed to xmlrpc::invoke is empty (see the two single quotes in Notice: xmlrpc::invoke last chars: ''). It seems, as the xml-rpc package has problems gracefully handling such cases. I've added a test that should handle this test more gracefully to CVS [1].

What we know now is that for whatever reason, an xml-rpc request with 100k size ends up as an empty string in xmlrpc::invoke. This means, that xmlrpc::get_content (called by the www/index.tcl returns empty). I would recommend to try to replace the body of xmlrpc::get_content by the single line (when you use NaviServer):

  return [ns_getcontent -as_file false -binary false]

Let's hope this helps.
-g

[1] http://cvs.openacs.org/browse/OpenACS/openacs-4/packages/xml-rpc/tcl/xml-rpc-procs.tcl?r1=1.9&r2=1.10