Forum OpenACS Q&A: xotcl-request-monitor blocking requests

We have recently added the xotcl-request-monitor package to our site and I am interested to learn more about it.

On our site, we have a few pages that make several calls to the same resource/page with different query strings. The request monitor blocks many of these requests complaining about too many requests and sighting "Sec-Fetch-Dest".

I have two questions:

Is there a way to get the monitor to include the query string in its comparison of requests so requests with the same resource but different query strings would not appear to be the same?
Is there a resource somewhere that explains more about what and how to configure the monitoring?  

I found https://openacs.org/xotcl/show-object?show_methods=2&show_source=1&object=throttle&show_variables=1 which gives some explanation but not much guidance.

For the time being we have had to disable the package because it has rendered these pages unusable. I would greatly like to learn to use this better.

Thank you,

-Tony Kirkham

Collapse
Posted by Gustaf Neumann on
The request monitor implements several features, where not all of these are wanted on every site.

The main features are
* performance overview
* performance statistics
* rescheduling between different connection thread pools
* request blocking
* request throttling
* value provider for munin-plugins

Performance overview

Performance statistics: display certain slow calls, causing queuing etc.

Connection thread pools: When certain requests take longer than a threshold value, move these on the fly to a "slow" pool, if such a pool is configured. This pool is not especially "slow", but it is typically defined with a larger queue, where requests are potentially rejected, if there are too many. But, the main point is, it avoids that the default pool is blocked by too many slow requests, such ensuring good service quality for most of the users.

To get some picture, below is a site having 5 connection pools defined, including a "slow" and a "fast" pool.

The request blocking can be optionally configured (parameter "do_double_click_prevention"). It is based on a white-list (e.g. embedded requests, members of the "fast" pool) and blocks URLs, which are executed by the same use on the same URL (see throttle_check). Your are right, this does not include the query parameters, and since this check happens in another thread, this thread is no access to the query parameters (if these are not passed explicitly). In general, it would be possible to include these).

The request throttling is something, i would not recommend to use, but i have not deleted it, since due to the new features in NaviServer 4.99.20 it in general possible to move some over-eager users (maybe temporarily) to some slow pool to ensure general liveliness. ... but other things got higher priority.

Concerning the munin-plugins: i thing, these are publically visible at https://openacs.org/munin/

So, in short, you can deactivate the request blocking, and still use the request monitor to get statistics, etc. One can in general extend the request blocking to include the query parameters, but has to check, what the consequences are. Probably the easiest way is to define this separated from the rest.

hope this helps
-gn

Collapse
Posted by Tony Kirkham on

This helps a great deal! We may have more questions as we put this to use, but for now this gets us going.

Thank you very much!

-Tony