Whisperer throughput limiter
Spider now includes a new circuit breaker to limit its impact: a throughput limiter.
When set, the Whisperer checks the rate of capture packets continuously and discards packets once the throughput is reached.
The first packets gets in, then it stops. This allows having complete communications for some time. A way of down-sampling.
Origin
This feature has been initiated when developing local agents
feature, for safety.
But it is available for all Whisperers.
Configuration
Configuration is made in Whisperer capture configuration tab, as part of the Circuit Breakers.
The configuration is by instance.
When you set 5 MB/min and there is 2 instances, you will get 10 MB/min total.
Implementation
Rate limiter is implemented using a ring buffer of 12 buckets of 5 seconds, with almost no noticeable impact on memory or on CPU usage.
Result
On the graph below, both Gossiper
and Development
Whisperers are capturing the same traffic, but Gossiper is configured with a throughput limit of 5MB/min. With 2 instances of it.
On the timeline, you can notice data flowing in and stopping.
The QualityLine tells you how many packets have been discarded by the limiter.
On the Whisperer status, the discarded packets are counted within the Dropped
count.