Skip to main content

Whisperer throughput limiter

· 2 min read

Spider now includes a new circuit breaker to limit its impact: a throughput limiter.

When set, the Whisperer checks the rate of capture packets continuously and discards packets once the throughput is reached.

The first packets gets in, then it stops. This allows having complete communications for some time. A way of down-sampling.

Origin

This feature has been initiated when developing local agents feature, for safety.
But it is available for all Whisperers.

Configuration

Configuration is made in Whisperer capture configuration tab, as part of the Circuit Breakers.

note

The configuration is by instance.
When you set 5 MB/min and there is 2 instances, you will get 10 MB/min total.

Implementation

Rate limiter is implemented using a ring buffer of 12 buckets of 5 seconds, with almost no noticeable impact on memory or on CPU usage.

Result

On the graph below, both Gossiper and Development Whisperers are capturing the same traffic, but Gossiper is configured with a throughput limit of 5MB/min. With 2 instances of it.

UploadGraph.png

On the timeline, you can notice data flowing in and stopping.
The QualityLine tells you how many packets have been discarded by the limiter.

Timeline.png

On the Whisperer status, the discarded packets are counted within the Dropped count.

Status.png