Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easier to configure rate limiter to never cross max_burst admitted cells in a specific period of time #219

Open
kskalski opened this issue Dec 21, 2023 · 0 comments

Comments

@kskalski
Copy link

I spent some time debugging why rate limiter allowed total n larger than max_burst within first minute when using Quota::per_minute(max_burst).

It seems this is actually an allowed and correct behavior according to the algorithm and careful reading of documentation. What happened was basically: some of the cost cells got replenished before client requested admitting the subsequent max_burst + Xth cell.

Maybe the documentation could be still improved, e.g. stating explicitly that above scenario is possible, but I'm wondering about the best way to configure rate limiter for a use case where the "moving window" admittance of cells in a period specified when creating quota is always below max_burst.
My current simple approach is "down-scaling" the period, i.e. something like this:

 // `governor` counts cost accumulated *minus* replenished in time, so it won't
 // strictly forbid going over max burst in specified period of time (during
 // steady ramp-up initial requests will be forgotten before full period).
 // Downscaling quota to smaller granularity period mostly prevents this problem.
 let divisor = 10.min((minute_limit + rate_limits::MAX_COST - 1) / rate_limits::MAX_COST);
 let max_period_burst =
     rate_limits::Cost::new(minute_limit / divisor).unwrap_or(rate_limits::Cost::MIN);
 let unit_replienish_in = std::time::Duration::from_millis(60000 / minute_limit as u64);
 governor::Quota::with_period(unit_replienish_in).unwrap().allow_burst(max_period_burst)

One idea that would make it easier is a version of above code to be added to quota, so I could do something like
governor::Quota::per_minute(minute_limit).with_granularity(Duration::from_secs(10))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant