Polly Core Rate Limiter

Feb 15, 2024 # Polly, Rate Limiter, Polly Core

Want to learn more about Polly? Check out my Pluralsight course on it.

Polly, the .NET resiliency framework has had a major update recently. All the same features are there, but some have changed names and the way you use them has changed. One of the changes is the rate limiter. It used to be called Bulkhead isolation.

As I was working on this blog post, I incorrectly used the rate limiter and ended up with an excessive number of calls being executed. This was due to how I was passing the parameters from my loop to the rate limiter.

Here is wrong way to do it -

for (int loop = 1; loop <= 10; loop++)
{
    results.Add(pipeline.ExecuteAsync(async (ct) => await SimpleDelayWithReturnValue(loop, ct)));
}

What happens here is the loop variable is captured by the lambda, and the lambda is executed after the loop has been incremented. So I saw executions of SimpleDelayWithReturnValue where 11 was passed to it. Not what I wanted. One of the big improvements with the newer version of Polly is the reduction in overloaded methods, but I still managed to use the wrong overload in this case.

Thank you to Martin Costello for pointing out my mistake.

This is the right way to do it -

for (int loop = 1; loop <= 10; loop++)
{
    results.Add(pipeline.ExecuteAsync(async (i, ct) => await SimpleDelayWithReturnValue(i, ct), loop));
}

Note, this is not the only way to do it, you could also use a foreach, and this problem would not occur.

To use the rate limiter, you need two NuGet packages - Polly.Core and Polly.RateLimiting.

Here is a fully working example -

 1using System.Threading.RateLimiting;
 2using Polly;
 3using Polly.RateLimiting;
 4
 5var rateLimiter = new RateLimiterStrategyOptions() // setup the rate limiter strategy
 6{
 7    DefaultRateLimiterOptions = new ConcurrencyLimiterOptions()
 8    {
 9        PermitLimit = 5,
10        QueueLimit = 3
11    },
12    OnRejected = (args) =>
13    {   
14        Console.WriteLine($"Rate limit exceeded");
15        return default;
16    }
17};
18
19var pipeline = new ResiliencePipelineBuilder() // add the rate limiter to the pipeline
20    .AddRateLimiter(rateLimiter)
21    .Build();
22
23await TryingRateLimiter(); // call the method to try the rate limiter
24
25async Task TryingRateLimiter()
26{
27    var results = new List<ValueTask<int>>();
28    
29    for (int loop = 1; loop <= 10; loop++)
30    {
31        results.Add(pipeline.ExecuteAsync(async (i, ct) => await SimpleDelayWithReturnValue(i, ct), loop));
32    }
33
34    foreach (var result in results)
35    {
36        try
37        {
38            Console.WriteLine($"Result: {await result}");
39        }
40        catch (RateLimiterRejectedException ex)
41        {
42            Console.WriteLine($"Exception: {ex.Message}");
43        }   
44    }
45}
46
47async ValueTask<int> SimpleDelayWithReturnValue(int number, CancellationToken token) // can return a Task or ValueTask
48{
49    return.WriteLine($"Called SimpleDelay with {number}");
50    await Task.Delay(1000);
51    return number;
52}

That’s it! Have fun with the new Polly!

Polly Rate Limiter Polly Core

Polly Core Rate Limiter

Related