Simmy Chaos Engine for .NET – Part 2, Resilience and Injected Faults

Full source code here.

I wrote a blog on using a Simmy Fault policy a few days ago, it is very simple to use by itself, but it is far more useful when used in combination with a resilience policy.

Take the scenario where I have added retry logic to my requests, but the requests never fail, how do I know the retry logic works the way I expect. This is where the fault policy shines.

I’m going to build on the previous post where I injected a fault 50 percent of the time when making http requests. In the previous post I had Simmy throw a HttpRequestException but this time I’m going to have Simmy return a HttpResponseMessage containing an InternalServerError.

Along with the fault policy, I am going add a simple retry policy, retrying up to three times.

Here’s how it will work –

The fault policy executes –

If the fault policy is active and a fault is to be returned (remember this happens 50% of the time) a InternalServerError is returned. The HttpClient does NOT make a request to the remote service

The retry policy executes. The handles clause sees the InternalServerError and the behavior clause of the retry kicks in and performs the retry

The fault policy executes again

The preceding two steps repeat until a request succeeds or the retry policy reaches is limit of 3 retries

 

If the fault policy is not active, or a fault is NOT to be returned

the HTTP client executes the request to the remote service

the handles clause of the retry determines whether a retry should be performed

 

This is what it looks like –

At the center is the HttpReqeust, around it is the fault policy and around everything is the retry policy.

To achieve all this in code is very simple, in ConfigureServices add the fault policy.

public void ConfigureServices(IServiceCollection services)
{
	HttpResponseMessage faultHttpResponseMessage = new HttpResponseMessage(HttpStatusCode.InternalServerError)
	{
		Content = new StringContent("Simmy swapped the Ok for an Internal Server Error")
	};
	AsyncInjectOutcomePolicy<HttpResponseMessage> faultPolicy = MonkeyPolicy.InjectFaultAsync(
		faultHttpResponseMessage,
		injectionRate: .5,
		enabled: () => true 
	);

When the fault policy executes, 50% of the time it will return an InternalServerError response, preempting the real request to the remote service.

Next is the retry policy, its behavior clause triggers if the HttpResponseMessage is not a success.

IAsyncPolicy<HttpResponseMessage> retryPolicy = Policy
	.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
	.RetryAsync(3, onRetry: (message, retryCount) =>
	{
		Console.WriteLine($"Retry: {retryCount}");
	});

Wrap the retry policy around the fault policy.

AsyncPolicyWrap<HttpResponseMessage> faultAndRetryWrap = Policy.WrapAsync(retryPolicy, faultPolicy);

Finally, create the HttpClientFactory, passing in the policy wrap. If you are not familiar with how the HttpClientFactory works read Steve Gordon’s series or my blog posts on it. If you want know how it works with Polly, take a look at my posts on that topic.

services.AddHttpClient("OpenBreweryDb", client =>
{
	client.BaseAddress = new Uri("https://api.openbrewerydb.org/breweries/");
	client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandler(faultAndRetryWrap);

To fire of the HttpClient request inside the BreweryController remains the same –

string requestEndpoint = $"?by_state={state}&by_name={name}";
var httpClient = _httpClientFactory.CreateClient("OpenBreweryDb");
var response = await httpClient.GetAsync(requestEndpoint);

Full source code here.

Simmy Chaos Engine for .NET – Part 1, Injecting Faults

Full source code here.

For quite a while I have been writing, presenting and making videos about using Polly to make applications more stable.

With this post I’m starting a series on how break your applications with a chaos engine, the kind you might have heard about from the likes of NetFlix.

The team behind Polly released a chaos engineering module in June 2019. It allows you to inject faults, latency and arbitrary behavior into your application.

There are two general ways of using these policies –
1. Within your own business logic (if you have access to it).
2. Within the calls to business logic (if you can’t or don’t want to change it).

I’m going to deal with the latter case in this post. I want to see how my code responds when failures occur in third party dependencies, since I have no control over those dependencies I have to introduce the errors myself.

Making a good service go bad
In the provided code I have dependency on remote service and I want make it seem that that service suffers from a high percentage of faults.
This example will make GET requests to an API that looks up breweries, from what I can tell it is a highly reliable API, but I will make look as though 50% of requests to it fail. The point of doing this is to see how my application reacts to these errors – does it do so gracefully, or does it fall on its face.

If you made requests to the API without interfering with them you should see a 100% success rate. But I want to see failures, so I’ll use the Fault Policy to inject exceptions.

First, I added two NuGet packages to my project – Polly.Contrib.Simmy and Microsoft.Extensions.Http.Polly to the project.

In Startup.cs I add the fault policy.

AsyncInjectOutcomePolicy<HttpResponseMessage> faultPolicy = MonkeyPolicy.InjectFaultAsync<HttpResponseMessage>(
	new HttpRequestException("Simmy threw an exception"), 
		injectionRate: .5,
		enabled: () => true
	);

The parameters specify the exception to throw, the percentage of requests that throw the exception, and whether the policy is enabled. You can use set the injection rate and whether the policy is enabled via configuration, but I’m not going to show that here.
In this case, I am throwing an HttpRequestException 50% of the time, and the policy is enabled.

Then create a HttpClientFactory for the OpenBreweryDb

services.AddHttpClient("OpenBreweryDb", client =>
{
	client.BaseAddress = new Uri("https://api.openbrewerydb.org/breweries/");
	client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandler(faultPolicy);

If you are not yet using the HttpClientFactory, the source code has an example for you too, check the BreweryNoHttpClientFactoryController.

Start up the application, it should open on port 5000 and make a request to look up Night Shift Brewing in Massachusetts. There is a 50% chance of an exception occurring with each request.

Try it out. You’ll see that my application does not handle these failures very well, it simply returns the error to original caller.

That’s it, you are now starting down the road of mixing in some chaos into your application.

What then?
Great, my application fails, but we always knew it could. There’s not much point introducing failures unless you plan to handle them in some way, and this is where you might add a retry policy to improve you likelihood of a successful request.

Of course, the Polly retry, or wait and retry policies would be ideal. I’ll show how to do just that in the next post.

Full source code here.

Selectively Caching Values Inside HttpResponseMessage with Polly – caching series 3/3

Full source code here.

This is the last of three posts on caching with Polly. The first showed how to selectively cache HttpResponseMessages based on the status code of the response.

The second showed how to cache values inside the HttpResponseMessage rather than the whole of the response.

This final post will show how to selectively cache values inside the response based on the status of the response.

If you have read the previous posts the content of this will feel familiar.

Changes to Startup
Like the previous post, add a memory cache, a policy registry and a HttpClientFactory to the service collection.

public void ConfigureServices(IServiceCollection services)
{
    services.AddMemoryCache();
    services.AddSingleton<IAsyncCacheProvider, MemoryCacheProvider>();

    IPolicyRegistry<string> registry = services.AddPolicyRegistry();

    services.AddHttpClient("RemoteServer", client =>
    {
        client.BaseAddress = new Uri("http://localhost:5000/api/");
        client.DefaultRequestHeaders.Add("Accept", "application/json");
    });

    services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
}

What’s new is in the Configure method, fist step is to add the registry to the method parameters, DI will take care of this for us.

public void Configure(IApplicationBuilder app, IHostingEnvironment env,
    IAsyncCacheProvider cacheProvider, IPolicyRegistry<string> registry)
{

Next is the cache filter, this one examines the tuple of int and HttpStatusCode and, if the HttpStatusCode is an OK the tuple will be cached for the duration specified. If the respones is not OK the filter will set the cache time to 0 which is interpreted as do not cache by the cache policy.

Func<Context, (int value, HttpStatusCode StatusCode), Ttl> cacheOnly200OkFilter =
	(context, result) => new Ttl(
		timeSpan: result.StatusCode == HttpStatusCode.OK ? TimeSpan.FromSeconds(5) : TimeSpan.Zero,
		slidingExpiration: true
	);

And this is the cache policy that uses the filter above to store the tuple.

IAsyncPolicy<(int, HttpStatusCode)> cachePolicy =
	Policy.CacheAsync<(int, HttpStatusCode)>(
		cacheProvider.AsyncFor<(int, HttpStatusCode)>(), 
		new ResultTtl<(int, HttpStatusCode)>(cacheOnly200OkFilter),
		onCacheError: null);

Lastly, add the policy to the registry.

registry.Add("cacheOnly200OkFilter", cachePolicy);

Changes to the Controller
Over in the CatalogController there many changes.

At the top I take in the HttpClientFactory and policy registry.

private readonly IHttpClientFactory _httpClientFactory;
private readonly IPolicyRegistry<string> _policyRegistry;

public CatalogController(IHttpClientFactory httpClientFactory, IPolicyRegistry<string> policyRegistry)
{
	_httpClientFactory = httpClientFactory;
	_policyRegistry = policyRegistry;
}

Inside the Get method, I setup grab the HttpClient from the HttpClientFactory, set the endpoint of the remote service, take the cache policy from the registry and setup the context for the cache policy to cache based on the incoming request. All of this is similar to normal usage of the Polly cache.

[HttpGet("{id}")]
public async Task<IActionResult> Get(int id)
{
    string requestEndpoint = $"inventory/{id}";
    var httpClient = _httpClientFactory.CreateClient("RemoteServer");

    var cachePolicy = _policyRegistry.Get<CachePolicy<(int, HttpStatusCode)>>("cacheOnly200Ok");
    Context policyExecutionContext = new Context($"GetInventoryById-{id}");	

As in the previous post I don’t call the HttpClient from inside the cache policy, instead I need to call something that returns the int and the HttpStatusCode. I use a local function that makes the request to the remote service, checks the response, deserializes the value and returns a tuple.

(int ItemsInStock, HttpStatusCode StatusCode) itemsInStockAndStatusCode =
    await cachePolicy.ExecuteAsync(context => MakeRequest(), policyExecutionContext);

// local function
async Task<(int count, HttpStatusCode statusCode)> MakeRequest()
{
    HttpResponseMessage response = await httpClient.GetAsync(requestEndpoint);

    if (response.IsSuccessStatusCode)
    {
        int itemsInStock = JsonConvert.DeserializeObject<int>(await response.Content.ReadAsStringAsync());
        return (itemsInStock, response.StatusCode);
    }

    return (0, response.StatusCode);
}

When the cachePolicy is executed it first checks if there is value already in the cache, if there is, that will be returned and the local function will not be called. If there is no value in the cache, the local function is called, if the response of the local function is in the 200 range, the value is stored in the cache.

At this point, the action method has not returned anything to its caller.

Below the local function I check the tuple returned from the call to the cache policy and return a response.

if (itemsInStockAndStatusCode.StatusCode != HttpStatusCode.OK)
{
     return StatusCode((int)itemsInStockAndStatusCode.StatusCode);
}
return Ok(itemsInStockAndStatusCode.ItemsInStock);

For clarity, here is the full Get method.

[HttpGet("{id}")]
public async Task<IActionResult> Get(int id)
{
    string requestEndpoint = $"inventory/{id}";
    var httpClient = _httpClientFactory.CreateClient("RemoteServer");

    var cachePolicy = _policyRegistry.Get<CachePolicy<(int, HttpStatusCode)>>("cacheOnly200Ok");
    Context policyExecutionContext = new Context($"GetInventoryById-{id}");

    (int ItemsInStock, HttpStatusCode StatusCode) itemsInStockAndStatusCode =
        await cachePolicy.ExecuteAsync(context => MakeRequest(), policyExecutionContext);

    async Task<(int count, HttpStatusCode statusCode)> MakeRequest()
    {
        HttpResponseMessage response = await httpClient.GetAsync(requestEndpoint);

        if (response.IsSuccessStatusCode)
        {
            int itemsInStock = JsonConvert.DeserializeObject<int>(await response.Content.ReadAsSt
            return (itemsInStock, response.StatusCode);
        }

        // it doesn't matter what int you use here, it won't be cached as the StatusCode is not a
        return (0, response.StatusCode);
    }

    if (itemsInStockAndStatusCode.StatusCode != HttpStatusCode.OK)
    {
         return StatusCode((int)itemsInStockAndStatusCode.StatusCode);
    }
    return Ok(itemsInStockAndStatusCode.ItemsInStock);
}

Here endeth the series on caching.

Full source code here.

Caching Values Inside HttpResponseMessage with Polly – caching series 2/3

Full source code here.

In this, the second of three posts on caching in Polly, I will show how to cache the values returned inside a HttpResponseMessage as opposed to caching the response with all its various elements.

The first post explained how to selectively cache a response, based on the Http StatusCode. The next post will show how to combine these two ideas and selectively cache the values inside the HttpResponseMessage.

Caching the whole response or the value
Caching the whole HttpResponseMessage is useful if your business logic depends on interrogating the response for values outside the main payload or you want a quick and easy caching solution.

If on the other hand, you only need the payload, there is no need to cache the whole response.

Both cases are plausible, and it is up to do decide which is more appropriate for your application.

How to cache the value inside HttpResponseMessage
The first step is to set the cache inside ConfigureServices.

public void ConfigureServices(IServiceCollection services)
{
    services.AddMemoryCache();
    services.AddSingleton<IAsyncCacheProvider, MemoryCacheProvider>();

Then in Configure, setup the policy to store the type you expect to find in the response. In this example it is in a int, but it could easily be a complex layered model representing anything.

public void Configure(IApplicationBuilder app, IHostingEnvironment env,
    IAsyncCacheProvider cacheProvider, IPolicyRegistry<string> registry)
{
    CachePolicy<int> cachePolicy =
        Policy.CacheAsync<int>(cacheProvider, TimeSpan.FromSeconds(10));

    registry.Add("CachingPolicy", cachePolicy);

Note, how I specified an int rather than a HttpResponseMessage.

In the previous post, I used the HttpClientFactory to select the policy from a Polly policy registry. I could do that because the type I was storing was HttpResponseMessage, that is not the case now and HttpClientFactory will not play nice when trying to cache anything other than a HttpResponseMessage. Instead I’m going to pass the policy registry directly to the controller and get the policy from this, I am still using the HttpClientFactory to pass HttpClient, but not for policy selection.

Here is how the controller starts.

public class CatalogController : Controller
{
    private readonly IHttpClientFactory _httpClientFactory;
    private readonly IPolicyRegistry<string> _policyRegistry;

    public CatalogController(IHttpClientFactory httpClientFactory, IPolicyRegistry<string> policyRegistry)
    {
        _httpClientFactory = httpClientFactory;
        _policyRegistry = policyRegistry;
    }

And here is the GET method.

[HttpGet("{id}")]
public async Task<IActionResult> Get(int id)
{
    string requestEndpoint = $"inventory/{id}";
    var httpClient = _httpClientFactory.CreateClient("RemoteServer");

    var cachePolicy = _policyRegistry.Get<CachePolicy<int>>("CachingPolicy");
    Context policyExecutionContext = new Context($"GetInventoryById-{id}");

    int itemsInStock =
        await cachePolicy.ExecuteAsync(context => GetItemsInStockCount(), policyExecutionContext);

    async Task<int> GetItemsInStockCount()
    {
        HttpResponseMessage response = await httpClient.GetAsync(requestEndpoint);

        if (response.IsSuccessStatusCode)
        {
            int itemsInStockCount = await response.Content.ReadAsAsync<int>();
            return itemsInStockCount;
        }

        return 0;
    }

    return Ok(itemsInStock);
}

The first few lines are straight forward, setup the endpoint, get the HttpClient, get the cache policy. Like in the previous post, I use the context when placing items into the cache and retrieving them.

Because I want to cache the value inside the HttpResponse (and not the response itself) I can’t call the HttpClient from inside the policy’s exec method. Instead I need to call something that returns the int from inside the HttpResponse.

The easiest way of doing this is to use a local function which makes the HttpClient request, deserializes the response and returns the int from the response.
In this post I am assuming that all values inside the response should be cached, I am NOT checking the status code of the response.

The policy then caches the int for the period specified in Startup.

That’s all there is to it.

If you are wondering how to selectively cache values inside the HttpResponse, tune in for the next post in this series.

Full source code here.

Selectively Caching a HttpResponseMessage with Polly – caching series 1/3

Full source code here.

When I give talks on Polly I show how to use it in a Web API application that calls another Web Api application. I give a simple example of how to use the cache policy where I store the whole of the HttpResponseMessage. At the end of the talk I often get a couple of questions –

1. is it possible to cache the response only when the status code is in the 200 range.
2. is it possible to cache the values inside the response instead of the whole response.

The answer to both of these is yes, and in this series of three articles I will explain how to do both of these things, and how how to combine them to cache a value inside a response only if the response is in the 200 range.

This article will demonstrate how to selectively cache HttpResponseMessages based on the Http StatusCode of the response.

As described in an earlier post, using a cache policy is a little more involved than using other policies. See this post for more info.

Changes in Startup
In Startup inside ConfigureServices I setup the memory cache, the HttpClientFactory and a policy registry (I’m not going to go into a detailed explanation of this code as it is already explained in the earlier article and you have all the source code).

All the action for this post takes place in the Configure method.

Creating the cache filter
The cache timeout filter, it sets the length of time to cache a value. The filter here will cache a response for 10 seconds if the response indicates a success, otherwise it caches the response for 0 seconds, in this case the Polly code does not perform any caching.

public void Configure(IApplicationBuilder app, IHostingEnvironment env,
    IAsyncCacheProvider cacheProvider, IPolicyRegistry<string> registry)
{
    Func<Context, HttpResponseMessage, Ttl> cacheOnly200OkFilter =
        (context, result) => new Ttl(
            timeSpan: result.StatusCode == HttpStatusCode.OK ? TimeSpan.
            slidingExpiration: true
        );

The cache policy says that it caches HttpResponseMessages and uses the filter to decide if a response should be cached.

IAsyncPolicy<HttpResponseMessage> cacheOnlyOkResponsePolicy =
    Policy.CacheAsync<HttpResponseMessage>(
        cacheProvider.AsyncFor<HttpResponseMessage>(), //note the .AsyncFor<HttpResponseMessage>
        new ResultTtl<HttpResponseMessage>(cacheOnly200OkFilter),
        onCacheError: null
    );

Note the AsyncFor, this is very important. Thanks to Dylan Reisenberger for his help here, I spent too many hours trying to figure the overloads with no success.

Try it out in the provided solution, hit F5 and put some breakpoints in the controllers and on the filter in Startup.

That’s it, you now have selective caching of responses.

In the next post, I’ll show you how to cache the value inside the response instead of the whole response.

Full source code here.

Testing Your Code When Using Polly

Full source code here.

When developing an application with Polly you will also probably want to write some unit tests.

Here are the scenarios I test for –

1. How my code behaves when the policy throws an exception, such as TimeoutRejectionException, BulkheadRejectedException or BrokenCircuitException.
2. How my code behaves when a policy becomes active and changes the outcome of a call, such as when an unreliable request works because Polly performs a retry.
3. What my code should do if there was no policy in place.

I have a few classes to demonstrate these scenarios, BusinessLogic.cs and OtherBusinessLogic.cs are the classes under test. ErrorProneCode.cs is the unreliable class that I will mock and pass mocked policies into.

For the first case I use Moq to mock the error prone code so it returns an incorrect value.

[Fact]
public void Should_Return_999_When_TimeoutRejectedException_Thrown()
{
    //Arrange 
    Mock<IErrorProneCode> mockedErrorProneCode = new Mock<IErrorProneCode>();
    mockedErrorProneCode.Setup(e => e.GetSomeNumber()).Returns(0);

    Mock<ISyncPolicy> mockedPolicy = new Mock<ISyncPolicy>();
    mockedPolicy.Setup(p => p.Execute(It.IsAny<Func<int>>())).Throws(new TimeoutRejectedException("Mocked Timeout Exception"));

    IBusinessLogic businessLogic = new BusinessLogic(mockedPolicy.Object, mockedErrorProneCode.Object);

    //Act
    // if there is a TimeoutRejectedException in this CallSomeSlowBadCode it will return 999
    int num = businessLogic.CallSomeSlowBadCode();

    //Assert
    Assert.Equal(999, num);
}

In the next case I verify that the application has correctly used the retry policy method. I use a seeded random number generator that produces an known sequence to return values from the ErrorProneCode class.

  
[Fact]
public void Should_Return_Odd_When_Retry()
{
    //Arrange 
    Mock<IErrorProneCode> mockedErrorProneCode = new Mock<IErrorProneCode>();

    Random rnd = new Random(1); // rnd.Next(10) retruns 2, 1, 4 

    mockedErrorProneCode.Setup(e => e.GetSomeNumber()).Returns(() => rnd.Next(10));

    ISyncPolicy<int> policy = Policy.HandleResult<int>(i => i %2 !=1) // retry if the number is not odd.
        .Retry(1);
    OtherBusinessLogic otherBusinessLogic = new OtherBusinessLogic(policy, mockedErrorProneCode.Object);

    //Act
    int num = otherBusinessLogic.CallSomeCodeThatNeedsToBeRetried();

    //Assert
    Assert.Equal(1, num % 2);
}

Finally, I want to verify that my code will work if no Polly policy is in use. To do this, I pass in a NoOp policy.

[Fact]
public void Should_Return_Even_With_NoOp()
{
    //Arrange 
    Mock<IErrorProneCode> mockedErrorProneCode = new Mock<IErrorProneCode>();

    Random rnd = new Random(1); // rnd.Next(10) retruns 2, 1, 4 

    mockedErrorProneCode.Setup(e => e.GetSomeNumber()).Returns(() => rnd.Next(10));

    ISyncPolicy<int> policy = Policy.NoOp<int>();
    OtherBusinessLogic otherBusinessLogic = new OtherBusinessLogic(policy, mockedErrorProneCode.Object);

    //Act
    int num = otherBusinessLogic.CallSomeCodeThatNeedsToBeRetried();

    //Assert
    Assert.Equal(0, num % 2); //even number
}

Full source code here.

Caching in Polly and the HttpClientFactory

Full source code here.

Polly allows you to cache a response for reuse by a subsequent request, it supports both local an distributed caches, full information can be found here https://github.com/App-vNext/Polly/wiki/Cache.

The HttpClientFactory lets you define polices in startup.cs and apply them to HttpClient requests anywhere in your application. With most policies this makes it easier for you to apply a policy to a request, but with the cache policy it complicates matters.

With the release of HttpClientFactory there are now two ways to use the cache policy, one taking advantage of HttpClientFactory to apply the policy to requests, and the other bypassing it when executing requests.

Caching with the cache policy inside the HttpClientFactory
First let’s look at how to use the Cache Policy using PolicySelector method on the HttpClientFactory to apply the policy to the request, see this post for more info.

If you are working in Polly V6+ add Polly.Caching.Memory package to your project, this is important, don’t add Polly.Caching.MemoryCache that is for Polly V5.9 and earlier, and many an hour was lost figuring that out.

In Startup.cs add the following to the ConfigureServices method –

public void ConfigureServices(IServiceCollection services)
{
    services.AddMemoryCache();
    services.AddSingleton<IAsyncCacheProvider, MemoryCacheProvider>();

    IPolicyRegistry<string> registry = services.AddPolicyRegistry();

    services.AddHttpClient("RemoteServer", client =>
    {
        client.BaseAddress = new Uri("http://localhost:5000/api/");
        client.DefaultRequestHeaders.Add("Accept", "application/json");
    }).AddPolicyHandlerFromRegistry(PolicySelector);

This sets up the in memory cache and policy registry, adding both to ServicesCollection. The cache policy is not added here, instead it is added in the Configure(..) method (thanks Andrew Lock for reminding me about this).

Next, add the HttpClientFactory and a method to pick the appropriate policy from the registry.

Inside the Configure(..) I setup the cache policy. The registry and cache provider are passed via dependency injection.

public void Configure(IApplicationBuilder app, IHostingEnvironment env, 
    IAsyncCacheProvider cacheProvider, IPolicyRegistry<string> registry)
{
    CachePolicy<HttpResponseMessage> cachePolicy = 
        Policy.CacheAsync<HttpResponseMessage>(cacheProvider, TimeSpan.FromSeconds(30));
    registry.Add("CachingPolicy", cachePolicy);

In the controller I have a constructor that takes one argument, the HttpClientFactory.

public CatalogController(IHttpClientFactory httpClientFactory)
{
    _httpClientFactory = httpClientFactory;
}

In the action method I get the HttpClient from the HttpClientFactory and specify the remote endpoint I want to call, but now I can’t use usual httpClient.GetAsync(..) because I can’t pass it a cache context key (i.e. the name of where I look for previously cached responses and store new responses).
Instead I build a HttpRequestMessage, specifying the HttpMethod, Uri and using a Polly extension method I set the cache context.

HttpRequestMessage httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, 
    new System.Uri(httpClient.BaseAddress + requestEndpoint));

httpRequestMessage.SetPolicyExecutionContext(new Context($"GetInventoryById-{id}"));

HttpResponseMessage response = await httpClient.SendAsync(httpRequestMessage);

This approach feels a bit awkward and not many people use the HttpRequestMessage to send requests to remote services.

Caching with the cache policy outside the HttpClientFactory
An alternative to the above is to add the cache to the registry, but not add the registry to the HttpClientFactory.

In this case the adding the HttpClientFactory to the services collection changes slightly –

services.AddHttpClient("RemoteServer", client =>  
{
    client.BaseAddress = new Uri("http://localhost:5000/api/");
    client.DefaultRequestHeaders.Add("Accept", "application/json");
}); // no policy selector

In the controller we now pass in the registry as HttpClientFactory as constructor arguments.

public StockMgtController(IHttpClientFactory httpClientFactory, IPolicyRegistry<string> policyRegistry) 
{
    _httpClientFactory = httpClientFactory;
    _policyRegistry = policyRegistry;
}

There are changes inside the action method also –

[HttpGet("{id}")]
public async Task<IActionResult> Get(int id)
{
    string requestEndpoint = $"inventory/{id}";
    var httpClient = _httpClientFactory.CreateClient("RemoteServer");

    var cachePolicy = _policyRegistry.Get<CachePolicy<HttpResponseMessage>>("CachingPolicy");
    Context policyExecutionContext = new Context($"GetInventoryById-{id}");

    HttpResponseMessage response = await cachePolicy.ExecuteAsync(
        context => httpClient.GetAsync(requestEndpoint), policyExecutionContext);
	//snip..	

The httpClient is retrieved from the HttpClientFactory as before, but now the cachePolicy is taken from the policy registry, the Context is defined and then we use the cachePolicy.ExecuteAsync(..) method to make call the httpClient.GetAsync(..) method.

The other way
You might have noticed that there is another way of using the cache policy. You could use the HttpClientFactory and a PolicySelector method to apply most policies to your HttpClient requests, but not for the cache policy. I wouldn’t do this, it will confuse everyone.

Conclusion
The outcome of both is the same, I think the first approach is better, even though the call to the httpClient is a little more convoluted, it will be more consistent with how you call other policies in your application, there is also a chance the Polly team will add an extension method to make the use of HttpRequestMessage transparent to us developers.

Full source code here.

Using Polly with Any Type of Request

Full source code here.

I recently presented a talk on Polly at the DevUp conference in St. Louis. In the presentation I showed examples using Polly with HttpClient requests only because this is my most common use case. I showed how to use retries, circuit breakers, fallbacks, caching, timeouts and bulkheads isolation. At the end of the presentation a few people asked me if Polly could be used with other types of request, of course it can and I should have said so during the talk. I’ve added new slide for the next time I present on Polly – (as of October 2018 that will be Granite State Code Camp, Boston Code Camp and NDC London).

Polly can be used with any type of request you make, whether it returns a value or not. You can use on code that throws exceptions from time to time (though I strongly recommend fixing your code), for database calls, any method call for that matter. Below are some examples of using the retry policy in a variety of scenarios.

Exceptions
This policy retries if an exception is thrown, you can be more specific about exception types.

 
RetryPolicy retryIfException = Policy.Handle<Exception>()
    .Retry(4, (exception, retryCount) =>
    {
        Console.WriteLine($"Got a response of {exception.Message} (expected 0), retrying {retryCount}");
    });

Int
Here I check check if the int returned is anything other than 0, or if there has been a DivideByZeroException. If so, I retry up to four times and also print some text to console.

 
RetryPolicy<int> retryPolicyNeedsAResponseOfOne = Policy.HandleResult<int>(i => i != 0)
    .Or<DivideByZeroException>()
    .Retry(4, (response, retryCount) =>
    {
        Console.WriteLine($"Got a response of {response.Result} (expected 0), retrying {retryCount}");
    });

IEnumerable
In this one I check that the IEnumerable has three items in it, if not, I retry and print some info to the console.

 
RetryPolicy<IEnumerable<int>> retryPolicyNeedsResponeWithTwoNumbers = Policy.HandleResult<IEnumerable<int>>(l => l.Count() != 3)
   .Retry(4, (response, retryCount) =>
   {
       Console.WriteLine($"Got a reponse with {response.Result.Count()} entries (expected 3), retrying {retryCount}");
   });

Bool
In this policy I check the bool returned, if it is false, I retry.

 
RetryPolicy<bool> retryPolicyNeedsTrueResponse = Policy.HandleResult<bool>(b => b != true)
   .Retry(4, (response, retryCount) =>
   {
       Console.WriteLine($"Got a reponse of {response.Result} entries (expected true), retrying {retryCount}");
   });

Polly can check the value of any return type or exception making it possible to use Polly for any call you can think of.

Full source code here.

Polly, HttpClientFactory and the Policy Registry in a console application

Full source code available here.

How to use the HttpClientFactory with a console application is not immediately obvious. I thought it would be a simple matter, but it’s not because it relies on the dependency injection infrastructure you get with a web application. I’ve written about using HttpClientFactory with Polly in a Web Api here.

The easiest way to use HttpClientFactory within a console application is inside a HostBuilder. This gives you access to the services collection, now everything is easy.

Start with a standard console application, if you’re wondering about the async Task on my Main method, this was introduced in C# 7.1.

static async Task Main(string[] args)
{
    var builder = new HostBuilder()
        .ConfigureServices((hostContext, services) =>
        {

Inside the ConfigureServices, we configure the HttpClientFactory in the same way I showed in my previous post. You can also configure other things like logging, configuration sources, more DI, etc.

But first off, I’m going to add a Polly registry –

IPolicyRegistry<string> registry = services.AddPolicyRegistry();
	
IAsyncPolicy<HttpResponseMessage> httWaitAndpRetryPolicy =
    Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
        .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(retryAttempt));

registry.Add("SimpleWaitAndRetryPolicy", httWaitAndpRetryPolicy);

IAsyncPolicy<HttpResponseMessage> noOpPolicy = Policy.NoOpAsync()
    .AsAsyncPolicy<HttpResponseMessage>();

registry.Add("NoOpPolicy", noOpPolicy);

Then add the HttpClientFactory, passing in the lambda to pick the right policy based on the HTTP verb.

services.AddHttpClient("JsonplaceholderClient", client =>
{
    client.BaseAddress = new Uri("https://jsonplaceholder.typicode.com");
    client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandlerFromRegistry((policyRegistry, httpRequestMessage) =>
{
    if (httpRequestMessage.Method == HttpMethod.Get || httpRequestMessage.Method == HttpMethod.Delete)
    {
        return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("SimpleWaitAndRetryPolicy");
    }
    return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("NoOpPolicy");
});

Next, add the hosted service we want to start.

services.AddSingleton<IHostedService, BusinessService>();

A hosted service is a class that implements IHostedService, more on this below.

Finally at the end of the the Main method, start the hosted service.

await builder.RunConsoleAsync();

For clarity, here is the full listing of the main method –

static async Task Main(string[] args)
{
    var builder = new HostBuilder()
        .ConfigureServices((hostContext, services) =>
        {
            IPolicyRegistry<string> registry = services.AddPolicyRegistry();

            IAsyncPolicy<HttpResponseMessage> httWaitAndpRetryPolicy =
                Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
                    .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(retryAttempt));

            registry.Add("SimpleWaitAndRetryPolicy", httWaitAndpRetryPolicy);

            IAsyncPolicy<HttpResponseMessage> noOpPolicy = Policy.NoOpAsync()
                .AsAsyncPolicy<HttpResponseMessage>();

            registry.Add("NoOpPolicy", noOpPolicy);

            services.AddHttpClient("JsonplaceholderClient", client =>
            {
                client.BaseAddress = new Uri("https://jsonplaceholder.typicode.com");
                client.DefaultRequestHeaders.Add("Accept", "application/json");
            }).AddPolicyHandlerFromRegistry((policyRegistry, httpRequestMessage) =>
            {
                if (httpRequestMessage.Method == HttpMethod.Get || httpRequestMessage.Method == HttpMethod.Delete)
                {
                    return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("SimpleWaitAndRetryPolicy");
                }
                return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("NoOpPolicy");
            });
                
            services.AddSingleton<IHostedService, BusinessService>();
        });

    await builder.RunConsoleAsync();
}

The hosted service
The hosted service is where you put your business logic, it is a simple c# class that implements IHostedService giving it two methods, StartAsync and StopAsync.

Its constructor takes an IHttpClientFactory as a parameter, which is satisfied by the dependency injection infrastructure.

public BusinessService(IHttpClientFactory httpClientFactory)
{
    _httpClientFactory = httpClientFactory;
}

From StartAsync, you can do anything you need.

In this example I call another method which in turn uses the HttpClientFactory to get an instance of a HttpClient to make requests to the the remote server. The requests are executed inside the appropriate Polly policy.

public async Task StartAsync(CancellationToken cancellationToken)
{
    await MakeRequestsToRemoteService();
}

public async Task MakeRequestsToRemoteService()
{
    HttpClient httpClient = _httpClientFactory.CreateClient("JsonplaceholderClient");
    var response = await httpClient.GetAsync("/photos/1");
    Photo photo = await response.Content.ReadAsAsync<Photo>();
    Console.WriteLine(photo);
}

Full source code available here.

Polly, HttpClientFactory and the Policy Registry – choosing the right policy based on the HTTP request

Full source code available here.

The release of .NET Core 2.1 has made using HttpClient much easier. If you have been using HttpClient for a while you will know about its limitations around reuse and DNS expiry, I wrote about this some time ago. The HttpClientFactory takes care of the problems addressed in that post.

Along with those improvements the HttpClientFactory now makes it very easy to add Polly policies that will be executed whenever you create a HttpClient with the factory. This means you define the polices in one place, add them to the factory and forget about them! No more code is needed to add resilience to each of your calls. The factory takes care of creating the client and applying the policy, you can’t even tell that Polly is protecting your call where you make it.

For more on the HttpClientFactory check out Steve Gordon’s series of articles.

A simple example
Here’s a simple example of the HttpClientFactory in use. This adds the factory to our dependency injection container, sets the base address of the remote server and lets us configure the client. It also adds a simple retry policy that checks if the response is NOT a success code and retries up to three times.

services.AddHttpClient("OrderApiServer", client =>
{
    client.BaseAddress = new Uri("http://localhost:57696/api/");
    client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandler(Policy.HandleResult<HttpResponseMessage>
    (r => !r.IsSuccessStatusCode).RetryAsync(3));

When you create a HttpClient now, it will include the policy.

var httpClient = _httpClientFactory.CreateClient("OrderApiServer");

And when you execute a request it the policy will retry up to three times if necessary.

var result = await httpClient.GetAsync("api/SomeEndpoint);

This might be great if all you are going to is perform GET’s and every call you make is safe and idempotent. But what if you want to use GET, PUT, POST and DELETE, you won’t want to retry all of those requests?

This is easy too. You add a policy registry will all the policies you want to use. Say, a retry policy, a wait and retry policy and no op policy, and provide a selector method to pick the right one based on the HTTP verb (or even the endpoint you are requesting).

Using a Policy Registry with the HttpClientFactory

Step 1

Add the policy registry to the Service Collection and add the policies to the registry.

public void ConfigureServices(IServiceCollection services)
{
    IPolicyRegistry<string> registry = services.AddPolicyRegistry(); // creates the registry and adds it to the service collection

    IAsyncPolicy<HttpResponseMessage> httpRetryPolicy =
        Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
            .RetryAsync(3);

    registry.Add("SimpleHttpRetryPolicy", httpRetryPolicy);

    IAsyncPolicy<HttpResponseMessage> httWaitAndpRetryPolicy =
        Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
            .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(retryAttempt));

    registry.Add("SimpleWaitAndRetryPolicy", httWaitAndpRetryPolicy);

    IAsyncPolicy<HttpResponseMessage> noOpPolicy = Policy.NoOpAsync()
        .AsAsyncPolicy<HttpResponseMessage>();

    registry.Add("NoOpPolicy", noOpPolicy);
    //snip..

Step 2
Now that we have the three polices and the registry, lets add the HttpClientFactory to the Service Collection.

services.AddHttpClient("OrderApiServer", client =>
{
    client.BaseAddress = new Uri("http://localhost:57696/api/");
    client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandlerFromRegistry(PolicySelector);

Note the PolicySelector, this is the method I use to choose the right policy for each request.

Step 3
This method is simple, it takes the registry and the HttpRequest as arguments and grabs the policy from the registry based on the HTTP verb.

private IAsyncPolicy<HttpResponseMessage> PolicySelector(IReadOnlyPolicyRegistry<string> policyRegistry, HttpRequestMessage httpRequestMessage)
{
    if (httpRequestMessage.Method == HttpMethod.Get)
    {
        return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("SimpleHttpRetryPolicy");
    }
    else if (httpRequestMessage.Method == HttpMethod.Post)
    {
        return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("NoOpPolicy");
    }
    else
    {
        return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("SimpleWaitAndRetryPolicy");
    }
}         

You could use a lambda instead of a method when creating the HttpClientFactory. For example if if you had two policies to choose from –

.AddPolicyHandlerFromRegistry((policyRegistry, message) => policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>(message.Method == HttpMethod.Get ? "SimpleHttpRetryPolicy" : "NoOpPolicy"));

That’s the hard work done.

Step 4
Let’s take a look at the controller.

public class OrderController : ControllerBase
{
    private readonly IHttpClientFactory _httpClientFactory;

    public OrderController(IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
    }


    [HttpGet("{orderId}")]
    public async Task<ActionResult> Get(int orderId)
    {
        //snip
        string requestEndpoint = $"invoice/{orderId}";

        var httpClient = _httpClientFactory.CreateClient("OrderApiServer");

        HttpResponseMessage response = await httpClient.GetAsync(requestEndpoint);

		//handle response
    }	
}

The only thing I’m passing into the constructor is the HttpClientFactory, I don’t have any Polly using statements.

In the GET method you can’t even tell that there is a policy around the HttpClient.GetAsync(..)

All the Polly work is done in the Startup.cs, the policies are defined, added to the registry, the registry is added to the HttpClientFactory and the HttpClientFactory is added to the services collections.

Now all your HttpClient requests are executed inside a policy.

For more on this, check out Dylan’s Polly and HttpClientFactory documentation.

Full source code available here.