Simmy Chaos Engine for .NET – Part 2, Resilience and Injected Faults

Full source code here.

I wrote a blog on using a Simmy Fault policy a few days ago, it is very simple to use by itself, but it is far more useful when used in combination with a resilience policy.

Take the scenario where I have added retry logic to my requests, but the requests never fail, how do I know the retry logic works the way I expect. This is where the fault policy shines.

I’m going to build on the previous post where I injected a fault 50 percent of the time when making http requests. In the previous post I had Simmy throw a HttpRequestException but this time I’m going to have Simmy return a HttpResponseMessage containing an InternalServerError.

Along with the fault policy, I am going add a simple retry policy, retrying up to three times.

Here’s how it will work –

The fault policy executes –

If the fault policy is active and a fault is to be returned (remember this happens 50% of the time) a InternalServerError is returned. The HttpClient does NOT make a request to the remote service

The retry policy executes. The handles clause sees the InternalServerError and the behavior clause of the retry kicks in and performs the retry

The fault policy executes again

The preceding two steps repeat until a request succeeds or the retry policy reaches is limit of 3 retries

 

If the fault policy is not active, or a fault is NOT to be returned

the HTTP client executes the request to the remote service

the handles clause of the retry determines whether a retry should be performed

 

This is what it looks like –

At the center is the HttpReqeust, around it is the fault policy and around everything is the retry policy.

To achieve all this in code is very simple, in ConfigureServices add the fault policy.

public void ConfigureServices(IServiceCollection services)
{
	HttpResponseMessage faultHttpResponseMessage = new HttpResponseMessage(HttpStatusCode.InternalServerError)
	{
		Content = new StringContent("Simmy swapped the Ok for an Internal Server Error")
	};
	AsyncInjectOutcomePolicy<HttpResponseMessage> faultPolicy = MonkeyPolicy.InjectFaultAsync(
		faultHttpResponseMessage,
		injectionRate: .5,
		enabled: () => true 
	);

When the fault policy executes, 50% of the time it will return an InternalServerError response, preempting the real request to the remote service.

Next is the retry policy, its behavior clause triggers if the HttpResponseMessage is not a success.

IAsyncPolicy<HttpResponseMessage> retryPolicy = Policy
	.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
	.RetryAsync(3, onRetry: (message, retryCount) =>
	{
		Console.WriteLine($"Retry: {retryCount}");
	});

Wrap the retry policy around the fault policy.

AsyncPolicyWrap<HttpResponseMessage> faultAndRetryWrap = Policy.WrapAsync(retryPolicy, faultPolicy);

Finally, create the HttpClientFactory, passing in the policy wrap. If you are not familiar with how the HttpClientFactory works read Steve Gordon’s series or my blog posts on it. If you want know how it works with Polly, take a look at my posts on that topic.

services.AddHttpClient("OpenBreweryDb", client =>
{
	client.BaseAddress = new Uri("https://api.openbrewerydb.org/breweries/");
	client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandler(faultAndRetryWrap);

To fire of the HttpClient request inside the BreweryController remains the same –

string requestEndpoint = $"?by_state={state}&by_name={name}";
var httpClient = _httpClientFactory.CreateClient("OpenBreweryDb");
var response = await httpClient.GetAsync(requestEndpoint);

Full source code here.

Simmy Chaos Engine for .NET – Part 1, Injecting Faults

Full source code here.

For quite a while I have been writing, presenting and making videos about using Polly to make applications more stable.

With this post I’m starting a series on how break your applications with a chaos engine, the kind you might have heard about from the likes of NetFlix.

The team behind Polly released a chaos engineering module in June 2019. It allows you to inject faults, latency and arbitrary behavior into your application.

There are two general ways of using these policies –
1. Within your own business logic (if you have access to it).
2. Within the calls to business logic (if you can’t or don’t want to change it).

I’m going to deal with the latter case in this post. I want to see how my code responds when failures occur in third party dependencies, since I have no control over those dependencies I have to introduce the errors myself.

Making a good service go bad
In the provided code I have dependency on remote service and I want make it seem that that service suffers from a high percentage of faults.
This example will make GET requests to an API that looks up breweries, from what I can tell it is a highly reliable API, but I will make look as though 50% of requests to it fail. The point of doing this is to see how my application reacts to these errors – does it do so gracefully, or does it fall on its face.

If you made requests to the API without interfering with them you should see a 100% success rate. But I want to see failures, so I’ll use the Fault Policy to inject exceptions.

First, I added two NuGet packages to my project – Polly.Contrib.Simmy and Microsoft.Extensions.Http.Polly to the project.

In Startup.cs I add the fault policy.

AsyncInjectOutcomePolicy<HttpResponseMessage> faultPolicy = MonkeyPolicy.InjectFaultAsync<HttpResponseMessage>(
	new HttpRequestException("Simmy threw an exception"), 
		injectionRate: .5,
		enabled: () => true
	);

The parameters specify the exception to throw, the percentage of requests that throw the exception, and whether the policy is enabled. You can use set the injection rate and whether the policy is enabled via configuration, but I’m not going to show that here.
In this case, I am throwing an HttpRequestException 50% of the time, and the policy is enabled.

Then create a HttpClientFactory for the OpenBreweryDb

services.AddHttpClient("OpenBreweryDb", client =>
{
	client.BaseAddress = new Uri("https://api.openbrewerydb.org/breweries/");
	client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandler(faultPolicy);

If you are not yet using the HttpClientFactory, the source code has an example for you too, check the BreweryNoHttpClientFactoryController.

Start up the application, it should open on port 5000 and make a request to look up Night Shift Brewing in Massachusetts. There is a 50% chance of an exception occurring with each request.

Try it out. You’ll see that my application does not handle these failures very well, it simply returns the error to original caller.

That’s it, you are now starting down the road of mixing in some chaos into your application.

What then?
Great, my application fails, but we always knew it could. There’s not much point introducing failures unless you plan to handle them in some way, and this is where you might add a retry policy to improve you likelihood of a successful request.

Of course, the Polly retry, or wait and retry policies would be ideal. I’ll show how to do just that in the next post.

Full source code here.

Streaming Results from Entity Framework Core and Web API Core

Full source code here.
The code provided will not compile until you make a change in seeder.cs, the way it’s written it generates 500,000 rows in a local db. Set this to whatever value you want.

In this post I’m going to show you how to return an unlimited number of results from a database via a Web API application while keeping your memory usage low and constant. In effect, you are going to stream results from the database with Entity Framework.

To achieve this you need to do two things, disable tracking and return a not materialize your data inside the action method.

Turn off tracking
By default Entity Framework tracks entities that you read from a database. Tracking allows EF to determine what, if any, properties have changed since being loaded, then EF can save just the relevant changes. But tracking takes up memory and CPU.

If you have no intention of changing these entities there is no point in tracking them.

There are two ways of doing this, at point where you make the request to the database, or globally for the whole context.

To use AsNoTracking for a single request it looks like this –
_salesContext.Products.AsNoTracking().Where(..)

To use it for all requests to that context set AsNoTracking in the constructor of the context.

public SalesContext(DbContextOptions<SalesContext> options) : base(options)
{
	ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;
}

That’s the first step taken care of, now we make sure not to turn our data into objects inside the controller.

Do not materialize
If you follow tutorials on Entity Framework Core and Web API you will see examples like this –

[HttpGet]
public async Task<ActionResult> Get()
{
    List<Product> products = await _salesContext.Products.ToListAsync();

    return Ok(products);
}

In this example, all the products in the database are read and put into a list of products, the return does not execute until after all the data has been read, so you are waiting for this to complete before getting any results. You are also storing the whole list of products in memory (you are probably also tracking the entities).

Instead of that you can do the following –

[HttpGet]
public ActionResult GetStreaming()
{
	IQueryable<Product> products = _salesContext.Products.AsNoTracking();

	return Ok(products);
}

Now, the products do not materialize, there is no list to store in memory, tracking is turned off and the action method begins returning data almost immediately.

Your memory profile will be almost same whether you are returning a few hundred or a few million rows of data.

Example of Memory Usage
Below are screenshots from Visual Studio 2017 of the application running showing the amount of memory consumed for a variety of requests. The first shows the memory usage when I requested a single row from the database

The following images show the memory usage when streaming and not streaming results sets of 100,000, 200,000, 300,000 and 500,000.

106 MB used for a single row

 

100,000 rows returned. On the left is with streaming, on the right without.

 

200,000 rows returned. On the left is with streaming, on the right without.

 

300,000 rows returned. On the left is with streaming, on the right without.

 

500,000 rows returned. On the left is with streaming, on the right without.

 

Summary of Results

It’s obvious that streaming maintains a uniform memory footprint while the memory consumed for non-streaming grows with the number of rows returned.

What you don’t see here is that streaming requests complete more quickly. If you download the attached code and use a local mdf file, you probably won’t see much of a difference in speed, but if your application connects to remote database you will see approximately a 20% improvement in speed.

Summary
For streaming to work you need to turn of tracking of entities and return an IQueryable from the action method.

Full source code here.
The code provided will not compile until you make a change in seeder.cs, the way it’s written it generates 500,000 rows in a local db. Set this to whatever value you want.

Getting Error Messages and Status Codes from Typed HttpClients

Full source code here.

I have been using HttpClientFactory for a while and have generally preferred using named clients over typed clients. With a named client you get the HttpClient just before you are going to use it, and you have full access to the HttpResponse at the point you make the call to the remote service.

var httpClient = _httpClientFactory.CreateClient("InvoiceApi");
HttpResponseMessage response = await httpClient.GetAsync(requestEndpoint);

This is great when things go wrong because you can check the HttpStatus code and the ReasonPhrase, any message, and then take appropriate action.

If you are using a typed client, it might look something like this –

public class InvoiceClient
{
	private readonly HttpClient _client;

	public InvoiceClient(HttpClient client)
	{
		_client = client;
	}
	
	public async Task<Invoice> GetInvoice(int orderId)
	{
		var response = await _client.GetAsync($"invoice/{orderId}");
		Invoice invoice = await response.Content.ReadAsAsync<Invoice>();
		return invoice;
	}
}

And then you use it –

var invoice = await invoiceClient.GetOrder(id);

If the request worked, you are fine, but if it didn’t you and got an error, then the deseriaization will fail and an exception will be thrown, but that exception won’t tell you very much about the cause of the original failure.

Returning a typed response
Here’s the alternative. I use a typed response that includes the HttpStatusCode, the ReasonPhrase and the content I actually want, in this case the Invoice.

public class TypedResponse<T>
{
	public T Content { get; set; }
	public HttpStatusCode HttpStatusCode { get; set; }
	public string ReasonPhrase { get; set; }
	public string Message { get; set; }
}

Here’s how to change the Get method in the Invoice.

[HttpGet("{orderId}")]
public async Task<ActionResult> Get(int orderId)
{
	// Pretend to load the order from the db. 
	Order order = new Order { OrderId = orderId, OrderDate = new DateTime(2018, 6, 14), CustomerId = 1001 };

	var invoiceResponse = await _invoiceClient.GetInvoice(orderId);

	if (invoiceResponse.Content != null)
	{
		order.Cost = invoiceResponse.Content.Cost;
		return Ok(order);
	}
	return StatusCode((int)invoiceResponse.HttpStatusCode, invoiceResponse.Message);
}

And finally, how to use it in the controller.

var invoiceAndStatus = await _invoiceClient.GetInvoice(orderId);

if (invoiceAndStatus.Content != null)
{
    order.Cost = invoiceAndStatus.Content.Cost;
    return Ok(order);
}
return StatusCode((int)invoiceAndStatus.HttpStatusCode, invoiceAndStatus.Message);

Run the code, it will open your browser to http://localhost:5000/api/order/2. The OrderController will call the InvoiceController, the first request will fail, but you will see the error message from the InvoiceController.

Hit F5 in the browser to make another request to the OrderController, again this calls the InvoiceController, but this time the response is a success.

Full source code here.

Selectively Caching Values Inside HttpResponseMessage with Polly – caching series 3/3

Full source code here.

This is the last of three posts on caching with Polly. The first showed how to selectively cache HttpResponseMessages based on the status code of the response.

The second showed how to cache values inside the HttpResponseMessage rather than the whole of the response.

This final post will show how to selectively cache values inside the response based on the status of the response.

If you have read the previous posts the content of this will feel familiar.

Changes to Startup
Like the previous post, add a memory cache, a policy registry and a HttpClientFactory to the service collection.

public void ConfigureServices(IServiceCollection services)
{
    services.AddMemoryCache();
    services.AddSingleton<IAsyncCacheProvider, MemoryCacheProvider>();

    IPolicyRegistry<string> registry = services.AddPolicyRegistry();

    services.AddHttpClient("RemoteServer", client =>
    {
        client.BaseAddress = new Uri("http://localhost:5000/api/");
        client.DefaultRequestHeaders.Add("Accept", "application/json");
    });

    services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
}

What’s new is in the Configure method, fist step is to add the registry to the method parameters, DI will take care of this for us.

public void Configure(IApplicationBuilder app, IHostingEnvironment env,
    IAsyncCacheProvider cacheProvider, IPolicyRegistry<string> registry)
{

Next is the cache filter, this one examines the tuple of int and HttpStatusCode and, if the HttpStatusCode is an OK the tuple will be cached for the duration specified. If the respones is not OK the filter will set the cache time to 0 which is interpreted as do not cache by the cache policy.

Func<Context, (int value, HttpStatusCode StatusCode), Ttl> cacheOnly200OkFilter =
	(context, result) => new Ttl(
		timeSpan: result.StatusCode == HttpStatusCode.OK ? TimeSpan.FromSeconds(5) : TimeSpan.Zero,
		slidingExpiration: true
	);

And this is the cache policy that uses the filter above to store the tuple.

IAsyncPolicy<(int, HttpStatusCode)> cachePolicy =
	Policy.CacheAsync<(int, HttpStatusCode)>(
		cacheProvider.AsyncFor<(int, HttpStatusCode)>(), 
		new ResultTtl<(int, HttpStatusCode)>(cacheOnly200OkFilter),
		onCacheError: null);

Lastly, add the policy to the registry.

registry.Add("cacheOnly200OkFilter", cachePolicy);

Changes to the Controller
Over in the CatalogController there many changes.

At the top I take in the HttpClientFactory and policy registry.

private readonly IHttpClientFactory _httpClientFactory;
private readonly IPolicyRegistry<string> _policyRegistry;

public CatalogController(IHttpClientFactory httpClientFactory, IPolicyRegistry<string> policyRegistry)
{
	_httpClientFactory = httpClientFactory;
	_policyRegistry = policyRegistry;
}

Inside the Get method, I setup grab the HttpClient from the HttpClientFactory, set the endpoint of the remote service, take the cache policy from the registry and setup the context for the cache policy to cache based on the incoming request. All of this is similar to normal usage of the Polly cache.

[HttpGet("{id}")]
public async Task<IActionResult> Get(int id)
{
    string requestEndpoint = $"inventory/{id}";
    var httpClient = _httpClientFactory.CreateClient("RemoteServer");

    var cachePolicy = _policyRegistry.Get<CachePolicy<(int, HttpStatusCode)>>("cacheOnly200Ok");
    Context policyExecutionContext = new Context($"GetInventoryById-{id}");	

As in the previous post I don’t call the HttpClient from inside the cache policy, instead I need to call something that returns the int and the HttpStatusCode. I use a local function that makes the request to the remote service, checks the response, deserializes the value and returns a tuple.

(int ItemsInStock, HttpStatusCode StatusCode) itemsInStockAndStatusCode =
    await cachePolicy.ExecuteAsync(context => MakeRequest(), policyExecutionContext);

// local function
async Task<(int count, HttpStatusCode statusCode)> MakeRequest()
{
    HttpResponseMessage response = await httpClient.GetAsync(requestEndpoint);

    if (response.IsSuccessStatusCode)
    {
        int itemsInStock = JsonConvert.DeserializeObject<int>(await response.Content.ReadAsStringAsync());
        return (itemsInStock, response.StatusCode);
    }

    return (0, response.StatusCode);
}

When the cachePolicy is executed it first checks if there is value already in the cache, if there is, that will be returned and the local function will not be called. If there is no value in the cache, the local function is called, if the response of the local function is in the 200 range, the value is stored in the cache.

At this point, the action method has not returned anything to its caller.

Below the local function I check the tuple returned from the call to the cache policy and return a response.

if (itemsInStockAndStatusCode.StatusCode != HttpStatusCode.OK)
{
     return StatusCode((int)itemsInStockAndStatusCode.StatusCode);
}
return Ok(itemsInStockAndStatusCode.ItemsInStock);

For clarity, here is the full Get method.

[HttpGet("{id}")]
public async Task<IActionResult> Get(int id)
{
    string requestEndpoint = $"inventory/{id}";
    var httpClient = _httpClientFactory.CreateClient("RemoteServer");

    var cachePolicy = _policyRegistry.Get<CachePolicy<(int, HttpStatusCode)>>("cacheOnly200Ok");
    Context policyExecutionContext = new Context($"GetInventoryById-{id}");

    (int ItemsInStock, HttpStatusCode StatusCode) itemsInStockAndStatusCode =
        await cachePolicy.ExecuteAsync(context => MakeRequest(), policyExecutionContext);

    async Task<(int count, HttpStatusCode statusCode)> MakeRequest()
    {
        HttpResponseMessage response = await httpClient.GetAsync(requestEndpoint);

        if (response.IsSuccessStatusCode)
        {
            int itemsInStock = JsonConvert.DeserializeObject<int>(await response.Content.ReadAsSt
            return (itemsInStock, response.StatusCode);
        }

        // it doesn't matter what int you use here, it won't be cached as the StatusCode is not a
        return (0, response.StatusCode);
    }

    if (itemsInStockAndStatusCode.StatusCode != HttpStatusCode.OK)
    {
         return StatusCode((int)itemsInStockAndStatusCode.StatusCode);
    }
    return Ok(itemsInStockAndStatusCode.ItemsInStock);
}

Here endeth the series on caching.

Full source code here.

dotnet new failing with Error: Invalid parameter(s)

If you are using Visual Studio Code and you want to add a new project to your workspace, the easiest thing to do is something like –

dotnet new xunit -f netcoreapp2.1

But you may get an error –

Error: Invalid parameter(s):
   -f netcoreapp2.1
      'netcoreapp2.1' is not a valid value for -f (Framework)

Here’s how to solve it.

Run –

dotnet --info

You will get a list of installed frameworks, your list will be different than mine –

This will show the list of frameworks installed 
.NET Core SDKs installed:
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.402 [C:\Program Files\dotnet\sdk]
  2.1.503 [C:\Program Files\dotnet\sdk]
  2.2.203 [C:\Program Files\dotnet\sdk]

Choose a version of the framework that you want, in my case I’ll use 2.1.503

Run this –

dotnet new globaljson --sdk-version 2.1.503

Now run –

dotnet new xunit -f netcoreapp2.1

and the new project will be created.

Caching Values Inside HttpResponseMessage with Polly – caching series 2/3

Full source code here.

In this, the second of three posts on caching in Polly, I will show how to cache the values returned inside a HttpResponseMessage as opposed to caching the response with all its various elements.

The first post explained how to selectively cache a response, based on the Http StatusCode. The next post will show how to combine these two ideas and selectively cache the values inside the HttpResponseMessage.

Caching the whole response or the value
Caching the whole HttpResponseMessage is useful if your business logic depends on interrogating the response for values outside the main payload or you want a quick and easy caching solution.

If on the other hand, you only need the payload, there is no need to cache the whole response.

Both cases are plausible, and it is up to do decide which is more appropriate for your application.

How to cache the value inside HttpResponseMessage
The first step is to set the cache inside ConfigureServices.

public void ConfigureServices(IServiceCollection services)
{
    services.AddMemoryCache();
    services.AddSingleton<IAsyncCacheProvider, MemoryCacheProvider>();

Then in Configure, setup the policy to store the type you expect to find in the response. In this example it is in a int, but it could easily be a complex layered model representing anything.

public void Configure(IApplicationBuilder app, IHostingEnvironment env,
    IAsyncCacheProvider cacheProvider, IPolicyRegistry<string> registry)
{
    CachePolicy<int> cachePolicy =
        Policy.CacheAsync<int>(cacheProvider, TimeSpan.FromSeconds(10));

    registry.Add("CachingPolicy", cachePolicy);

Note, how I specified an int rather than a HttpResponseMessage.

In the previous post, I used the HttpClientFactory to select the policy from a Polly policy registry. I could do that because the type I was storing was HttpResponseMessage, that is not the case now and HttpClientFactory will not play nice when trying to cache anything other than a HttpResponseMessage. Instead I’m going to pass the policy registry directly to the controller and get the policy from this, I am still using the HttpClientFactory to pass HttpClient, but not for policy selection.

Here is how the controller starts.

public class CatalogController : Controller
{
    private readonly IHttpClientFactory _httpClientFactory;
    private readonly IPolicyRegistry<string> _policyRegistry;

    public CatalogController(IHttpClientFactory httpClientFactory, IPolicyRegistry<string> policyRegistry)
    {
        _httpClientFactory = httpClientFactory;
        _policyRegistry = policyRegistry;
    }

And here is the GET method.

[HttpGet("{id}")]
public async Task<IActionResult> Get(int id)
{
    string requestEndpoint = $"inventory/{id}";
    var httpClient = _httpClientFactory.CreateClient("RemoteServer");

    var cachePolicy = _policyRegistry.Get<CachePolicy<int>>("CachingPolicy");
    Context policyExecutionContext = new Context($"GetInventoryById-{id}");

    int itemsInStock =
        await cachePolicy.ExecuteAsync(context => GetItemsInStockCount(), policyExecutionContext);

    async Task<int> GetItemsInStockCount()
    {
        HttpResponseMessage response = await httpClient.GetAsync(requestEndpoint);

        if (response.IsSuccessStatusCode)
        {
            int itemsInStockCount = await response.Content.ReadAsAsync<int>();
            return itemsInStockCount;
        }

        return 0;
    }

    return Ok(itemsInStock);
}

The first few lines are straight forward, setup the endpoint, get the HttpClient, get the cache policy. Like in the previous post, I use the context when placing items into the cache and retrieving them.

Because I want to cache the value inside the HttpResponse (and not the response itself) I can’t call the HttpClient from inside the policy’s exec method. Instead I need to call something that returns the int from inside the HttpResponse.

The easiest way of doing this is to use a local function which makes the HttpClient request, deserializes the response and returns the int from the response.
In this post I am assuming that all values inside the response should be cached, I am NOT checking the status code of the response.

The policy then caches the int for the period specified in Startup.

That’s all there is to it.

If you are wondering how to selectively cache values inside the HttpResponse, tune in for the next post in this series.

Full source code here.

Selectively Caching a HttpResponseMessage with Polly – caching series 1/3

Full source code here.

When I give talks on Polly I show how to use it in a Web API application that calls another Web Api application. I give a simple example of how to use the cache policy where I store the whole of the HttpResponseMessage. At the end of the talk I often get a couple of questions –

1. is it possible to cache the response only when the status code is in the 200 range.
2. is it possible to cache the values inside the response instead of the whole response.

The answer to both of these is yes, and in this series of three articles I will explain how to do both of these things, and how how to combine them to cache a value inside a response only if the response is in the 200 range.

This article will demonstrate how to selectively cache HttpResponseMessages based on the Http StatusCode of the response.

As described in an earlier post, using a cache policy is a little more involved than using other policies. See this post for more info.

Changes in Startup
In Startup inside ConfigureServices I setup the memory cache, the HttpClientFactory and a policy registry (I’m not going to go into a detailed explanation of this code as it is already explained in the earlier article and you have all the source code).

All the action for this post takes place in the Configure method.

Creating the cache filter
The cache timeout filter, it sets the length of time to cache a value. The filter here will cache a response for 10 seconds if the response indicates a success, otherwise it caches the response for 0 seconds, in this case the Polly code does not perform any caching.

public void Configure(IApplicationBuilder app, IHostingEnvironment env,
    IAsyncCacheProvider cacheProvider, IPolicyRegistry<string> registry)
{
    Func<Context, HttpResponseMessage, Ttl> cacheOnly200OkFilter =
        (context, result) => new Ttl(
            timeSpan: result.StatusCode == HttpStatusCode.OK ? TimeSpan.
            slidingExpiration: true
        );

The cache policy says that it caches HttpResponseMessages and uses the filter to decide if a response should be cached.

IAsyncPolicy<HttpResponseMessage> cacheOnlyOkResponsePolicy =
    Policy.CacheAsync<HttpResponseMessage>(
        cacheProvider.AsyncFor<HttpResponseMessage>(), //note the .AsyncFor<HttpResponseMessage>
        new ResultTtl<HttpResponseMessage>(cacheOnly200OkFilter),
        onCacheError: null
    );

Note the AsyncFor, this is very important. Thanks to Dylan Reisenberger for his help here, I spent too many hours trying to figure the overloads with no success.

Try it out in the provided solution, hit F5 and put some breakpoints in the controllers and on the filter in Startup.

That’s it, you now have selective caching of responses.

In the next post, I’ll show you how to cache the value inside the response instead of the whole response.

Full source code here.

Testing Your Code When Using Polly

Full source code here.

When developing an application with Polly you will also probably want to write some unit tests.

Here are the scenarios I test for –

1. How my code behaves when the policy throws an exception, such as TimeoutRejectionException, BulkheadRejectedException or BrokenCircuitException.
2. How my code behaves when a policy becomes active and changes the outcome of a call, such as when an unreliable request works because Polly performs a retry.
3. What my code should do if there was no policy in place.

I have a few classes to demonstrate these scenarios, BusinessLogic.cs and OtherBusinessLogic.cs are the classes under test. ErrorProneCode.cs is the unreliable class that I will mock and pass mocked policies into.

For the first case I use Moq to mock the error prone code so it returns an incorrect value.

[Fact]
public void Should_Return_999_When_TimeoutRejectedException_Thrown()
{
    //Arrange 
    Mock<IErrorProneCode> mockedErrorProneCode = new Mock<IErrorProneCode>();
    mockedErrorProneCode.Setup(e => e.GetSomeNumber()).Returns(0);

    Mock<ISyncPolicy> mockedPolicy = new Mock<ISyncPolicy>();
    mockedPolicy.Setup(p => p.Execute(It.IsAny<Func<int>>())).Throws(new TimeoutRejectedException("Mocked Timeout Exception"));

    IBusinessLogic businessLogic = new BusinessLogic(mockedPolicy.Object, mockedErrorProneCode.Object);

    //Act
    // if there is a TimeoutRejectedException in this CallSomeSlowBadCode it will return 999
    int num = businessLogic.CallSomeSlowBadCode();

    //Assert
    Assert.Equal(999, num);
}

In the next case I verify that the application has correctly used the retry policy method. I use a seeded random number generator that produces an known sequence to return values from the ErrorProneCode class.

  
[Fact]
public void Should_Return_Odd_When_Retry()
{
    //Arrange 
    Mock<IErrorProneCode> mockedErrorProneCode = new Mock<IErrorProneCode>();

    Random rnd = new Random(1); // rnd.Next(10) retruns 2, 1, 4 

    mockedErrorProneCode.Setup(e => e.GetSomeNumber()).Returns(() => rnd.Next(10));

    ISyncPolicy<int> policy = Policy.HandleResult<int>(i => i %2 !=1) // retry if the number is not odd.
        .Retry(1);
    OtherBusinessLogic otherBusinessLogic = new OtherBusinessLogic(policy, mockedErrorProneCode.Object);

    //Act
    int num = otherBusinessLogic.CallSomeCodeThatNeedsToBeRetried();

    //Assert
    Assert.Equal(1, num % 2);
}

Finally, I want to verify that my code will work if no Polly policy is in use. To do this, I pass in a NoOp policy.

[Fact]
public void Should_Return_Even_With_NoOp()
{
    //Arrange 
    Mock<IErrorProneCode> mockedErrorProneCode = new Mock<IErrorProneCode>();

    Random rnd = new Random(1); // rnd.Next(10) retruns 2, 1, 4 

    mockedErrorProneCode.Setup(e => e.GetSomeNumber()).Returns(() => rnd.Next(10));

    ISyncPolicy<int> policy = Policy.NoOp<int>();
    OtherBusinessLogic otherBusinessLogic = new OtherBusinessLogic(policy, mockedErrorProneCode.Object);

    //Act
    int num = otherBusinessLogic.CallSomeCodeThatNeedsToBeRetried();

    //Assert
    Assert.Equal(0, num % 2); //even number
}

Full source code here.

Calling Generic Methods Using Reflection

Full source code here.

Accessing generic methods by reflection is not easy. A while ago I needed to do just that and found relatively little information out there. So I dug in a figured it out for myself.

Here are some of the possible ways that you could execute generic methods by reflection –

  • Execute a Generic Instance Method in a Non-Generic Class
  • Execute Generic Static Method In a Non-Generic Class
  • Execute Generic Overloaded Static Method
  • Execute NonGeneric Method in a Generic Class

There are other variations, but you should be able to figure out how to write them based on the ones shown here.

In the classes PrinterA and PrinterB, there are a variety of generic and one non-generic methods that I will call using reflection from Program.

The code will explain things better than I can, so here is the listing.

using System;
using System.Collections.Generic;

namespace ExecuteGenericMethodByReflection
{
    public class PrinterA<T>
    {
        public void Print(IEnumerable<T> items)
        {
            Console.WriteLine();
            foreach (var item in items)
            {
                Console.Write($"{item} ");
            }
        }
    }
}
using System;
using System.Collections.Generic;

namespace ExecuteGenericMethodByReflection
{
    public class PrinterB
    {
        public void PrintSimple<T>(IEnumerable<T> items)
        {
            Console.WriteLine();

            foreach (var item in items)
            {
                Console.Write($"{item} ");
            }
            Console.WriteLine();

        }

        public static void PrintPrefixAndSuffix<T>(IEnumerable<T> items, string prefix, string suffix)
        {
            Console.Write(prefix);
            foreach (var item in items)
            {
                Console.Write($" {item} ");
            }
            Console.Write(suffix);
            Console.WriteLine();
        }

        //This and the next method are overloads
        public static void PrintPrefix<T>(IEnumerable<T> items, string prefix)
        {
            Console.Write(prefix);

            foreach (var item in items)
            {
                Console.Write($" {item}");
            }
            Console.WriteLine();

        }

        public static void PrintPrefix<T>(IEnumerable<T> items, int prefix)
        {
            Console.WriteLine(prefix);

            foreach (var item in items)
            {
                Console.Write($" {item}");
            }
            Console.WriteLine();
        }

    }
}

And here is program where all the crazy reflection takes place.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;

namespace ExecuteGenericMethodByReflection
{
    class Program
    {
        static void Main(string[] args)
        {
            Program p = new Program();

            List<int> numbers = new List<int>() { 1, 2, 3, 4, 5 };

            p.ExecuteGenericInstanceMethodInNonGenericClass(numbers);
            p.ExecuteGenericStaticMethodInNonGenericClass(numbers);
            p.ExecuteGenericOverloadedStaticMethod(numbers);

            p.ExecuteNonGenericMethodInGenericClass(numbers);

            Console.WriteLine("\n\nPress any key to exit...");
            Console.ReadKey();
        }

        private void ExecuteGenericInstanceMethodInNonGenericClass(object objectToPrint)
        {
            Type typeToPrint = objectToPrint.GetType().GetGenericArguments()[0];

            PrinterB genericPrinter = new PrinterB();
            MethodInfo printSimpleMethod = typeof(PrinterB).GetMethod("PrintSimple");
            MethodInfo printSimpleMethodToExecute = printSimpleMethod.MakeGenericMethod(typeToPrint);

            printSimpleMethodToExecute.Invoke(genericPrinter, new object[] { objectToPrint});
        }

        private void ExecuteGenericStaticMethodInNonGenericClass(object objectToPrint)
        {
            Type typeToPrint = objectToPrint.GetType().GetGenericArguments()[0];
            MethodInfo printPrefixAndSuffixMethod = typeof(PrinterB).GetMethod("PrintPrefixAndSuffix");
            MethodInfo genericprintPrefixAndSuffixMethodToExecute = printPrefixAndSuffixMethod.MakeGenericMethod(typeToPrint);

            genericprintPrefixAndSuffixMethodToExecute.Invoke(null, new object[] { objectToPrint, "My Prefix", "My Suffix" });
        }

        // Executing an overloaded method is more diffcult, the method has to be identified by its parameters
        private void ExecuteGenericOverloadedStaticMethod(object objectToPrint)
        {
            string prefixToPrint = "My Prefix";
            Type overloadType = prefixToPrint.GetType();
            Type typeToPrint = objectToPrint.GetType().GetGenericArguments()[0];
            MethodInfo genricMethodToExecute = GetOverloadedMethod(typeToPrint, overloadType);
            genricMethodToExecute.Invoke(null, new object[] { objectToPrint, prefixToPrint });
        }

        private void ExecuteNonGenericMethodInGenericClass(object objectToPrint)
        {
            Type typeToPrint = objectToPrint.GetType().GetGenericArguments()[0];

            Type printerAType = typeof(PrinterA<>);
            Type genericPrinterAType = printerAType.MakeGenericType(typeToPrint);

            object genericPrinterAInstance = Activator.CreateInstance(genericPrinterAType);
            MethodInfo printMethod = genericPrinterAType.GetMethod("Print");
            printMethod.Invoke(genericPrinterAInstance, new object[] { objectToPrint });
        }

        private MethodInfo GetOverloadedMethod(Type typeToPassToGenericMethod, Type overloadType)
        {
            IEnumerable<MethodInfo> printMethods = typeof(PrinterB).GetMethods().Where(m => m.Name == "PrintPrefix");
            MethodInfo printMethod = printMethods.Single(m => m.GetParameters().Any(p => p.ParameterType == overloadType));
            MethodInfo genericPrintMethod = printMethod.MakeGenericMethod(typeToPassToGenericMethod);
            return genericPrintMethod;
        }
    }
}

Full source code here.