Simmy Chaos Engine for .NET – Part 2, Resilience and Injected Faults

Full source code here.

Want to learn more about Polly? Check out my Pluralsight course on it.

I wrote a blog on using a Simmy Fault policy a few days ago, it is very simple to use by itself, but it is far more useful when used in combination with a resilience policy.

Take the scenario where I have added retry logic to my requests, but the requests never fail, how do I know the retry logic works the way I expect. This is where the fault policy shines.

I’m going to build on the previous post where I injected a fault 50 percent of the time when making http requests. In the previous post I had Simmy throw a HttpRequestException but this time I’m going to have Simmy return a HttpResponseMessage containing an InternalServerError.

Along with the fault policy, I am going add a simple retry policy, retrying up to three times.

Here’s how it will work -

The fault policy executes -

If the fault policy is active and a fault is to be returned (remember this happens 50% of the time) a InternalServerError is returned. The HttpClient does NOT make a request to the remote service

The retry policy executes. The handles clause sees the InternalServerError and the behavior clause of the retry kicks in and performs the retry

The fault policy executes again

The preceding two steps repeat until a request succeeds or the retry policy reaches is limit of 3 retries

If the fault policy is not active, or a fault is NOT to be returned

the HTTP client executes the request to the remote service

the handles clause of the retry determines whether a retry should be performed

This is what it looks like -

At the center is the HttpReqeust, around it is the fault policy and around everything is the retry policy.

To achieve all this in code is very simple, in ConfigureServices add the fault policy.

 1public void ConfigureServices(IServiceCollection services)
 2{
 3	HttpResponseMessage faultHttpResponseMessage = new HttpResponseMessage(HttpStatusCode.InternalServerError)
 4	{
 5		Content = new StringContent("Simmy swapped the Ok for an Internal Server Error")
 6	};
 7	AsyncInjectOutcomePolicy<HttpResponseMessage> faultPolicy = MonkeyPolicy.InjectFaultAsync(
 8		faultHttpResponseMessage,
 9		injectionRate: .5,
10		enabled: () => true 
11	);

When the fault policy executes, 50% of the time it will return an InternalServerError response, preempting the real request to the remote service.

Next is the retry policy, its behavior clause triggers if the HttpResponseMessage is not a success.

1IAsyncPolicy<HttpResponseMessage> retryPolicy = Policy
2	.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
3	.RetryAsync(3, onRetry: (message, retryCount) =>
4	{
5		Console.WriteLine($"Retry: {retryCount}");
6	});

Wrap the retry policy around the fault policy.

AsyncPolicyWrap<HttpResponseMessage> faultAndRetryWrap = Policy.WrapAsync(retryPolicy, faultPolicy);

Finally, create the HttpClientFactory, passing in the policy wrap. If you are not familiar with how the HttpClientFactory works read Steve Gordon’s series or my blog posts on it. If you want know how it works with Polly, take a look at my posts on that topic.

1services.AddHttpClient("OpenBreweryDb", client =>
2{
3	client.BaseAddress = new Uri("https://api.openbrewerydb.org/breweries/");
4	client.DefaultRequestHeaders.Add("Accept", "application/json");
5}).AddPolicyHandler(faultAndRetryWrap);

To fire of the HttpClient request inside the BreweryController remains the same -

1string requestEndpoint = $"?by_state={state}&by_name={name}";
2var httpClient = _httpClientFactory.CreateClient("OpenBreweryDb");
3var response = await httpClient.GetAsync(requestEndpoint);

Full source code here.