Polly, HttpClientFactory and the Policy Registry in a console application

Full source code available here.

How to use the HttpClientFactory with a console application is not immediately obvious. I thought it would be a simple matter, but it’s not because it relies on the dependency injection infrastructure you get with a web application. I’ve written about using HttpClientFactory with Polly in a Web Api here.

The easiest way to use HttpClientFactory within a console application is inside a HostBuilder. This gives you access to the services collection, now everything is easy.

Start with a standard console application, if you’re wondering about the async Task on my Main method, this was introduced in C# 7.1.

static async Task Main(string[] args)
{
    var builder = new HostBuilder()
        .ConfigureServices((hostContext, services) =>
        {

Inside the ConfigureServices, we configure the HttpClientFactory in the same way I showed in my previous post. You can also configure other things like logging, configuration sources, more DI, etc.

But first off, I’m going to add a Polly registry –

IPolicyRegistry<string> registry = services.AddPolicyRegistry();
	
IAsyncPolicy<HttpResponseMessage> httWaitAndpRetryPolicy =
    Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
        .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(retryAttempt));

registry.Add("SimpleWaitAndRetryPolicy", httWaitAndpRetryPolicy);

IAsyncPolicy<HttpResponseMessage> noOpPolicy = Policy.NoOpAsync()
    .AsAsyncPolicy<HttpResponseMessage>();

registry.Add("NoOpPolicy", noOpPolicy);

Then add the HttpClientFactory, passing in the lambda to pick the right policy based on the HTTP verb.

services.AddHttpClient("JsonplaceholderClient", client =>
{
    client.BaseAddress = new Uri("https://jsonplaceholder.typicode.com");
    client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandlerFromRegistry((policyRegistry, httpRequestMessage) =>
{
    if (httpRequestMessage.Method == HttpMethod.Get || httpRequestMessage.Method == HttpMethod.Delete)
    {
        return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("SimpleWaitAndRetryPolicy");
    }
    return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("NoOpPolicy");
});

Next, add the hosted service we want to start.

services.AddSingleton<IHostedService, BusinessService>();

A hosted service is a class that implements IHostedService, more on this below.

Finally at the end of the the Main method, start the hosted service.

await builder.RunConsoleAsync();

For clarity, here is the full listing of the main method –

static async Task Main(string[] args)
{
    var builder = new HostBuilder()
        .ConfigureServices((hostContext, services) =>
        {
            IPolicyRegistry<string> registry = services.AddPolicyRegistry();

            IAsyncPolicy<HttpResponseMessage> httWaitAndpRetryPolicy =
                Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
                    .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(retryAttempt));

            registry.Add("SimpleWaitAndRetryPolicy", httWaitAndpRetryPolicy);

            IAsyncPolicy<HttpResponseMessage> noOpPolicy = Policy.NoOpAsync()
                .AsAsyncPolicy<HttpResponseMessage>();

            registry.Add("NoOpPolicy", noOpPolicy);

            services.AddHttpClient("JsonplaceholderClient", client =>
            {
                client.BaseAddress = new Uri("https://jsonplaceholder.typicode.com");
                client.DefaultRequestHeaders.Add("Accept", "application/json");
            }).AddPolicyHandlerFromRegistry((policyRegistry, httpRequestMessage) =>
            {
                if (httpRequestMessage.Method == HttpMethod.Get || httpRequestMessage.Method == HttpMethod.Delete)
                {
                    return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("SimpleWaitAndRetryPolicy");
                }
                return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("NoOpPolicy");
            });
                
            services.AddSingleton<IHostedService, BusinessService>();
        });

    await builder.RunConsoleAsync();
}

The hosted service
The hosted service is where you put your business logic, it is a simple c# class that implements IHostedService giving it two methods, StartAsync and StopAsync.

Its constructor takes an IHttpClientFactory as a parameter, which is satisfied by the dependency injection infrastructure.

public BusinessService(IHttpClientFactory httpClientFactory)
{
    _httpClientFactory = httpClientFactory;
}

From StartAsync, you can do anything you need.

In this example I call another method which in turn uses the HttpClientFactory to get an instance of a HttpClient to make requests to the the remote server. The requests are executed inside the appropriate Polly policy.

public async Task StartAsync(CancellationToken cancellationToken)
{
    await MakeRequestsToRemoteService();
}

public async Task MakeRequestsToRemoteService()
{
    HttpClient httpClient = _httpClientFactory.CreateClient("JsonplaceholderClient");
    var response = await httpClient.GetAsync("/photos/1");
    Photo photo = await response.Content.ReadAsAsync<Photo>();
    Console.WriteLine(photo);
}

Full source code available here.

Using the HttpClientInterception to Test Methods That Use a HttpClient

Full source code available here.

In my previous post I showed a way of testing a controller that uses a HttpClient.

I had to mock the HttpMessageHandler pass that to the HttpClient and set a bunch of properties. It works well, but is a bit long winded.

I received a comment from a reader who suggested that I try the JustEat.HttpClientInterception library. It allows you to setup responses to specified requests, and pass these to a HttpClient. Then the HttpClient is passed to the controller.

Here is how the test method looks –

[Fact]
public async Task GetTest()
{
    //Arrange
    List<int> myList = new List<int>() {1, 2, 3, 4, 5};
    
    // setup the interceptor
    HttpRequestInterceptionBuilder builder = new HttpRequestInterceptionBuilder()
        .ForHost("localhost.something.com")
        .ForPath("/v1/numbers")
        .WithJsonContent(myList);
    
    // create the HttpClient from the builder
    // and setup the HttpClientBaseAddress
    HttpClient client = new HttpClientInterceptorOptions()
        .Register(builder).CreateHttpClient("http://localhost.something.com/v1/");

    ValuesController controller = new ValuesController(client);

    //Act
    IActionResult result = await controller.Get();

    //Assert
    OkObjectResult resultObject = result as OkObjectResult;
    Assert.NotNull(resultObject);

    List<int> numbers = resultObject.Value as List<int>;
    Assert.Equal(5, numbers.Count);
}

Briefly, here is the constructor of the values controller. It takes the HttpClient as a parameter, usually passed by dependency injection.

public class ValuesController : Controller
{
    readonly IAsyncPolicy<HttpResponseMessage> _httpRetryPolicy;
    private readonly HttpClient _httpClient;

    public ValuesController(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }
	//snip..

Full source code available here.

Unit Testing a Method That Uses HttpClient

Full source code available here.

In this post I’m going to show you how test an action method of controller that uses a HttpClient. When performing this test you want to isolate just the code of the action method for testing, you want to remove the dependency on the HttpClient. I hoped it would be simple, that there would be an IHttpClient, but there is not.

Instead I mocked the HttpMessageHandler, then pass it to the constructor of the HttpClient. The HttpMessageHandler is set to return a list of numbers in response to any request.

The constructor of the controller takes the HttpClient as a parameter, usually passed by dependency injection.

public class ValuesController : Controller
{
    readonly IAsyncPolicy<HttpResponseMessage> _httpRetryPolicy;
    private readonly HttpClient _httpClient;

    public ValuesController(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }
	//snip..

In the test class I mock the HttpMessageHandler, set its SendAsync method to return OK and a list of numbers in response to any request. I then pass the mocked HttpMessageHandler to the constructor of the HttpClient.

Now the HttpClient will return the list of numbers I expect and I my test just tests the code of the action method.

The rest of the test is written as normal.

[Fact]
public async Task GetTest()
{
    //Arrange
    string numberJson= JsonConvert.SerializeObject(new List<int>() { 1, 2, 3, 4, 5 });

    var httpMessageHandler = new Mock<HttpMessageHandler>();
    httpMessageHandler.Protected()
        .Setup<Task<HttpResponseMessage>>("SendAsync", ItExpr.IsAny<HttpRequestMessage>(),
            ItExpr.IsAny<CancellationToken>())
        .Returns(Task.FromResult(new HttpResponseMessage
        {
            StatusCode = HttpStatusCode.OK,
            Content = new StringContent(numberJson, Encoding.UTF8, "application/json"),
        }));

    HttpClient httpClient = new HttpClient(httpMessageHandler.Object);
    httpClient.BaseAddress = new Uri(@"http://localhost:63781/v1/");
    httpClient.DefaultRequestHeaders.Accept.Clear();
    httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

    ValuesController controller = new ValuesController(httpClient);

    //Act
    IActionResult result = await controller.Get();

    //Assert
    OkObjectResult resultObject = result as OkObjectResult;
    Assert.NotNull(resultObject);

    List<int> numbers = resultObject.Value as List<int>;
    Assert.Equal(5, numbers.Count);
}

Full source code available here.

Reusing HttpClient with Dependency Injection

Full source code available here.

If you are using HttpClient to make requests for you, you might have come across some articles discussing how to reuse HttpClient. They strongly advocate for using a single HttpClient for as many requests as possible, i.e. not creating a new HttpClient for every request.
Not having to create/dispose of the HttpClient for every request should improve the performance of you application. One estimate states that every time you instantiate a HttpClient takes 35ms.

In this article I will show you how to use dependency injection to reuse the HttpClient in .Net Core, but the same principle applies in Framework 4.x applications.

The one advantage of creating a new HttpClient for every request is that you don’t need to worry about the DNS record of an endpoint changing during the lifetime of the application, this is common if you are swapping staging and production instances during a deployment. But this is easyly(ish) handled by the ServicePointManger.

Adding HttpClient to the DI Container
In Startup.cs add the below lines.

public void ConfigureServices(IServiceCollection services)
{
    Uri endPointA = new Uri("http://localhost:58919/"); // this is the endpoint HttpClient will hit
    HttpClient httpClient = new HttpClient()
    {
        BaseAddress = endPointA,
    };

    ServicePointManager.FindServicePoint(endPointA).ConnectionLeaseTimeout = 60000; // sixty seconds

    services.AddSingleton<HttpClient>(httpClient); // note the singleton
    services.AddMvc();
}

This approach is ideal if you have a limited number of endpoints and you know them at application startup. If you don’t know then endpoints at startup you can add the call to ServicePointManager where you HttpClient requests occur.

Using the HttpClient
Now I have the HttpClient registered with the Dependency Injection container, let’s take a look at the controller that uses it to make a request. See the inline comments.

public class ValuesController : Controller
{
    private readonly HttpClient _httpClient; // declare a HttpClient

    public ValuesController(HttpClient httpClient) // this is the singelton instance of HttpClient
    {
        _httpClient = httpClient; // assign it to the local HttpClient
    }

    // GET api/values
    [HttpGet]
    public async Task<IActionResult> Get()
    {
        string requestEndpoint = "api/products";

        HttpResponseMessage httpResponse = await _httpClient.GetAsync(requestEndpoint); // make request
        List<Product> products = await httpResponse.Content.ReasAsJsonAsync<List<Product>>();
        return Ok(products);
    }
}

Full source code available here.