Loading Config from Multiple Sources with .NET Core 2.x Web Api or MVC

Full source code available here.

.NET Core 2 and .NET Core 2.1 offer many ways to load configuration and they are well documented by Microsoft. But there is one scenario that I didn’t see explained.

If you want to supplement the configuration in appsettings.json with more from a remote service, database or some other source, you first need to know where that source is, then make a request to it and add it to your configuration. You are probably going to put the location of the remote configuration appsettings.json, then you call the remote config source.
Sounds easy? It is, but not obvious.

First, a little background.

In an out of the box Web API or MVC application you get a Program.cs that looks like this –

public class Program
    public static void Main(string[] args)

    public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>

On line 9, we have WebHost.CreateDefaultBuilder(args), if you peek the definition of this with Resharper you will some something like –

The highlighted code is how your appsettings.json is loaded into the configuration and added to services collection, this call happens just as you leave Program.cs

But you want to read from the appsettings.json and use a value from it to make another call to get more configuration and then add the whole lot of the services collection. You might also want to setup some logging configuration in Program.cs, so you need access to all configuration settings before calling


Here’s how you do it.

Step 1
Inside the main method in Program.cs, build the configuration yourself –

IConfigurationBuilder builder = new ConfigurationBuilder()
Configuration = builder.Build();

Get the location of the other configuration service, in my example it is another json file.
Add it to the builder and call build.

string otherConfigService = Configuration["otherConfigService"]; // this could be a database or something like consul

Configuration = builder.Build();

Now you have access to the configuration values from the second source.


So far so good, but as mentioned above, the CreateDefaultBuilder adds the configuration to the ServiceCollection, making it available by DI. But that only happens for appsettings.json (and appsettings.{env.EnvironmentName}.json, not the config you loaded from the other source.

If you tried to access the a config setting of your from inside Startup.cs or a controller, it would not be there.

Step 2
Let’s make our configuration available via the ServicesCollection.

In the first block of code above there was a call – CreateWebHostBuilder(args).Build().Run();

I added line 4 below, this will add the configuration to the services collection.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
        .UseConfiguration(Configuration); // add this line to add your configuration to the service collection

Now, the config values loaded from appsettings.json and your secondary config source will be available throughout your application.

Full source code available here.

Using Dependency Injection with Startup.cs in ASP.NET Core

Full source code available here.

Did you know that .NET Core 2 can pass in an instance of any type you want to the Startup.cs constructor? Well you can! Here’s how.

Out of the box the Startup.cs comes like this –

public class Startup
	public Startup(IConfiguration configuration)
		Configuration = configuration;

The IConfiguration is passed in by dependency injection. In Program.cs you can add other types to the DI container, and then add that type to the constructor parameters of Startup.cs.

Here’s how it works. The Microsoft.AspNetCore.Hosting.WebHostBuilder has a ConfigureServices(...) method –

public IWebHostBuilder ConfigureServices(Action<IServiceCollection> configureServices)
    if (configureServices == null)
        throw new ArgumentNullException(nameof(configureServices));

    return ConfigureServices((_, services) => configureServices(services));

This lets you add services to the dependency injection container from the Program.cs.

WebHost.CreateDefaultBuilder(args) returns an IWebHostBuilder and that lets you use the ConfigureServices(...) method. Easy!

public static IWebHost BuildWebHost(string[] args) =>
        .ConfigureServices(serviceCollection =>
            serviceCollection.AddScoped<ISomeService, SomeService>())

And in Startup.cs

public class Startup
    private ISomeService _someService;
    public Startup(IConfiguration configuration, ISomeService someService)
        _someService = someService;
        Configuration = configuration;
// snip..	

The instance of ISomeService is of course available for DI everywhere in the application.

public class ValuesController : Controller
    private readonly ISomeService _someService;
    public ValuesController(ISomeService someService)
        _someService = someService;

Full source code available here.

Getting .NET Core 2.1 Preview 2 Working with Visual Studio 2017

About a year ago I wanted to start using .NET Framework 4.7, it should have been an easy process, but wasn’t. After some trial and error if figured it out and wrote a blog post explaining how to get it working.

Now with the release of .NET Core 2.1 Preview 2, I have hit the familiar problems – no obvious instructions from Microsoft, no one place to download all everything that needed and a Visual Studio install that does not include what you the latest SDK or runtime, and errors like – The specified framework 'Microsoft.AspNetCore.App', version '2.1.0-preview2-final' was not found. or, 'dotnet.exe' has exited with code -2147450730 (0x80008096).

After a few hours messing around and installing the wrong versions of the right software I figured it out.

Step 1

Install the latest version of Visual Studio Preview https://www.visualstudio.com/vs/preview/, at the time of writing this was version 15.7.0 Preview 4.0

I’m interested in developing Web API applications, so I check that box.

BUT, version 15.7.0 Preview 4.0 comes with .NET Core Version 2.1.0 Preview 1. So you don’t get the fancy new features like HttpClientFactory. Read on…

Step 2

Go to
https://github.com/dotnet/core/blob/master/release-notes/download-archives/2.1.0-preview2-download.md and download the SDK and the Runtime for your architecture.

You can verify the checksum of the downloads if you want to by comparing your SHA sum to these ones – https://dotnetcli.blob.core.windows.net/dotnet/checksums/2.1.300-preview2-008530-sdk-sha.txt

Install both the SDK and the runtime.

Step 3

To verify that that you are running .NET Core Preview 2, open visual studio and create a new .NET Core Web API application.

Once it has been created, open the .csproj file, you should see this block –

  <PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.0-preview2-final" />

If you don’t see Version="2.1.0-preview2-final", something has gone wrong.

You can verify what packages are installed by going to your start menu and opening Apps and Features/Add or remove programs.

If you have installed the right SDK and runtime you should see the following –

The Circuit Breaker pattern with Polly

Full source code available here.

This post on the Polly circuit breaker is part of a larger series of post on the Polly Resilience Framework, see here for the others, or check out my Pluralsight course.

The circuit breaker controls the flow of requests from a source to one or more downstream system and cuts the connection when some failure condition is met and resumes the connection after a period. This lets us fail quickly when we know that some remote endpoint or host is not responding as we expect to requests. In this post I am only dealing with the basic circuit breaker, I’ll post another about the advanced circuit breaker later.

Benefits of a circuit breaker

  • Stops requests to the downstream system
  • Reduces load on a failing downstream system, giving it a chance to recover
  • The circuit breaker immediately returns an an error informing the source system that the circuit is open, the source system doesn’t have to wait for a timeout to occur
  • The application with the circuit breaker does not resource like memory, ports and threads when

Background and Terminology
The name, circuit breaker, comes from electrical circuit breakers in your home or office, in some countries these are referred to as trip switches. If the electrical system is working correctly, the circuit is closed and electricity flows to outlets and appliances. But if something goes wrong, the circuit opens, preventing electricity from flowing, thus protecting people and appliances.

The Polly circuit breaker has the corresponding closed and open positions. When closed, the circuit breaker allows requests to be sent, when open, nothing can be sent and an exception is immediately thrown if a request is send to the circuit breaker.

The Polly circuit breaker has one more status, half-open. When in this state Polly will allow the next request to be sent, and if it succeeds the circuit is closed (and normal operation resumes), but if it fails the circuit returns to open (preventing requests from being sent).
The circuit transitions from closed to open when the failure condition is met, from open to half-open when the specified break time is reached.

Here is the full set of possible transitions:
Closed to open – when the failure condition occurs
Open to half-open – when the duration of break is reached
Half-open to closed – when the first request is a success
Half-open to open – when the first request is a failure

The Policy
When creating a circuit breaker policy you specify the condition under which the circuit opens (breaks) and for how long. To be of any use the same instance of the policy must be used across multiple requests, so you cannot just instantiate it inside a controller.

This has implications, if you use the same circuit breaker policy when calling multiple downstream systems, if the circuit breaks for one, it breaks for all.
On the other hand, if you use a circuit breaker on just one downstream system, it can only break the circuit for that one.

I recommend passing policies into the controllers with dependency injection, in the example below I add the policy directly to the DI container, but you should consider using the Polly registry.

As mentioned above the circuit breaker opens when a specified failure condition is met. In the example policy the condition is two consecutive failure responses, the policy also specifies the duration of the break, in this case 10 seconds.

CircuitBreakerPolicy<HttpResponseMessage> breakerPolicy = Policy
    .HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
    .CircuitBreakerAsync(2, TimeSpan.FromSeconds(10), OnBreak, OnReset, OnHalfOpen);

The policy is passed into the DI container this way –

In the controller, assign the circuit breaker to a local variable.

private readonly CircuitBreakerPolicy<HttpResponseMessage> _breakerPolicy;

public CatalogController(HttpClient httpClient, CircuitBreakerPolicy<HttpResponseMessage> breakerPolicy)
    _breakerPolicy = breakerPolicy;
    _httpClient = httpClient;

Then use it to make calls to the downstream systems, here –

public async Task<IActionResult> Get(int catalogId)
    string inventoryEndpoint = $"inventory/{catalogId}";

    HttpResponseMessage inventoryResponse = await _breakerPolicy.ExecuteAsync(
             () => _httpClient.GetAsync(inventoryEndpoint));


And here –

public async Task<IActionResult> GetPrice(int catalogId)
    string requestEndpoint = $"price/{catalogId}";

    HttpResponseMessage pricingResponse = await _breakerPolicy.ExecuteAsync(
            () => _httpClient.GetAsync(requestEndpoint));


Full source code available here.

How to install .NET Framework 4.7 in Visual Studio 2017

Microsoft just released .NET Framework 4.7, but surprisingly it is not the easiest to install.

Here are the steps for Windows 10, as of this writing, it is not available for earlier versions for of Windows.

1. Install Windows 10 Creators Update

Go here https://www.microsoft.com/en-us/software-download/windows10, download and install. This will take a while and will reboot your computer a few times.

2. Install Visual Studio 2017 Updates

In Visual Studio, go to Tools -> Extensions and Updates, install any available updates.

(You might have the option here to install 4.7, see the image below in step 4.)

3. Open Visual Studio 2017 Installer

Use the Start Menu to search for the Visual Studio Installer.

In the installer, click Modify on the version of Visual Studio you have installed.

4. Install .NET framework 4.7

Click .NET desktop development.

On the right select .NET Framework 4.7 and finally install.

A simple Polly example with WebApi 2

Download full source code.

If you are calling APIs and you haven’t heard about The Polly Project, you should check it out. It helps take care of many common problems like unreliable connections, retries and circuit breaking. It can also be used to wrap code where other forms of transient exception may occur.

In this blog post I’m going to show the wait and retry policy when calling an unreliable Web Api endpoint.

When using Polly there are two pieces that need you need to code – the policy and the execution of code wrapped by the policy.

In this example the policy is created in the constructor of the controller, this is NOT what you should do in a real application. For an example using DI to share policies see “Reusing Polly Policies with Dependency Injection”.

readonly RetryPolicy<HttpResponseMessage> _httpRequestPolicy;

public ValuesController()
    _httpRequestPolicy = Policy.HandleResult<HttpResponseMessage>(
            r => r.StatusCode == HttpStatusCode.InternalServerError)
            retryAttempt => TimeSpan.FromSeconds(retryAttempt));

This policy states that in the event that a HttpResponseMessage has a of InternalServerError, the request should be tried up to three times with a back off increasing by one second each time.

That’s the policy in place, now we need to call the web service endpoint.

public async Task<IHttpActionResult> Get()
   var httpClient = GetHttpClient();
   string requestEndpoint = "numbers/"; // numbers is the name of the controller being called

   HttpResponseMessage httpResponse = await _httpRequestPolicy.ExecuteAsync(() => httpClient.GetAsync(requestEndpoint));

   IEnumerable<int> numbers = await httpResponse.Content.ReadAsAsync<IEnumerable<int>>();
   return Ok(numbers);

If you execute the code it makes a GET request to http://localhost:2351/api/values, despite two failures when calling the NumbersController you will get a 200 response and a list of numbers.

If you want to verify this, step through the provided solution with the debugger. You’ll see that the first two requests to the NumbersContoller return 500 errors and that the third succeeds and returns an array of ints.

With just a few lines of code Polly provided us with a robust wait and retry mechanism for web requests.

In another blog I’ll show a better way of defining policies so they can be used in any controller.

Download full source code.

Locating and checking an executing DLL on a running web server or other application

Edit – even though the steps described below related to an example with IIS, you can use this technique for any running process, it’s just a matter of identifying the process id. In some cases you will not even need the process id, e.g. if all processes are using the same version of a DLL.


Figuring out what dll is actually executing on a running web server is no easy task; it’s not as simple as looking in the bin directory of the deployed application as the dlls are copied from there to a set of temporary directories within the windows systemroot.

I know of two ways of figuring this out, the first requires some code changes and the second does not.

I wrote some time ago about how to find which dll is really executing when running an application, it works just fine if you can alter the source code to include the snippet I proposed.

Clearly that is not a lot of use in a deployed application when you cannot change the code or even redeploy.

No code change needed

Here is the alternative, no code changes needed. The application I am interested in is called MyFancyApp, there are other applications running on the server too. Let’s say that the dll I am interested in is AutoMapper.dll, I need to verify that the running dll is really the version I expect.

1. Open a command prompt and navigate to %systemroot%\System32\inetsrv.
2. Execute appcmd list wp, this will show all the running application pools and the associated process ids. The output will look something like this –
find dll appcmd

You can see that MyFancyApp is running with process id of 1224.

Now switch to Process Explorer.

3. Open Process Explorer and hit CTRL-F, type in AutoMapper.dll. In the results PID 1224 is shown, in fact it will be there a few times. At this point the file location of the dll is visible.
find dll search

4. Double click one of the entries for PID 1224 where the Type is DLL.

5. In the lower pane of Process Explorer the AutoMapper.dll is selected and the version is shown, I had to right click on the lower pane and the version column to make it visible.

find dll results

That’s it. No redeploy, no code editing, no remote debugging. You now know for sure which dll is running and where to find it. If the dll is your own and you are concerned about what might be in the code you can open the file in dotpeek to decompile it.

Getting Web API Exception Details from a HttpResponseMessage

The Problem

It’s hard to get the details of an exception from a Web Api response when calling Web Api from a C# program. (Skip to the solution if you don’t care about the background), it even handles inner exceptions!

Some background

If you are working on a Web Api project and testing with a web browser you get a wonderful error page when an exception occurs. It gives you the message, exception message, exception type and the stack trace. Pretty much all you need to get started figuring out what has gone wrong.
Exception in browser

Same thing with fiddler, get a 500 back and you’ll even be treated to a Json version of the above.

Exception in fiddler

What about calling the action method from inside a c# program? Should be easy, you just create a client, setup the query, let it rip and examine the response for a success status and then read the content to get the returned values. Great, works fine.

What if the server threw an exception like the ones shown above, I thought it would be a simple thing to call response.Exception or the like and get all the details. But easy it is not.
I rooted around in the response for a while but found nothing that was simple to use.

The Solution

Instead I have added an extension method to HttpResponseMessage to parse the details of the exception from the Json in the response.Content.

using System.Net.Http;
using System.Threading.Tasks;
using Newtonsoft.Json;

namespace SimpleWebApiClient
    public static class HttpResponseMessageExtension
        public static async Task<ExceptionResponse> ExceptionResponse(this HttpResponseMessage httpResponseMessage)
            string responseContent = await httpResponseMessage.Content.ReadAsStringAsync();
            ExceptionResponse exceptionResponse = JsonConvert.DeserializeObject<ExceptionResponse>(responseContent);
            return exceptionResponse;

    public class ExceptionResponse
        public string Message { get; set; }
        public string ExceptionMessage { get; set; }
        public string ExceptionType { get; set; }
        public string StackTrace { get; set; }
        public ExceptionResponse InnerException { get; set; }

Usage is simple.

    HttpResponseMessage response = await httpClient.GetAsync(query).ConfigureAwait(false);
    if (response.IsSuccessStatusCode)
        // return the value
    // But if an error occurred read the details           
    ExceptionResponse exceptionResponse = response.ExceptionResponse();



Why you should use IDictionary, IList, etc

When returning objects from a method try to use IList, IDictionary, etc instead of List and Dictionary. This is especially important when the method is inside a class library which you distribute.

I had to edit a method that was returning a Dictionary. Inside of which some complex work was being done to populate the dictionary.
The changes needed were going to be much easier if I could use and return a ConcurrentDictionary. See here for what I was doing.

But because the method was returning a simple Dictionary it was not as straight forward as I hoped. More work needed to be done and I had the choice of:

  1. Doing the processing in a more difficult way, not involving a ConcurrentDictionary.
  2. Moving the entries from the new ConcurrentDictionary to a simple Dictionary just before calling return.
  3. Changing the return type of the method (and the associated interface).

I went with option three, this happened to be the easiest, but it was also the best. If the code had initially been written to return IDictionary I would only have had to concern myself with required logic changes inside the method.

Here is a quick, but contrived, example of the flexibility gained by using IDictionary.

The class has a single public method which returns an IDictionary. Depending on the boolean value of getConcurrentDictionary it does its work using either as a simple Dictionary or a ConcurrentDictionary, but the caller does not need be aware of this. This gives great flexibility, the method GetDataDictionary can change its internal implementation and even the type of dictionary it returns without any negative impact on the caller.

The caller also has the flexibility to cast the IDictionary to a ConcurrentDictionary as needed.

    class GetDataWithInterfaceCollections : IGetDataWithInterfaceCollections
        public IDictionary<int, int> GetDataDictionary(int start, int end, bool getConcurrentDictionary)
            IDictionary<int, int> dictionaryToReturn;

            if (getConcurrentDictionary)
                dictionaryToReturn = GetConcurrentDictionary(start, end);
                dictionaryToReturn = GetDictionary(start, end);

            return dictionaryToReturn;

        private IDictionary<int, int> GetConcurrentDictionary(int start, int end)
            //ConcurrentDictionary offers a lot of features not available in Dictionary
            ConcurrentDictionary<int, int> dictionary = new ConcurrentDictionary<int, int>();
            for (int loop = start; loop <= end; loop++)
                dictionary.AddOrUpdate(loop, loop*10, (i, i1) => loop * 10 ); // AddOrUpdate is specific to a ConcurrentDictionary

            return dictionary;

        private IDictionary<int, int> GetDictionary(int start, int end)
            IDictionary<int, int> dictionary = new Dictionary<int, int>();
            for (int loop = start; loop <= end; loop++)
                dictionary.Add(loop, loop * 10);

            return dictionary;

Here is how to call the method shown above.

            IDictionary<int, int> concurrentDictionary = _getDataWithInterfaceCollections.GetDataDictionary(1, 10, true);

            IDictionary<int, int> simpleDictionary = _getDataWithInterfaceCollections.GetDataDictionary(11, 20, false);

            ConcurrentDictionary<int, int> explicitConcurrentDictionary = _getDataWithInterfaceCollections.GetDataDictionary(21, 30, true) as ConcurrentDictionary<int, int>; 

Using the Watch window in Visual Studio we can see more info.

Dictionary Types

Entity Framework Migrations with an existing database


If you read my post on Entity Framework migrations, you might be saying, “that’s amazing, but that’s all very well for new projects, but we’ve got eighty tables in place already and that code first migration ship has sailed”.
But no, you can still make use of Entity Framework migrations, it takes a some extra work, but it is well worth the effort.

If you already have POCOs that represent your databases tables and they really match your tables, you might be able to skip this step.

Reversing the Database

If your POCOs don’t match the database you can use the database tables to generate POCOs for you.

There are plenty of tutorials on Entity Framework’s reverse engineering tool. So I won’t go into too much detail.

Create a new solution with a console application as the startup project.
Set the connection string in your app.config to point to the your reference database (this is the one that has the all the tables you are currently using).

Grab the Entity Framework Power Tools Beta 4 (https://visualstudiogallery.msdn.microsoft.com/72a60b14-1581-4b9b-89f2-846072eff19d/). Use this to reverse engineer the tables to POCOs. Review the outputted classes, mappings and context to make sure that they were generated the way you expect.

Notice that there is nothing on the POCOs to suggest where there should be indexes. Inferred primary key and foreign keys will be added in by the migration by default. But others will not.

First Migration

Go to the package manager and Enable-Migrations.

Then add the first migration, Add-Migration InitalCreate, this is the migration that can be used to generate the tables.

But, it is probably not complete, it’s going to be missing indexes as mentioned above, stored procs and more.

For example the Vehicle table looks like this
Table Layout

And has an indexes on VehicleID, TyreTypeIDa and Name, but the migration did not pick up the Name index.

Testing the migration

Change the connection string to point to new database.
And run Update-Database

You now have a new database based on the migration.
Open your SQL Server Management Studio, right click on the reference database and click Tasks, then Generate Scripts…
Generate-Script A

Click next and the choose the tables to script out.
6.Generate-Script B

Click next and then Advanced, scroll to the bottom of the list and turn on Script Indexes and any other feature you may need (eg, triggers).

7.Generate-Script C
Save the output to single file named ReferenceDB.sql

Follow the same steps to generate a script based on the new database.

Compare the files in a text comparison tool and you’ll see differences like this

The index on the Name column is missing in the new database.

No problem. Go back to the migration and add it in.

Now you have two choices here, you can add a new migration to alter the existing table, or you can change the initial migration.
I suggest changing the InitalCreate migration, because you should try to get the this first migration to be as close as possible to the existing database.

Adding an index to a CreateTable call is as simple as adding one line at the end.

			c => new 
					VehicleID = c.Int(nullable: false, identity: true), 
					Name = c.String(nullable: false, maxLength: 50), 
					Description = c.String(maxLength: 200), 
					MSRP = c.Double(nullable: false), 
					TyreTypeID = c.Int(nullable: false), 
			.PrimaryKey(t => t.VehicleID) 
			.ForeignKey("dbo.TyreType", t => t.TyreTypeID, cascadeDelete: true) 
			.Index(t => t.TyreTypeID) 
                         // this last line is the index I added by hand. 
			.Index(t => t.Name, name: "IX_Name");

Drop the new database the run Update-Database.

The Vehicle will now be created with the new index.
You could add the index to the POCOs, but you might want to consider how this would affect any move away from Entity Framework. See Code First Data Annotations for more.

For a full list of what you can do with migrations see DbMigration Methods.

Remember that you can run arbitrary sql with the Sql method or even from the SqlFile method. This gives you full flexibility when using Entity Framework migrations.