Loading Config from Multiple Sources with .NET Core 2.x Web Api or MVC

Full source code available here.

.NET Core 2 and .NET Core 2.1 offer many ways to load configuration and they are well documented by Microsoft. But there is one scenario that I didn’t see explained.

If you want to supplement the configuration in appsettings.json with more from a remote service, database or some other source, you first need to know where that source is, then make a request to it and add it to your configuration. You are probably going to put the location of the remote configuration appsettings.json, then you call the remote config source.
Sounds easy? It is, but not obvious.

First, a little background.

In an out of the box Web API or MVC application you get a Program.cs that looks like this –

public class Program
{
    public static void Main(string[] args)
    {
        CreateWebHostBuilder(args).Build().Run();
    }

    public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
        WebHost.CreateDefaultBuilder(args)
            .UseStartup<Startup>();
}

On line 9, we have WebHost.CreateDefaultBuilder(args), if you peek the definition of this with Resharper you will some something like –

The highlighted code is how your appsettings.json is loaded into the configuration and added to services collection, this call happens just as you leave Program.cs

But you want to read from the appsettings.json and use a value from it to make another call to get more configuration and then add the whole lot of the services collection. You might also want to setup some logging configuration in Program.cs, so you need access to all configuration settings before calling

CreateWebHostBuilder(args).Build().Run();

Here’s how you do it.

Step 1
Inside the main method in Program.cs, build the configuration yourself –

IConfigurationBuilder builder = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json");
Configuration = builder.Build();

Get the location of the other configuration service, in my example it is another json file.
Add it to the builder and call build.

string otherConfigService = Configuration["otherConfigService"]; // this could be a database or something like consul

builder.AddJsonFile(otherConfigService);
Configuration = builder.Build();

Now you have access to the configuration values from the second source.

Configuration["SomeOtherConfigItem1"]}
Configuration["SomeOtherConfigItem2"]}

So far so good, but as mentioned above, the CreateDefaultBuilder adds the configuration to the ServiceCollection, making it available by DI. But that only happens for appsettings.json (and appsettings.{env.EnvironmentName}.json, not the config you loaded from the other source.

If you tried to access the a config setting of your from inside Startup.cs or a controller, it would not be there.

Step 2
Let’s make our configuration available via the ServicesCollection.

In the first block of code above there was a call – CreateWebHostBuilder(args).Build().Run();

I added line 4 below, this will add the configuration to the services collection.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
        .UseStartup<Startup>()
        .UseConfiguration(Configuration); // add this line to add your configuration to the service collection

Now, the config values loaded from appsettings.json and your secondary config source will be available throughout your application.

Full source code available here.

Setting the Kestrel Port From Appsettings.json

Full source code available CoreWithKestrelFromConfighere.

In my previous post I explained how to host Kestrel web server running on (the default) port 5000 as a Windows service. But what if you want to run the server on a different port?

There are a few ways to do this, but the easiest is to add an entry to the appsettings.json file specifying the urls the server should listen to.

In this example, I’m setting it to http://localhost:62000.

The appsettings.json file looks like this –

{
"urls": "http://localhost:62000/"
}

I also removed applicationUrl setting from the from the launchSettings.json file, now Visual Studio and the command line will set the application to the same port.

To add the settings in appsettings.json to the configuration, call .AddJsonFile("appsettings.json") when creating the ConfigurationBuilder.

public class Program
{
    public static IConfiguration Configuration { get; set; }
    public static void Main(string[] args)
    {
        var builder = new ConfigurationBuilder()
            .AddJsonFile("appsettings.json");
        Configuration = builder.Build();

        BuildWebHost(args).Run();
    }

    public static IWebHost BuildWebHost(string[] args) =>
        WebHost.CreateDefaultBuilder(args)
            .UseStartup<Startup>()
            .UseConfiguration(Configuration)
            .Build();
}

You don’t have name the file appsettings.json, you can put it any file you like, but this is convenient.

To run the application from the command line you can do one of two things –

1. Go to the directory where the csproj file is located and type:
dotnet run

2. Go to the bin\debug\netcoreapp2.0 directory and type:
dotnet CoreWithKestrelFromConfig.dll

You can set Kestrel to listen on multiple urls, the format is this –

{
   "urls": "http://localhost:62000/;http://localhost:62002/"
}

Full source code available CoreWithKestrelFromConfighere.

The type ‘xxxx’ is defined in an assembly that is not referenced. System.Runtime.

If you recognize the error from the title of this post, you can jump to the solution.

The problem

I have a ASP.NET 5 solution with two projects, a web application project and a class library project.
After adding the class library I was very surprised to get this error –

The type 'IEnumerable<>' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Runtime, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

All I was doing was querying an Entity Framework context for some simple data and returning the result as an IEnumerable.

This is not the code but is close enough.

var members = _context.Members.Where(m => m.FirstName.Contains(firstName));

To verify that I wasn’t messing up something very simple I tried the same code from the web api controller in the web application, it compiled and worked fine. I looked in the references for both projects for an indication that I had left something out of my class library but could find nothing.

I checked the project.json for both and messed around in nuget for a while but all looked fine.

The project.lock.json files

Finally, I had a look in the project.lock.json files and noticed this section in the web application file.

{
  "locked": false,
  "version": 2,
  "targets": {
    "DNX,Version=v4.5.1": {
      "EntityFramework.Core/7.0.0-rc1-final": {
        "type": "package",
        "dependencies": {
          "Ix-Async": "1.2.5",
          "Microsoft.Extensions.Caching.Abstractions": "1.0.0-rc1-final",
          "Microsoft.Extensions.Caching.Memory": "1.0.0-rc1-final",
          "Microsoft.Extensions.DependencyInjection": "1.0.0-rc1-final",
          "Microsoft.Extensions.DependencyInjection.Abstractions": "1.0.0-rc1-final",
          "Microsoft.Extensions.Logging": "1.0.0-rc1-final",
          "Microsoft.Extensions.Logging.Abstractions": "1.0.0-rc1-final",
          "Microsoft.Extensions.OptionsModel": "1.0.0-rc1-final",
          "Remotion.Linq": "2.0.1",
          "System.Collections.Immutable": "1.1.36"
        },
        "frameworkAssemblies": [
          "Microsoft.CSharp",
          "mscorlib",
          "System",
          "System.Collections",
          "System.ComponentModel.DataAnnotations",
          "System.Core",
          "System.Diagnostics.Debug",
          "System.Diagnostics.Tools",
          "System.Globalization",
          "System.Linq",
          "System.Linq.Expressions",
          "System.Linq.Queryable",
          "System.ObjectModel",
          "System.Reflection",
          "System.Reflection.Extensions",
          "System.Resources.ResourceManager",
          "System.Runtime",
          "System.Runtime.Extensions",
          "System.Threading"
        ],

 

There in the project.lock.json for the web application project is "System.Runtime" in the "frameworkAssemblies" section.

But the same section in the class library’s file did NOT have a "System.Runtime".

{
  "locked": false,
  "version": 2,
  "targets": {
    ".NETFramework,Version=v4.5.1": {
      "EntityFramework.Core/7.0.0-rc1-final": {
        "type": "package",
        "dependencies": {
          "Ix-Async": "1.2.5",
          "Microsoft.Extensions.Caching.Abstractions": "1.0.0-rc1-final",
          "Microsoft.Extensions.Caching.Memory": "1.0.0-rc1-final",
          "Microsoft.Extensions.DependencyInjection": "1.0.0-rc1-final",
          "Microsoft.Extensions.DependencyInjection.Abstractions": "1.0.0-rc1-final",
          "Microsoft.Extensions.Logging": "1.0.0-rc1-final",
          "Microsoft.Extensions.Logging.Abstractions": "1.0.0-rc1-final",
          "Microsoft.Extensions.OptionsModel": "1.0.0-rc1-final",
          "Remotion.Linq": "2.0.1",
          "System.Collections.Immutable": "1.1.36"
        },
        "frameworkAssemblies": [
          "Microsoft.CSharp",
          "mscorlib",
          "System",
          "System.ComponentModel.DataAnnotations",
          "System.Core"
        ],

I added "System.Runtime" into the project.lock.json of class library project and everything compiled and worked.

But, and this is a big but, the project.lock.json file is generated from the project.json file and any changes to nuget or to the porject.json will lead to my project.lock.json being overwritten.

At least I know the culprit and now I had to figure out how to get that entry  for "System.Runtime" to be generated and put into "frameworkAssemblies".

Note also that line 5 for the two files also differs, this I think is more telling problem.

 "DNX,Version=v4.5.1": { 

vs

 ".NETFramework,Version=v4.5.1": { 

Time to compare the project.json files.

The project.json files

This is project.json in the web application project

{
  "version": "1.0.0-*",
  "compilationOptions": {
    "emitEntryPoint": true
  },

    "dependencies": {
        "EntityFramework.MicrosoftSqlServer": "7.0.0-rc1-final",
        "Member.Business": "1.0.0-*",
        "Member.DataLayer": "1.0.0-*",
        "Member.Domain": "1.0.0-*",
        "Microsoft.AspNet.IISPlatformHandler": "1.0.0-rc1-final",
        "Microsoft.AspNet.Mvc": "6.0.0-rc1-final",
        "Microsoft.AspNet.Server.Kestrel": "1.0.0-rc1-final",
        "Microsoft.AspNet.StaticFiles": "1.0.0-rc1-final",
        "Microsoft.CSharp": "4.0.0",
        "Microsoft.Extensions.Configuration.FileProviderExtensions": "1.0.0-rc1-final",
        "Microsoft.Extensions.Configuration.Json": "1.0.0-rc1-final",
        "Microsoft.Extensions.Logging": "1.0.0-rc1-final",
        "Microsoft.Extensions.Logging.Console": "1.0.0-rc1-final",
        "Microsoft.Extensions.Logging.Debug": "1.0.0-rc1-final"
    },

  "commands": {
    "web": "Microsoft.AspNet.Server.Kestrel"
  },

  "frameworks": {
    "dnx451": { },
    "dnxcore50": { }
  },

  "exclude": [
    "wwwroot",
    "node_modules"
  ],
  "publishExclude": [
    "**.user",
    "**.vspscc"
  ]
}

And this is in the class library project.

{
  "version": "1.0.0-*",
  "description": "Member.Business Class Library",
  "authors": [ "bryan" ],
  "tags": [ "" ],
  "projectUrl": "",
  "licenseUrl": "",
    "frameworks": {
        "net451": { },
        "dotnet5.4": {
            "dependencies": {
                "Microsoft.CSharp": "4.0.1-beta-23516",
                "System.Collections": "4.0.11-beta-23516",
                "System.Linq": "4.0.1-beta-23516",
                "System.Runtime": "4.0.21-beta-23516",
                "System.Threading": "4.0.11-beta-23516"
            }
        }
    },
    "dependencies": {
        "EntityFramework.MicrosoftSqlServer": "7.0.0-rc1-final",
        "Member.DataLayer": "1.0.0-*",
        "Member.Domain": "1.0.0-*"
    }
}

There are some serious differences between the two, most notably around the “frameworks” and “dependencies”. I have no idea why the two project.json files are so different.

Solution A and B

I changed my project.json in the class library project to the below adding the "frameworkAssemblies" node.

{
  "version": "1.0.0-*",
  "description": "Fund.FundEntitlement.Business Class Library",
  "authors": [ "bryan" ],
  "tags": [ "" ],
  "projectUrl": "",
  "licenseUrl": "",
    "frameworks": {
        "net451": {
            "frameworkAssemblies": {
                "System.Runtime": "4.0.10.0"
            }
        },
        
        "dotnet5.4": {
            "dependencies": {
                "Microsoft.CSharp": "4.0.1-beta-23516",
                "System.Collections": "4.0.11-beta-23516",
                "System.Linq": "4.0.1-beta-23516",
                "System.Runtime": "4.0.21-beta-23516",
                "System.Threading": "4.0.11-beta-23516"
            }
        }
    },
    "dependencies": {
        "EntityFramework.MicrosoftSqlServer": "7.0.0-rc1-final",
        "Fund.FundEntitlement.DataLayer": "1.0.0-*",
        "Fund.FundEntitlement.Domain": "1.0.0-*"
    }
}

An alternative that also works is changing the project.json in the class library to look more like the on from the web application project.

{
    "version": "1.0.0-*",
    "description": "Fund.FundEntitlement.Business Class Library",
    "authors": [ "bryan" ],
    "tags": [ "" ],
    "projectUrl": "",
    "licenseUrl": "",
    "frameworks": {
        "dnx451": { },
        "dnxcore50": {
            "dependencies": {
                "Microsoft.CSharp": "4.0.1-beta-23516"
            }
        }
    },
    "dependencies": {
        "EntityFramework.MicrosoftSqlServer": "7.0.0-rc1-final",
        "Fund.FundEntitlement.DataLayer": "1.0.0-*",
        "Fund.FundEntitlement.Domain": "1.0.0-*",
        "System.Collections": "4.0.11-beta-23516",
        "System.Linq": "4.0.1-beta-23516",
        "System.Runtime": "4.0.21-beta-23516",
        "System.Threading": "4.0.11-beta-23516"
    }
}

Now everything compiles and all is good. I’m sadly sure it won’t be the last time I have screw around with project.lock.json and project.json. For more info about those files see http://davidfowl.com/diagnosing-dependency-issues-with-asp-net-5/