Lambda cold starts for .NET applications are not so bad

Want to learn more about AWS Lambda and .NET? Check out my A Cloud Guru course on ASP.NET Web API and Lambda.

The advice here should all be viewed in the light of the wonderful hedge - “it depends”. This post does not have any hard and fast rules for how you should build your functions, or how to handle cold starts.

It is more of a suggestion that cold starts may not be the problem you think they are, why that’s the case, and some advice on alleviating the problem if it exists.

Your mileage may vary.

Introduction

There is a lot of discussion and opinions around using .NET with Lambda functions and their cold starts. From what I’ve seen there seems to be more concern than is warranted. Yes, Lambda functions will have cold starts, but they are probably not as frequent as you think or take as long as you think.

So cold starts, may not be a problem at all. If that’s the case, there is nothing to stop you from moving some of your .NET applications to Lambda functions.

If cold starts are taking too long, or occur more frequently than you would like, there are things you can do to mitigate the problem.

Why use Lambda functions?

The AWS Lambda service is function as a service offering for your .NET code. They are a great way to run code in response to events from other applications, AWS services, HTTP requests, direct invocations, and polling. You only pay for what you use, so they can be a cost-effective way to run applications that do not need to be up 24/7.

You don’t need to manage any infrastructure, you focus on writing the code, and the Lambda service takes care of provisioning, scaling, and managing the execution environments.

Many people use Lambda functions to host ASP.NET Web API applications. This can be a very efficient way to host such applications. Compare the price of running an application on a server 24/7 vs running it on Lambda only as requests are received.

If you want your Lambda function to be invoked by other AWS services, it is as simple as subscribing to the events they raise.

In the rest of this post I’m going to cover:

• What a cold start is
• Scenarios where cold starts may not be a problem
• Scenarios where cold starts may be a problem
• What can you do about cold starts

What a cold start is

The first time a Lambda function is invoked, an execution environment is created. Creating this environment takes a bit of time and is referred to as a “cold start”. The application is downloaded, memory is allocated, the application is JITed, and the initialization code is run. Initialization code refers to the constructor, initializers, etc. This all happens in response to the first invocation of the function, but before the function handler is run.

The initialization code is run only once for an execution environment, that’s why it’s a good idea to put things like database connection initialization, S3 client creation, etc, outside the function handler.

Once this execution environment is up and running the function is referred to as “warm”, and it will stay warm for an undefined period. The initialization code will not be run again, only the function handler will be run. The function will remain warm if it is being called regularly. An execution environment will be reused for subsequent invocations of the same function.

But an execution environment can handle a single request at a time. If you have a single execution environment warm, and it is handling a request, then a second request to the function will cause a new execution environment to be created, this second execution environment will now go through a cold start.

Even if an execution environment is warm and being called regularly, it will be recycled by the Lambda service at some point. If that was the only execution environment available, then a cold start will occur on the next invocation.

It is possible that through normal use of the Lambda function, you will have a pool of warm execution environments available to handle incoming requests and all requests will be handled by these execution environments. Over time the pool will grow and shrink as the rate of incoming requests changes. If requests stop entirely, the pool will shrink to zero after some period of time, and the next invocation will cause a cold start.

How long a cold start takes varies depending on the size of the application and your initialization code. But it is generally in the order of a few hundred milliseconds to a couple of seconds.

Here is an example of the output after a cold start. You will see this in CloudWatch logs, or at the command line if you invoke a function from there.

REPORT RequestId: 651cd093-9aef-4e3b-b7e8-39cd94ce6cd5  Duration: 385.08 ms     Billed Duration: 386 ms Memory Size: 256 MB     Max Memory Used: 61 MB  Init Duration: 207.84 ms 

• Init Duration is the time it took to create the execution environment, and run the initialization code.
• Duration is the time it took to run the function handler.
• Billed Duration is the time it took to run the function handler, rounded up to the next millisecond. You are not charged for initialization time when using a managed runtime, only for the time it takes to run the function handler.

Here is the output after the same function is invoked again. The execution environment is already warm, so the initialization code is not run again.

REPORT RequestId: f37b73e6-666a-4e7f-a5e4-a2758854af6e  Duration: 1.43 ms       Billed Duration: 2 ms   Memory Size: 256 MB     Max Memory Used: 61 MB 

You will see that the Init Duration is absent, and the Duration is very short.

Keep in mind that during development and testing, you are likely to see far more cold starts than you will see in a production environment. This is because the Lambda service will recycle execution environments every time you update the function code, and because you are likely to be invoking the function less frequently than in production.

Scenarios where cold starts may not be a problem

These are not the only scenarios where you should use Lambda functions, but they are the ones where cold starts may not be a problem:

• your function is invoked infrequently, responds quickly, the response is time sensitive, but you can handle the occasional slow response during a cold start.
• your function is invoked infrequently, and the response is not time sensitive.
• your function is invoked frequently, but not in bursts, it responds quickly. Your execution environments may stay warm and result in few cold starts.
• you have unpredictable bursts of requests, but the responses are not time sensitive. The Lambda service will automatically scale to meet these bursts much faster than other technologies such as virtual machines or containers.
• your function takes a request and queues a job on some other system and doesn’t send a response.
• the caller can handle an occasional slow response during a cold start.
• your function takes items from a queue, but the queue is not going to reach capacity.
• your function handler is very fast, so once you have a few warm execution environments they will be able to handle all the requests received.
• you have a steady rate of requests, and you don’t have unpredictable bursts of requests.

In all the above, cold starts may not be something you need to concern yourself with at all. Your application may respond quickly enough with infrequent delays.

Scenarios where cold starts may be a problem

Here are a few scenarios where cold starts may be a problem, of course, there are many more, but in these scenarios, you should consider some of the steps in the next section, or choose an alternative technology for hosting your .NET application.

• your function is invoked infrequently, responds quickly, but the response is time sensitive. But you can’t handle the occasional slow response during a cold start.
• your function is invoked infrequently, but the function handler is slow to respond. The slow response may cause many execution environments to be created, and each will have a cold start.
• your function is called frequently but is slow to respond due to heavy processing work, slow I/O, etc. In this case, a different technology may be a better fit, consider using Fargate or App Runner.
• you have unpredictable bursts of requests that need to be handled quickly. When you get a burst of requests, each may cause an execution environment to be created, and each will have a cold start.
• your function performs a long-running task. In this case, the execution environment will be held by that task, and any subsequent requests will cause a new execution environment to be created, and each will have a cold start. Long-running tasks are generally not a good fit for Lambda functions. Consider using Step Functions with Lambda, or move to a different technology such as Fargate or App Runner.

What can you do about cold starts

There are several steps you can take to reduce the number of cold starts, and how long they take.

The simplest and first to consider is provisioned concurrency. It requires no changes to your code.

• use provisioned concurrency, this will keep a set number of execution environments warm, and ready to handle requests. There is a cost for this, but it can be as little as $3 a month for a single execution environment.
• if you have many functions, and they each do very little and are called infrequently, consider combining them into fewer functions. This will reduce the number of cold starts because the combined functions will be called more often.
• add provisioned concurrency to the combined functions (previous suggestion). This will reduce cold starts and be more cost-effective than using provisioned concurrency on many functions.
• prewarm execution environments. See these posts related to prewarming for examples of how to do this.
• keep your function code (handler and initialization code) as small as possible. The smaller the binary, the faster it will be to download and JIT compile.
• add more memory to your function, this will increase the power of the allocated CPU, reduce the time it takes to JIT, and initialize your code. This increases the price per execution.
• optimize your code in the same way you would for any other application.
• consider using PublishReadyToRun if your application uses .NET 6 or newer.
• consider using PublishTrimmed.
• consider using native AoT compilation if you application uses .NET 7.
• consider using arm64 architecture.

Summary

Don’t worry too much about cold starts until your application is under a production-like load. Even then, there are multiple ways to reduce the impact of cold starts.

Try it out, and see how it works for you. If you have any questions or suggestions, leave a comment below.