Putting Tasks in a Cache, and Computing Only Once, When First Requested

Most of the time when I cache a value it is something that I pull from a slower data source like a database or an API. Other times the value to store is something that takes significant work to compute, so I don’t want to recompute it each time it is requested. Normally the value stored in the cache is some concrete value, like a number, string, or business object.

But you can also store an unstarted Task<T> in a cache. So when the key is requested, the value is returned if it is available, or if the value is not yet computed, you trigger the code to get it.

Why would you do this? Well, you may never want to go to the expense of getting/computing the value if it is never needed. By storing the unstarted Task itself in the cache, rather than the result of the task you achieve significant performance improvements. If multiple requests for the same key come in around the same time, they will all wait for the one task to complete, rather than starting multiple tasks that perform the same work.

Here is the basic code -

1if(cache.TryGetValue(id, out Task<Task<string>> doWorkAsyncTask))
2{
3    if(doWorkAsyncTask.Status == TaskStatus.Created)
4    {
5        doWorkAsyncTask.Start();
6    }
7    return await await doWorkAsyncTask;
8}
9return $"Value {id} not found";

That’s the core of it, but you haven’t seen how to add the Tasks to the cache, and the way they are added is important.

A Fuller Example

To explain everything I have a Web API application.

Its GET request takes an id parameter. The cache is checked for that id, and if it is there, the task is returned. It is started if it is not already running or complete. Then the task is awaited and the result is returned.

Sounds simple enough, but I don’t see any examples like this on the web.

The basic API code

Here is the usual start Program.cs code for a Web API application, but with a cache added -

using Microsoft.Extensions.Caching.Memory;

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

MemoryCache cache = new MemoryCache(new MemoryCacheOptions());

The method that takes a while to run

Here is the method that takes a while to run. For the purposes of the example, it is a simple asynchronous method that takes an int, and returns a Task<string>.

async Task<string> DoWorkAsync(int number)
{
    Console.WriteLine($"Running DoWorkAsync with a {number} second delay");
    DateTime started = DateTime.Now; 
    await Task.Delay(number * 1000);
    return $"Started at {started:HH:MM:ss}, finished at {DateTime.Now:HH:MM:ss}. This value took {number} seconds to generate";
}

Adding unstarted tasks to the cache

Here is the code that adds the unstarted tasks to the cache -

1for (int loop = 1; loop <= 20; loop++)
2{
3    int delay = loop; // this is important
4    var taskToRunLater = new Task<Task<string>>(() => DoWorkAsync(delay));
5    cache.Set(loop, taskToRunLater);
6}

There are a few things to note here.

• Line 3, assigns the loop variable to the delay variable. This is important because if I don’t do this, the value of loop will be 21 for all tasks when they are run.
• Line 4, creates a new Task that returns a Task<string>. The outer Task takes a Func<Task<string>> as a parameter. I pass this a Lambda. The Func will run when the task is started.
• Line 5, adds the unstarted task to the cache.

The API method

Here is the code that runs when the API is called -

 1app.MapGet("/{id}", async (int id) => {
 2    if(cache.TryGetValue(id, out Task<Task<string>> doWorkAsyncTask))
 3    {
 4        if(doWorkAsyncTask.Status == TaskStatus.Created)
 5        {
 6            doWorkAsyncTask.Start();
 7        }
 8        return await await doWorkAsyncTask;
 9    }
10    return $"Value {id} not found";
11});

• Line 2, checks the cache for a key with the value of id. If it is found, the Task<Task<string>> is returned. Note, it is a nested task.
• Line 4, checks the status of the task. If it is in the Created status, then line 6 starts it.
• Line 8, has two awaits, the first one returns the Task<string>, and the second one returns the string.

The first time a key that is in the cache is requested, the task is started. If another request comes in for the same key the task will not be in the Created state, so it will not be started again. The second request will wait for the first one to finish. **In this way, the same key could be requested a thousand times while the value is being computed, but there will be a single invocation of the method that computes the value. ** Once computed, all subsequent requests for the key will get the value immediately.

The full code

For clarity, here is the full code -

 1using Microsoft.Extensions.Caching.Memory;
 2
 3var builder = WebApplication.CreateBuilder(args);
 4var app = builder.Build();
 5
 6MemoryCache cache = new MemoryCache(new MemoryCacheOptions());
 7
 8for (int loop = 1; loop <= 20; loop++)
 9{
10    int delay = loop; // this is important
11    var taskToRunLater = new Task<Task<string>>(() => DoWorkAsync(delay));
12    cache.Set(loop, taskToRunLater);
13}
14
15app.MapGet("/{id}", async (int id) => {
16    if(cache.TryGetValue(id, out Task<Task<string>> doWorkAsyncTask))
17    {
18        if(doWorkAsyncTask.Status == TaskStatus.Created)
19        {
20            doWorkAsyncTask.Start();
21        }
22        return await await doWorkAsyncTask;
23    }
24    return $"Value {id} not found";
25});
26
27app.Run();
28
29async Task<string> DoWorkAsync(int number)
30{
31    Console.WriteLine($"Running DoWorkAsync with a {number} second delay");
32    DateTime started = DateTime.Now; 
33    await Task.Delay(number * 1000);
34    return $"Started at {started:HH:MM:ss}, finished at {DateTime.Now:HH:MM:ss}. This value took {number} seconds to generate";
35}