Caching Secrets Manager Secrets with a Lambda Layer

Want to learn more about AWS Lambda and .NET? Check out my A Cloud Guru course on ASP.NET Web API and Lambda.

Download full source code.

This post builds on earlier posts I wrote on accessing Secrets Manager secrets from Lambda functions. In those posts, I showed how to use the AWS SDK to access secrets. This post shows how to use a Lambda layer to cache secrets in memory and access them with an HTTP request. This post builds on earlier posts I wrote on accessing Secrets Manager secrets from Lambda functions. In those posts, I showed how to use the AWS SDK to access secrets. This post shows how to use a Lambda layer to cache secrets in memory and access them with an HTTP request using the AWS Parameters and Secrets Lambda Extension.

The earlier post showed how to create a secret, and deploy a Lambda function that retrieved the secret.

You are going to follow a few steps from that post.

Steps 1, 2, 3

Follow the steps from the linked post to get the tools, create the secret, and create the Lambda function. But change the name of the function name to LambdaSecretsManagerCachedLayer.

Open the Function.cs file, and replace the content with -

 1using System.Net.Http.Json;
 2using Amazon.Lambda.Core;
 3using Amazon.SecretsManager.Model;
 4
 5// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
 6[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
 7
 8namespace LambdaSecretsManagerCachedLayer;
 9
10public class Function
11{
12    private readonly HttpClient _httpClient;
13    public Function()
14    {
15        _httpClient = new HttpClient() { BaseAddress = new Uri("http://localhost:2773") };
16        _httpClient.DefaultRequestHeaders.Add("X-AWS-Parameters-Secrets-Token", Environment.GetEnvironmentVariable("AWS_SESSION_TOKEN"));
17    }
18    
19    public async Task<string> FunctionHandler(ILambdaContext context)
20    {
21        string secretId = "my-credentials";
22
23        var secret = await _httpClient.GetFromJsonAsync<GetSecretValueResponse>($"/secretsmanager/get?secretId={secretId}");
24        return secret.SecretString;
25    }
26}

4. Deploy the function

To deploy your application to the Lambda service, run -

dotnet lambda deploy-function LambdaSecretsManagerCachedLayer

You will be asked to select an IAM role, or create a new one, at the bottom of the list will be *** Create new IAM Role ***, type in the associated number.

You will be asked for a role name, enter LambdaSecretsManagerCachedLayerRole.

After this you will be prompted to select the IAM Policy to attach to the role, choose AWSLambdaBasicExecutionRole, it is number 6 on my list.

After a few seconds, the function will be deployed.

5. Add permissions to the role

You can add the relevant permissions to the role via the AWS Console, or the command line.

In the attached zip file there is a policies directory, open the ReadCredentialsFromSecretsManager.json and change the ARN to the one returned when you created the secret in step 2.

The following command will create an inline policy, granting LambdaSecretsManagerCachedLayerRole access to the secret you created.

aws iam put-role-policy --role-name LambdaSecretsManagerCachedLayerRole --policy-name ReadCredentialsFromSecretsManager --policy-document file://policies/ReadCredentialsFromSecretsManager.json

Now the Lambda function will be able to access the secret.

For completeness, here is the content of ReadCredentialsFromSecretsManager.json -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "secretsmanager:GetSecretValue",
            "Resource": "arn:aws:secretsmanager:us-east-1:xxxxxxxxxx:secret:my-credentials-xxxxxx"
        }
    ]
}

6. Add the Secrets Manager cache layer

You need to look up the ARN of the AWS Systems Manager Parameter Store and AWS Secrets Manager extension layer.

When you find the one for your region run the following (this example uses us-east-1, change as appropriate) -

aws lambda update-function-configuration --function-name LambdaSecretsManagerCachedLayer --layers arn:aws:lambda:us-east-1:177933569100:layer:AWS-Parameters-and-Secrets-Lambda-Extension:2

7. Invoke the function

Try invoking the function -

dotnet lambda invoke-function LambdaSecretsManagerCachedLayer 

The request will succeed and you will see -

Amazon Lambda Tools for .NET Core applications (5.3.0)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet

Payload:
"'{\"username\":\"bryan\",\"password\":\"A-COMPLEX-PASSWORD123!\"}'"

bad session token or header key

If you are trying to use the AWS Systems Manager Parameter Store and AWS Secrets Manager extension layer and you are getting this error -

bad session token or header key

You are missing the X-AWS-Parameters-Secrets-Token header. You need to add this header to your request, setting the value to the environment variable AWS_SESSION_TOKEN. See the code example above.

Download full source code.