C# and AWS Lambdas, Part 5 – Updating the Zip in S3 and Updating the Running Lambda, with Pulumi IaC
Full source code available here.
- Part 1 - Hello World
- Part 2 - Web API and an API Gateway
- Part 3 - Pulumi IaC for Web API and an API Gateway
- Part 4 - Storing the Zip in S3, Setup with Pulumi IaC
- Part 5 - Updating the Zip in S3 and Updating the Running Lambda, with Pulumi IaC
- Part 6 - .NET 5 inside a Container inside a Lambda
- Part 7 - .NET 5 Web API inside a Container inside a Lambda, with API Gateway in front
- Part 8 - .NET 6, inside a Container, inside a Lambda
This post pulls together a few threads I’ve been working on - the creation of Lambda to run .NET, storing the zip in S3, and updating the .NET Lambda when the zip in S3 is updated.
This one took quite a while to put together - the permissions, roles, and policies were not obvious and I hope it will be of help to you. This is not a blog post on CI/CD, I am cutting corners by using Pulumi to upload the zip files initially, and then use the AWS command line to send zips to S3. In a future set of posts I will show how to use GitHub Actions to build the infrastructure, and to compile and deploy the .NET Lambda directly to S3 from GitHub.
I want to have a Lambda that runs .NET code stored in a zip file in S3. I want to be able to update the zip and have the .NET Lambda run the code in the new zip. I had hoped this would be a little tick box on the Lambda, but sadly there is no such box.
Instead, I have a second Lambda (referred to as the updater Lambda) that is triggered by an update on a specified bucket in S3. This updater Lambda in turn calls an update on the .NET Lambda and within a few seconds, the .NET Lambda will be running the new code. Doesn’t sound easy, but I didn’t think it would be too hard, but take a look at the number of resources needed!
- A role to run .NET Lambda.
- A role to run Lambda that updates the .NET Lambda, I’m calling this the updater.
- A policy to give the updater permissions to update the .NET Lambda and S3.
- A policy attachment for the .NET Lambda.
- A policy attachment for the updater Lambda.
- An S3 bucket.
- An S3 bucket object.
- The .NET Lambda pointing at the bucket and bucket object.
- The zip file for the .NET Lambda.
- The updater Lambda with variables passed in to verify the update of the .NET Lambda.
- The zip file for the updater Lambda - Node.js.
- Permission for the bucket to call the updated Lambda.
- A bucket notification with attached permissions.
- Reduce the bucket accessible to the public (not necessary, but good).
That’s a lot more than the tick box I was hoping for.
Below is the code of the updater lambda. The
if checks to make sure that the
lambda.updateFunctionCode(..) runs only if the expected file in S3 is updated. The environmental variables were passed in via the Pulumi code above.
The zip attached to this blog post has all the source code needed, you don’t have to add or change anything.
From the console, run -
Note the outputs. These are the name of your Lambdas and the s3 bucket and key -
Outputs: LambdaHelloWorldFunctionName: "PulumiHelloWorldAutoUpdate_LambdaHelloWorldFunction-???????" LambdaUpdateFunctionName : "PulumiHelloWorldAutoUpdate_LambdaUpdateFunction-???????" S3Bucket : "pulumi-hello-world-auto-update-s3-bucket" S3Key : "helloworld.zip"
Go to the AWS console, and test the Lambda as shown in part 1 of this blog series.
You should get output like this - “HELLO WORLD”.
Updating the zip in S3
Now to try out the real functionality, updating the zip in S3 and see if it runs in the .NET Lambda.
In the attached source there is a
Lambdas directory with two subdirectories -
helloworld_with_date. They contain two variations of the .NET application. The first converts the input text to uppercase, the second converts the input text to uppercase and adds the current date and time.
You can run the below commands to upload each zip file and try out the Lambda. A few seconds after you upload, the .NET Lambda will use that zip.
// no date aws s3 cp ./Lambdas/helloworld_no_date/helloworld.zip s3://pulumi-hello-world-auto-update-s3-bucket/helloworld.zip // with date aws s3 cp ./Lambdas/helloworld_with_date/helloworld.zip s3://pulumi-hello-world-auto-update-s3-bucket/helloworld.zip
If you don’t want to go into the AWS UI console to try out the Lambda you can invoke it from the command line, but you need to swap the function name below for the one in the output of the
pulumi up command -
aws lambda invoke --function-name PulumiHelloWorldAutoUpdate_LambdaHelloWorldFunction-??????? --payload '"hello world"' /dev/stdout
This was a long tough one, but I’ve learned a lot about AWS, Pulumi, and even GitHub Actions (more on that soon).
Full source code available here.
- C# and AWS Lambdas, Part 4 – Storing the Zip in S3, Setup with Pulumi IaC
- C# and AWS Lambdas, Part 3 – Pulumi IaC for Web API and an API Gateway
- GitHub Actions with .NET, Part 4 - Building an S3 bucket with Pulumi
- C# and AWS Lambdas, Part 2 - Web API and an API Gateway
- C# and AWS Lambdas, Part 1 - Hello World