Getting the JSON sent to a Lambda Function when Deserialization Fails

Want to learn more about AWS Lambda and .NET? Check out my A Cloud Guru course on ASP.NET Web API and Lambda.

When working with Lambda functions that handle events raised by other services (AWS or third party), it can sometimes be difficult to work out what the right model is to use as the input to the function.

I hit this problem recently when using a Lambda function to handle an event sent via EventBridge from another service, but the technique will work with JSON sent from any source. If you are not aware, Lambda lets you run your .NET code in a function as a service manner.

My function handler looked like this -

public void FunctionHandler(CloudWatchEvent<SomeServiceStatusChange> someServiceStatusChange, ILambdaContext context)
{
    // work with someServiceStatusChange...
}

I thought I had the correct model in my function, but I got the following errors in the CloudWatch logs -

Error converting the Lambda event JSON payload to type...
System.Text.Json.JsonException: The JSON value could not be converted to...

I made a minor change to my model but still got the same error.

At this point, it was clear that my model didn’t match the JSON the function was receiving. Deserialization was not going to work until I fixed the model. But to fix the model, I needed to know what the JSON looked like.

There are two approaches you can take, the first is to set an environment variable in the Lambda function which causes the raw JSON to be logged to CloudWatch before the function handler is called.

The second is to change the function handler input parameter type to Stream and read the raw JSON from the stream.

Setting the environment variable

This is the easier of the two approaches, it requires no code changes, but be sure to turn it off when you’re done. You may have sensitive data in the JSON that you don’t want to be logged all the time.

From the command prompt, run the following command -

aws lambda update-function-configuration --function-name MyLambdaFunction --environment "Variables={LAMBDA_NET_SERIALIZER_DEBUG=true}"

Then open up the CloudWatch logs for the Lambda function and you will see the raw JSON logged before the function handler is called.

Changing the parameter type to Stream

This approach requires a code change.

In the function handler I change the input parameter type to Stream. .NET Lambda functions support the Stream type without any serialization/deserialization.

After receiving the stream, I read it to a string, and log it to CloudWatch. No more deserialization errors and I can see the raw JSON.

public void FunctionHandler(Stream someServiceStatusChange, ILambdaContext context)
{
    StreamReader reader = new StreamReader(someServiceStatusChange);
    string json = reader.ReadToEnd();
    context.Logger.LogInformation($"someServiceStatusChange:{json}");
}

I invoked the service that uses EventBridge to send the event to the Lambda function, then I opened the logs of the Lambda function in CloudWatch.

The raw JSON is now visible in the logs. I can use that to create the correct model.

2023-02-17T11:01:311Z   a4ea2b10-b5ba-4a12-b182-df7a12bbede1    info    input:
{
    "version": "0",
    "id": "45a19a57-ea9c-4d9d-9e01-4465f9840e84",
    "detail-type": "Some Service State Change",
    "source": "someservice",
    "account": "694977046108",
    "time": "2023-02-17T11:01:51Z",
    "region": "us-east-1",
    "resources": [],
    "detail": {
        "SomeServiceJobName": "8591b623-207f-4537-9dbe-8f1aeb28a258",
        "SomeServiceJobStatus": "COMPLETED"
    }
}

Now it’s a simple matter of updating the model to match the JSON.