Lambda insights to the rescue

In previous episode we enabled tracing and we were able to see more information about our platform, requests, etc. Still, it is not much, but it is better than it was! Let's continue then.

Lambda Insights

The best description of Lambda insights will be the enhanced monitoring. According to AWS documentation, Insights is a service which collects, aggregates and summarized the system-level metrics, like CPU usage, memory usage, disk and network, and also collects diagnostic information like Lambda workers shutdown and cold start.

Ok, looks like this helps to understand better what is going on in the application.

Enable Lambda Insights

Similarly like you did it for X-Ray, navigate to your Lambda, Configuration and finally to Monitoring and operations tools.

AWS tries to update your roles and policies accordingly. When completed, you need to do some requests and wait some time before you start using Insights.

Also, what is most important, you should see a new layer added to your Lambda

In order to see Insights you can go there through CloudWatch console, or directly from your Lambda (from Monitoring tab).

...

...

...

...

...

...

...

...

Finally, after many, many minutes (in my case when I prepared this example, it was more than 30 minutes), Insights are generated.

Lambda Insights console

AWS gives the possibility to check mulitple functions at once, but I will go with single function view, as I have only one function enabled at the moment.

Metrics are available and are clear and readable. Invocations, errors, duration, and all infrastructure related metrics have their own panel.

Below these panels, AWS placed two tabs. One is for application logs, you saw it already, so I will not focus on it. The second (well, in fact, the first tab) is about Invocations.

This view gives us a lot of information. Yes, part of it is available in logs, but here we have it without parsing. Duration, consumed memory, CPU time, network usage, error information.

Additionally, we can see Init duration, which indicates cold start. Nice.

Let's explore it further. On top right corner of this part of the view we have two controls.

Both will take you to Logs Insights with two different queries.

Performance logs

This link will go to Logs with query shown below

fields @timestamp, @message, @logStream
| filter function_name = "lambdaDemo-lambdaFunction-qnettGrKoNgh"
| sort @timestamp desc
| limit 20

Here we have well formatted metrics about the whole invocation. Not just the Report generated (like in logs), but full information. For example, quite interesting data, like cold start, agent version, timestamp in UNIX format, traceID and many more.

Example of returned record is shown below

Application logs

Executes another query

fields @timestamp,@message,@logStream
| sort @timestamp desc
| limit 20

This also shows a lot of interesting information. First, visualisation (histogram) is available. Very nice.

Second, we have all logs collected in one place.

Of course we can add it to SAM template. Here are the needed changes:

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Description: Simple Lambda

Resources:
  lambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: simplefunction.handler
      CodeUri: lambdafunction/
      Policies:
        - CloudWatchLambdaInsightsExecutionRolePolicy
      Runtime: python3.8
      AutoPublishAlias: live
      Description: Simple demo function
      MemorySize: 128
      Timeout: 10
      Tracing: Active
      Layers:
        - !Sub "arn:aws:lambda:${AWS::Region}:580247275435:layer:LambdaInsightsExtension:14"
      Events:
        simpleApi:
          Type: Api
          TracingEnabled: true
          Properties:
            Path: /
            Method: get

I added the policy CloudWatchLambdaInsightsExecutionRolePolicy and layer LambdaInsightsExtension:14 to the Lambda definition. What is important here, I used the syntax ${AWS::Region} in order to attach the layer from the region in which my Lambda is executed. This is why I used !Sub to "translate" this information during the actual execution of the template.

Ok, all of this is very nice. Again, we know more about our system. But still, something is missing, something big. In next episode we start to add more detailed tracing / loging / metrics to the Lambda.

Cover image by Hebi B. from Pixabay
Waiting image by Shlomaster from Pixabay