DEV Community

Vadym Kazulkin for AWS Community Builders

Posted on • Updated on

AWS SnapStart - Part 22 Measuring cold and warm starts with Java 17 using synchronous HTTP clients

Introduction

In the previous parts we've done many measurements with AWS Lambda using Java 17 runtime with and without using AWS SnapStart and additionally using SnapStart and priming DynamoDB invocation :

In this article we'll now add another dimension to our Java 17 measurements : the choice of HTTP Client implementation. Starting from AWS SDK for Java version 2.22 AWS added support for their own implementation of the synchronous CRT HTTP Client. The asynchronous CRT HTTP client has been generally available since February 2023. In this article we'll explore synchronous HTTP clients first and leave asynchronous ones for the next article.

I will also compare it with the same measurements for Java 21 already performed in the article Measuring cold and warm starts with Java 21 using different synchronous HTTP clients

Measuring cold and warm starts with Java 17 using synchronous HTTP clients

In our experiment we'll re-use the application introduced in part 8 for this. Here is the code for the sample application. There are basically 2 Lambda functions which both respond to the API Gateway requests and retrieve product by id received from the API Gateway from DynamoDB. One Lambda function GetProductByIdWithPureJava17Lambda can be used with and without SnapStart and the second one GetProductByIdWithPureJava17LambdaAndPriming uses SnapStart and DynamoDB request invocation priming.

As we did our measurements for Java 17 in the previous articles of the series, we have always used the default HTTP Client implementation which is Apache HTTP Client (we'll use the measurements for the comparison in this article), now we'll explore 2 other options as well.

There are now 3 synchronous HTTP Clients implementations available in the AWS SDK for Java.

  1. Url Connection
  2. Apache (Default)
  3. AWS CRT

This is the order for the look up and set of synchronous HTTP Client in the classpath.

Image description

Let's figure out how to configure the HTTP Client. There are 2 places to do it : pom.xml and DynamoProductDao

Let's consider 3 scenarios:

Scenario 1) Url Connection HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>url-connection-client</artifactId>
     </dependency>
Enter fullscreen mode Exit fullscreen mode

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(UrlConnectionHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();
Enter fullscreen mode Exit fullscreen mode

Scenario 2) Apache HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>apache-client</artifactId>
     </dependency>
Enter fullscreen mode Exit fullscreen mode

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(ApacheHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();
Enter fullscreen mode Exit fullscreen mode

Scenario 3) AWS CRT synchronous HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>aws-crt-client</artifactId>
     </dependency>
Enter fullscreen mode Exit fullscreen mode

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(AwsCrtHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();
Enter fullscreen mode Exit fullscreen mode

For the sake of simplicity, we create all HTTP Clients with their default settings. Of course, there is the optimization potential there to figure out the right HTTP Client settings.

The results of the experiment below were based on reproducing more than 100 cold and approximately 100.000 warm starts with experiment which ran for approximately 1 hour. For it (and experiments from my previous article) I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman. I ran all these experiments for all 3 scenarios using 2 different compilation options in the AWS SAM template.yaml each:

  1. no options (tiered compilation will take place)
  2. JAVA_TOOL_OPTIONS: "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling)

We found out in the article Measuring cold and warm starts with Java 17 using different compilation options that with them both we've got the lowest cold and warm start times. We’ve also got good results with "-XX:+TieredCompilation -XX:TieredStopAtLevel=2” compilation option but I haven’t done any measurement with this option yet.

Let's look into the results of our measurements.

Cold (c) and warm (m) start time with compilation option "tiered compilation" without SnapStart enabled in ms:

Scenario Number c p50 c p75 c p90 c p99 c p99.9 c max w p50 w p75 w p90 w p99 w p99.9 w max
Url Connection 2615.48 2672.78 2726.22 3660.9 3817.73 3993.28 6.82 7.57 8.80 22.01 50.82 2172.5
Apache 2831.33 2924.85 2950.12 3120.34 3257.03 3386.67 5.73 6.50 7.88 20.49 49.62 1355.08
AWS CRT 2340.71 2406.5 2482.01 2578.71 2721.06 2890.88 5.73 6.61 8.00 21.07 70.39 980.93

Cold (c) and warm (m) start time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) without SnapStart enabled in ms:

Scenario Number c p50 c p75 c p90 c p99 c p99.9 c max w p50 w p75 w p90 w p99 w p99.9 w max
Url Connection 2610.59 2700.55 2800.53 3028.36 3184.08 3326.09 7.04 7.88 9.31 22.55 55.04 1286.36
Apache 2880.53 2918.79 2974.45 3337.29 3515.86 3651.65 6.11 7.05 8.94 23.54 62.99 1272.96
AWS CRT 2268.78 2314.49 2341.29 2461.23 2613.98 2754.08 5.55 6.30 7.57 20.49 75.70 1010.95

Cold (c) and warm (m) start time with compilation option "tiered compilation" with SnapStart enabled without Priming in ms:

Scenario Number c p50 c p75 c p90 c p99 c p99.9 c max w p50 w p75 w p90 w p99 w p99.9 w max
Url Connection 1510.72 1566.07 1797.68 2006.60 2012.63 2014.23 6.93 7.87 9.38 23.92 935.81 1343.25
Apache 1506.20 1577.06 1845.01 2010.62 2280.46 2281 5.82 6.72 8.39 22.81 798.46 1377.54
AWS CRT 1196.86 1313.44 1584.96 1781.58 1872.88 1873.52 5.55 6.41 7.87 21.40 681.26 1164.44

Cold (c) and warm (m) start time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) with SnapStart enabled without Priming in ms:

Scenario Number c p50 c p75 c p90 c p99 c p99.9 c max w p50 w p75 w p90 w p99 w p99.9 w max
Url Connection 1567.63 1647.96 1889.80 2026.76 2075.97 2076.57 7.10 8.00 9.69 25.41 953.93 1190.54
Apache 1521.33 1578.64 1918.35 2113.65 2115.77 2117.42 6.01 7.05 8.94 23.92 101.41 1077.45
AWS CRT 11176.70 1259.45 1621.82 1854.25 1856.11 1857.59 5.55 6.30 7.63 21.40 670.53 990.96

Cold (c) and warm (m) start time with compilation option "tiered compilation" with SnapStart enabled and with DynamoDB invocation Priming in ms:

Scenario Number c p50 c p75 c p90 c p99 c p99.9 c max w p50 w p75 w p90 w p99 w p99.9 w max
Url Connection 666.97 745.23 965.42 1084.10 1108.20 1108.66 7.21 8.07 9.61 24.22 145.49 377.43
Apache 708.90 790.50 960.61 1041.61 1148.80 1149.91 5.64 6.61 8.38 21.07 141.53 373.37
AWS CRT 679.76 851.18 1026.11 1102.68 1111.53 1111.64 5.92 6.72 8.26 22.09 171.22 1065.32

Cold (c) and warm start (m) time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) with SnapStart enabled and with DynamoDB invocation Priming in ms:

Scenario Number c p50 c p75 c p90 c p99 c p99.9 c max w p50 w p75 w p90 w p99 w p99.9 w max
Url Connection 673.67 748.22 946.31 1184.96 1213.73 1214.34 7.16 8.13 9.83 25.89 141.53 275.35
Apache 692.79 758.00 1003.80 1204.06 1216.15 1216.88 6.21 7.27 9.38 25.09 103.03 256.65
AWS CRT 640.19 693.49 1022.02 1229.60 1306.90 1307.14 5.64 6.51 8.13 22.81 171.22 877.24

Conclusion

In terms of the HTTP Client choice for Java 17, AWS CRT HTTP Client is preferred choice in case SnapStart isn't enabled or SnapStart is enabled but no priming is applied. In case of priming of the DynamoDB invocation, the results in terms of the cold starts for all 3 HTTP Clients are close to each other as the initialization of the DynamoDB Client with the HTTP Client and the most expensive first invocation (priming) happens already during the deployment phase of the Lambda function and doesn't impact the further invocations that much. The Apache HTTP Client is probably the most powerful choice, but it shows the worst results for the cold starts for SnapStart not being enabled.

We observed the same also for Java 21, see the measurements in the article Measuring cold and warm starts with Java 21 using different synchronous HTTP clients.

The warm execution times are more or less close to each other for all 3 HTTP clients and compilation options and very a bit in favor of one or another HTTP Clients depending on the percentile and compilation option. We observed the same also for Java 21

Can we reduce the cold start a bit further? From our article Measuring cold starts with Java 17 using different deployment artifact sizes we know that smaller deployment artifact sizes lead to the lower cold start times. The usage of AWS CRT HTTP Client adds 18 MB to the deployment artifact size for our sample application (total size 32MB versus 14 MB for URL Connection and Apache HTTP Clients). If we look into the deployment artifact with AWS CRT HTTP Client, we'll discover the following additional packages for each operating system : linux, osx and windows.

Image description

If we take a look into those folders, we'll see for example the following content for the linux folder (the same applies for windows and osx folders) :

Image description

As we see the content of such folders is natives file for each operating system and processor architecture: for osx it's libaws-crt-jni.dylib file, for windows - aws-crt-jni.dll and for linux - libaws-crt-jni.so. If we already know that we'll run our Lambda only on Linux x86 architecture, we can delete the osx and windows folders completely and subfolders for arm architecture in the linux folder. This will reduce the deployment artifact size from 32 to 19 MB for AWS CRT HTTP Client and further reduce the cold start time a bit.

The choice of HTTP Client is not only about minimizing cold and warm starts. The decision is much more complex end also depends on the functionality of the HTTP Client implementation and its settings, like whether it supports HTTP/2. AWS publshed the decision tree which HTTP client to choose depending on the criteria.

In the next article of the series we'll make the same measurements for Java 17 but using the asynchronous HTTP Clients.

Update on 06.06.2024. For the CRT client we can set classifier (i.e. linux-x86_64) in our POM file to only pick the relevant binary for our platform. See here. Big thanks to Maximilian Schellhorn for the hint!

Top comments (0)