AMM Performance Testing Report

Overview

In the rippled 1.12.0 release, the AMM amendment stands out as a significant feature in both size and scope. Since September 2022, the RippleX performance team has collaborated closely with the engineering team responsible for the AMM feature implementation. This report presents a thorough overview of our testing approach, findings, and key takeaways. Based on our analysis, we are confident that activating the AMM amendment will not negatively impact the network's consensus performance or its ability to process other transactions.

Objective

Our primary objective was to assess transaction performance and its effect on consensus processing, particularly concerning the DEX upon implementing the AMM amendment. Specifically, we aimed to ascertain if the AMM amendment could possibly impede the performance of the XRP Ledger network at the worst case scenarios. To achieve this goal, we adopted a new testing methodology compared to the previous performance tests which primarily focused on exploring the maximum software processing capacity. In this testing, our focus was on maintaining a 5-second network consensus processing latency as the capacity threshold.

Testing Methodology

Capacity Criteria

Given the objective to measure the maximum performance impact of integrating AMM transactions in the DEX, we use the 5-seconds consensus latency limit to gauge the consensus capacity while ensuring the network is in its optimal condition. During testing, both the ledger's and consensus's performance and latency were assessed to determine whether the network load surpassed this latency criteria.

Load Modeling

Research was undertaken to understand the distribution of payment transaction path sizes. Our findings showed that over 99.7% of transactions did not specify the Paths field. When the Paths field is not specified in cross currency payments, it is assumed that there is one path and one step. As a result, we came up with the following load scenarios.

• Scenario #1: XRP-XRP direct payments
• Scenario #2: Payment transactions involving the sending and receiving of two IOU tokens
  • 2.1: Using only the AMM-based DEX
  • 2.2: Using only the LOB-based DEX
• Scenario #3: Mixed load scenario to emulate a heavy load condition
  • 3.1: Using only the AMM-based DEX
  • 3.2: Using only the LOB-based DEX

For scenario #3, the complexity and the proportions of each condition is laid out in the table below.

Test Environment

Our testing infrastructure simulated a private XRPL environment with 9 nodes, matching Ripple's MainNet in terms of hardware specifications. The key features of this environment are the following.

• Network Setup
  • 5 function as validator nodes
  • 4 serve as client P2P nodes, interfacing with load generators
• Uniform System Specifications: Each system mirrors the hardware specifications found in Ripple’s MainNet rippled infrastructure. Specifically, they are hosted on AWS's EC2 instance type, z1d.2xlarge, boasting 8 CPU cores, 64GB RAM, and 300GB NVMe SSD storage, dedicated to the rippled database.
• Network Configuration: All nodes operate within the same AWS region and are interconnected through a shared LAN.
• Load Submission: Between 1 and 4 load servers are actively engaged in transmitting transactions to the 4 P2P nodes.
• Account Setup: A total of 100K synthetic accounts were established, built atop a public ledger synchronized from the MainNet as of July 3rd, 2023. Provisions were made in the simulation logic to ensure the uniqueness of source and destination accounts for every consensus cycle.
• Configuration Continuity: The rippled configuration file was sourced directly from a Ripple MainNet validator. While the core configurations remained unchanged, requisite modifications were made to align with the specific needs of our environment.

Test Data Setup

A total of 100K synthetic accounts were established based on a public ledger synced from the MainNet as of July 3rd, 2023. To maintain the integrity of the simulation, we have implemented logic ensuring that unique accounts are designated as sources and destinations for simultaneous transactions throughout each consensus cycle. The following describes the setup.

• Rippling activated across all synthetic accounts
• 15 IOU Tokens
• Designated two synthetic accounts as the “genesis account” and the “liquidity provider”
• Accounts have trustlines with the issuers of all the IOU Token in the network
• Accounts are sufficiently funded with the IOU Tokens
• Created AMM instances to allow intricate payment paths as shown in the “AMM/LOB IOU Tokens Topology” graph
  • A bi-directional arrow shows that there is a corresponding AMM instance between the two IOU tokens

Test Results

Here are the test results from the scenarios discussed in the Load Modeling section.

Conclusions

To underscore the scale of the load test, we have juxtaposed the load test throughput with the year-to-date throughput, represented as “Current TPS”, of Payment transactions in the graph below.

From our comprehensive testing and subsequent evaluation, it is clear that, despite the computational intricacies associated with the transactions that interact with the AMM-based DEX, we have made significant advancements in XRPL's performance over the last 18 months. This has propelled the transaction throughput beyond 100 TPS, even under the most challenging conditions. Considering these outcomes, we are confident that incorporating the XLS-30 AMM feature into rippled will not introduce risks to the network in the near term. We remain steadfast in our dedication to rigorous research and development, aiming to continually elevate the performance and adaptability of the XRP Ledger.