Partnered With:

Comparative analysis of the five major DEX aggregators: What is the golden indicator to measure the pros and cons of an aggregator?

Read this article in 34 Minutes

This article aims to reveal the current state of the aggregator industry and how to objectively measure and compare the performance of aggregators.

Original title: "Introduction|Comprehensive Analysis of DEX Aggregator Performance"

Original source: Ethereum enthusiasts

Main conclusion:

·In 70% of cases, the adjusted price provided by 0x API is better than 1inch, Dex Ag, Paraswap and Uniswap. The so-called adjusted price is the price after paying the transaction fee.

0x API 1.0 leads the market in gas efficiency. 0x API provides a highly optimized custom architecture. Therefore, the cost for users to access Uniswap liquidity through this API is as low as or even lower than accessing Uniswap directly.

The aggregator eliminates the psychological barriers for users when choosing a trading platform by promising to allow users to obtain the best quotes from the entire ecology. However, not all aggregators are the same: they differ to varying degrees in terms of pricing, quote accuracy, gas usage, revert rates, and response times.

·The best offer is not necessarily the best adjusted price. Like buying an airline ticket, there may be a bunch of follow-on fees/costs. While the base fare may be cheap, when you factor in all the extras (baggage fees, priority boarding fees, reserved seat fees, etc.), the actual cost can be high.

0x API and Matcha (using 0x API in the background) are two products launched by 0x Labs that allow DeFi developers and traders to easily buy and sell assets at the best price among all DEX liquidity sources.

DEX Liquidity War and the Rise of Aggregators

The past few A plethora of new sources of liquidity have sprung up this week. There are more than $4 billion in funds spread across 15 sources, with new ones popping up every week. As a result, DEX liquidity has become more fragmented than ever, making it harder for DeFi users to find the best price for a particular transaction.

This has promoted the rise of the aggregator. Aggregators have grown significantly over the past 12 months, accounting for around 9.6% of total transactions in Q3. However, people get a little misunderstood about aggregators. For example, while quotes are often seen as the only metric to compare aggregator performance, it is not uncommon to see some aggregators quote $100 and settle at $90. Quotes are not actual settlement prices.

This begs the question: what is the correct metric to measure the performance of an aggregator?

This article aims to reveal the current state of the aggregator industry and how to objectively measure and compare the performance of aggregators.

Adjusted Prices – The Golden Metric for Comparing Aggregators

We thought it important to build a A high-performance professional-grade aggregator must follow the following principles:

1. Accuracy:

1. Accuracy: aggregator should have accuracy, i.e. the settlement price should be equal to the quote. Slight discrepancies are acceptable because there is often market fluctuation between when a transaction is submitted and when it is confirmed on-chain. However, all other things being equal, an aggregator with more accurate quotes is better. What you see is what you get.

2. Low cost: The more operations performed on the chain, the more gas usage That is, the more, the higher the gas cost. Aggregators should aim to keep costs as low as possible to ensure users get the best deal possible. Aggregators should adapt to the current gas price market and adjust paths accordingly. Every penny spent on gas should bring more value to users.

3. Low Rollback rate: To protect users, aggregators must keep the rollback rate of transactions of any size low. If the transaction fails, the user will be very frustrated because the gas fee has already been paid.

Finally, we summarize the above principles into a golden indicator: Adjusted Price .

While we have analyzed the above indicators separately, this article focuses primarily on adjusted prices. The adjusted price refers to the price after the user pays the transaction cost (gas fee and handling fee) when the transaction is completed on the chain. It is calculated as follows:

Adjusted Price = USD Value of Asset Sold / (USD Value of Asset Bought - Transaction Cost dollar value)

I believe this metric is better used to compare aggregators as this is the price at which users end up impacting user revenue. Let's look at an example:

A user wants to spend 100 DAI to buy USDC. Aggregator 1's offer is 100 USDC with a transaction cost of $5. From the above formula, we can get Adjusted Price = 100/(100 - 5) = $1.05.

Look at another aggregator. For the same 100 DAI, the quote from Aggregator 2 is 104 USDC, but the transaction cost is $10. In this case, adjusted price = 100/(104 - 10) = $1.06.

In the example above, Aggregator 1 found a cheaper price. Although Aggregator 2 quotes are higher, we can see the effect of transaction costs on the adjusted price. All in all, paying an extra $5 to buy 4 USDC is obviously slightly more expensive.

We simulated 59,975 trades and calculated the adjusted prices on different exchanges and came to the following conclusions:

_0x API、1inch、 Adjusted price wins for Dex Ag, Paraswap, and Uniswap. 0x API provides adjusted prices that outperform other aggregations 72% of the time_

On average, 0x API outperforms other aggregates 70% of the time other exchanges. That said, a trader using a 0x API aggregator such as Matcha has a 70% chance of getting the best price of any DEX.

In terms of the adjusted price, we found that the win rate of the exchange changes as the trading volume increases. In terms of small transactions, Uniswap does better because its transaction costs are lower than most aggregators. As the volume of transactions increases, aggregators will perform their functions and start splitting transactions to different sources. The 0x API performs well for transactions of any size. As transaction volume increases, aggregators begin to show their competitiveness. For example, in terms of large transactions, 1inch is more competitive than 0x API because it has more sources.

Methodology

This The data shared in this section comes from a simulation tool built by us at 0x Labs. In the face of different token pairs, transaction volume and confirmation delay, we can continue to quote and execute transactions through this tool. If you're tech-savvy, I'll tell you: the simulation is to execute an eth_call call on a Geth node and use state overrides to fund and track transactions, including buying and selling Metrics such as quantity and gas usage.

Exchanges we track include 0x API, 1inch, Dex Ag, Paraswap, and Uniswap. In this study, we selected the 20 most common trading pairs on these DEXs, and sampled an average of 11,995 transactions per exchange via public APIs. Trade volumes ranged from $1 million to $1 million, and settlement delays ranged from 0 to 180 seconds (to simulate wait times in a trading pool). The simulation lasted about 78 hours and the gas price fluctuated between 47 and 194 gwei, which ensured that our research reflected market conditions.

In the charts within this report, the green bars and Matcha represent both 0x APIs. Only the exchange whose adjusted price outperforms other exchanges by 0.005% will be considered the ultimate winner. If the difference is within 0.005%, it is considered a tie. A "tie including Metcha" refers to a tie between 0x API and one or more other exchanges. A "tie excluding Metcha" refers to a tie between two or more exchanges other than 0x API. All charts use the same color.

Accuracy, gas usage, rollback rate and response time

In this study In , we analyze three different prices. Therefore, we give the following definitions:

Quote: The initial advertised price for a particular transaction— The list price the user sees in the UI.

Actual price: the actual settlement price after the delay. Typically it takes between 0 and 180 seconds from transaction submission to confirmation on-chain.

Adjusted price: When a transaction is completed, the price after the user has paid the transaction cost (gas fee and handling fee). In other words, it is the actual price minus the cost.

Let's take a look at the quotation and actual price first.

0x API, 1inch, Dex Ag, Paraswap and Uniswap quotes and actuals Price wins. Many aggregators seem to be optimized for quotes, but the best quote doesn't necessarily mean the best adjusted price

In the graph above, We can see that the overall performance of Uniswap is not outstanding. The chart above is in stark contrast to the adjusted price equity chart. In other words, while Uniswap may not be able to provide the best quotes, it can sometimes provide the best adjusted prices for trades under $5,000 due to low transaction fees. As mentioned above, as transaction volume increases, Uniswap's win rate decreases significantly because aggregators are able to split transactions to different sources to find the best price.

From these charts, we can also see that the 0x API's win rate on the quoted and actual price is much lower than its win rate on the adjusted price —— The difference is 19% on average, and if a draw is taken into account, the winning rates of the quoted price and the actual price are 57% and 63% respectively. 0x API has an average win rate of about 72% at adjusted prices. The reason why the 0x API has such a low win rate in terms of quotes and actual prices is that we made an explicit trade-off when designing the routing logic of the smart contract, taking gas costs into account when calculating the transaction path of the order. By taking gas costs into account in the smart order routing of the 0x API, we can guarantee that the price advantage outweighs the cost of sourcing liquidity from different sources.

Unfortunately, many aggregators seem to be optimized only for winning bid competitions, as this gets them more users. However, as shown in the Aggregator 2 example above, aggregators that do not consider transaction costs can provide better offers because they are willing to pay more, but this is actually not the most favorable solution for users. Therefore, the best offer does not necessarily result in the best adjusted price.

A very apt analogy is booking a plane ticket. Have you ever booked on an airline because it seemed to be cheaper than another airline, only to find out that you have to pay extra for seat selection and a little more for checked bags? While the base fare might be cheaper, you end up paying more when you add in the miscellaneous fees. The same is true for some aggregators: it seems that the price quoted by other aggregators is higher than that of other aggregators, but once the transaction is completed, the actual money received is lower than expected.

But why is this happening?

Take 1inch as an example, because it has the highest winning rate in terms of quoted and actual prices. From quote to actual price to adjusted price, 1inch’s win rate decreased stepwise, from 22% to 19% to 14%. This means that 1inch appears to have a 20% chance of getting the best deal. However, this is often not the case once the costs are deducted once the transaction is completed.

Generally speaking, the reason for the difference between the quotation and the actual price is that the quotation is not accurate, or the market price during the period from quotation to settlement on the chain change, causing the quotation to become invalid. The difference between the actual price and the adjusted price is the additional cost of using the aggregator.

Let's take a look at the accuracy of all exchanges. What is the probability that the exchange will settle according to the quoted price?

Accuracy is important because It builds user trust and reflects whether the price quoted by the aggregator matches the price actually paid by the end user. Given enough time and samples, the difference tends to zero.

As shown in the figure above, 0x API and Uniswap always have a positive bias (that is, greater than zero), which means that the quotation of the transaction goes to the chain Price fluctuations between settlements are beneficial to users. This is the so-called positive slippage. Once this happens, the 0x API will always transmit positive bias to the user.

Interestingly, the deviation of 1inch is always less than zero, which means that no matter how the market price fluctuates, users will not get a more favorable price. A deviation of less than zero also indicates that the aggregator is quoting with outdated information, or is not responding fast enough. These problems may lead to a higher probability of transaction rollback. In the event of a transaction rollback, the user still needs to pay the transaction fee even though no token sale has occurred. A detailed explanation will be given later.

Now, let's go back to the cost, which is measured in gas usage on Ethereum. In the graph below, we can see that there is a big difference between the gas usage of these exchanges.

As far as the aggregator is concerned, it corresponds to the old saying "less is more". A bit more cost is fine if the aggregator delivers more value to the user, but as we showed earlier in our adjusted price win chart, most aggregators fail to do so. Optimizing the code to reduce gas usage increases the likelihood that the aggregator will split transactions to more sources for more value.

Uniswap has one of the lowest gas usage exchanges, so this is something all aggregators should aim for, especially for those smaller transactions. A transaction worth $100 is simply impossible to split across multiple exchanges, even with a gas price of only 1 gwei.

In general, 0x API costs more than Uniswap. However, as the adjusted price odds graph shows, although the 0x API is slightly higher in gas usage, it has a 70% probability of getting the user the best adjusted price. All other aggregators cost at least twice as much as the 0x API, i.e., double the cost to users, but create little additional value. 1inch and Paraswap are the exchanges with the highest gas usage, as can be seen from their low win rates on adjusted prices.

For an objective comparison, here is the average cost of all simulated trades worth $500-$1000:

0x API — $6.43
1inch — $13.58
Dex Ag — $13.41
Paraswap — $18.59
Uniswap — $3.93 (We’ll come back to this stat later )

Reducing gas usage has been a focus of our team for the past 8 months. Our initial goal is to reduce transaction costs by 50%. However, in 0x API 1.0, we managed to reduce the average transaction cost by around 76%. For a $50,000 transaction, the gas usage for 0x API 1.0 is about 460,000 gwei, and the gas usage for 0x API 0.0 is about 1.5 million gwei. These optimizations have turned the 0x API into the most efficient aggregator, regardless of transaction size.

In fact, the gas price has been well above 100 gwei in the last month. We want to show you the adjusted price win ratio of 0x API in this case. We ran a second simulation, manually raising the gas price, and found that the 0x API still outperformed other competitors at higher gas prices—providing The probability of optimal adjusted price is 80%.

Now, back to the gas usage of 0x API and Uniswap. Typically, 0x API has higher gas usage than Uniswap. However, 0x API offers higher adjusted prices than Uniswap in 70% of cases. How does the 0x API do this? Our 0x Labs team built a highly optimized contract that allows 0x API to obtain 100% liquidity from Uniswap in a way that is more gas-efficient than the default Uniswap contract. This is especially beneficial for small transactions (less than $5000) that are routed to a single source.

The following two transactions are examples:

The left side of the picture is the ETH<>UNI transaction routed directly to Uniswap. right is 100% routed to Unisw through 0x API AP's ETH<>UNI trade. As shown in the "Gas usage of transaction" field in the figure, the transaction cost of Uniswap is about 114,000 gwei, and the transaction cost of 0x API is about 110,000 gwei, which is 3.7% lower than the former. Certain on-chain states and conditions may change the outcome (for example, if users need to set new limits), but in general, routing transactions to Uniswap via the 0x API will only be cheaper than trading directly on Uniswap.

Now that we've discussed accuracy and gas usage, it's time to talk about transaction rollbacks. When a user trades on these exchanges, how likely is the trade to fail?

Aggregators should keep the rollback rate as low as possible, because paying for gas after a failed transaction is bad for users. Common causes of transaction rollbacks include market volatility between quotes and on-chain settlement, and inaccurate quotes (especially if quotes were provided using outdated information).

Our simulation tool always executes transactions under perfect conditions, i.e., sufficient funds, unlimited gas and no front-running transactions (i.e., no news leaks into the dark in the forest). We ruled out transaction rollbacks caused by these reasons as well as real-life user behavior. Therefore, for all exchanges, the simulated rollback rate is lower than the actual rollback rate. Nonetheless, simulated rollback rates can be a good indicator of how exchanges are performing.

In the graph above, the 0 to 10 second delay is the most interesting. Latency below 10 seconds means that the transaction is executed immediately, or the transaction is included in the next block as soon as it is submitted. Therefore, a high rollback rate means inaccurate quotes during this lag period.

Generally speaking, we found that Paraswap has the highest rollback rate compared to other exchanges, followed by Dex Ag and 1inch. In a simulated environment where human action does not cause a transaction to roll back, one of the possible reasons for a rollback to occur is stale quotes (as indicated by high-latency periods) or inaccurate quotes (as indicated by low-latency periods). If you recall, Dex Ag and Paraswap had the lowest accuracy rates.

Then look at the impact of the rollback rate on the end user. If a user makes 100 trades worth $500 to $1000 per exchange, the average loss from a failed trade is:

0x API — $6.89
1inch — $62.52
Dex Ag — $3.47
Paraswap — $119.22
Uniswap — $0.21

In the real world, we It is seen that Uniswap's daily rollback rate is as high as 30%, but in our simulation environment, the highest rollback rate under longer delays is about 1.3%. The discrepancy between actual and simulated rollback rates means there is a lot of room for improvement in the user experience of all DEXs, but that's another topic.

The result of the simulation experiment is that the simulated rollback rate of our 0x API is the lowest under different delay conditions. The simulated rollback rates for the 0x API range from 0.6% (shorter latency) to 1.8% (longer latency). In the real world, we have seen average weekly rollback rates for applications integrating the 0x API, such as Matcha, range between 3% and 4%.

Now, there's another important metric we haven't mentioned yet: response time. End users of applications such as Matcha may not have a strong sense of this metric. However, when you build a product on third-party infrastructure, it must be as reliable and fast as possible. Faster aggregators allow users to complete transactions faster, reduce slippage and rollback rates, and improve user experience.

Because our 0x API requires liquidity from more than 13 sources (including systems provided by professional market makers RFQ ) sampling, keeping response times low is a technical challenge. However, our goal is to make 0x API the most professional liquidity aggregation API. Currently, the response time of the 0x API is about 1.5 seconds, which we are very proud of.

Among all the exchanges involved in this research, 1inch has the longest response time, which is 2.7 times higher than that of 0x API . 1inch has a response time of 4 seconds, about 1/3 of the time it takes to mine a new block, which may cause time-sensitive transactions to be rolled back, for example, during periods of high volatility where market conditions are constantly changing.

Given all the information above, how do you choose the best aggregator? We can calculate the average gain or loss of transfers under various transaction scenarios, and thus come up with the expected value of the return generated from switching from one exchange to another. This is the user's expected benefit.

For those switching from other aggregators to an app like Matcha that integrates the 0x API, the average expected return on transactions under $10K is:

1inch — $7.17
Dex Ag — $10.36
Paraswap — $14.89
Uniswap — $17.14

Overall, if users who trade on 1inch switch to an application like Matcha that integrates 0x API, they can expect to earn an average of $7.17 per transaction. If they had moved from Uniswap to the 0x API, they could expect to earn $17.14 per transaction on average.

Summary

First of all, glad you're here!

Our goal is to share what we've learned from building the 0x API and illuminate key differences that aren't obvious between aggregators. As we mentioned above, we want to make the industry more transparent and spark a discussion on how to objectively measure and compare the performance of different aggregators so that DeFi developers and traders can make informed decisions .

By making our data public, we hope to advance the DEX aggregator industry and continue to build the best applications for DeFi developers and users.

If you are a trader, you might as well come to Matcha to experience it!

If you are a DeFi developer, you can read our 0x API 1.0 announcement and 0x API documentation. If you have already integrated 0x API 0.0 into your project, the migration process to 1.0 is very simple. Just in case, we wrote this guide.