Partnered With:

Conducting differential fuzz testing on Solidity fixed-point library.

Read this article in 10 Minutes

In the world of blockchain, code is law, and ensuring the security of the code is crucial.

Original Title: "Differential Fuzzing On Solidity Fixed-Point Libraries"

Source: 0xNorman

Introduction

Security is one of the most important aspects of writing smart contracts. Developers may be familiar with traditional unit testing to test their code, but this only provides basic coverage to test the functionality and business logic of the code. However, they are assuming that the smart contract behaves as expected under given conditions. To further enhance security, we need to adopt more advanced testing methods, such as differential fuzzing.

Differential fuzz testing is a type of testing that involves executing different implementations of the same function or feature and comparing the results. This type of testing allows us to verify if different implementations are equivalent and behave consistently when provided with invalid or random inputs. This is different from regular fuzz testing, which typically tests a single implementation by providing a wide range of random inputs and monitoring for unexpected behavior, crashes, or security vulnerabilities.

In this project, we use Foundry to perform differential fuzz testing on different fixed-point libraries (OpenZeppelin, Solmate, Solady, and Prbmath). Please note that in several major versions of the Solmate library, we use the latest (v6) codebase because we believe that most new projects choosing the Solmate library will choose this version. We want to examine whether different code libraries have significant differences in the implementation of the same function. After running ten million tests, we found that these libraries have wide compatibility, but there are some differences in handling edge cases and gas consumption.

Specific Differences We will illustrate the differences in implementation of these libraries through the following examples. In the Log2 function (which returns the logarithm to base 2), if the input is 0, the implementations of OpenZeppelin and Solady will return different results. OpenZeppelin will simply return 0, while Solady will return 0x5be3aa5c (the signature of Log2Undefined()).

Therefore, we added if (num != 0) in the test case to avoid this difference. A similar difference also appears in the mulDiv() function. We need to ensure that the divisor is not zero and there is no overflow. Otherwise, OZ and Solady will report errors. For example, Solady will return 0xad251c27, which is the signature of MulDivFailed(), so we also added an if statement. If you want to reproduce the error situation in the test, please comment out the if statement and use specific test cases.

Please refer to the library implementation to understand why they handle it this way.

Gas Report

Due to the high compatibility between the functions in these libraries, we compared their gas consumption to detect which database is the most gas-efficient. The gas consumption snapshot can be found in the .gas-snapshot file. For better readability, we have provided a simplified gas report (average gas consumption per function call).

Or you can run this command to get it.

We found that the gas consumption of OpenZeppelin is much higher than that of other code libraries. In particular, functions such as OzLog2, OzLog2Up, OzMulDivDown, OzMulDivUp, and OzSqrt consume more gas than their corresponding functions in other libraries. Among the remaining libraries, solady has the most optimal gas consumption.

The gas consumption of all comparison functions in the Solady library is relatively low, and it provides the most comprehensive functionality, making it the preferred choice for advanced developers. However, it is worth noting that only OpenZeppelin and Solmate have been audited by external security teams.

Even Solmate, as a member of the encryption industry, was not designed to be simple and easy to use from the beginning. Therefore, before you lift a stone and hit your own foot, you should thoroughly read the documentation and understand the codebase.

The good news is that Cantina has decided to audit the Solady library as a public project, and you can monitor their audit status here.

足够的模糊测试

translates to

Sufficient Fuzz Testing

To ensure the correctness of fuzz testing, we conducted 9999999 rounds of fuzz testing. Usually, it is not necessary to run so many rounds during the development process, but as security researchers, we hope to cover as many random inputs as possible. You can change the number of runs in [foundry.toml]. More runs mean more random inputs are fed into the function. If you want to run quick tests, such as for continuous integration, adjust the configuration according to your needs.

Built on top of our foundation, Fixed-point arithmetic is just one aspect of the vast Solidity library. To expand the scope of this research, we can compare libraries that handle different functionalities, using different pragma versions. This will provide us with a secure and broader understanding across different use cases and Solidity versions. Additionally, we can also record the random inputs used by Foundry and analyze them. This will enable us to determine if these inputs are truly random and ensure a fair and comprehensive evaluation of the library.

Conclusion

Our main goal in conducting differential fuzz testing is to reveal significant differences in output results among the Solidity fixed-point libraries of OpenZeppelin, Solmate, Solady, and PRB-Math. Although these libraries perform well in terms of compatibility, our testing has found subtle differences in their handling of edge cases, particularly in the Log2 and mulDiv functions.

Although these differences may seem small, they can lead to unexpected behavior in smart contracts, making them significant.

Through this work, we hope to provide developers with detailed insights to guide their decision-making when choosing libraries for smart contract applications. Our research did not find significant deviations between these libraries.

However, the subtle differences we observe remind us of the importance of thoroughly testing and understanding the code we use.

In the world of blockchain, code is law, and ensuring the security of the code is crucial. Performing differential fuzz testing on Solidity libraries is one step towards achieving this goal, providing a higher level of assurance than unit testing. We look forward to seeing a more secure cryptographic ecosystem.