OpenZeppelin on Security: How to Make Security Testing More Fun with Hacker Mindset?

During the hotly contested crypto event in Paris this week, in addition to the highly anticipated Ethereum EthCC, another eye-catching event was the 2023 DeFi Security Summit. At this summit, Felix, the Security Services Manager for OpenZeppelin EMEA, gave a speech that BlockBeats has translated and organized for you below:

Hello everyone, I'm Felix from OpenZeppelin. You may know us for providing a security library for smart contracts, but our services go far beyond that, including auditing, monitoring, and many other areas. Today, I want to discuss with you the topic of testing.

This is an important issue that we are currently facing: if we achieve 100% test coverage, have we truly achieved security? If we have, is it limited security or extensive security? I will attempt to answer this question in today's speech. However, before delving into this issue, let's take a step back and answer why we need to conduct tests.

Generally speaking, the purpose of testing is to verify whether your codebase has implemented specific functions or whether you expect it to have these functions.

This is the original intention of the test, but if you dig deeper, you will find that testing has other purposes. You may not only want to do unit testing, but also some integration testing or end-to-end testing, which actually have "descriptive" properties. Therefore, you can clearly see what the normal running path of your code library is, and how potential end users will interact with your code library. This "descriptive" property is also part of it. However, does it really help with security?

We hope it can enhance security. However, we cannot provide an answer to this question on the current slide. So let's continue the discussion and return to this question later. Now, let's talk about the quality of testing. Over time, you may want to complete testing. Therefore, we have various quality metrics.

The first metric is "functionality", which refers to the test coverage of the codebase. This is typically expressed as line coverage or branch coverage. This is a good reference value. We assume that this number has reached 100% in our test sample, so we are doing well in this regard.

"Descriptive" means that you have some interesting test cases, not just very small unit tests, but some very interesting end-to-end test cases. Now the question arises, do we need an additional metric, or have we accidentally achieved perfect security? Let's take a small example: this is a very small ERC20 token that I wrote:

As can be seen in the top half of the slide, 100% test coverage is guaranteed by Foundry (Solidity framework). From the bottom half of the slide, it can be seen that this token has three use cases: minting, burning, and transferring. They all work and are covered in this test case. Basically, we have done well in terms of "functionality" and "descriptiveness", but are we secure?

No, there is a publicly known vulnerability for destroying tokens of any user. This is a critical vulnerability with almost 0% security. You need to fix this and cannot deploy in this form. So what is missing exactly?

The above test case is missing. In Foundry syntax, if a function name starts with "test fail", it usually means that it is used to check specific failure cases. A typical test may check whether an arbitrary address can destroy another user's 1000 tokens. Obviously, you don't want this to happen, so you write this negative test. This is just a simple example, let's take a look at the overall principle.

When people say they are conducting tests, we usually see that most of them belong to the left category. They perform functional tests and focus on coverage. This is indeed a part of testing and is done very well, but it is not directly related to security. If you want to conduct security testing, you need to look to the right.

You need to ask yourself, have I thoroughly tested the functionality of the application? For example, have I really tested the scenario where an address cannot interact with the contract in a specific way for unauthorized functions? Therefore, application-aware testing is what we refer to as security testing here.

Now I want to take a step back and consider another question: how is testing conducted in theory and practice?

IBM's theoretical numbers suggest that we should invest heavily in testing, as this will bring actual returns. If you discover problems early in the design phase, implementation phase, or dedicated testing phase, the cost is relatively low. However, in the deployment phase, costs may rise sharply. This is not only applicable to blockchain systems, in which case the number "100" you see may further increase due to the TVL brought by the protocol.

This principle applies to any software system. Therefore, in theory, everyone should do their best to test, because economic principles clearly tell us that this will bring returns. However, in practice, testing is not a glamorous thing. Testing is not "cool". No one will say, "I am really passionate about QA testing, I am a tester". All "cool" things are basically related to hackers. Of course, being attacked by hackers is a bad thing for your protocol. But hackers are "cool", and testing is a necessary but unpopular task.

So how do we solve this problem? Here's the secret to making testing more appealing: treat hackers as part of the testing process.

To do really good security testing, we need to start with "while true" because you can iterate this loop forever. Your goal is to create a brand new POC vulnerability and try to attack the protocol you are developing. You may validate that it doesn't work or find something that needs to be fixed. Then you can integrate it as a test case into your security testing suite.

By applying hacker thinking to daily activities, the attractiveness of testing can be improved. This is not just about unit testing or boring testing that you know, that is a typical college course. This security testing is more like the application of modern hacker thinking, just applied to a artifact that can be integrated into your test suite.

This is a good theory, but now we need to add some practical perspective. Let's take a look at which projects this year have vulnerabilities at the code level. I'm not talking about low-level errors like private key leaks, but bugs in the code. If the testing method mentioned above had been used, it might have prevented the theft incidents. This year, the stolen funds have reached $250 million, which I think is a strong argument for integrating this security testing into the testing suite.

If you are a decision maker, you may not want to remember all the details, but just want a simple standard. According to the concept of vulnerability proof, count the number of interesting test cases in your test suite and use this as a measure of security testing. Overall, 100% test coverage is for functional testing and has nothing to do with security. If you want to do security testing, you should test the parts that do not exist. To do security testing in a good way, you need to adopt the mindset of a hacker and really include POC vulnerabilities as part of your test suite.

However, unfortunately, this is not a one-size-fits-all solution. You can use it as a starting point for your project, and then it is best to contact your security partner to further enhance this testing method.

Welcome to join the official BlockBeats community:

Telegram Subscription Group: https://t.me/theblockbeats

Telegram Discussion Group: https://t.me/BlockBeats_App

Official Twitter Account: https://twitter.com/BlockBeatsAsia