Partnered With:

What is "data availability"?

Read this article in 15 Minutes

Explain in detail the data availability problems faced by the expansion plan.

Original title: "Popular Science | What is "Data Availability"?" "

Original source: Ethereum enthusiasts

"data availability" and "data "Availability problem" refers to a problem faced by some blockchain expansion solutions. Specifically, when a new block is created, how does a node ensure that all data in that block has been published to the network? The difficulty is that if the block producer does not publish all the data in the block, no one can find out if a malicious transaction is hidden in the block.

In this article, I will delve into the importance of data availability issues, and how to address them.

How do blockchain nodes work?

Each block on the blockchain consists of two parts:

The block header, i.e., the block's metadata, consists of some basic information about the block's content, including the Merkle root of the transaction. Transaction data, i.e., the main component of a block, consists of actual transaction data.

There are two main types of nodes in the blockchain network:

Full nodes (also known as fully validating nodes). Such nodes download every transaction in the blockchain and verify its validity. This consumes a lot of resources and hundreds of gigabytes of disk space, but these nodes are the most secure as they will not accept blocks containing invalid transactions. light client. If your computer doesn't have enough resources to run a full node, you can run a light client. Light clients do not need to download or verify any transactions. They only download block headers and assume that the transactions contained in the block are valid. Therefore, light clients are less secure than full nodes.

Fortunately, there is a way for light clients to indirectly check that all transactions in a block are valid. Light clients can rely on full nodes to send them fraud proofs about invalid transactions, without having to check the validity of transactions themselves. A Fraud Proof is a small proof that a certain transaction in a block is invalid. Regarding the working principle of fraud proof, this article does not give a detailed introduction, please read this article.

There is only one problem here: if the full node wants to generate a fraud proof for a block, it needs to know the block transaction data. If the block producer only publishes the block header and does not publish the transaction data, the full node will not be able to verify the validity of the transaction and generate fraud proofs for invalid transactions. This requires block producers to publish all data for a block, but we need to find a way to enforce this.

In order to solve this problem, the light client needs to find some way to check whether the transaction data of the block is real Published to the network for full node verification. However, we also want to avoid letting light clients download entire blocks, because this would make light clients useless.

How can we solve this problem? First, let's discuss the relevance of the data availability problem and how to address it.

What programs are data availability issues related to?

In the previous section we covered the data availability issue. Let's discuss its importance for scalability solutions.

Increase block size

in Bitcoin In blockchains such as , most ordinary laptops can run full nodes and verify the entire chain, because there is an artificial upper limit on the block size to prevent the blockchain from becoming too large.

But what if we want to increase the block size cap? Only more people can afford to run full nodes and independently verify the blockchain, and most people will run less secure light clients. This is bad for decentralization as it makes it easier for block producers to change protocol rules and insert invalid transactions to trick light clients. Therefore, it is important to provide fraud proof support for light clients, but as we have already discussed, light clients need a way to verify that all data in a block has been published to the network.

Sharding

Improve blockchain throughput One way to do this is to split the blockchain into multiple chains, ie, sharding. These shards all have their own block producers and can communicate with each other to transfer tokens between shards. The significance of sharding is to group block producers in the network, so that each block producer does not need to process each transaction, but only needs to distribute them to different shards. Each shard only needs to process some transactions.

Generally speaking, on a sharded blockchain, validators only need to be one or a few Shards run full nodes and light clients for other shards. After all, if every validator has to run a full node for every shard, it defeats the purpose of sharding — splitting the overhead of the network among different nodes.

However, this approach has its own flaws. What if the block producers on the shard go awry and start accepting invalid transactions? This is more likely to happen with sharded systems than non-sharded ones, which are easier to attack with only a small number of block producers per shard. Remember that block producers are continuously assigned to different shards.

In order to detect whether there is a shard accepting invalid transactions, we must ensure that all data in the shard is publicly available so that fraud proofs can be used to prove all invalid transactions.

Rollup

Optimistic rollup is a Novel scalability strategy for rollup sidechains (similar to sharding). These sidechains have their own exclusive block producers and can transfer assets to and from other sidechains.

However, if a malicious block producer packs an invalid transaction into a block, all What should I do if the user's funds are stolen? To solve this problem, we can use fraud proofs to detect such cases. But, again the same old problem, sidechain users need to find some way to ensure that the data of all blocks on the sidechain are publicly visible in order to detect invalid transactions. To solve this problem, Rollup on Ethereum publishes all rollup blocks to the Ethereum blockchain, relying on Ethereum for data availability. That is, using Ethereum as a data availability layer.

ZK-rollup is similar to optimistic rollup. The difference is that the former does not use fraud proofs to find invalid blocks, but uses validity proofs to prove the validity of blocks. Proof of validity itself does not require data availability. In general, however, ZK-rollup requires data availability, because if a block producer creates a valid block and generates proofs of validity for it, but does not publish the block data, users have no way of knowing the block The state of the chains and their balances, it is impossible to interact with the blockchain.

Explore further

rollup is designed to The blockchain is used as the data availability layer to store transactions, but the actual transaction processing and calculations all happen on rollup. This is a very interesting way of thinking: the blockchain does not actually need to perform any calculations, but at least it needs to divide transactions into blocks and ensure the data availability of transactions.

This is also the design idea of LazyLedger, that is, a "lazy" blockchain that only needs to complete the block The two core tasks of the block chain are to sort transactions in a scalable way and to achieve data availability of transactions. This makes LazyLedger the smallest "pluggable" component in systems like rollup.

Solutions to Data Availability Problems

Download all data

As discussed above, the most straightforward way to address data availability is to require everyone, including light client) to download all data. Obviously, this approach does not have good scalability. Most blockchains like Bitcoin and Ethereum take this approach.

Data availability certificate

Data availability Proof of availability is a new technique: a client only needs to download a small portion of the data in a block to check that all the data in that block has been published.

Proofs of data availability employ a mathematical element known as an erasure code. Erasure codes are widely used in information technologies ranging from CD-ROMs to satellite communications to QR codes. Erasure codes can expand the original 1 MB block data to 2MB, and the extra 1 MB is special data called erasure codes. If any bytes in the block are lost, erasure codes can help you retrieve them. Even if the data of the entire block is lost, the erasure code can help you retrieve all the data. Likewise, with erasure coding, even if the data in the CD-ROM is erased, your computer can read it becomes 2 MB, you still need to get at least 1MB of data to recover the original data, although the 1 MB of data is not required to be continuous).

This means that to achieve 100% data availability, block producers only need to 50% of the data in a block is published to the network. If a malicious block producer wants to successfully conceal 1% of the data, it must conceal more than 50% of the data, otherwise the 1% of the data can be retrieved with the remaining 50% of the data (Translator's Note: The first sentence of this paragraph Doubtful).

With this knowledge, the client can take steps to ensure that the data in the block is not hidden. Clients can try to download random blocks of blocks, and if they fail to download a block (i.e., it shows that the block belongs to the undisclosed 50% of the malicious block producer), they will Deny the availability of data for this block. If a random block of data is downloaded, the client has a 50% chance of finding an invalid block. If two chunks are downloaded, there is a 75% probability. If three data chunks are downloaded, there is an 87.5% probability. And so on until after seven data blocks have been downloaded, there is a 99% probability. In this way, the client only needs to download a small part of the data in the block, and can effectively check the data availability of the entire block.

The full details of the proof of data availability are a bit more complex and rely on other assumptions, e.g. The number of light clients in can't fall below a certain lower limit, so that enough light clients request the data block in order to restore the data of the entire block. If you want to learn more, you can check out the paper on data availability proofs.