Partnered With:

Foresight Ventures: read Arweave, key infrastructure for Web3.0

Read this article in 43 Minutes

Web3.0 is likely to arrive within 2 years, and blockchain infrastructure like Arweave will certainly show its merits by then.

Reading Arweave: critical infrastructure for Web3.0

Original author: MsFew & Amp; Yuanqi Li, Foresight Research

1. An overview

Arweave is a decentralized network focused on permanently storing data, akin to a global hard drive that never loses data compared to the bitcoin global ledger and the Ethereum global computer. Arweave will most likely be Web3.0's "Layer 0" of the stored public chain as a significant infrastructure presence.

2. Why is it important to store data permanently?

Before the Web era

Before the Advent of the Web, data was usually recorded in books such as history books, and libraries were the storage centers of today. In ancient times, the Library of Alexandria in Egypt was a wellspring of Western knowledge, containing hundreds of thousands of rare handwritten volumes. But after several wars, the 600-year history of the Library of Alexandria was burned, and the precious data was destroyed. Because of the burning of these data, human history is likely to be set back decades.

Web 1.0 - The Web 2.0 era

In the Web2 era, there is a similar site to Arweave: the Internet Archive (https://archive.org). It keeps snapshots of almost every site from time to time (for example, we can see the Apple website in 2001), as well as music, books and other data. It is the permanent storage of the Web2 era, holding all the valuable data such as websites and media, and can see it as it was when the Internet speed changed. But it is run by centralized databases and servers. At the time of this writing, Internet Archive's web site snapshot service is not accessible, so its service is somewhat unstable. At the same time, the organizations behind Internet Archive have to pay a large amount of money to maintain servers and databases, which is another destabilizing factor. At the same time, although the Internet Archive is an open source non-profit organization behind it, the data stored on its servers is still their own and there is a problem of centralization. Because of these factors, the Internet Archive may lose data under extreme conditions, although it must perform disaster recovery and backup operations.

Web 3.0

In the Web3 era, Arweave came along with a perfect solution. Through a unique consensus mechanism to ensure the permanent storage of data, through Token economy and reasonable mining mechanism to allow miners to store data and expand and activate the developer ecosystem, solve the server and development funding problems. Through the decentralized architecture, users' data sovereignty is guaranteed. Arweave also continues to work actively with the Internet Archive to slowly and permanently store data on the web. Arweave is the Alexander library of data stored forever for the Web3 era.

3. Data structure and consensus mechanism

Arweave implements permanent file storage through Blockweave data structure and SPoRA consensus mechanism.

A. Blockweave

Arweave differs from the block structure of traditional blockchains, where blocks are stacked one after another, thus forming a chain structure. On this basis, Arweave also points to a random previous recall block (formerly known as Recall Block, now known as Recall Chunk), which is therefore a spindly structure from a three-dimensional perspective. From the point of view of the storage content, the conventional chain storage is mainly index, data distributed among nodes, while Blockweave blocks store data information, you can see the image type and other content from Arweave's blockchain browser. Blockweave is designed with its own sharding property, and its performance is highly extensible. Compared with other storage projects, Arweave can improve TPS many times and greatly reduce long-term storage costs.

The most important part of Blockweave's design is the consensus mechanism that serves Arweave.

B. SPoRA consensus mechanism

Before explaining the SPoRA consensus mechanism Arweave uses today, let's look at the PoA + PoW mechanism it used before. We are already familiar with the PoW mechanism and have seen the classic mechanism in many blockchains, so let's take a look at the additional PoA mechanism.

PoA stands for Proof of Access, Proof of Access. If a miner wants to generate a new block, he must Access a random recall block. If we recall the data structure of Blockweave, in an ideal world, if a miner wants to satisfy the CONDITIONS of this PoA mechanism, he will choose to store all blocks of data, because each next block points to the current block and a random block before it. PoA encourages miners to store more blocks of Arweave, and it also encourages miners to store blocks that are not stored to increase the probability of meeting the PoA. Of course, PoA alone is not feasible, because if everyone stores everything, then everyone's probability of producing blocks is 1. So in addition to PoA, there are PoW mechanisms to ensure decentralization, under the consensus of PoA and PoW, &NBSP; Probability of miner's block production = probability of having random recall block * probability of finding hash first. Thus, the unique design of the PoA encourages miners to store more data and store rare data, enabling permanent storage of data.

In addition, in order to adjust the generation speed of blocks in the network, PoW algorithm allows to adjust the difficulty, the greater the difficulty of PoW, the longer the calculation cycle, if the generation rate of blocks in the network exceeds the target frequency, it increases the difficulty of future BLOCK generation PoW problem. Also, as the speed of block generation in the network slows down, the difficulty Settings are adjusted downward; In this way, the network can adjust the block generation rate without being limited by the number of nodes and computing power in the network.

Arweave upgraded the PoA to THE SPoRA (Succinct Proofs of Random Access) in the first half of this year, building on the CONSENSUS mechanism of the PoA. PoA + PoW alone does the job for permanent storage of data, but there is no incentive for speed of data access, so miners are likely to put their databases elsewhere, where costs are lower, rather than locally. In this way, the network access speed will slow down due to distance and other factors. The mechanism of SPoRA reduces the previous weight of the probability of miners producing blocks and adds the consideration of data access speed. The weight of PoW in the probability of generating blocks decreases, which greatly reduces the computation power and energy wasted in searching hash, reduces energy consumption and is more friendly to the environment. Upgrading PoA to SPoRA allows miners to focus more on maintaining their own local hardware and nodes, avoiding the need for miners to store their data in the same low-rate data center, diversioning node locations, and increasing decentralization.

Through these unique and ingenious consensus mechanisms, Arweave ensures data permanence, access speed and block time.

C. Decentralized storage

Another feature of Arweave is that the miners' experience is very good. Unlike traditional blockchains, Arweave does not require miners to download data from all the nodes of the entire blockchain. Instead, Arweave only needs to download a few blocks to start mining (although the probability of producing blocks is low at this point), which is actually very important. If there is no such design for a storage chain, it is particularly difficult to add new nodes.

Through the design of block spinning and consensus mechanism,& NBSP; Miners will spontaneously download rare data, which will ensure that the average number of data stored remains uniform. The first line below shows the various nodes of a traditional blockchain. Each node needs to download all the data, which wastes a lot of resources. The second line represents the situation before miners download rare data spontaneously. Rare data may be omitted by miners, leading to possible data loss and excessive storage of some data. The third line shows the current situation of Arweave, which shows the balance that miners have reached after spontaneously downloading rare data. Each data has the same number of copies, avoiding resource waste while maximizing data storage security.

In addition, Arweave's miners, with SPoRA's consensus, can focus more on upgrading hard drives. In future releases, Arweave will continue to reduce CPU configuration requirements and focus on improving the data storage and read/write experience of Arweave.

4. Network and application architecture

Arweave's DApp doesn't run directly on the Arweave blockchain, which is just a blockchain similar to a hard drive or database. On top of Arweave's blockchain network layer, there's also permaWeb, the permaWeb layer where Arweave's decentralized application runs, DApp smart contracts in Ethereum all run in Ethereum's EVM, which runs directly in the Ethereum network. Arweave's blockchain structure and store location are very similar to the paper tape of a Turing machine. Developers can use code to self-manage application states and use Arweave to record these states on the chain.

An architecture like Arweave gives developers a lot of freedom, Just as in the Web2.0 era, it doesn't matter if the back end of an application uses PHP or Java. Any developer who is familiar with Arweave's HTTP API (which costs developers almost nothing to learn) can use Arweave as a blockchain database. The storage-based consensus paradigm (SCP) is a smart contract design idea that brings Arweave unlimited ecological potential.

The problem with this flexibility is that since the DApp does not run on Arweave, Arweave cannot directly dictate DApp contracts and Token standards. We will discuss Arweave smart contracts in more detail in a later section.

DApp = front-end page = smart contract

· Definition of smart contract

When Nick Szabos coined the term "smart contract" in 1996, he defined that a smart contract should have the following four characteristics: a set of promises; Specified in digital form, not paper; Inclusion agreement; The personnel involved in the contract fulfill their commitments. In addition, I think smart contracts also need to be trusted with data storage and computing.

· Traditional DApp and Arweave DApp

A traditional DApp usually consists of a front end of the app (web page) and a smart contract that sends the user's data to the smart contract. Such a DApp meets the four conditions and two features of a smart contract, according to the developers of Arweave: The front-end page is the smart contract. The idea is that instead of having both the front-end page and the smart contract on the chain, a DApp can simply combine the front-end page and the smart contract into one.

· How do front-end pages and smart contracts become one?

As mentioned earlier, Arweave can store files of different content-types directly. We can store front page directly to the Arweave, open the transaction data can be directly see a complete page (such as the deal the data stored in the open is a rendering good front page) directly through this example, we can understand why the front page can be a smart deal. As we open the Arweave code data, the browser also renders the page for us. We can not only see the source of the page, but also see the page. You can't do that on Ethereum, where a smart contract is just a string of text in a browser that doesn't render a page, As developers develop, the logic of the DApp and the logic of the contract can be written in the front end, and the transactions and data that need to be stored can be stored in Arweave via the HTTP API.

· How to ensure reliable data storage and computing?

The consensus preferred by Arweave eco-developers and Sam Williams is data-based "consensus" (first proposed by Arweave eco-developer, everFinance Founder Outprog). Contract page source code stored in Arweave with certainty, we can see, at the same time, the data of each interaction is stored in the chain, it ensures the reliable data storage, at the same time, we apply colours to a drawing gives the contract page modern browser provides us with an almost completely unified execution environment, this ensures the reliable calculation, By design like this, Arweave's smart contracts also guarantee data permanency/immutability/traceability/low data cost.

· How to upgrade the contract code?

When you want to update your contract or front end style, you just need to re-publish the new version of the contract to Arweave. This will generate a new Arweave transaction and update the front end and the contract content.

Different implementations of smart contracts

· Smartweave (Arweave is a Turing machine tape, Smartweave is dedicated to managing the state of the Turing machine)

The issue:

Remember the Turing machine tape concept we mentioned in the previous article? To make a Turing machine, Arweave tape recording is not enough. We need a way to effectively manage and change the state of the application. Smartweave is an important part of this missing piece. Smartweave is Arweave's official smart contract standard, which uses the concept of front-end code = smart contract. Developers can use JavaScript, the front-end scripting language, to manage state. The front-end page itself is the logic and state of the UI, and it is much easier for front-end developers to write smart contracts to manage state using Smartweave than Solidity.

Design ideas and features:

Smartweave is designed to execute lazy-execution when it has to, when the user has to get the latest contract status on the front end. Because of the JavaScript language, the user can run all transactions directly from the browser, without the need to run on the chain. Smartweave stores the initial status and the transactions associated with the contract (an ordered list of transactions 1, 2, 3). To get the latest status of the contract, the user will run all contract-related transactions in the browser, acting as the user himself as a node, and finally get the latest status and submit a new transaction.

Case analysis:

CommunityXYZ is the dashboard and management platform for the Arweave profit sharing community. CommunityXYZ's Smart Contract is a standard implementation of Smartweave that uses TypeScript (an extended version of JavaScript). Through jSON-like data structure, it contains the user's balance, vault, role, voting status and other information. The contract also provides the required Transfer, increaseVault, check balance and so on. The SmartWeave contract implemented by CommunityXYZ can be put into the front-end page code base, so that the contract logic can be saved in the front end.

Disadvantages and optimization:

If you read the design of Smartweave carefully, you will see that the upper limit of TPS becomes the upper limit of browser performance and network bandwidth. However, leaving all transactions to the user presents a problem. If a contract has too many transactions, will it take too long to get the latest status and cause congestion?

While in reality, normal but high numbers of transactions do not cause significant user experience headaches (the size of JavaScript files loaded and running on modern web pages is inherently large, O < span style = "box-sizing: border-box; line-height: 22px; display: block; word-break: inherit! Important;" To solve this problem, we can filter the transaction data through a separate Smartweave contract before submitting the transaction, receive a large amount of the transaction data, and then rollup to the AR in bulk.

Redstone's Smartweave contract is designed to reduce reloading and running of unnecessary transaction data through multiple layers of browser caching, which can greatly optimize client performance. Or a trusted user can generate a snapshot of the transaction status to reduce the transaction operation.

· Tok-Demo of everFinance (another store-based consensus computing paradigm)

EverFinance tok-Demo is designed in the same way as Smartweave. It is also based on the idea that Arweave is a Turing machine tape. Tok-demo contains two application modules: Issuer and Detector provide the Token WEB page and provide the Token interface for checking balances, transaction records, and submitting transactions. The Token validator program automatically loads data from Arweave (lazy-execution as described in Smartweave) and provides an interface to query Token transactions.

In theory, Arweave's smart contract can be implemented in countless different forms, so each project may have its own implementation, but most projects use Smartweave. Arweave actually has a Token standard like ERC-20, which specifies various apis, Smartweave's smart contracts on Arweave simplify the development process so much that if Arweave were used for storage, the project team might not need a contract engineer at all.

5. PSC (Profit sharing community) interpretation

PSC (Profit sharing community) introduction

The PSC is a profit-sharing community on Arweave and is a new, more equitable structure for startups. These communities give founders more active control and flexibility over their projects, while empowering the contributors to those projects. In traditional start-ups, employee-owned equity is unlikely to become liquid anytime soon, will not be granted governance, and will receive little in the way of dividends. Meanwhile, THE PSC combines THE DAO and Token, and the VOTING power of the PSC depends on how many community PST a person holds and how long they hold those tokens.

By contrast, All PSC contributors earn PST tokens that give them fair governance rights, instant liquidity and interest generated by using the app, and more PSC projects can be seen on communityXYZ, the Hub of the profit-sharing community.

Introduction to PST (Profit sharing Token)

PST is an additional mechanism that Arweave allows developers to attach PST attributes to the tokens that belong to a protocol when they create a protocol. When a user interacts with the application on Arweave and generates a service fee, the holder of the PST attribute Token will get a refund in AR from the service fee at the proportion agreed upon when the agreement is created. The refund will be distributed randomly according to the proportion of the PST holder's relative PST holdings. The amount of PST held is large. In line with the high probability of receiving distribution (similar to PoS), Arweave is incentivized to build a good user base through fee refund, which can be used as an important source of income for project developers, and the coin holder can also receive dividends in addition to the increase of coin price according to the development of the project.

The development of the PST

You may wonder why such tokens are not designed and standardized by Arweave. Is it safe to let projects set up? In fact, the PST on Arweave also has the same standard as ERC20, there will be some state general interface. However, there are no restrictions for us to make applications. Developers can use the advantages of Smartweave and PST flexibly, so Arweave does not impose standards. In terms of delegating control to the project Settings, we mentioned earlier that the developers of Arweave generally believed that the front end of the project could be a smart contract, so if the profit distribution was written in the contract, it would be safe. The user can verify that the Token is assigned correctly and that the Code is Law.

Can PST land on the central trading platform?

From a technical point of view, PST-type tokens can meet the requirements for logging into a centralized trading platform, but the main constraints for their presence are the following two points:

First, Arweave ecology as a whole is in an early stage, with poor mobility of PST and activity of PSC. In Verto, PST's decentralized trading platform, only one transaction has taken place for all PST's this month, and Evermore, for example, has not moved in price since its launch in April, reflecting poor liquidity, which may be resolved if it is launched on the trading platform.

Second, the technical limitations of PST. The maximum value of PST is that PST holders can receive tips from the application. However, PST income distribution is calculated based on the address holding OF PST. That is to say, if PST is transferred to the trading platform, users cannot receive PST income without targeted adaptation. To solve this problem, the trading platform needs to calculate the share of users and then distribute corresponding profits to users (similar to mining pools). In addition, because different projects may have different smart contract implementations (such as Smartweave and token-demo), token implementations may also be different, so the trading platform may need special adaptation (in fact, similar to the various application API in the Web2.0 era).

6. Current storage status

Node distribution

The current number of nodes is 1019, including 868 nodes in mainland China, 103 nodes in The United States, and the rest scattered in various countries. The current number of nodes is still small, and the upper limit of single node storage is low, but compared with 150 nodes in April this year, it has achieved a great leap.

The total storage

The explosion of NFT boosted Arweave's storage, followed by more blockchains backing up data, and the total amount of storage jumped to 27.5TiB, compared to Filecoin's 13.5EiB, more than half a million times. Filecoin, however, stores a low percentage of valid information due to its incentive system, while Arweave can upload 43.88GB of data on a single block.

Store price

Arweave's consensus cost is low, with permanent storage of 1GB of data (which can process millions of transactions) costing about $20. By comparison, Filecoin costs about $0.06 per gigabyte of storage, but Filecoin offers an annual billing model that doesn't guarantee permanent storage, and Arweave is a buy-out model that guarantees at least 200 years of storage for a single fee. So Arweave pays a reasonable premium for data storage.

To ensure that Arweave's storage fees for miners are competitive in the long term, Arweave is depositing most of the transaction fees currently paid by users into the endowment, which will be released to miners over time as its eco-Dapp grows.

7. Network project ecology

Ecological overview

ArDrive

Baidu web disk is baidu cloud storage front end, the corresponding ArDrive is Arweave storage front end. ArDrive can be regarded as the baidu web disk stored on Arweave. It is a set of applications for PC and mobile terminals, which can upload text, pictures, videos and other forms of data to Arweave for permanent storage. Currently, ArDrive is a one-time, permanent device that costs about $20 for 1 gigabyte of storage, slightly more than traditional storage, but the value of permanent storage and data security far exceeds that of traditional storage.

It's not just for the coin-sphere, but for all the good that's worth recording forever.

KYVE

The advantages of Arweave storage make it favored by different blockchains, and KYVE Network can be regarded as the middle layer of Arweave's cross-chain storage. Through KYVE, data of other blockchains can be permanently stored on Arweave as backup. Different blockchains can also interact with each other through the KYVE network. In addition, KYVE ensures the accuracy of uploaded data through a storage pool. The uploader is responsible for storing the latest data, and the verifier ensures the accuracy of uploaded data. Both parties pledge KYVE tokens to the storage pool to ensure accuracy and fairness.

The public chain integration will cover Avalanche, Cosmos, Near, Polkadot and Solana.

EverPay

Everpay is a DApp for cross-chain transfer. Currently, real-time Token transfer has been implemented on Ethereum and Arweave. Users can recharge their assets on Ethereum or Arweave into Everpay and take out any assets from the other end through mapping. With no waiting and no Gas fees, Everpay is more like a Layer2 application, with off-chain calculations to ensure speed, rollup data to Arweave for security, and a gateway to the Arweave ecosystem and other blockchains.

Everpay will host and integrate more blockchains and support smart contract accounts in the future, providing a seamless transfer experience for investors who want to participate in Arweave's growth dividend.

Verto

The article mentioned above that PST-type tokens have not landed on the centralized trading platform yet, while Verto is a decentralized trading platform of PST-type tokens, facing the profit sharing mechanism of PST. Verto's more important role is as a vehicle for yield swaps, in which Verto holds high-volume agreements for higher fee returns. In addition, its VRT can get 0.5% of all PST transaction fees. After Verto completed the Ethereum bridge in February this year, it allows users to exchange ETH through Verto while allowing VRT holders to get transaction fees denominated in Ethereum.

Although the current number of Verto transactions is very small, but with the enhancement of Arweave economic effect, PST type promotion and VRT value capture method, Verto is bound to usher in its highlight moment.

ArConnect

If you want to use Arweave, you will have to get around ArConnect. As the first plug-in wallet of the Arweave ecosystem, ArConnect ends the awkward situation where you need to paste the key into the application page to use the DApp. It is currently deeply bound in Arweave ecosystem, as the official and many protocols of the main launch wallet, it is worth mentioning that there is no charge for AR and PST transfer in the program, the fees generated when interacting with DApp will be partially distributed to randomly selected (POS) VRT holders.

Pianity

Since Arweave ecology as a whole is in the early category, its flagship project should belong to Pianity except for infrastructure agreement. For music NFT platform, Pianity platform proposes a model that can bring collectors a certain annual income. Pianity platform obtains certain commission income through primary sales and secondary transactions of music NFT. The proceeds are then distributed to each NFT holder in proportion to the user's value at the time of purchase and the total value of the NFT. Since the proportion calculation method is the price when buying NFT, users may deliberately raise the PRICE of NFT in order to obtain a higher proportion of allocation, which in turn stimulates the market distributable income to increase gradually and attracts traffic attention. In addition, its Token PIA also belongs to the PST type, so users will also get the realization of Pianity's traffic in addition to the NFT income they can get.

Pianity has a chance to stand out in the relatively blue-ocean market of music NFT with its permanent storage and dual revenue model.

Does Arweave require smart contracts and Token standards?

Before I got to know Arweave, I always believed that smart contracts and Token standards were necessary for a public chain ecosystem to develop. But Arweave is a much lower-level chain, and dApps don't run inside the chain, so they certainly don't have the ability to restrict developers in certain secure sandbox environments like EVM, But in fact, Arweave developers' "front end = smart contract", "store-based consensus computing", and "Arweave as Turing machine tape" ideas, in fact, slowly let developers reach a consensus, so more and more project ecosystem are using Smartweave, Implement the design idea of "front-end = smart contract".

8. Bottom line: Arweave's true value will explode in Web3.0

We can start by enumerating the benefits of Arweave. Arweave's original consensus and miner incentives guarantee the permanence of storage. The permanent preservation of data has always been a major issue in human history. It is extremely important for human beings. Arweave can serve as the Alexander library for the Web3.0 era, storing data and continuing to accumulate human wisdom. Arweave's decentralized feature can overturn the monopoly of centralized cloud service vendors in the storage market, and users' data can be stored in the cloud without censorship, bringing freedom of data distribution. Arweave's storage positioning is low enough that it can be used as "Layer0" and become an indispensable part of blockchain development. It is also versatile and extremely easy to get started, making it an ideal hotbed for all kinds of decentralized applications. Next, let's look at the disadvantages of Arweave. Arweave is too low-level, unable to restrict the standards of Token and contract due to architectural reasons, and the development ecology is too flexible, which may not be unified, leading to the failure of the ecology.

We may not understand Arweave at the moment, but we didn't see through Bitcoin, either. Charlie Munger has said, "The human brain is reluctant to make changes to save space, a form of avoiding inconsistency." We can't look at Arweave through the lens of a normal blockchain, because that would miss out on a lot of the important value of Arweave. Web3.0 is likely to come in 2 years, by which time a blockchain infrastructure like Arweave will show its merits and be understood by everyone.

The relevant data

· Arweave technical details:

https://www.chainnews.com/articles/543779897127.htm

https://www.youtube.com/watch?v=YO3zyQvL3Yg

https://esl-conf-static.s3.eu-central-1.amazonaws.com/media/files/000/000/841/original/Sam_Williams_-_Blockchains__block weaves__and_the_decentralised_revolution.pdf?1513004189

https://arweave.medium.com/the-arweave-network-is-now-running-succinct-random-proofs-of-access-spora-e2732cbcbb46

· Arweave storage:

https://news.ycombinator.com/item?id=24791621

· Arweave ecosystem:

https://arweave.medium.com/nfts-and-attention-rewards-verto-and-koi-spotlight-c7fa1ca94827

https://arweave.news/profit-sharing-communities-a-deep-dive-by-arweave/

https://www.verto.exchange

https://arweave.medium.com/profit-sharing-tokens-a-new-incentivization-mechanism-for-an-open-web-1f2532411d6e

https://coinmarketcap.com/alexandria/article/profit-sharing-communities-a-deep-dive-by-arweave

Arweave true value:

https://www.chainnews.com/articles/124011762292.htm

https://www.chainnews.com/articles/930028948285.htm