Partnered With:

Road to mass adoption: Storage consensus-based under - chain computing paradigm using Substrate

Read this article in 55 Minutes

At present, the basic design of these blockchains (public chains) cannot carry the number of Internet users, which has structural and institutional problems.

Article title: Road to Mass Application: Using Substrate to Implement Storage Consensus-based Chain-based Computing Paradigm.

Mike Tang, Web3 Explorer

preface

A successful Web2.0 To C app in the mass market should be at least on the order of a million live months. We are familiar with the top Web2.0 applications Twitter, Google, Facebook, WeChat, Tiktok and so on, the monthly live of more than 1 billion magnitude. That said, C-oriented applications have to handle high concurrency and large data storage and retrieval, among other things.

And let's take a look at the blockchain-based star applications so far (written: 2021.10.15) :

●Audius, which currently has 4 million monthly users, claims to be built on Top of Ethereum and Solana, but is not transparent about how it uses the chain and how much it relies on blockchain

●Axie Infinity, which has 1.7 million users, runs on its own Ethereum chain Ronin, which is also less publicly available

● The head of 24 DeFi applications on Ethereum has a combined user address number of 3 million [1]

In general, the number of users on blockchain is in a low state.

There are two fundamental reasons for this, in addition to the lack of attention from users themselves:

1. The UX experience of most existing DApps is difficult for ordinary Internet users to understand and operate

2. The low computing performance and high data storage cost of existing blockchain make it too expensive to use

In other words, the basic design of these blockchains (public chains) cannot actually carry the number of Internet users. There are structural and institutional problems in this.

This article attempts to discuss how blockchain can be adapted for large-scale applications.

Basic requirements for millions of monthly user applications

We will only briefly analyze the Web2.0 monthly million user entry level applications.

1. Good UXwith

The UX experience of Web apps and mobile apps should be smooth, the process should be smooth, and the pre-requirements should not be too many (such as the need for a wallet account, the installation of a browser plug-in), so that users can use it immediately. And the account does not need to be re-registered, can easily use the existing third party login is the best.

2. Can support high concurrency, low request delay

Can support a large number of users to access at the same time, there is no jam, the delay is not too high (more than a few seconds). This is reflected in the UX user experience of the product. In the face of a large number of requests, there can be no service crash, interruption.

3. Calculate low cost

The computing cost is related to the computing performance. The computing performance of the server system should not be too low, or it cannot handle high concurrent requests. The cost of computing should not be too high, or the cost of cloud servers will skyrocket.

4. Low cost of storage

Users will generate a large amount of data in the process of using App. These data include not only UGC data directly created by users, but also a lot of service state data and log data at various levels. This data will eventually be stored in SQL or NoSQL databases, object storage services, file storage services, and so on. Some indexing services such as ElasticSearch also need to store indexed data separately. These stores take up a lot of storage space. Considering the reliability of data, redundancy processing is also required. Do redundant processing, equivalent to the space occupied by data to several times larger. All these require the storage cost to be kept at a relatively low level, otherwise the overall cost of the service will be too high, and the operation and maintenance of App will be difficult.

The requirements of Web3.0

In fact, the goal of Web3.0 is only one: Openness. Openness is the only core goal of Web3.0. So what does the word openness mean?

1. Openness of data

2. Openness of data operation

3. Openness of organizational form

This article is not intended to be discussed here. For detailed discussion, please refer to my two books: a review of Web3.0 concepts [2] and why Web3.0 needs blockchain [3].

Bottlenecks in current blockchain computing methods

We are here to discuss the major problems with the technical solution of Ethereum, the mainstream blockchain.

1. Contract VIRTUAL machine, time-sharing multiplexing, limited computing capacity

All ethereum nodes in the world run a sequence of contracts in EVM as a single thread. From the point of view of a contract program, each call to a contract function is executed in a queue, one after the other, the equivalent of time-sharing reuse of the same CPU. This computing model determines that the computing capacity of a node is limited by the computing capacity of a node's single CPU, and its performance is very limited. Ethereum currently has a processing power TPS of around 13.

2. High storage cost

It is currently calculated that each 1K byte of data stored on Ethereum costs more than: $100 (one-time payment). The cost of storing 1 GIGAByte of data for traditional cloud storage services is about $0.02 per month. Over 10 years, the cost of storage on Ethereum is 41,666,666 times higher than cloud storage.

In order to prevent excessive use of computing and storage resources, Ethereum designed a Gas fee mechanism.

Expansion of the road

With ethereum's limited computing and storage capabilities, other public-link projects have an opening. Different public chains have made some degree of innovation in different directions.

Solana

Solana thought is to achieve the best performance of Layer1, known as the highest TPS public chain. Its PoH consensus assist mechanism enables some consensus sub-processes to use GPU for parallel verification. Solana's TPS is around 2000.

Solana abandons some of the decentralized features to some extent, and its node configuration is extremely demanding. These are the costs of pursuing high TPS.

Ethereum 2.0, NEAR, Polkadot

All three of them took the Sharding route. Sharding is very difficult, probably the most difficult capacity expansion solution. Ethereum 2.0 is slow and slow. While NEAR is said to be launching a six-shard network by the end of 2021, it is unclear how effective the shard network will actually be, how much advantage it will bring, and whether transaction costs will increase. Polkadot's parallel links are now well on their way, with several already on the pilot network, but not yet proven in a real application.

For the difficulty of sharding in principle, see NEAR's White Paper on Night Shadow Protocol [4]. At present, there is no real stable operation of the sharding network, the future of the sharding network will encounter any new problems, whether it is practical, these need time to verify.

L2 group male: Polygon, Arbitrum, Optimistic Rollup, ZK Rollup

L2's expansion of Ethereum is the current mainstream expansion direction. For details, see "the road to expansion of ethereum's Layer2" [5].

Cosmos, Octopus

The brothers' project is multi-chain and cross-chain. The basic starting point is that the smallest unit supporting an application is an Appchain, not a contract. Each application has one chain, and then all application chains can communicate with each other directly through the IBC protocol.

All of these projects have had a significant impact so far. The road to expansion is not so easy. In the end, only time will tell who can come to the end.

Computing paradigm for Web3.0 applications

As mentioned earlier, the core of Web3.0 is openness, not any of the following terms (though each is important) :

Low security

Low block chain

● Decentralization

Low cryptography

Security is an extended attribute, cryptography and blockchain are tools, decentralization is the means, and openness is the end.

As for blockchain, all the mainstream blockchains we mentioned above are actually designed for assets, not applications. If you look back in history, starting with the creation of Bitcoin in 2008, all the major blockchain projects have actually done one thing for more than a decade: deal with assets. One result of 2012, just right, is the DeFi craze of summer 2020.

Even an asset is a big deal to get right (think of global finance as a particularly big industry in its own right). In other words, ethereum is no longer enough even if it only deals with assets and financerelated businesses, and there are still so many L2, side chain, new public chain and other projects to share the processing. So many players together is still far from the goal of solving the financial business is too far away.

In other words, the efforts made so far on blockchain have not been able to handle even the basic financial sector, let alone innovation at the level of Web3.0 adoption.

On-chain logic is only suitable for handling asset business

In my book why blockchain needs Web3.0 [6], I put forward that contractual public chain is only suitable for DeFi (asset and finance) related business. In this article, we will further argue that all on-chain logic is suitable only for DeFi (asset, finance) related business.

Since we are talking about large-scale applications in this article, the starting point is the standard of large-scale applications, not the possibility of doing a Demo.

For a common chain of contracts, you can think of all the applications of contracts, combined, as one big application, as on-chain logic. This is determined by the underlying account model. For Appchain, an application chain developed by Substrate, the on-chain logic is still only suitable for handling asset-related business. The on-chain logic of blockchain cannot be used to deal with general-purpose large-scale applications. The reasons are as follows:

1. The calculation amount of logic on the chain is limited by the node performance and block spacing;

2. On-chain logic must be completed within the VIRTUAL machine system without any side effects;

3. The on-chain state storage is realized by MPT, which has a series of excellent features (such as easy proofreading, easy incremental hash, easy access to historical versions), but also increases the intermediate storage and computation amount;

4. The on-chain data structure is limited by storage characteristics and Gas charges, with fewer types and primary forms. There are many details to be paid attention to, which is very limited in business processing.

5. Due to the security requirements of blockchain (encryption, encoding and decoding), the chain cannot have a built-in SQL, the most basic and powerful retrieval engine in Web2.0, which makes it very awkward to manually establish updated index objects when data is inserted when writing business codes.

Due to the above problems, we believe that the traditional blockchain on-chain computing paradigm cannot support the large-scale practice of Web3.0 applications. For Web3.0 applications, we need to find alternative computing and storage paradigms.

Storage consensus paradigm based on Arweave

Storage Consensus paradigm (formerly storage Computing paradigm) : Consensus is achieved through Arweave permanent storage and off-chain computing.

The Stored Consensus paradigm is the next generation blockchain application development paradigm proposed by everFinance. In the Ethereum model, calculations are performed by all nodes in the blockchain network, all of which generate and store global state for query. Unlike the Ethereum model, the storage consensus paradigm separates computation and storage. Blockchain only stores data without performing any computation. All computation is performed by the client/server below the chain, and the generated state is also saved by the client/server below the chain.

The stored consensus paradigm uses off-chain smart contracts, which can be programs written in any language that take all their input parameters from a stored blockchain. In the paradigm, a blockchain is more like a computer's hard drive, and off-chain smart contracts can take place on any machine with computing power. Using blockchain as a hard disk, the data stored has the characteristics of immutable and traceable, and blockchain gives the data the characteristics of trust. The off - chain smart contract computs the trusted data to ensure the final consistency of the state.

Applications developed with the stored Consensus paradigm have the transparency and trust characteristics of blockchain, while having the high performance and availability of traditional Internet applications. The stored consensus paradigm has the following advantages:

1. Composability: business connection between traditional financial services requires accounting reconciliation, which is complicated and error-prone. With the storage consensus paradigm, financial transaction records are fully linked, traditional financial services are transformed into blockchain applications, automatic reconciliation is realized, and fast collaboration and efficient combination between applications is achieved.

2. Low barriers to development: Writing smart contracts with Ethereum virtual machines requires Solidity language, while off-chain calculations can be developed in any language and developers can convert traditional apps to blockchain apps.

3. No performance limitation: The performance of off-chain computing is the same as that of traditional applications. The off-chain server can carry a large number of transactions. TPS depends on the performance of the machine that provides the service and the technical architecture that builds the application.

4. Consensus costs are extremely low: $1 for 1 million transactions.

Reference: Storage-based Consensus Paradigm[7].

OffChain properties of Substrate frames

OffChain properties are a fairly powerful set of infrastructure provided by Substrate. After all, for blockchain, there is very limited space for logical manipulation on the chain, and some things have to be done down the chain. Before the OffChain Worker (OCW), this kind of thing was usually done by Oracle. Prognosticators are external services that submit transactions to the blockchain via the RPC interface of the blockchain node to transmit information from the outside world to the chain. While this approach is feasible, it is still not good enough for security, integration, scalability, and infrastructure efficiency.

To make off-chain data integration more secure and efficient, Substrate provides offchain-specific characteristics. Its architecture diagram is as follows:

Off-chain Worker architecture diagram

The Offchain feature has three components:

Low OffChain Worker

Low OffChain Storage

Low OffChain Indexing

OffChain Worker is used to implement the off-chain logic. The code is written with Runtime code, compiled into the same WASM bytecode string, and propagated across the network in the same transaction. However, the OffChain Worker code is executed in a separate VM, that is, completely separated from the execution of the Runtime logic. Specifically, OffChain Worker can realize the following functions:

● Includes a full-featured HTTP client that can access data from external services

● Access the local node's keystore so that transactions can be verified and signed

● Local KV database can be accessed and shared among all offchain workers

● Local secure entropy source, used to generate random numbers

● Can access the local time of the node ● Can sleep and resume work

OffChain Storage is an OffChain Storage space that is logically independent from the Storage on the chain. It has the following characteristics:

● Can be read and written by OffChain Worker

● It is stored locally in node and will not be transmitted to other nodes in the network. It will not participate in the network consensus

● Shared access by all OffChain Workers running at the same time (hence lock operation required). Therefore, it can be used to communicate between different Workers

● Can be written by Runtime code, but cannot be read. Therefore, it can realize certain chain-to-chain interaction functions based on it

● Can be read by code in nodes outside the WASM environment, and therefore can be read by RPC

Indexing OffChain indexes provide the ability to write data to OffChain Storage in the Runtime environment. But you can't read it. Data in OffChain Storage. This opens up the possibility of new programming paradigms.

Others include, for example, a complete OCW integration testing framework.

Substrate's OffChain properties are very powerful and impressive.

Underchain computing paradigm based on storage consensus based on Substrate implementation chain

Taking inspiration from the Arweave stored consensus paradigm, we will look at how to implement a similar stored consensus based under chain computing paradigm in Substrate.

The Substrate framework is actually divided into three chunks of programmable logic:

1.Runtime

2.Offchain Worker

3. The Node container

And two large blocks of storage:

1. Status Storage on the Runtime Storage chain

2.Offchain Storage

See below for details:

Substrate application development framework

As stated earlier, The Runtime in Substrate is only suitable for handling asset-class logic and the Runtime Storage is only suitable for storing asset-class data; they are not suitable for handling and storing logic and data for other large-scale applications.

In why Web3.0 needs blockchain, there is a general architecture diagram for Web3.0 applications:

Web3.0 App architecture

As you can see, off-chain computing and off-chain storage play a very important role in Web3.0 applications. In this section, a new calculation mode can be designed by refining the Substrate.

We understand that a Block is a historically segmented package of the application's operations and input data (note, not its historical state). It is similar to the SQL log of a database, except that:

1. Segmented packaging, discrete, postmarked

2. Each segment is confirmed by multiple nodes, forming a network-wide consensus, and finalized in its own node. The process of finalization is also the process of ordering the transactions in the transaction pool at each node

Each size contains three hashes:

1. Hash of the previous block to determine to which block the current block is chained (each block has a parent block, thus forming a chain, hence called blockchain)

2. Total Hash of the transaction sequence contained in the current block, merkle_root. Used for authentication to prevent tampering with transactions in transit over the network

3. State_root of the state space on the global chain in the current node. For synchronization of the global state space between different nodes, each block verifies state_root to maintain a strictly synchronized and consistent on-chain global state within all participating nodes.

Based on these understandings, we can make the following design based on the Substrate application architecture diagram:

Strip logic from the Runtime. The Runtime logic only serves to:

1. As a proxy function, write the name and input data of the method invoked by the external transaction to Offchain Storage (use the aforementioned Offchain indexing feature)

2. Verify input parameters

3. Upgrade the entire WASM code (including the offchain worker code where the main application logic resides). This mechanism is built into substrate Runtime and does not need to be developed separately

On-chain storage is no longer required (but not 0, Substrate has some by default, so we don't need to worry about that).

The real logic is executed in the Offchain Worker. This part of the code is WASM bytecode, by the chain transaction unified management upgrade, to ensure the consistency of all nodes. The Offchain Worker code executes at block import time. The execution logic is as follows:

1.Offchain worker entry function execution;

2. Check whether the offchain storage contains storage contents that comply with certain key-value pair specifications, including method name and parameter table (input parameter data). If so, fetch the method name and argument list (possibly serialized);

3. Route according to method name in Offchain worker, and transfer the deserialized parameter list to the corresponding route handler. The routing handler code can be written in different files, imported in the Runtime main file (which will also be compiled into WASM);

4. The Handler can use offchain storage data to compute, aggregate, and renormalize data.

5.Handler notifies the gateway of the result to be returned in the following ways:

A. Send the result to the gateway through the HTTP request capability of offchain worker. After the gateway listens to the result, it returns the data to the front-end according to the URI in the result.

B. Cache the result to the offchain storage. Gateway An RPC interface of a substrate node, through which a result is obtained from the offchain storage and returned to the front-end.

6. The main process is complete.

This is the main process for implementing the storage consensus-based under - chain computing paradigm using Substrate. It is necessary to explain why this approach is called down the chain based on stored consensus:

1. The code in the execution logic offchain worker is submitted to the chain through substrate trading and updated to each node through P2P network after each node passes the verification. This ensures consistent code execution (as Arweave does)

2. Indexes feature consistency. There is a passage in Substrate. Dev [8] : "Unlike OCWs, which are not executed during Initial Blockchain Synchronization, off-chain indexing is populating the storage every time a block is processed, So the data is always consistent and will be exactly the same for every node with indexing enabled." This ensures that calls to methods stored in offchain storage and passed parameters are consistent across all nodes. That is to ensure the consistency of the input data (which plays the role of Arweave)

3. Once the Offchain worker starts to execute, it is executed under the chain, completely separated from the on-chain logic and storage. This corresponds to the sub-chain execution part of the Arweave store consensus paradigm

4. It is very important that the result of calculation will not be re-linked (because there is no on-chain storage required). In this respect, the Arweave stored consensus paradigm is also the same.

So, having said that, what are the advantages of using Substrate to implement a store-based consensus-based under chain computing paradigm that we should pay attention to? In the following section, we focus on the comparison with the existing typical calculation schemes.

Comparison with the current typical calculation scheme

Contrast with traditional blockchain computing paradigm

Here, when we say traditional blockchain computing paradigm, we refer to on-chain computing paradigm.

Advantages of Substrate based Storage consensus under chain computing paradigm:

1.Offchain worker calculation is no longer limited by the requirement of block output interval, and can realize long calculation or waiting;

2.Offchain worker is an independent lightweight thread that can execute multiple lightweight threads concurrently to make full use of the computing power of nodes;

3.Offchain worker has the ability to access external services as well as the ability to produce external side effects, which is very important in actual business.

Offchain Storage, no longer restricted by on-chain design, can make full use of the existing historical accumulation and cutting-edge innovation in the field of database, paving the way for large-scale data Storage; 5. The Storage content in Offchain Storage can be divided into three parts. Only the method name call and input data sections need strong consistency. Logical operation results require access consistency. In the third part, local node data can be accessed/cached flexibly to achieve more flexible programming;

6. The nodes used for Substrate chain calculation can be part of the nodes in the whole blockchain large network, which can effectively control the computing cost and storage cost, so that the scale of content processing and storage can be expanded on a large scale.

Disadvantages of Substrate based Under-chain computing paradigm:

1.Substrate Storage is currently a simple KV database, which has lost some inherent characteristics of MPT structure (such as convenient historical version access, fast incremental hash, fast comparison, etc.);

2. On-chain storage state has state_root synchronization in each block, whereas Substrate chain computing paradigm loses this feature. This is not mandatory for synchronization of storage state;

3. The Substrate chain computing paradigm requires a set of custom specifications for the storage of calculation results and intermediate states, which are only defined in the offchain worker code without the process of mandatory verification by other nodes;

4. In Substrate worker, it should be noted that side effects such as random numbers and external requests should not affect storage areas requiring consistency. So when you're writing code, you need human attention, you need conventions or best practices.

Contrast with Arweave based stored consensus paradigm

Advantages of Substrate chain computing paradigm:

1. Higher chain integration. Substrate nodes themselves constitute a blockchain, which no longer requires a separate chain as a service;

2. The validation process of Substrate chain transactions is more detailed than the validation process of Arweave data upload requests. The Substrate Runtime allows you to write custom validation code for transactions, which are performed on-chain, whereas Arweave is an on-chain storage service and cannot do this;

3. Indexes actively write content down the chain, whereas Arweave requires underchain computing clients to actively pull data from the Arweave network. This process is less automated, at least with more polling. In addition, there is network access to pull data from Arweave, which may cause network access failure. If there is a large amount of data, it will take considerable time to complete the download. Indexing, on the other hand, operates entirely within a node, providing better performance, greater integration, and greater control for developers.

4. Based on Substrate nodes, on-chain logic and off-chain logic are unified in operation and maintenance, with better integration and reliability of operation and maintenance (less trust, less possibility of disputes);

5. Logic under the Substrate chain enforces wASM bytecode to run. At this point, Arweave is not asking. Webassembly is a set of internationally widely supported bytecode standards on which there is a lot of innovation and a rapidly evolving ecosystem. Deploying applications with WASM is an advantage in terms of operational efficiency and future ecology

In short, Substrate underchain computing paradigm based on memory consensus implements some of the functions of Arweave, but does not provide the persistent properties of Arweave. In this regard, the persistence advantage of Arweave can be fully exploited if the block history of Substrate chain is uploaded to Arweave for storage.

Disadvantages of Substrate chain computing paradigm:

1. No language. Substrate can only be developed in Rust language at present, which means that in order to use Substrate to achieve Web3.0 application development, Rust language must be learned. The Arweave stored consensus paradigm is language neutral and can be developed in any language.

2. The Substrate chain computing paradigm has new requirements on programming mode, and programming development based on Arweave storage consensus paradigm is similar to Web2.0 development, requiring only the invocation of Arweave related third-party services;

3. Perpetuation implemented by Arweave has the advantage of better historical reliability for content that requires consistent consensus (code and input). Even if the blockchain project fails, its content is still available on Arweave (meaning open to debate);

4. As a storage computing Service, Arweave is similar to XaaS (X as a Service), which is convenient in some scenarios.

As we described earlier, the undersubstrate chain computing paradigm requires some artificial conventions, which are also encountered in the Arweave storage consensus paradigm. That's the same thing.
From The Graph/SubQuery The contrast Substrate is a kit of tools (like LAMP in those days), and The Graph/SubQuery is a service. The Graph/SubQuery is an external index service, which is mainly used to index events thrown on The chain and extract event information by writing Mapping method to generate schema structured data and store it in The database. They provide the GraphQL interface and are themselves traditional SQL or NoSQL databases.

Advantages of Substrate based Storage consensus under chain computing paradigm:

1. Higher integration, The Graph/SubQuery requires a third party external service to get data;

2. The mode of Substrate can provide more flexible interface forms. Configure RPC, Restful, GraphQL on the gateway.

3.Substrate is more controllable and decentralized. The Graph/SubQuery is a service. Everyone uses this service, so they are more dependent on this service and worry about its stability. As for the Substrate set of tools, they are only responsible for their own chains, and the problems of others will not affect them, and the problems of their own will not affect others. Decentralizing is better when you deploy your own separate services (if you try to use the same tools, or at least follow the same protocols).

Disadvantages of Substrate based Under-chain computing paradigm:

1. Currently Offchain Storage replacement/upgrade to support SQL engine Storage has not been implemented, the index still have to maintain;

2.The Graph can index The contract public chain (for developing Dapp), while The undersubstrate chain computing paradigm is only applicable to Appchain implemented by Substrate framework;

3.The Graph/SubQuery index The information thrown on The chain, and The way The code is written on The chain is unaffected, i.e., non-intrusive. However, the Substrate chain computing paradigm has requirements on the way of using Substrate framework to develop code, and code should be written in accordance with the new paradigm, so it is intrusive (although this paper believes that the new computing paradigm is more suitable for Web3.0 applications).

By contrast, both have their own applicable scenarios. The Graph/SubQuery is more suitable for traditional contract-based Dapp development, while Substrate based storage consensus chain computing paradigm is more suitable for specific scenario-oriented Web3.0 App development.

Contrast with traditional centralized applications

Due to the nature of off-chain computing, mature infrastructure for traditional centralized applications (such as distributed databases) can theoretically be used. Utilization of Substrate based storage consensus under - chain computing paradigm applications.

Advantages of Substrate based Storage consensus under chain computing paradigm:

1. Compared with the traditional centralized application, the under-chain computing paradigm application of Substrate is a decentralized application, in which the business logic runs on each node and the consistency of business and operation results is maintained through the mandatory characteristics of blockchain. The application of Substrate chain computing paradigm automatically has all the advantages of decentralized application;

2. Openness is guaranteed. The openness of the three levels mentioned above (data, operation of data, and form of organization) can be effectively guaranteed under this computing paradigm, which is exactly what we pursue in Web3.0. Anyone interested can run Substrate nodes to synchronize all historical blocks and run out all data. The person accessing the data does not necessarily participate in the chain's consensus. Traditional centralized applications, on these three levels, are closed.

Disadvantages of Substrate based Under-chain computing paradigm:

1. The same logic requires repeated calculation of nodes under each chain and storage at each node. This is really not an advantage in calculating cost and energy consumption. However, since the number of nodes participating in the off-chain calculation can be controlled, the redundancy of this part is similar to the data redundancy and logical redundancy required by traditional centralized applications (to prevent service interruption caused by the service of one node).

2. At the infrastructure level, centralized application can be used to fully mature infrastructure supporting large-scale application, while the infrastructure of Substrate chain computing paradigm is still immature.

Current deficiencies in Substrate Offchain infrastructure

Offchain Storage is currently only the most basic kv database implementation (Rocksdb). The infrastructure needs to be developed to support truly real-world applications, and there are several directions to consider:

1.Redis: As an in-memory KV database, Redis is supported by rich built-in data structures, which makes it widely applied in various fields;

2.Mongodb: aS an object database, Mongodb is especially suitable for some fields;

3.Tikv + Tidb: As a representative of the new generation of distributed database, Tikv has made the distributed protocol layer on top of Rocksdb, and Tidb has further made the SQL engine layer on top of Tikv. This is a very effective and reasonable hierarchy and is particularly suitable for the future improvement of Substrate Offchain Storage. If TikV/TiDB can be grafted into Substrate, many ecological facilities on TIkV/TIDB can be used for calculation under the chain of Substrate, which is a wonderful situation.

In addition, there are some changes in the thinking of code writing: Offchain Worker is timer thinking, or flow programming thinking. Different from the traditional server on_request/response thinking. More research is needed on how best to do this.

Definition of Web3.0 App

Based on the discussion in the previous chapters of this paper, we can further distinguish blockchain as follows: the Chain used to process the assets on the Chain is called Asset Chain. The Chain that uses the down - Chain computing paradigm for processing data from Web3.0 applications is called the Dataset Chain. Based on this distinction, we can make the following schematic diagram:

This graph defines: Web3.0 App = Dataset Chain + Asset Chain + part of existing Web2.0 infrastructure

Dataset Chain is the core carrier of Web3.0 App. Web3.0 apps based on the storage consensus off-chain computing paradigm are located at the boundary of the blockchain ecosystem, connecting the asset blockchain on one side and the existing Web2.0 infrastructure on the other.

Tools or services?

At the end of this article, let's do a quick tool versus service debate. By services, WE mean infrastructure services at the B level, not business services at the C level.

Usually, you have two options for getting something done:

1. Use a tool (downloaded from a website, or given to you) to run the calculations on your own computer and complete 2. Using a service (usually through a browser page), run the calculation on the server where the service is located and return the results to you

That's the difference between a tool and a service: the calculation, whether it's done on your own turf or on a third party's turf.

The cloud service IaaS has been a huge success. From the previous model of buying their own server hosting to computer room, to cloud service IaaS, opened the cloud era, and then to the current K8S, this road is still in sight.

In contrast to IaaS, the higher level OF PaaS does not seem to be successful. SaaS has had some success.

Service generation is always delayed, with some tools (development frameworks) being implemented first. Let's not forget, what ushered in the Web2.0 era? Is LAMP (Linux operating system, Apache Web server, MariaDB or MySQL database, PHP, Perl or Python scripting language). As of today, 30% of the world's websites are built on Wordpress, a PHP site builder (see more data here [9]).

The Java Spring framework, Ruby on Rails, Nodejs and other technical stack frameworks have played a very important role in the development of the Internet (Web2.0). These frameworks give you:

1. Develop and debug the process and ability of deploying a complete application (Web, App) locally;

2. 100% control of your own code logic;

3. Run on the cloud server (cluster) applied for by oneself, and have 100% control over application data;

4. The operation cost of Internet service can be accurately estimated. 5.

These frames don't give you:

1. As services expand, you need to maintain server clusters.

2. Maintain server security.

1. 3. Smart refrigerator

And third-party services give you:
1. Ready-made functional components, low-cost out of the box;

2. There is no need to pay attention to operation and maintenance. All capacity expansion is automatically completed.

3. Fast test environment;

4. 4. Smart refrigerator

Third party services do not give you:
1. Transparent implementation;

2. Accurate calculation of cost (if not free, there is more concern about free formal use);

3. Go deep into the custom optimization space 4.

For the entrepreneurial team, in some scenarios, the short and fast service startup will tend to directly use the third-party service implementation. However, when the business scale reaches a certain stage, and strive to achieve full control of the operation of each component, it tends to use the supporting components of the development framework to build their own services. Of course, the actual situation is often mixed up use.

If more factors such as culture, region and national policy are taken into account, the demand for the ability to build independently becomes even stronger.

The most important point is that it is relatively easy for all parties to build relatively complete services, which is a very important measure for the realization of "decentralization". If everyone is using the same service, is it another level of centralization (whether the implementation of the service itself is centralized or decentralized)?

Unless you have a very specialized service, in general, services are optional and development frameworks (tools) are fundamental.

Investing in a development framework should not be overlooked.

conclusion

In this paper, we try to design a technology route for large-scale application of Web3.0 apps: the store consensus based off the chain computing paradigm. This article uses Substrate framework as an example. In theory, this computing paradigm can be implemented by any framework -- only the differences of integration and maturity. You can try to incorporate this computing paradigm into your own framework.

This article was written by Mike Tang and Outprog. Our email is: mike@oct.network, outprog@ever.finance, welcome to contact us.

The resources

[1]: https://cryptobriefing.com/ethereum-defi-ecosystem-has-hit-3m-users/[2]: https://mp.weixin.qq.com/s/vJM6TIZT2f-tnQ49cpMnrw[3]: https://mp.weixin.qq.com/s/h76lTnFWlvpXs72aBVs3FA[4]: https://near.org/papers/nightshade/[5]: https://www.theblockbeats.com/news/24443[6]: https://mp.weixin.qq.com/s/zXqJ0BGdEbvZypAwLWKmtQ[7]: https://mirror.xyz/0xDc19464589c1cfdD10AEdcC1d09336622b282652/KCYNKCIhFvTZ1DmD7IpXr3p8di31ecC283HgMDqasmU[8]: https://substrate.dev/docs/en/knowledgebase/learn-substrate/off-chain-features#off-chain-indexing[9]: https://techjury.net/blog/percentage-of-wordpress-websites/#gref