Three factors restricting the development of Filecoin: user experience, information security, and storage technology solution

Author: Frank Li, Vice President of Investment at ParallelVC

What exactly is Web3.0 that everyone is talking about?

The development of the Internet has gone through two stages: Web1.0 and Web2.0. Web sites in the stage of Web1.0 are mostly static sites, and there is no interaction process between network users and network information. Users can only obtain the information displayed on the network, and the effectiveness and efficiency of information acquisition are relatively low.

With the increase of Internet speed and bandwidth, people and Internet gradually began to interact. In 2003, dale ducherty, vice President of O "Reilly Media, introduced the concept of Web2.0.

Web2.0 is called the read-write web. In the early days of Web2.0, compared with Web1.0, all web users could create their own content and upload it to the network instead of getting information from the network, which greatly increased the richness of the network information. With the further development of AI, big data and other technologies in recent years, human-computer interaction has been promoted to a new stage. The browsing information, clicking, searching and other online behavior data generated by users on the Internet are captured and recorded. The technical background can analyze the accurate user portrait through the real-time data of users combined with their past information, and recommend corresponding products or information to users according to the user portrait. In this way, it not only improves the conversion efficiency of merchants' purchase, but also enables users to find the goods they may want to buy more quickly, so that users can have a better experience.

However, information centralization, while convenient, also has a big disadvantage, that is, all the data of users are used by the platform mobile phones without their perception, and even the ownership of the data is ambiguous.

In the early days of Web1.0 and Web2.0, the amount of user data was small and the data dimensions were relatively small, so users' personal data could not generate much value. However, with the increase of people's access to the Internet in recent years, the value of personal information on the Internet has become significant. In the past two years, there have been cases of Internet companies violating personal privacy data and user data theft in various countries. In the future, with the development of artificial intelligence Internet of Things (AIoT) and 5G networks, the dimension of personal network data will be more comprehensive and valuable, making data security and data privacy more important.

The emergence of Web3.0 is to solve the current problems faced by Web2.0. Blockchain network, due to its functions and properties of de-trust, non-tamper and confirmation of rights, well meets the requirements as the underlying technology of Web3.0. At the same time, due to the change of network architecture, data is no longer a mere number but a commodity with value attributes, and our existing data network is gradually transformed into a value network.

Web3.0 requires decentralized storage

As mentioned above, if you want to build a decentralized network to guarantee the security and privacy of data, you need to ensure that the data can only be controlled by its owner (the data producer), including the right to obtain or authorize the use of others. Unfortunately, the storage solutions in the Internet are all centralized storage, and the main storage providers are amazon, Aliyun, Google Cloud and other central organizations. At the same time, the personal data of users generated on the Internet are now owned by all platforms and apps. Even if the ownership of personal data can be returned to the users in the future, users cannot ensure that their data will not be used or modified by the APP platform or even the storage provider in the case of central storage.

In addition, 5G and AIoT cutting-edge technologies are developing rapidly. In the future, the dimension of personal data in the network will be further increased, and the value of data will also increase rapidly. Only the use of decentralized storage can further ensure the security and privacy of data.

Without the decentralized storage technology as the underlying technology support of the decentralized network, even if the decentralized information transmission and decentralized computing are realized, the data cannot be truly decentralized. Therefore, decentralized storage will be an essential technology component in the future Web3.0 ecosystem.

Current status of traditional cloud storage

At present, the traditional cloud storage field can be divided into public cloud, private cloud and hybrid cloud. Public cloud is the most common form of cloud service at present, which is held and managed by public cloud and technology providers and serves multiple customers. At the same time, public cloud can be divided into three technical forms: SaaS, PaaS and IaaS.

SaaS is the abbreviation of "Software as a Service". This kind of service provides applications as services to users, mainly for individual home users and small and medium-sized enterprises. The main service providers in China are Baidu Network Disk and Tencent Micro cloud, while the main service providers abroad are Dropbox and iCloud.

PaaS is the abbreviation of "Platform as a Service". Such a service provides the development Platform as a service to users, mainly targeting at small and medium-sized enterprises and individual developers. PaaS platform in China has Baidu Cloud, while Openshift platform developed mainly for Google App Engine and Red Hat in foreign countries.

IaaS is short for "Infrastructure as a Service", which mainly provides users with cloud computing resources such as virtual machines and cloud storage elements as a service network. Domestic leading enterprises include Aliyun and Huawei Cloud, while amazon EC2 provides such services abroad.

Unlike public clouds, private and hybrid clouds tend to serve large enterprise users. Private cloud is used and maintained by an enterprise or organization, and users have more control over personalization, etc. Hybrid cloud is a solution that mixes and matches public and private clouds to achieve relatively high cost performance.

First of all, we can analyze the data of domestic individual users on cloud storage: according to aurora MAU share data in 2019, the top four are baidu disk, Tencent micro cloud, colorful cloud and tianyi cloud disk.

Among them, Baidu network disk active user share reached 82.9 percent. Continue to observe the user interest of Baidu network disk TGI, you can get the user's main storage data for movies, animation and sports related content.

At the corporate level, data shows that 39% of enterprises are using cloud storage-related services (such as AWS, Azure, Google Cloud, etc.), and it is predicted that the number will reach 60% by 2022. Among the top three U.S. cloud storage companies, Microsoft Azure has 44% of the market, AWS has 32%, and Google Cloud has 19%. Azure was also named the most competitive cloud service provider in the Spiceworks2019 public cloud report. The survey also found that 79% of business owners use Windows as the main operating system for their business, and That Microsoft's Azure cloud storage service has a better product experience with Windows.

The current cloud storage service architecture has been relatively mature, and users can choose to use public cloud, private cloud or hybrid cloud service according to their own needs. In the public cloud service, users can choose different storage modes based on the frequency of data invocation, which can save the cost to the greatest extent without affecting the usage.

Compared with cloud storage, nearly 81% of enterprises are still using traditional storage matrix, namely hardware storage. Dell EMC's market share now tops 42%. According to the survey, 80% of enterprises choose the traditional storage matrix because of its higher reliability. At the same time, they believe that compared with cloud storage, the privacy and security of traditional hardware storage are higher.

At the technical level, traditional cloud storage has been in three phases of iteration. The first stage is to consider NAS (Network Attached Storage) and SAN (Storage Area Network) as the underlying storage architecture scheme. This architecture has initially formed a cloud storage scheme, but it is difficult to flexibly call different storage clusters due to the high geographical location of servers, so there is a big bottleneck in the expansion of capacity and performance.

The second phase began with the emergence of the EMC VPLEX architecture. Based on the original storage architecture, VPLEX realizes the function of virtual storage and heterogeneous storage, liberates the geographical location limitation of storage hardware, and solves the operation problem across clusters. Through intelligent distributed cache integration, access and sharing of data across hosts, clusters and data centers are realized, and cloud storage capacity and scalability are greatly improved. The services it provides are similar to those now provided by IAAS.

The third stage is the current stage of cloud storage. The current underlying architecture of cloud storage is a cloud-like structure, in which multiple storage devices are interconnected, and hardware devices are intelligently switched according to different requirements. On the basis of the original, multi-copy consistency, disaster recovery, elastic expansion and other aspects are greatly improved. The current cloud storage SaaS, PaaS platform can provide users with easy-to-use storage services.

The development of decentralized storage

In the current cloud storage infrastructure, distributed architecture has become standard due to its advantages of high performance, consistent multi-distribution, and support for hierarchical storage. Decentralized storage can be classified as distributed storage.

Compared with other distributed storage solutions, the data center in a decentralized storage architecture will not be completely owned by the cloud provider, but will be jointly provided by multiple nodes in the network, and the data will be stored by encrypting. In this way, the privacy and security of data can be better addressed, even if the data center or storage hardware provider cannot access the saved data.

At present, the most representative items of decentralized storage are IPFS and Filecoin. IPFS was called "Inter Planetary File System" and "Star File System" in Chinese. IPFS is an underlying network transport protocol equivalent to HTTP (hypertext) in the current Internet world. HTTP is a relatively simple request-response protocol for user-server interaction.

IPFS functions like HTTP, but adds the architectural features of p2p networks. IPFS protocol is more efficient than HTTP protocol. HTTP is single-threaded communication, and only one task can be performed on a server at a time, while IPFS USES p2p for multi-threaded download, which can save more than 50% of the bandwidth cost. At the same time, due to the centralized property of the current Internet server, the information in the network can be completely controlled and hard to save. However, if the decentralized protocol such as IPFS is used, as long as the information is owned by any user in the network, the whole network can obtain this information.

In the decentralized storage ecosystem, IPFS mainly plays the role of the underlying technical protocol, and other specific business-side solutions will be implemented by the Layer2 project. Currently, Filecoin, the incentive layer developed by IPFS team, is the most likely to be implemented first based on IPFS protocol. The test network has been started for this project, and the main network will be started as soon as this year.

There are a number of projects beyond IPFS that are trying to implement decentralized storage on the ground, but none of them have made any real progress so I won't go into it.

Read Filecoin

Filecoin is a decentralized storage project based on IPFS protocol, which builds a bridge between users and storage service providers to provide users with de-centralized storage services.

Participants in the Filecoin network can be divided into three parts: storage miners, retrieval miners, and users. Storage miners provide storage space for a decentralized storage network and pledge a portion of the tokens in the network to prevent miners from doing evil. Storage miners need to prove to the network that they have stored the user's target data for a specified period of time, and if they cannot, they will be punished by deducting a percentage of the collateral.

Miners retrieval is mainly responsible for providing users with data retrieval services in the network. After users initiate demands, miners retrieval needs to find relevant information in the network and send it to users. At the same time, there is no requirement for miners to participate in the network. Miners can be storage miners and retrieval miners at the same time, or they can only participate in one of them.

Users also have a lot of flexibility with Filecoin. On the one hand, users can use Filecoin network as a network disk. Users only need to indicate the data they want to store and the quotation, and the system will match the quotation of users and miners. When prices are agreed, the system sends data from users to storage miners.

Miners are stored in different sectors (storage units) according to the size of the data, which are then encapsulated and sliced, and the transaction information is stored in a hash table in the network. As the order continues, the node will constantly challenge the miner to check if the miner has saved the file. This completes the entire process of storage.

In terms of data privacy, users can choose whether the data they store can be used for personal viewing or disclosed to the whole network when creating a storage subscription. This data can be queried by anyone in the network by retrieving miners if it is made available to the whole network. The user stores the data in several different nodes.

In terms of Consensus algorithms, Filecoin USES three different Consensus algorithms: Expected Consensus, proof-of-Replication and proof-of-Spacetime.

Expected Consensus is the Consensus algorithm used to specify block rules in Filecoin, which is a probabilistic Byzantine fault-tolerant algorithm. According to the Expected Consensus, when each new block is generated, a leader will be elected to make the block. Similar to BTC mining, the probability of being selected is proportional to the proportion of computing power in the whole network. However, in Filecoin network, computing power is based on the effective storage space provided by miners rather than the computing power in BTC network. Before each block is generated by the network, miners will generate newticket according to the ticket value of the prove in the previous block through the linear process of VRF and VDF, and then confirm whether they are eligible for block extraction by comparing the newticket value and the ratio of effective computing force to the total network final strength. If the Newticket is less than its effective computing power/net computing power, the miner is elected as a block miner.

Different from BTC network, Filecoin may have no one to block or many leaders to block together. When no one leaves the block, the height in the network is displayed as an empty block. Because there was also the possibility of multiple people contributing blocks at the same time, the Filecoin team designed the TipSet -- a collection of blocks. As with the BTC network, there may be two Tipsets at the same time due to network latency problems, but in the end the network will elect the only Tipset based on Weight and proceed down the chain. The calculation formula of weight for each block is as follows:

ParentWeight + ECV + ECPrM * ratio

At present, ECV is set to 10 and ECPrM to 100. Ratio is the storage efficiency of the current node (storage capacity of the node/storage capacity of all nodes). In other words, under the current algorithm, if the node ratio is high, Weight will be high. When weight is the same, a Tipset with a small ticket value will be elected. Because of this technical design, filecoin's underlying architecture is more DAG.

The proof-of-Replication consensus algorithm is used to prove that the miners' work has indeed been performed. In the Filecoin network, the miner's main job is to store the user's data. Miners need to seal user data in a sector. In the process, the system will generate a DRG (Depth Robust Graph), also known as replica value. Only the storage space corresponding to the generated replica value can be regarded as effective storage space.

The proof-of-Spacement consensus is ultimately responsible for the steps of validation. In the generation of each new block, the network will randomly challenge the miners, and the miners need to provide replica value for verification. The zero-knowledge proof will be used for verification. If the result is correct, then the miner normally stores the encapsulated data and the effective storage share remains the same. If the replica value cannot be verified correctly, the storage space corresponding to the replica value will be removed from the effective storage space.

What's the future of Filecoin?

There is still a big gap between the current Filecoin technical solution and the traditional mature cloud storage solution. The following is a simple analysis of Filecoin's current problems.

1. Information security issues

Compared with traditional centralized storage, the biggest advantage of decentralized storage is that it can better protect the privacy of data security. But Filecoin's current design raises questions about the security and privacy of data.

Consider the process of user storage: after a successful order match, the user sends the raw data directly to the miner, where the user's data is not encrypted or sharded. Miners have direct access to complete user data and cannot be detected by networks or users. According to this technical scheme, the privacy security of users' personal data in Filecoin will be greatly threatened, and its security and privacy are even weaker than traditional cloud storage schemes. At the same time, in the early stage of the storage process, there is a vulnerability that causes direct data leakage, so the encryption, sharding and other methods to ensure data privacy in the subsequent process also become meaningless.

On the other hand, because a user all data for a single store if the miners in a miner's hard drive for some reason to give up continue to dig, the data on the user's presence here would be completely lost from the network, and the related mechanism in the network is not set allows users to retrieve their data before miners launched. Such unpredictability also makes the data in the Filecoin network much less secure and reliable.

Meanwhile, according to the information mentioned in the previous AMA of this project, Filecoin is only a protocol to coordinate the needs of storage providers and consumers, and cannot require storage providers to make specific actions. While that reduce the risk of decentralization, the platform control user data from happening, but reduce the cost of storage providers to do evil, even apply request storage provider delete or destroy the stored data, storage providers can also cheat users copy private user data even forcibly do not perform user requirement. The traditional cloud storage technology and equipment or service providers are big company, can immediately after the problem occurs with clear responsibility and begin to solve the problem, but like Filecoin decentralized network of miners are relatively fragmented and unknown and solution to a problem after the follow-up this also caused a lot of obstacles, make its comprehensive security have weaker than centralized storage.

2. User experience

As described in the white paper as a whole, Filecoin users should have a poor experience. First, Filecoin projects in addition to the simplest storage capabilities, to provide customers with no additional technical services, disaster such as scheme, users can only through to save more than your own file as a way to deal with several different node storage unit drops, damage cannot be accessed, and even lead to data loss. For users who do not understand the technology, do not understand disaster recovery, may lead to the permanent loss of user data.

In addition, because of the information security issues mentioned above, if users want to further ensure the security of their data, they need to encrypt the files before storing them. In this way, the operation requirements for users are higher, which makes the user experience worse.

3. The storage technology scheme is not complete

At present, Filecoin technology solution is not complete, which is an important reason for users to use their storage network, such as disaster recovery mentioned above. Under Filecoin's current design, the user's data will be stored by one miner at a time, and no other miner will actively store a backup of the file. This requires more disaster recovery technology than traditional cloud storage, because the traditional cloud storage is mainly for the technical reasons of the equipment itself. However, Filecoin needs to prevent the failure of the technology of the equipment itself as well as the occurrence of the storage miners' service suspension due to various main related reasons. There will be far more factors contributing to this than mere technical failure, and the solutions will be far more complex.

Currently Filecoin method to solve the problem of disaster is to let the user to do multiple data miners in active backup, but early in the network, because the miners instability, cannot be ruled out multiple backups data loss, or all of the lost, so for the operation of the user and the network will be a very big blow.

At the same time, the redundancy of the storage network cannot be solved because of the simple active backup. In traditional storage, because storage is centralized, the system can analyze, process and optimize all data in the background to remove duplicate data and improve storage network efficiency. Although the non-public data stored by users cannot be processed in this way due to the nature of decentralization, how to optimize the data storage structure of user-stored public data combined with the characteristics of IPFS network is a direction that the team needs to study carefully.

The three issues mentioned above restrict Filecoin's future development at different levels. From the analysis of the traditional storage market at the beginning of this article, it can be seen that the current storage market is basically divided into two categories: the first category mainly serves individuals and small business users, and mainly exists in the form of SaaS platform technology. After years of development, the user experience and product diversity of SaaS platform have become relatively mature. Meanwhile, cloud disk applications such as baidu web and Google Drive have their own characteristics and advantages in file transmission, file search or collaborative office. Filecoin itself does not have any advantages in its competition. If there are vulnerabilities in data security, Filecoin won't survive the competition.

Even if the data security issue is resolved, Filecoin's current development is not easy to be optimistic from the perspective of real storage requirements. It can be seen from the storage data of Chinese network disk users provided at the beginning of the article that the first three TGI users of cloud disk are mainly movies, sports and animation. And these types of related video or image data pay more attention to dissemination rather than privacy, compared to decentralized storage, can be Shared or query the traditional network disk will be more advantageous.

The real demand for decentralized storage can be roughly divided into two categories: first, there is a greater demand for decentralized storage for data such as personal photos and private files, but such data is relatively small in the current stock market. The second is personal data generated by personal IoT devices in the future. However, as head IoT device companies like Xiaomi now regard user data as an important asset through which huge value can be generated, they will not return user data to customers without specific regulations. If the head company is unwilling to return the number of individuals and does not apply to decentralized storage, then even if the individual users are willing to use it, it does not make sense because its fundamental purpose is not achieved.

The second type of users are large and medium-sized enterprises. Large and medium-sized enterprises mainly use cloud storage to store relevant data of the company, so they have high requirements on the reliability, security and privacy of storage schemes. The traditional solution including public clouds and private clouds, a hybrid cloud and four kinds of traditional storage matrix solution, the private cloud and the collocation of traditional storage hardware to solve the company's customer concerns about public clouds above data privacy, but also can meet customer demand for storage solution reliability and security. Currently Filecoin lags behind traditional cloud storage solutions in three areas, so it cannot compete. Compared with the traditional storage matrix, the benefits brought by decentralized storage are not what the traditional storage matrix users pursue, so it is difficult to transform among these users. For decentralized storage, it is possible to develop TOB-related businesses only when the data reliability, security and privacy can be guaranteed in the future and the price can be lower than the current traditional cloud storage scheme.

In addition to technical factors, Filecoin project also has many uncertainties.

The first is the team factor. The two main network launch promises in 2018 have not been fulfilled. The mining is changed from CPU mining to GPU mining, and the mining rules are changed in the test network stage. Although later changes were made to prevent miners from spamming data, such frequent changes are less trusted by users and network participants.

In addition, the Filecoin mining incentive mechanism and distribution rules have not been determined so far, which adds to the uncertainty of the project.

Finally, Filecoin mining logic is different from other POW mining types. The project requires long-term participation of miners and cannot be withdrawn in the middle, because the withdrawal of miners may lose data forever. However, the specific details of Filecoin mining are too complex, and the income is difficult to budget, which also increases uncontrollable factors. Whether enough miners will be able to participate and stay is a matter of concern, and the impact on the network of dozens of miners will only be determined after the main network comes online.

In terms of price, Because Filecoin has various technical and product problems mentioned above, it needs a very attractive price compared with traditional cloud storage to reduce users' cost.

Based on the above analysis, it can be known that at present, Filecoin's main competitor will be SaaS applications in the traditional cloud storage system. Then, the following will summarize the current mainstream toB and toC SaaS service charging schemes in the market for readers' reference:

Dropbox:

Personal version of the free storage space, 2 g version Plus provide 2 t storage space and provide a text search, mobile terminal such as offline storage service is RMB 78 yuan per month, professional versions offer 3 t of storage space, at the same time, on the basis of Plus provides AutoOCR, collaborative editing, file locking and other services, is RMB 130 yuan per month.

The toB version, plus, costs 81 yuan per month, provides 5T of storage space and 2 gigabytes of file transfer capacity, starting with 3 people. The Professional version costs 130 yuan per person per month, provides wireless data space and 300 gigabytes of file transfer capacity, also starting at 3 people.

The Box:

The personal version offers 10 gigabytes of storage for free, and uploads are limited to 250 megabytes per file size. There are 5 dollars/month, 15 dollars/month,25 dollars/month and 35 dollars/month. The $5 plan provides 100G of storage space and limits the size of individual files uploaded to 2GB. At the same time, box can connect to an app and store app data directly into Box. The $25 plan allows you to connect to 3 apps, while the $35 plan allows you to connect to unlimited apps.

Baidu network disk:

The payment plan of Baidu personal network disk is shown in the figure above. The monthly fee for super members is 18 yuan, while the monthly fee for ordinary members is 8 yuan. Non-members have 15G of storage space before completing tasks, and will get 2T of storage space after completing a series of sharing tasks.

The above chart shows the introduction and price of Baidu enterprise network disk.

Compared to the three centralized storage providers listed above, Filecoin is hard to compete with in terms of enterprise-class services. On personal services, Filecoin can only compete with Filecoin by offering a more flexible pay plan and a lower price. Meanwhile, Filecoin and other distributed storage projects also need to enrich their additional functions to improve their competitiveness.

The prospect of decentralized storage

Compared with the centralized cloud storage industry, which has been relatively mature in terms of technical architecture and operation mode, the concept of decentralized storage was proposed only in the past two years, and there is great room for improvement in both technical design and operation mode. The development of decentralized storage should draw lessons from the mature cloud storage solutions and improve the solutions according to their own characteristics.

It is too hasty to completely deny all the previous technical solutions of centralized storage, and try to directly replace the original solution with the new imperfect solution, which is difficult to be recognized by the market. Only by finding a good balance between innovation and tradition can we create products that meet the needs of the market.

For now, it is a good attempt to re-establish the underlying network protocol like IPFS, but its usability is relatively poor and it is difficult to meet the fundamental needs of storage users. Relatively speaking, if can be used in the current public clouds block structures, an insulation layer, the chain technology may be a better option, because this approach can not only comprehensive dispatching high-performance storage resources, also can ensure the privacy of user information, at the same time also can fully enjoy traditional cloud storage complete storage system. Such a landing scheme can realize the requirement of decentralization relatively quickly, ensure the security and privacy of data, and give sufficient time for the development of decentralized storage technology. It not only gives users time and opportunity to fully understand centralized storage, but also facilitates the large-scale implementation of decentralized technology in the future. But there is no denying that only through continuous innovation and experimentation can decentralized ecological storage blaze a trail suitable for itself.

Welcome to join the official BlockBeats community:

Telegram Subscription Group: https://t.me/theblockbeats

Telegram Discussion Group: https://t.me/BlockBeats_App

Official Twitter Account: https://twitter.com/BlockBeatsAsia