header-langage
简体中文
繁體中文
English
Tiếng Việt
Scan to Download the APP
Partnered With:

CoinFund: AI is popular again, building a more fair and open AI system through Web3

Read this article in 9 Minutes
Integrating Web3 into AI's infrastructure layer is a critical step in ensuring that AI systems are built in a fair, open, and accessible way.

Original title: "Open neural networks: the intersection of AI and web3"
Original author: Rishin Sharma & Jake Brukhman @CoinFund
Original Compilation: Peter Pan@BlockBeats


There is no end to technological innovation, especially artificial intelligence (AI). Over the past few years, we have seen the resurgence of deep learning models as AI pioneers. These models are composed of densely interconnected layers of nodes that pass information to each other. They are roughly similar to the structure of the human brain, so they are also called neural networks.


In the early 2010s, state-of-the-art deep learning models had only a few million parameters , are used for specific sentiment analysis and high-level supervision. Today's state-of-the-art models are approaching a trillion parameters, such as DreamStudio, GPT-3, DALL-E 2 and Imagen, which are doing tasks comparable in complexity to humans , and even more creative tasks.


Take the following picture as an example, it is generated by AI, the lines are highly detailed, sharp. Additionally, we are beginning to see the social and cultural impact of these models, which shape the way we learn new things, interact with each other, and express ourselves creatively.


AI-generated images for the search engine Lexica.art


However, training some large "neural Many of the core technologies, critical data sets, and computing power of the Internet are almost all closed source and controlled by big tech companies such as Google and Meta. Although GPT-NeoX, DALLE-mega and BLOOM and other copied open source models have been provided by StabilityAI, EleutherAI and HuggingFace lead the way, but open source AI can still be further enhanced through Web3.


Specifically, the Web3 infrastructure layer for AI can introduce open source development, community ownership and governance and elements of universal access, thereby creating new models and efficiencies in the development of these new technologies.


In addition, the adoption of AI technology will also enhance many key use cases of Web3, such as fromGenerate Art NFT to Metaverse. And open source AI is also in line with the spirit of openness, decentralization and democratization of Web3, which represents the AI alternative provided by Big Tech.


Various basic AI models


The underlying models are "neural networks" trained on massive datasets to perform tasks that typically require intelligent human behavior, and these models create some impressive Impressive results.


OpenAI's GPT-3, Google's LaMDA and language models like Nvidia's Megatron-Turing NLG have the ability to understand and generate natural language, summarize and synthesize text, and even The ability to write computer code.


DALLE-2 is OpenAI's text-to-image diffusion model, which can generate unique images from written text. At the same time, Google's AI department DeepMind has produced competing models, including PaLM (a 540B parameter language model), and its own image generation model Imagen, which has outperformed DALLE-2 on the benchmarks of DrawBench and COCO FID, and Imagen can especially generate More realistic effects and spell ability.


Additionally, Google's AlphaGo have also defeated the human world champion at Go while discovering novel strategies and playing techniques never seen in Go's 3,000-year history.


The race to build complex foundational models is on, and Big Tech is at the forefront of innovation. But while the progress in the field is exciting, there remains one key theme that deserves attention. That is, over the past decade, as AI models have become more complex, they have become less and less accessible to the public.


On the other hand, the tech giants are investing heavily in producing such models and integrating data and code Retain as proprietary technology while maintaining its competitive moat through its economies of scale in model training and computation. Generating the underlying model is a resource-intensive process for any third party, with three main bottlenecks: Data, Compute, and Monetization< /b>.


However, we have seen early progress that some Web3 themes are addressing some of these issues.


Dataset production can be aggregated through Web3 ownership


Labeled datasets are critical to building effective models, AI systems learn by generalizing from Continuous improvement over time.


In addition to computing resources, however, high-quality dataset compilation and labeling require dedicated knowledge and processing. Large tech companies often have internal data teams dedicated to using large proprietary data sets and IP systems to train their models, and have little incentive to open up access to the production or distribution of their data.


However, there are a number of model trainings that are being made open and accessible to the global community of researchers, As in the following example:

- Common Crawl is a public repository of ten years of Internet data that can be used for general training. (Although research shows that more precise and leaner datasets can improve the general cross-domain knowledge of the model)


- LAION is a A non-profit organization that aims to make large-scale machine learning models and datasets available to the public, and released LAION5B, which is A dataset of 5.85 billion CLIP-filtered image-text pairs. LAION5B became the world's largest publicly accessible image-text dataset after its release.


- EleutherAI is a decentralized dataset that publishes the largest open source text data The Pile. The Pile is an 825.18 GiB English language dataset for language modeling using 22 different data sources.


Currently, these communities are organized informally and rely on contributions from a wide range of volunteers. To make their contributions more valuable, Token rewards can be used as an incentive mechanism to create open source datasets. For example, tokens can be issued based on specific contributions, and large text image datasets can be labeled, and the DAO community can verify such claims.


Finally, large-scale deep learning models can issue Tokens from a public pool, and based on the The downstream income of model building products can also be captured in the value of Token. In this way, dataset contributors can even hold shares in large models through their tokens.


In addition, well-compiled open-source datasets are useful for scaling up research accessibility and Improving model performance is critical. At the same time, the text image dataset can be expanded by increasing the size and filters of different types of images to obtain more refined results, such as non-English datasets will be needed to train natural language models that can be used by non-English speakers. Over time, we will be able to achieve these results faster and more openly using the Web3 approach.


AI computing will move to the decentralized network


The computation required to train large-scale "neural networks" is one of the biggest bottlenecks in the underlying model. Over the past decade, the computational demands of training AI models have doubled every 3.4 months. During that time, AI models have gone from image recognition to using reinforcement learning algorithms to beating human champions in strategy games to training language models with transformers.


For example, OpenAI's GPT-3 has 175 billion parameters and a training time of 3640 petaFLOPS- day. That would take two weeks on the world's fastest supercomputer, but more than a thousand years on a standard laptop. As model sizes continue to grow, computation remains a bottleneck for the development of this field.


< br>

AI supercomputers require specific hardware, such as graphics processing units (GPUs) or application-specific integrated circuits (ASICs), optimized to perform the mathematical operations required to train "neural networks." Today, most hardware optimized for this type of computing is controlled by a handful of oligopolistic cloud service providers such as Google Cloud, Amazon Web Services, Microsoft Azure, and IBM Cloud.


This is also the next major intersection where we will see decentralization through the public open network Gaining momentum in decentralized computing distribution, decentralized governance can be used to fund and allocate resources to train community-driven projects. Additionally, the decentralized market model is publicly accessible across geographies so that any researcher can access computing resources.


Imagine a bounty system for crowdfunding model training by issuing Tokens, successful crowdfunding Will get priority computing for its models and drive innovation where demand is high. For example, if a DAO has a strong desire to generate Spanish or Hindi GPT models to serve a wider population, research could be focused on this area.


And like Companies like GenSyn are already working to launch protocols to incentivize and coordinate access to alternative, cost-effective, and cloud-based hardware for deep learning computing. Over time, as the frontiers of AI continue to be explored, shared, decentralized global computing networks built using Web3 infrastructure will become more cost-effective, scale and serve us better.


Open Access and Product Harmonization


Large models have become increasingly private over the past few years, as the investment of resources required to produce them has driven projects to become closed source . Thus, the above discussion can be made possible by open source artificial intelligence models.


Take OpenAI as an example. OpenAI was founded in 2015 as a non-profit research laboratory with a mission to produce general-purpose AI, which was in stark contrast to the AI leaders at the time, Google and Facebook.


Intense competition and funding pressures have eroded transparency and open source code over time Ideal, as OpenAI moves to a for-profit model and signs a massive $1 billion business deal with Microsoft. Also, the most recent controversy is the general censorship around their text-image model DALLE-2.


DALLE-2 Prohibition of "gun", "execution", "attack", "Ukraine ” and celebrity imagery, the brutish censorship blocks content such as “LeBron James attacks the rim” or “programmer executes lines of code,” and access to private beta versions of these models has implicit implications for Western users. Geographic bias, which cuts off large segments of the global population from interacting with these models.


This is not how AI should spread: protected and regulated by just a few big tech companies. As with blockchain, new technology should be applied as fairly as possible so that it is not concentrated among the few who can use it. Co-discover the most compelling use cases and build consensus on the fair use of AI through the compounding of advances in the open use of AI across industries, geographies, and communities.


With the Token structure of the general basic model, it will be possible to gather a larger pool of contributors, These contributors can monetize their work while releasing their code open source. Projects like OpenAI, built with open source in mind, had to turn to an independent funding company to compete for talent and resources.


Web3 allows open source projects to be just as financially profitable and is further developed with private Investment-led project competition. Furthermore, innovators building products on top of an open-source model can confidently demonstrate the transparency of building the underlying AI, which can then be rapidly adopted and marketed downstream by a number of novel AI use cases.


In the Web3 space, this includes predictive analysis of smart contract vulnerabilities and , an image generator that can be used to mint NFTs and create metaverse landscapes Digital AI Personality and more.


Conclusion


< /p>

AI is one of the fastest growing technologies today and will have a huge impact on our society as a whole. Today, the space is dominated by large tech companies, as financial investments in talent, data, and computing create significant moats for open source development. Integrating Web3 into AI's infrastructure layer is a critical step in ensuring that AI systems are built in a fair, open, and accessible manner.


We've seen open models quickly take over public innovation in open spaces like Twitter and Hugging Face position, and Web3 can move those efforts forward.


< p>

Original link



Welcome to join the official BlockBeats community:

Telegram Subscription Group: https://t.me/theblockbeats

Telegram Discussion Group: https://t.me/BlockBeats_App

Official Twitter Account: https://twitter.com/BlockBeatsAsia

举报 Correction/Report
Choose Library
Add Library
Cancel
Finish
Add Library
Visible to myself only
Public
Save
Correction/Report
Submit