Partnered With:

In 2023, what made MosaicML sell for $1.3 billion in the AI industry's largest acquisition?

Read this article in 16 Minutes

What has supported MosaicML's high valuation in just two years since its establishment with over 60 employees?

Original Title: "Why did MosaicML, worth $21 million per employee after only 2 years, sell for $1.3 billion?"

Original Source: R3PO

Original Author: elizeliu

Recently, there has been a wave of investment and acquisitions in the AI industry. Global renowned enterprise Salesforce invested $450 million in Anthropic, while Runway successfully raised $141 million in funding. In addition, Snowflake announced the completion of its acquisition of Neeva, and Chinese domestic giant Meituan acquired AI company Lightyear for $2.065 billion.

However, the most eye-catching deal is undoubtedly the acquisition of the startup MosaicML. It is reported that MosaicML was acquired by big data giant Databricks for about $1.3 billion, and its valuation increased sixfold in this transaction, making it the largest acquisition in the first half of this year. What supported MosaicML's high valuation in just two years with over 60 employees?

Databricks acquires MosaicML to accelerate democratization of generative AI technology

Databricks recently announced its acquisition of MosaicML, a generative AI startup, for approximately $1.3 billion USD (approximately 930 million RMB). The acquisition aims to provide services for enterprises to build ChatGPT-like tools.

After the acquisition, MosaicML will become part of the Databricks Lakehouse platform, and the entire team and technology of MosaicML will be incorporated into Databricks, providing enterprises with a unified platform to manage data assets and enabling them to build, own, and protect their proprietary generative AI models using their own proprietary data.

MosaicML is a very young generative AI company, founded in San Francisco in 2021. It has only disclosed one round of financing and has only 62 employees. In the last round of financing, its valuation was $220 million, which means that the valuation of the acquisition of MosaicML has directly increased sixfold. This transaction is the largest publicly announced acquisition in the generative AI field so far this year. Just recently, cloud computing giant Snowflake announced the acquisition of another generative AI company, Neeva. After experiencing several months of investment fever, it seems that large enterprises are starting a wave of massive acquisitions of generative AI startups.

Databricks originated from UC Berkeley and was involved in the development of the Apache Spark project. As a data storage and analysis giant, it is valued at $31 billion as of 2022 and helps large companies such as AT&T, Shell, and Walgreens process data. Recently, it open-sourced its large model Dolly, which aims to achieve similar effects to ChatGPT with fewer parameters. After cloud computing became more popular, Spark's "lakehouse" concept deeply influenced a group of big data startups. Since its establishment in 2013, Databricks has rapidly grown into the world's hottest Data Infra company. Last year, Databricks announced annual revenue of over $1 billion, and after completing its latest round of financing in August 2021, its latest valuation reached $38 billion.

MosaicML MPT Series Model Advantages

MosaicML's MPT series models are subclassed from the HuggingFace PretrainedModel base class and fully compatible with the HuggingFace ecosystem. The MPT-7B model is one of MosaicML's most popular models, with billions of parameters and the ability to handle over 2,000 natural language processing tasks. The optimization layers of MPT-7B include FlashAttention and low-precision layer norms, which can make the model 2-7 times faster than traditional training methods, and the near-linear scalability of resources ensures that models with billions of parameters can be trained in a matter of hours rather than days. MosaicML has also released a new commercially available open source large language model, MPT-30B, with 30 billion parameters and better performance than GPT-3.

Data source: MT-Bench evaluation of mainstream models for MosaicML

The advantage of the MPT series models lies in their efficiency and low cost. The complexity of artificial intelligence models trained with large amounts of data has sharply increased, and it now costs at least millions of dollars to train a model. Except for large companies, other small and medium-sized enterprises generally cannot afford it. However, MosaicML's MPT series models can allow enterprises to train their own language models at lower costs and higher efficiency, making it easier to apply generative AI technology and achieve better business performance. Most open source language models can only handle sequences with up to a few thousand tokens (see Figure 1). However, with the MosaicML platform and a single node of 8xA100-40GB, users can easily fine-tune MPT-7B to handle up to 65k context lengths. The ability to adapt to such extreme context lengths comes from ALiBi, which is one of the key architecture choices in MPT-7B.

For example, the full text of "The Great Gatsby" is less than 68k tokens. In a test, the StoryWriter model read "The Great Gatsby" and generated an ending. One of the endings generated by the model is shown in Figure 2. StoryWriter read "The Great Gatsby" in about 20 seconds (about 150,000 words per minute). Due to the long sequence length, its "typing" speed is slower than other MPT-7B models, at about 105 words per minute. Although StoryWriter's context length was fine-tuned to 65k, ALiBi enabled the model to infer longer inputs than the training data: up to 84k tokens in the case of "The Great Gatsby" in the test.

Figure 2: MPT-7B-StoryWriter-65k+ wrote an epilogue for "The Great Gatsby". The result of the epilogue is to provide the full text of "The Great Gatsby" (approximately 68k tokens) as input to the model, followed by the word "epilogue", allowing the model to continue generating.

Popularization of Generative AI Technology

Generative AI technology is a branch of artificial intelligence that uses large amounts of data and deep learning algorithms to automatically generate original content such as text, images, and computer code. This technology allows for more convenient data processing and analysis, better serving human needs. With the rapid development of big data and AI technology, generative AI has been widely applied in fields such as natural language processing, image recognition, and virtual reality. For example, in the field of natural language processing, GPT-4 has become one of the most popular generative AI models, capable of generating articles, translating languages, and answering questions. In the field of image recognition, StyleGAN2 can generate high-quality images, which can be used in game development, film production, and virtual reality.

MosaicML's CEO Naveen Rao previously stated that since 2018, the complexity of artificial intelligence models trained with large amounts of data has sharply increased, and it now costs millions of dollars to train a model. Except for large companies, other small and medium-sized enterprises generally cannot afford it. After this acquisition, the joint product of Databricks' Lakehouse platform and MosaicML technology will enable enterprises to use their proprietary data to train and build generative AI models simply, quickly, and at low cost, allowing users to have control and ownership of their data and customize AI model development. According to Databricks, with the support of their platform and technology, the cost of training and using LLMs for enterprises will be significantly reduced, expected to be around a few thousand dollars. This provides convenience for the popularization of generative AI.

Databricks Acquiring MosaicML: What It Means

The main purpose of Databricks' acquisition of MosaicML is to accelerate the development and democratization of generative AI technology. By integrating the technologies and resources of both companies, Databricks can better meet the needs of its customers and provide more efficient and convenient solutions. Specifically, this acquisition will bring about the following changes:

1. More efficient large language models

After Databricks' acquisition of MosaicML, it can integrate the MPT series models into its Lakehouse platform, providing customers with more efficient and cost-effective large language models. This will help enterprises better handle natural language processing tasks, improve business efficiency and accuracy.

2. Faster Model Training Speed

MosaicML's MPT series models have the feature of fast training, which will help Databricks provide faster model training services. This is particularly important for enterprises that need to quickly respond to market demands, as it can help them better meet customer needs.

3. Higher degree of democratization

The acquisition of MosaicML by Databricks also means that the democratization of generative AI technology will be further enhanced. MosaicML's MPT series models can make it easier for small and medium-sized enterprises to train their own language models, thereby better applying generative AI technology and achieving better business performance. This will help promote the development and application of generative AI technology, and promote the popularization and development of artificial intelligence technology.

Summary

Generative artificial intelligence applications aim to generate raw text, images, and computer code based on natural language prompts from users. Since the launch of the AI chatbot ChatGPT by artificial intelligence startup OpenAI in November last year, interest in this technology has skyrocketed. "Every organization should be able to benefit from the AI revolution and have more control over how they use their data. Databricks and MosaicML have an incredible opportunity to democratize AI and make Lakehouse the best place to build generative artificial intelligence," said Ali Ghodsi, co-founder and CEO of Databricks.

The significance of Databricks' acquisition of MosaicML lies not only in accelerating the development and democratization of generative AI technology, but also in integrating the technologies and resources of the two companies to provide customers with more efficient and convenient solutions. With the rapid development and application of artificial intelligence technology, generative AI technology will play an increasingly important role, and Databricks' acquisition of MosaicML reflects the importance and investment of various enterprises in this direction. Companies like Anthropic and OpenAI license their existing language models to enterprises, which then build generative AI applications on top of them. Driven by strong commercial demand for these models, opportunities are created for startups like MosaicML. From the successive acquisitions of Snowflake and Databricks, we can see that large technology companies are gradually moving from independent research and strategic investment towards mergers and acquisitions in generative AI technology.