Meituan Open Sources Trillion-Parameter Large Model LongCat-2.0, Powered by Domestic Chips

According to Dongzha Beating monitoring, Meituan has open-sourced the large-scale hybrid expert (MoE) model LongCat-2.0. The model has a total of 1.6 trillion parameters, with about 480 billion parameters activated per token, supporting 1M ultra-long context.

This is the industry's first trillion-parameter large model that relies on domestic computing power to complete the entire process of training and inference. It has completed pre-training on 35 trillion tokens on a cluster of over 50,000 domestic AI chips, successfully demonstrating the engineering stability of domestic computing power for cutting-edge large models.

The core updates of LongCat-2.0 focus on long context and inference efficiency. LongCat Sparse Attention (LSA) addresses the memory read and compute overhead caused by sparse attention indexes by introducing flow-aware indexes, cross-layer indexes, and hierarchical indexes, making the index reading during long-text inference more continuous and allowing for the reuse of partial index results between adjacent layers.

The model also integrates a 5-gram embedding module with 135 billion parameters, expanding the embedding space by modeling combinations of adjacent tokens to enhance local context representation. Compared to relying solely on MoE expert routing, such pre-embedding can reduce some memory read-write pressure in large-batch inference.

In mainstream evaluation benchmarks such as SWE-bench Pro, LongCat-2.0 performs close to or even surpasses some mainstream closed-source models.