Xiaomi first unveils MiMo-V3's new architecture: computing workload for million-token prefill reduced by 80%.

Beating AI News Flash: The Xiaomi MiMo team has unveiled the HySparse2 architecture that will be adopted by the next-generation MiMo-V3. It primarily addresses the issue of Agents becoming increasingly expensive to run. Agents only issue very short instructions each time, yet may retrieve large chunks of content from web pages, terminals, and tools. The model must first read through all this new content; the longer the context, the more compute this step consumes, and the more VRAM the KV Cache occupies.

HySparse2 first splits the model into two segments. The front half handles input processing, while the back half continues reasoning and generation. The context information needed by the back half can be directly generated from the results already computed by the front half, so prefill no longer needs to run the back half in full. This idea comes from YOCO, proposed by Microsoft Research in 2024, whose name stands for "You Only Cache Once." The core is to let the back half share the information already computed by the front, storing less cache and performing fewer redundant computations.

It also redesigns sparse attention. Ordinary full attention must look at the complete context every time, whereas HySparse2 only lets a small number of layers do so. They first select the 1024 most relevant tokens from the long context, and subsequent layers directly reuse this batch of results while always retaining the most recent 128 tokens. The previous generation HySparse grouped every 64 tokens into a block before selecting, so as long as a block contained important information, the entire block had to be kept. Now it has been changed to token-by-token selection, so the same amount of computation can be devoted to more genuinely relevant content. After testing this change in isolation, two long-text retrieval scores improved by 6.57 points and 8.14 points respectively.

The paper uses a model with 80 billion total parameters and about 3 billion activated parameters per pass as a comparison. The 49-layer model only needs to run the first 25 layers during the prefill stage. When input reaches 1 million tokens, compared with the hybrid sliding window attention adopted by the MiMo-V2 series, prefill compute drops to about 1/5, and KV Cache falls from 12.09GB to 2.69GB.