Microsoft Completes Pre-training of 2.8 Trillion Parameter Model: Stronger Than GPT-5.6 Base, Using Only 1/4 the Compute

动察 Beating AI News Flash: Microsoft announced the completion of pretraining for its new-generation in-house model MAI-Base-2. The model has a total of 2.8 trillion parameters and adopts a MoE mixture-of-experts architecture, activating only 80 billion parameters at a time.

From August to September this year, Microsoft used 16,000 NVIDIA GB200 GPUs, trained for 38 days, and processed approximately 50 trillion tokens. The new model also introduced native visual pretraining for the first time, with visual data accounting for 18% of the training volume.

Microsoft's published internal tests show that MAI-Base-2 surpasses models such as DeepSeek-V4-Pro-Base, Kimi K3, and Qwen3.8 on text prediction metrics for code, question answering, mathematics, and STEM subjects.

Microsoft also claims that if it only wanted to train a model at the same level as DeepSeek-V4-Pro-Base, its method would require only about 1/4 to 1/12 of the compute. But Microsoft continued to invest more compute to make the model larger and stronger. MAI-Base-2's actual training compute consumption is still higher than DeepSeek's.

Microsoft AI chief Mustafa Suleyman said at an internal meeting on October 15 that MAI-Base-2's pretrained base is already stronger than the OpenAI model supporting GPT-5.6 Sol, while using only one quarter of the latter's training compute.

Microsoft improved MoE expert scheduling, the Muon optimizer, and data filtering methods to reduce VRAM waste and improve the stability of large-scale training. Microsoft also emphasized that all training data was processed by itself, and that it did not acquire capabilities by distilling other companies' models.

MAI-Base-2 is currently still in the post-training stage. Microsoft plans to first use it in its own products, and then license the model weights to enterprise partners through Foundry.