header-langage
简体中文
繁體中文
English
Tiếng Việt
한국어
日本語
ภาษาไทย
Türkçe
Scan to Download the APP
Partnered With:

Milestone in Domestic Computing Power: Feitian Ascend 910C Successfully Runs Full Training of DeepSeek's 1.6 Trillion Parameter Model

According to Dongcha Beating monitoring, a joint research team composed of Shenzhen Graduate School, Harbin Institute of Technology (Shenzhen), Shenzhen Big Data Research Institute, and a Huawei-related team, in collaboration with the Shenzhen AI Computing Power Platform, announced the successful full-parameter post-training of the 1.6 trillion-parameter large model DeepSeek-V4-Pro on a domestic AI computing power platform.

This marks the first time a third-party organization globally has completed full-parameter post-training of a 1.6 trillion-parameter model on a domestic computing power platform.

Compared to pre-training from scratch, the post-training stage (mainly including supervised fine-tuning SFT and reinforcement learning RL) focuses on aligning the model with high-quality instructions and human preferences, teaching the model to follow instructions and perform specific tasks. However, for a 1.6 trillion-parameter MoE architecture model, full-parameter post-training still places very stringent requirements on underlying hardware such as GPU memory capacity, inter-card communication bandwidth (such as the all-to-all communication triggered by MoE routing), and the stability of large-scale clusters. Leveraging the Huawei Ascend 910C computing power cluster with over a thousand chips, the joint research team successfully overcame communication bottlenecks through optimized distributed load-bearing and load balancing strategies. During the training process lasting over 1500 steps, the system did not encounter any interruptions, the model's computing power utilization rate (MFU) exceeded 30%, the key operator efficiency improved by 14%, and all indicators met industrial operational standards.

Industry analysis points out that the successful operation of the Huawei Ascend 910C cluster in training trillion-scale models confirms the technical feasibility of domestic AI chips in handling ultra-large-scale model deep training tasks. Prior to this, the development of large models heavily relied on NVIDIA GPU clusters, and domestic computing power mainly supported inference or small parameter fine-tuning tasks. The success of this joint research marks the acceleration of the domestic computing power ecosystem's transition from "inference-only support" to "carrying out full-parameter training of ultra-large models."

举报 Correction/Report
RelatedRecommended
Technology should never be an exclusive luxury.
A16Z founder believes DeepSeek R1 is one of the most amazing and impressive breakthroughs he has ever seen, and as an open-source project, it is a profoundly meaningful gift to the world.
By lowering the barriers to training and inference, more domestic storage, chip, and networking vendors are brought into the competition for AI infrastructure
If all goes well, as early as the end of this year.
Correction/Report
Submit
Add Library
Visible to myself only
Public
Save
Choose Library
Add Library
Cancel
Finish