Tencent MegaFormer Hy4 Open Source: 770 Billion Total Parameters, 1 Million Contexts

Dynamic Beating AI News Flash: Tencent has released and open-sourced Hy4 preview, a new model using the MoE architecture with a total of 770B parameters, 49B per activation, and an expanded context to 1M. Compared to Hy3 with 295B total parameters, 21B activation parameters, and 256K context, this generation's scale has more than doubled directly, and the long context has also expanded to about 4 times.

This time, Tencent did not just emphasize general benchmarking but specifically strengthened real tasks such as coding, office work, gaming, and research. For example, the model can build a Three.js 3D website from scratch, create playable game demos in Unity, and process 72 financial documents at once, checking for duplicate reimbursements, exceeding limits, and budget anomalies. Hy4 is also trained and iterated with Tencent products like WorkBuddy and CodeBuddy.

A particularly noteworthy point is that Hy4 has begun to participate in its own research and development. Tencent stated that it will help optimize training methods, data strategies, evaluation systems, and low-level operators, propose solutions, run experiments, and then continue to adjust based on the results. The code, logs, and feedback generated from experiments will enter the next round of R&D, forming a preliminary self-improvement cycle.

Hy4 preview has been integrated into WorkBuddy, CodeBuddy, Yuanbao, ima, Tencent Cloud TokenHub, and OpenRouter. Tencent had previously hinted in its financial report that Hy4 would be larger than Hy3 and that real product feedback would be an important source for model training.