According to Dynamic Beating Monitor, the Financial Times reported, citing three sources familiar with the matter, that ByteDance has started pre-training a mega-scale model with up to 10 trillion parameters. The project is still in its early stages, and the final scale has not been determined yet. If trained to the upper limit, it would surpass the Kimi K3 by more than three times, becoming the largest known model in terms of parameters among Chinese teams.
This time, ByteDance has directly pushed the model size to near Anthropic's flagship. Anthropic has never disclosed the parameter count of Mythos 5, but the Financial Times, citing industry estimates, said Mythos 5 has about 800 billion parameters, and Fable 5 has about 500 billion. If ByteDance's new model reaches 10 trillion, it has already entered the same order of magnitude as the most advanced closed-source models in the United States in terms of parameter scale.
This model is currently in pre-training (the fundamental ability to train a model using massive data). The Financial Times reported that this stage usually takes 3 to 6 months, after which it will need to continue fine-tuning to be successfully launched. Having more parameters does not necessarily mean a stronger capability; the ultimate performance still depends on the architecture, training data, and training methods.
Recently, Zhang Yiming explicitly opposed distilling competitor models within Seed, demanding that the team accept short-term setbacks and strive to impact the world's top tier through self-developed efforts. This new model with up to 10 trillion parameters marks ByteDance's most aggressive scale bet to date.
