ByteDance Trains 10 Trillion-Parameter AI Model

ByteDance, the Chinese technology group that owns TikTok, is reportedly training an artificial intelligence model with as many as 10 trillion parameters. This scale could approach Anthropic's most advanced Mythos system.

The parameter count of an AI model refers to the numerical settings it learns from training data to recognize patterns, generate responses, and carry out tasks. At 10 trillion parameters, ByteDance's model would be more than three times the size of Moonshot AI's Kimi K3, a 2.8 trillion-parameter model.

Industry estimates put Anthropic's Mythos 5 at around 8 trillion parameters, with its more widely available sibling Fable 5 estimated at about 5 trillion parameters. However, Anthropic does not officially disclose parameter counts for its models, so these figures remain industry estimates.

The ByteDance model is currently in the pre-training phase, a process that typically takes three to six months. After this phase, it would move to fine-tuning before any potential public release. The exact final parameter count has not been locked in and could change before training concludes.

This development comes days after ByteDance founder Zhang Yiming reportedly told employees to avoid relying on AI distillation techniques purely to chase short-term gains. This follows allegations from a senior US official that Moonshot AI had distilled Anthropic's Fable model while building Kimi K3.

Mythos is Anthropic's frontier model class, positioned above its Opus tier, and has drawn attention for strong agentic coding, reasoning, and cybersecurity capabilities. Access to Mythos has largely been restricted to trusted partners under Anthropic's Project Glasswing, given concerns about misuse in hacking and vulnerability exploitation.

ByteDance's push comes as Chinese technology firms accelerate their model release cycles to keep pace with US rivals in an increasingly expensive race to build larger and more capable large language models. Several Chinese labs are already working on models in the roughly 5 trillion-parameter range associated with Fable, with ByteDance now positioning itself as one of the most ambitious, aiming closer to the scale of Mythos.

The development of large language models is a rapidly evolving field, with companies competing to build the most advanced models. The race to build larger and more capable models is driven by the potential applications of these models in various industries, including technology, healthcare, and finance.

In the context of the global AI landscape, ByteDance's push to develop a 10 trillion-parameter model is significant. It highlights the company's ambition to compete with US rivals in the AI space and its commitment to investing in research and development.

The implications of this development are far-reaching, with potential applications in various industries. As the AI landscape continues to evolve, it will be interesting to see how ByteDance's model compares to other models in the market and how it will be used in real-world applications.

In conclusion, ByteDance's decision to train a 10 trillion-parameter AI model is a significant development in the AI space. It highlights the company's ambition to compete with US rivals and its commitment to investing in research and development. As the AI landscape continues to evolve, it will be interesting to see how this model compares to other models in the market and how it will be used in real-world applications.