Alibaba, ByteDance, DeepSeek race to train 5-10 trillion parameter AI models at huge cost

The Return of "Bigger is Better"

At the Alibaba Cloud Computing Conference on September 22, Alibaba reaffirmed its commitment to the "large parameter" approach to AI model development.

Alibaba CEO Eddie Wu disclosed that the company is preparing to train a new generation of AI model with parameters ranging between 5 trillion and 10 trillion. Later that day, Liu Dayiheng, head of the Qwen LLM project at Alibaba's ATH business group's Token Foundry, stated that scaling laws represent the critical path toward Artificial Superintelligence (ASI), with plans for Qwen 4.5 and Qwen 5 to target parameter scales of 5 trillion to 10 trillion.

On the same day, another report circulated in the industry: according to The Information, DeepSeek is currently training a new model with approximately 2 trillion parameters. DeepSeek founder Liang Wenfeng reportedly told investors at a recent meeting that the company's next step is to push the model to 8 trillion parameters.

Two months earlier, in August, the Financial Times reported, citing sources, that ByteDance is training a large model with up to 10 trillion parameters.

Understanding the Scale

Parameters can be understood as a model's "brain capacity"—the total weight of experience it calls upon during inference calculations. To put 10 trillion in perspective: DeepSeek-R1, which ignited the industry in 2025, had only 671 billion parameters—less than one-fifteenth of 10 trillion.

From Alibaba to DeepSeek to ByteDance, Chinese domestic large-model companies are collectively reviving the belief that "bigger is better."

The Other Side of the Scale Race

However, larger parameters come with significantly higher resource and cost requirements, while intelligence does not scale proportionally. As models expand, data must expand accordingly—not just in quantity but also in quality. When everyone is stacking parameters, the true differentiator becomes computational efficiency: how much intelligence can be extracted per unit of computing power.

Current Landscape

Among released models, the domestic peak is held by Moonshot AI's Kimi K3, launched in July, with 2.8 trillion total parameters—the world's first open-source model at the 3 trillion level. Alibaba's current flagship, Qwen 3.8-Max, has approximately 2.4 trillion parameters, while DeepSeek's current V4-Pro stands at 1.6 trillion.

Looking overseas, closed-source models have even higher parameter counts. Industry estimates suggest Anthropic's Mythos 5 has approximately 8 trillion parameters, and Fable 5 around 5 trillion.

This means the parameter ceiling for released domestic models still lags behind estimated overseas leaders by a factor of one to two. Targeting 10 trillion, current flagship models would generally need to scale up approximately fivefold.

Does Bigger Really Mean Better?

The logic supporting "bigger is better" points to the Scaling Law, which suggests that model capability improves when model size, data, and computing power all increase simultaneously.

Over the past few years, each parameter leap—from hundreds of billions to trillions—has been accompanied by visible jumps in capability. Theoretically, larger parameters do mean higher performance. This is why Liu Dayiheng described scaling as Alibaba's key path to achieving ASI.

However, the Scaling Law is not infinitely effective. It requires that as parameters grow, data volume and computing power must keep pace proportionally—and with "high-quality data." At the trillion-parameter level, simply stacking parameters yields clearly diminishing marginal returns. From a practical standpoint, a 10-trillion-parameter model fed with insufficient high-quality corpus will not see proportional capability improvements.

Other variables include data, training methods, architecture, and inference algorithms.

For example, Kimi K3 has 2.8 trillion total parameters but uses a Mixture of Experts (MoE) architecture, with 896 experts but only 16 activated per inference, resulting in approximately 104 billion activated parameters. In practice, the model uses far fewer than its nominal 2.8 trillion.

In real-world usage, large parameters do not necessarily mean "easy to use." Larger models are slower at inference, with noticeably longer user wait times. Inference costs also rise accordingly, ultimately passed on to users through pricing. Even flagship models with just hundreds of billions or 2-3 trillion parameters already carry significant usage costs.

Setting aside usability, the parameter race determines capability ceilings, and ceilings determine rankings. This is why Alibaba, ByteDance, and DeepSeek—despite knowing about diminishing returns and high costs—are still pushing toward 5 to 10 trillion parameters. The large-parameter competition is not about current user experience but about securing tickets to the next generation of models.

The 10-Trillion Bill: Who Can Afford the Cost?

Super models may reach higher intelligence, but larger foundations require more training and deployment resources. That large models are expensive is an unspoken consensus among major companies.

Overseas, Microsoft, Google, Meta, and Amazon have combined capital expenditure guidance for 2026 surging to approximately $720–745 billion.

The domestic market is equally aggressive: Alibaba's single-quarter capital expenditure in Q2 2026 reached 67.7 billion yuan, up 75% year-over-year, with free cash flow swinging from positive 73.8 billion yuan to negative 46.6 billion yuan. ByteDance's 2026 capital expenditure ceiling could reach up to $70 billion—roughly triple the previous year—with potential to hit $100 billion in 2027.

Where Does the Money Go?

Infrastructure is the most direct expense. When OpenAI trained GPT-4, computing demands were measured in trillions of tokens, requiring thousands of GPUs in high-speed interconnected clusters running for weeks or months—and GPT-4 had only 2 trillion parameters.

Pre-training a 10-trillion-parameter model would require tens of thousands of high-end GPUs running continuously for months. Electricity, cluster maintenance, and chip procurement each represent astronomical figures.

Regarding data center investment alone, Wu Yongming announced at the Cloud Computing Conference that Alibaba Cloud plans to operate global data centers exceeding 20GW capacity by 2032. Goldman Sachs previously estimated Alibaba's current online capacity at approximately 3–4GW. This means adding approximately 16–17GW over the next six years—expanding capacity more than fivefold.

According to NVIDIA's Jensen Huang, building 1GW of computing capacity costs approximately $50–60 billion. By this estimate, Alibaba's additional 16–17GW over six years could require total construction investment on the order of $800 billion to $1 trillion.

Inference costs follow. Model training is not the end but the beginning. Larger parameters mean more computing power is called upon for each generated response. When models are deployed across millions of daily user calls, inference-side electricity and computing costs often exceed training costs in both duration and magnitude.

Data represents another often-overlooked hidden cost. As models expand, training data must expand proportionally—not just in quantity but also in quality. Industry consensus holds that high-quality, meticulously cleaned and labeled data is scarcer than raw computing power.

Can the Players Afford It?

Cash flow has already issued warnings. Over the past five quarters, Alibaba's free cash flow was negative for four quarters. The most recent quarter saw capital expenditure of 67.7 billion yuan. The company's approximately 474.5 billion yuan in cash and liquid investments provides some buffer, and last month it raised approximately 80 billion Hong Kong dollars through a Hong Kong stock placement for AI infrastructure.

However, against trillion-dollar-level computing investments, these funds are far from sufficient. Continued external financing appears almost inevitable.

ByteDance faces a similar situation. According to The Information, ByteDance's net profit in the first half of this year fell by a single-digit percentage year-over-year to approximately $20 billion. It is the largest AI capital spender among domestic tech giants, with AI-related spending of 150 billion yuan in 2025, increasing to 200 billion yuan in 2026.

To bridge the gap, ByteDance recently completed approximately $30 billion in syndicated loans—the largest in tech company history—with part of the funds expected to go toward expanding AI infrastructure.

An Unstoppable Arms Race: From Flash to 10 Trillion Parameters

Less than a month before the Cloud Computing Conference, Alibaba had just unveiled Qwen 3.8-Flash-Next: 125 billion total parameters, with only 6 billion activated per inference. Training costs were reduced by nearly 90% compared to the previous generation, and API pricing dropped to as low as one-third of DeepSeek-V4-Flash.

On the same day, Zhipu AI's GLM-5.3-Flash appeared on open-source leaderboards with 320 billion total parameters and 18 billion activated.

During those two weeks, the industry buzzword was "cost revolution": the decisive factor in the model competition was who could be more economical, faster, and drive unit costs lowest in high-frequency usage scenarios.

A month later, the Cloud Computing Conference pushed "big" back to center stage. Two approaches now coexist.

From a model perspective, Flash-type models optimize for "high-frequency, real-time, cost-sensitive" scenarios where users want speed and affordability, hence minimizing activated parameters.

Large-parameter models, by contrast, optimize for the hardest, most complex, and broadest scenarios—requiring the toughest reasoning, longest agent chains, and widest capability coverage.

The technology enabling these two approaches to coexist—and even switch between—is the MoE (Mixture of Experts) architecture. It splits the model into numerous "experts," activating only a subset per inference. Its core contribution is decoupling total parameters from actual inference parameters: total parameters can continue expanding to support capability ceilings, while per-inference computation is contained, not growing proportionally with total scale.

For large-model companies, MoE transforms the cost structure itself: training costs remain controllable, and inference is billed by activated parameters. This allows betting on both "small and fast" and "large and strong" routes using the same foundation.

Both Flash and large-parameter models ultimately answer the same question: how to maximize intelligence output per dollar spent.

However, large parameters remain an aspiration for now. Ten trillion is still a number on PowerPoint slides. The current ceiling for domestic flagship models is 2.8 trillion. By the time such models are actually deployed, the playing field may have already been reshuffled. At that point, 10 trillion will no longer be the finish line—it will be the new starting line.