Skip to the yard
Waxsealai

Alibaba Qwen4-Max Uses 2.4 Trillion-Parameter Backbone and Lower-Token Inference

Shipped
Shipper
Maria Deutscher
Readers
403

Qwen4-Max combines four-channel training shortcuts and selective attention; Alibaba reports lower token use on internal and coding benchmarks.

Alibaba Qwen4-Max Uses 2.4 Trillion-Parameter Backbone and Lower-Token Inference

Qwen4-Max’s main practical claim is efficiency: Alibaba says its newest large language model delivers higher-quality output than Qwen3.8-Max while using fewer tokens on two evaluations. The company attributes those gains to its Qwen4 architecture, changes to training and attention, and a large model backbone that activates only part of its parameter set at a time.

The scale is substantial. Qwen4-Max’s backbone contains 2.4 trillion parameters, but it activates 71 billion for any given computation. Alibaba also describes 328 billion N-gram parameters as a cache that stores results of frequently repeated calculations. Reusing those results can avoid doing the same work again, though the provided figures alone don’t show how much that cache contributes to end-to-end speed or cost.

Training changes target efficiency and quality

Alibaba’s training explanation centers on gradients, the signals used to adjust a model’s parameters as it learns. In a conventional neural network, those signals pass through layers to refine their outputs. Residual connections provide shortcuts across layers, helping information and gradients travel through the network along alternate paths.

Residuals are already used in many language models. Alibaba says Qwen4-Max changes the arrangement by using four gradient channels instead of one, with a gate that can adjust how gradients move through them. The intended result is fewer training errors and, in turn, lower training costs and better output. Those are the design goals described for the mechanism; no separate training-cost figure or controlled comparison for this component is provided.

The four-channel design is therefore best read as an architectural choice aimed at the training process, rather than as a standalone performance result. It may matter most to teams building or training models at scale, where training efficiency can affect the resources required to produce a capable system. For users accessing the finished model, the available comparison focuses instead on task scores and token consumption.

Selective attention addresses long prompts

Qwen4-Max also changes how it processes prompts. Its attention mechanism includes Gated DeltaNet, which Alibaba says reduces the hardware required to handle large prompts, and QSA, which skips data the model doesn’t need to answer a particular request. Both are intended to reduce infrastructure demands during inference, when the model is generating responses.

These methods address a recurring cost in language model deployment: a prompt’s length and the information it contains can shape the computation required to respond. Selectively processing relevant information can help limit that work. But the description doesn’t provide hardware comparisons, latency measurements or prompt-length thresholds, so it’s not possible to quantify the savings or determine which workloads benefit most.

For developers assessing the model, token use is one useful measure, but it isn’t a complete substitute for cost or speed data. Fewer generated tokens can reduce the volume of output a system handles, yet actual infrastructure costs also depend on pricing, hardware and the details of a workload. Alibaba’s reported results give a reason to investigate Qwen4-Max for tasks where its quality holds up while its token use falls; they don’t establish a universal cost advantage.

Reported benchmark gains need workload context

Alibaba compared Qwen4-Max with Qwen3.8-Max on an internal benchmark covering knowledge-work tasks. The company reports that Qwen4-Max scored higher and used up to 35% fewer tokens. On Terminal-Bench 4.0, a suite of coding challenges, the token reduction reached 49%.

Those figures point to a potentially meaningful reduction in output volume, especially on the coding suite. Still, the results are reported by Alibaba, and the source provides no raw scores, task-by-task breakdown, evaluation setup or independent replication. “Up to” also describes a maximum reported saving rather than a result guaranteed across every task. Teams should treat the figures as an initial comparison and check the model against their own prompts, quality criteria and usage patterns.

Qwen4-Max is available through Alibaba’s cloud platform. Alibaba also plans to open-source a collection of smaller models built with the same technologies, though no release timing is given. It is open-sourcing Qwen-Live-Delivery-Plugin, an extension designed for lengthy tasks: the model can return an early draft, keep improving it in stages and incorporate user feedback during the process.

That plugin changes how users interact with a long-running task by making intermediate work available before the final result. It could help people steer an output while it is still being developed, although the description doesn’t specify which tasks it supports or how its performance is measured. For Qwen4-Max, the next useful evidence will be independent benchmark results and practical measurements of quality, latency and resource use across real workloads.

Source: Maria Deutscher · siliconangle.com

Same vesselRelated cargo