Forget the LLM leaderboard: Does the AI actually work?

For years, the AI industry has worshipped at the altar of benchmark scores. MMLU, HumanEval, SWE-bench – a few points higher on any of these could make or break a model's reputation. But at World AI Conference (WAIC) 2026 in Shanghai, a growing chorus of Chinese AI players is pushing back against the assumption that standardized tests tell the whole story.

"The benchmark race is not the whole point," said Yang Minghui from StepFun, the company behind the popular Step series models. "We care far less about leaderboard scores and much more about whether our model actually works well across our own products – phones, cars and robots. The real test is just using it."

StepFun, which showcased its agent operating system STEPX Neo at WAIC 2026, argues that benchmark homogeneity is especially problematic for companies like theirs that run multiple vertical-specific models. "Each of our models excels in different domains – no single benchmark can capture that diversity," Yang added.

MiniMax's Bai Chuanxu made the case from a different angle. The Shanghai-based firm's flagship M3 – a natively multimodal model with a million context and frontier coding capabilities – competes efficiently despite a smaller parameter count compared to some rivals. "Our model supports images, video and audio natively," Bai explained. "But benchmark scores often fail to reflect that and can even create misleading impressions.”

"With fewer parameters, we require less hardware. That makes us a cost-effective choice, but that advantage doesn't show up on most leaderboards," he told CGTN.

Web giant Baidu, whose universal agent "DuMate" was named a WAIC 2026 official "treasure of the hall," also sees limits in conventional evaluation. Li Jingqiu, product lead for DuMate, put it differently. "The best way to judge a model today is to give it a complex, time-consuming task. Even an ordinary person can tell the difference in quality," she told CGTN in an exclusive interview. Baidu founder Robin Li said the industry should consider daily active agents as a score, just like daily active users in the age of web.

These critiques come as the industry confronts a well-documented problem: benchmark contamination and saturation. As evaluation platform Singularity Moments recently noted, "benchmark contamination is a persistent issue" and "differences within a few points [on MMLU] are often meaningless" given how easily top models can memorize test data. Industry analyst site 80AJ observed that "static benchmarks are rapidly losing value due to data leakage and overfitting," forcing a shift toward human-preference arenas like Chatbot Arena.

What unites these Chinese voices is a pragmatic turn: AI is not a monolith. There's no one-size-fits-all score. When models are deployed in real products serving millions of users, the metric that matters is no longer a synthetic score – it is whether the model actually delivers.