HUAWEI and HONOR bend the rules to catch up to Qualcomm

Update, September 5, 2018 (10:15AM): HUAWEI has reached out to Android Authority with a comment on the matter:

HUAWEI always prioritizes the user experience rather than pursuing high benchmark scores – especially since there isn’t a direct connection between smartphone benchmarks and user experiences. HUAWEI smartphones use advanced technologies such as AI to optimize the performance of hardware, including the CPU, GPU and NPU.

When someone launches a photography app or plays a graphically-intensive game, HUAWEI’s intelligent software creates a smooth and stable user experience by applying the full capabilities of the hardware, while simultaneously managing the device’s temperature and power efficiency. For applications that aren’t as power intensive like browsing the web, it will only allocate the resources necessary to deliver the performance that’s needed.

In normal benchmarking scenarios, once HUAWEI’s software recognizes a benchmarking application, it intelligently adapts to “Performance Mode” and delivers optimum performance. HUAWEI is planning to provide users with access to “Performance Mode” so they can use the maximum power of their device when they need to.

HUAWEI – as the industry leader – is willing to work with partners to find the best benchmarking standards that can accurately evaluate the user experience.

Original article, September 5, 2018 (07:09AM): The HONOR Play bills itself as a major gaming device, but the handset might not have the chops to make good on all those promises. Thorough research by friends over at AnandTech caught the company cheating on benchmarks and pushing GPU performance well past the recommended power limits for sustainable smartphone performance and battery life.

The article is well worth a read if you want the fine details, but the gist is as follows:

• HONOR and HUAWEI phones boost performance when they detect benchmarking applications.
• HONOR blames unscrupulous Chinese rivals for also cheating on benchmarks.
• Higher performance is achieved by ignoring thermal design power (TPD) recommendations for smartphones and allowing the chip to run at full tilt. This results in unsustainable performance and much higher power consumption.
• The bottom line is HONOR’s and HUAWEI’s benchmark scores won’t reflect real-world performance, and the actual results put them well behind their rivals.

The benchmark cheating scandal will feel very familiar to industry followers who remember what happened in 2013. The topic reared its head again as recently as last year, with OnePlus and Meizu in the spotlight. Benchmark cheating is unfortunately rather common it seems, so it’s best not to rely too much on them, or at least only pay attention to benchmark tests at least somewhat relatable to real-world usage.

Comments from HONOR actually indicate the company would rather see “real-world” tests take precedent over opaque benchmark scores, though it didn’t offer any solutions. Benchmark cheating isn’t the most revealing thing about this news, it also points to a deeper head-to-head going on inside some of our smartphones.

Mali vs Adreno

Actual graphics performance scores place recent Kirin 970-powered HONOR and HUAWEI devices behind their competitors, which predominantly utilize Qualcomm’s Snapdragon 845 SoC. This isn’t an issue limited to HUAWEI phones. We’ve also previously highlighted similar performance discrepancies with Samsung’s Exynos processors too, despite their inclusion of more Mali GPU cores than HUAWEI’s Kirin chips.

Qualcomm’s latest Adreno graphics architecture is just more power efficient than Arm’s Bifrost Mali GPUs. This is why we see not only higher scores, but also a more sustainable performance from recent Qualcomm-powered phones versus handsets equipped with Kirin and Exynos SoCs.

The balance of power doesn’t look set to change drastically with the arrival of Arm’s Mali-G76 GPUs either. I’m expecting the new Kirin 980, which features a 10-core Mali-G76, to place around the performance levels of Qualcomm’s current Adreno 630. This is despite some very notable performance and even higher energy efficiency improvements over the Kirin 970. Even if the chip can take a lead, the next generation Adreno GPU arriving later this year will likely keep Qualcomm out in front throughout 2019.

To help make up this gap, HUAWEI has turned to its GPU Turbo technology. The company is also offering gamers the option to push performance higher still, at the expense of battery life, through its “Game acceleration” mode.

GPU Turbo and Game acceleration

During the unveiling of the Kirin 980, HUAWEI acknowledged that its latest chip will only push past the Snapdragon 845 when utilizing the company’s GPU Turbo technology. GPU Turbo is a software level solution that optimizes SoC voltages, clock speeds, and power consumption on a per title basis.

HUAWEI achieves this using neural network training to create a DVFS profile that loads up for the title. In other words, it trains a neural network to figure out if a game requires a higher CPU frequency or more power dedicated to the GPU to perform optimally on its hardware. This means the technology is limited to just the few games HUAWEI works closely with. Even then, the benefits only offer performance and efficiency gains in the region of 10 to 15 percent.

Game acceleration is a different beast. It affects all titles played through HUAWEI’s Game Suite application, boosting the phone’s performance at the expense of battery life. Throwing out some regard for TDP limits will obviously help HUAWEI close the gap further, but phones will struggle with heat, battery life, and sustained performance. In this sense, throwing full unthrottled performance at benchmarks isn’t so far away from what HUAWEI is offering its actual customers, though that’s hardly an excuse for being misleading.

HUAWEI has to find alternatives to bridge the hardware gap with Qualcomm’s Adreno GPU. This involves both new neural networking-based software optimizations and pushing energy consumption above what’s typical in a smartphone. The Kirin 980 will help HUAWEI a lot in this regard, but pressure is surely mounting on Arm to provide its customers with graphics components that can match the growing demand for high-performance gaming.