HUAWEI phones delisted from 3DMark (Update: Statement from HUAWEI)

Update, September 7, 2018 (02:21PM EST): Earlier this week, 3DMark removed several HUAWEI/HONOR listings from its benchmark lists. The removal was the result of discovering that the HUAWEI devices in question pushed the handset processors beyond their limits in order to artificially inflate “normal” usage scores. You can read the original article below to get further details on the scandal.

Today, HUAWEI posted a response to the delisting of its devices. While the company does not admit any wrongdoing in the matter, it does concede that it will give users the option to access the so-called “Performance Mode” that enabled its benchmark scores to be so high. By giving users access to this special mode, the very-high benchmark scores would likely be within the benchmarking rules established by 3DMark — and thus, get those HUAWEI/HONOR devices relisted.

You can read HUAWEI’s statement in full below:

HUAWEI and UL (creators of 3DMark) have held comprehensive discussions on benchmarking practices this week, and have reached a positive agreement on the next steps in working together.

In the discussion, HUAWEI explained that its smartphones use an artificial intelligent resource scheduling mechanism. Because different scenarios have different resource needs, the latest HUAWEI handsets leverage innovative technologies such as artificial intelligence to optimize resource allocation in a way so that the hardware can demonstrate its capabilities to the fullest extent, while fulfilling user demands across all scenarios.

UL understands the intent of HUAWEI’s approach, but is opposed to forcing the use of a “Performance Mode” by default when a benchmarking application is detected by the device. UL rules require a device to run the benchmark as if it were any other application.

HUAWEI respects consumers’ right to choose what to do with their devices. Therefore, HUAWEI will provide users with open access to “Performance Mode” in EMUI 9.0, so that the user can choose when to use the maximum power of their device.

HUAWEI and UL have also discussed current common benchmark testing methodologies in general. UL and HUAWEI would like to participate in an industry movement to develop benchmarking standards that best serve the needs of manufacturers, press, and consumers.

To prevent confusion around current benchmarking results, after discussion, UL and HUAWEI have temporarily delisted the benchmark scores of a range of HUAWEI devices, and will reinstate them after HUAWEI grants all users of HUAWEI handsets access to the Performance Mode.

Original Article, September 6, 2018 (12:18PM EST): Today, several Huawei phones (including one Honor device) were delisted from smartphone benchmarking hub 3DMark.

The delisting of the devices is in response to an exposé-style article published earlier this week by AnandTech. In the article, AnandTech suggests that HUAWEI artificially inflates benchmark scores through software trickery in an attempt to paint HUAWEI devices as superior to other brands’ devices.

The three HUAWEI devices that are now delisted are the HUAWEI P20, the HUAWEI P20 Pro, and the HUAWEI Nova 3. The brand new HONOR Play is also delisted.

3DMark has now decided to delist the HUAWEI devices after reading AnandTech’s article as well as conducting its own investigation into the previously-published scores.

With these devices now delisted, they will appear unranked and without scores at the bottom of 3DMark’s popular smartphone rankings.

According to AnandTech (and now 3DMark), HUAWEI has software on its devices that detects when benchmarking applications are running. During the benchmark test, the device will push all processing power to the max, ignoring things like thermal design power (TPD) recommendations. What results is a very high benchmark score that wouldn’t work long-term in real-world situations.

3DMark points to the fact that it has very clear rules on how benchmarking scores should be obtained, and that HUAWEI violates those rules with this processor-pushing software.

You can see how wildly different the benchmark scores are below:

Android Authority reported on the AnandTech article yesterday and received a statement from HUAWEI in response. In the statement, HUAWEI defends its practices and says, “HUAWEI – as the industry leader – is willing to work with partners to find the best benchmarking standards that can accurately evaluate the user experience.”

This is not the first time a company has been caught gaming the benchmarking system. In 2013, Samsung was called out for its benchmark cheating, and both OnePlus and Meizu were caught running similar tactics in 2017.

NEXT: Fact check: HUAWEI flaunts Kirin 980 gains over Snapdragon 845, but are they meaningful?