Best of Android 2016: Performance

We know how important performance is when it comes to a high-end Android phone. No one wants to fork out several hundred dollars for a phone and then suffer from lag. And if there’s a phone out there that costs half the price of most mainstream flagships and has better performance we know you’re going to want to know all about it.

In this installment of the Best of Android 2016, we’re tackling the sticky issue of Android performance. Do the best specs on paper always result in the best stats under pressure? Or is software more important? What about RAM? Is more always better or can optimization be the key to heavy workload dominance?

We’ve run 10 of the biggest Android phones of 2016 through a series of popular benchmark tests, all available in Google Play so you can install them too and see how your phone stacks up against the best of the best in 2016. Results in each individual benchmark are ranked and at the end those results are averaged out to declare an overall winner.

At the heart of every smartphone is a System-on-a-Chip (SoC) that contains the CPU, the GPU and a whole bunch of other stuff like a DSP, ISP and the cellular modems. Obviously the choice of SoC will have a significant impact on the overall device performance, however it isn’t the only factor. The speed of the internal flash storage is certainly a factor, especially when it comes to app loading times. Secondly, the screen resolution will also influence performance as the CPU and GPU will need to work harder to maintain smooth animations.

Looking at the table above you can see that the Snapdragon 820/821 is the dominant SoC. The 821 is a slightly tweaked version of the 820 with improved performance and power efficiency. The only device not using a Qualcomm SoC is the HUAWEI Mate 9 which uses HUAWEI’s own Kirin 960. There are two versions of the Samsung Galaxy S7 Edge, one with a Snapdragon 820 (which we used for the results in this test) and another that uses Samsung’s own Exynos 8890 SoC (which scored slightly lower on most benchmarks, hence using the 820 results here) .

AnTuTu

AnTuTu is one of the “standard” benchmarks for Android, it measures both CPU and GPU usage as well as a few other things like RAM bandwidth and I/O throughput. Although all of its workloads are completely artificial, meaning they don’t reflect real world usage, the benchmark is still useful for establishing a baseline about the general performance of a device.

As you can see the OnePlus 3T gets the highest score from AnTuTu followed by the Moto Z Force Droid and the LG V20. Having said that, all the devices performed amazingly for AnTuTu. Last year’s winner was the Galaxy Note 5 with its Exynos 7420 SoC, however it scored less than 70,000. Now all the devices in our test score over 120,000! The Xiaomi Mi 5 was the weakest performer in AnTuTu.

GeekBench 4

GeekBench is another popular Android benchmarking tool, however this app only tests the CPU performance. The test is split into two parts, the single-core tests, which measures the speed of an individual core, regardless of how many cores there are on the SoC; and the multi-core tests, which exercises all the cores on the SoC simultaneously.

The Geekbench king is the HUAWEI Mate 9 with its Kirin 960 SoC. The Kirin 960 uses four ARM Cortex-A73 cores and four ARM Cortex-A53 cores plus the Mali G71 MP8 GPU. The Mate 9 gets the top score for both the single-core tests and the multi-core tests.

Next up is the OnePlus 3T and the Samsung Galaxy S7 Edge. The former does better on the single-core tests but comes third in the multi-core tests. Conversely, the Samsung comes third overall for the single-core tests but second for the multi-core tests. The weakest single-core performance was shown by the Mi 5, with the HTC10 producing the poorest multi-core results.

Basemark OS II

Basemark OS II is an “All-In-One” benchmark that tests the overall performance of a device including system, memory, graphics, and web browsing. As well as these individual scores there is a overall rating calculated (graph below includes Vellamo scores).

The OnePlus 3T comes out on top of the Basemark OS II tests with a score of 2,719. Next comes the Samsung Galaxy S7 Edge with 2,659 and the HUAWEI Mate 9 with 2,588. Interestingly there is only a 5% difference in score among the top three devices.

At the other end of the scale, the Xiaomi Mi 5 was weakest (2.091), followed by the LG V20 (2,188) and HTC10 (2,197) in second and third last places, respectively.

Vellamo

Vellamo is Qualcomm’s own benchmarking utility, which breaks tests down into three main “chapters”: Browser, Multicore and Metal. We ran the Browser test on the default internet app on each device to see how well they handle web content and then ran Metal to evaluate single-core performance of the mobile processor.

For the Metal test of Vellamo the top scoring device is once again the OnePlus 3T with 4,238, the only device to break the 4K barrier. Next comes the ZTE Axon 7, the first time the ZTE has appeared on the podium! In third place is the LG V20. The poorest result in Metal was the Xiaomi Mi 5 which was the only device to not make 3K (interestingly, the Pixel XL only just scraped by with 3,040).

Turning to the Vellamo web browsing tests the top place goes to the Samsung Galaxy S7 Edge with 7,778. In second place comes the HUAWEI Mate 9 and after that the OnePlus 3T. The weakest Vellamo browsing result was from the ZTE Axon 7 (3,756).

Jet Stream

JetStream is a JavaScript benchmark tool that focuses on advanced web applications, performing tests for latency and throughout (sustained peak performance). Jet Stream covers a variety of advanced workloads and programming techniques to avoid the possibility of software tweaks to “game” individual benchmarks and runs 39 different tests to produce a single overall score.

Like the Vellamo web browsing test, the top three phones are the S7 Edge, the Mate 9 and the OnePlus 3T, but this time the ordering is different. First came the HUAWEI Mate 9, then the OnePlus 3T and in third place the Samsung Galaxy S7 Edge. The HTC10 was the worst and the Xiaomi Mi 5 just a little better.

3DMark

3DMark is probably the world’s leading benchmark tool because of its cross-platform nature and because of its emphasis on 3D graphics. The Slingshot Extreme test is designed specifically for the OpenGL ES 3.1 standard. 3DMark not only has rules for manufacturers to avoid OEMs modifying the way the system runs when tests are being performed, but you can compare 3DMark results with over 3,000 Android devices as well as the latest iPhones and iPads.

It is worth re-emphasizing at this point that devices with lower screen resolutions rank better as the GPU has less work to do. In fact QHD displays have 77 percent more pixels than Full HD displays!

As we can see the top result goes to a device with Full HD display: the OnePlus 3T. The ZTE Axon 7, with a QHD display, comes next, followed by the Full HD HUAWEI Mate 9. However, what is even more impressive is that the score for the Axon 7 is only 3.5 percent less than the OnePlus 3T, despite pushing around 77 percent more pixels! The worst 3DMark performance came from the HTC10 by a wide margin.

GFXBench

Like 3DMark, GFXBench is primarily concerned with GPU performance. It includes a new 1440p Manhattan 3.1.1 for OpenGL ES 3.1 test and other assorted benchmarks that test Android Extension Pack features like hardware tessellation on game-like content. We ran the demanding T-Rex and Manhattan 3.1 benchmarks for our tests. Again, be mindful of display resolution differences.

The winner for the T-Rex test is the ZTE Axon 7, in second is the Xiaomi Mi 5 and in third is the Sony Xperia XZ. The Axon 7 has a QHD display but the next two are Full HD. Next up is the Moto Z Force Droid, which is just 2fps slower than the Sony Xperia XZ and only 1fps slower than the OnePlus 3T.

As for the Manhattan test, the top performers are the Sony Xperia XZ and the OnePlus 3T which both scored 32fps. Again, both devices have a Full HD display. Next comes the Pixel XL (30fps with a QHD screen) and in third is the HUAWEI Mate 9 (28fps and Full HD again).

Wrap up

Overall, the best performer came first by quite a wide margin. It came first in AnTuTu, Basemark OS II, 3DMark and GFXBench Manhattan (tied with the Sony). It also managed a top three result for all the other tests with the exception of the GFXBench T-Rex test, where it came fourth. So which Android phone has the best performance in 2016?

The OnePlus 3T

The OnePlus 3T sports the brand-new Snapdragon 821 SoC, combined with 6 GB of RAM and a Full HD display, a killer combination that made it unbeatable where these tests were concerned.

In second place is the HUAWEI Mate 9. It has the highest Geekbench score plus it came in first in the JetStream JavaScript benchmark suite. It also finished in the top three repeatedly across other tests, helped in part by its Full HD display (a feature our top three all share).

In third overall place is the Sony Xperia XZ. While it didn’t win any of the tests outright, it did come in joint first for the GFXBench Manhattan test and scored well for the GFX T-Rex test.

It is also worth mentioning the ZTE Axon 7, which took fourth place overall and was the highest performing device with a QHD display. The Moto Z Force Droid was next, followed by the Pixel XL, both of which also feature QHD displays.

A note on results: Our overall performance scores were calculated by assigning points between 1-10 to each device for each tested benchmark. The best performer scored 10 points, second place 9 points and so on, regardless of how close the margin was. Those category scores were then averaged for the final grading you see above.

There are obviously a few things that can affect how well any given device performs in benchmarks. Screen resolution, CPU, GPU, RAM, firmware, software optimizations and so on can all affect the results.

As you can see above, devices with Full HD displays did better overall due to having fewer pixels to push around but we also had some standout results from QHD devices. If you don’t think 2K is worth the drain on performance and battery life, then the OnePlus 3T is the obvious choice. But if you’re looking for a QHD smartphone with top notch performance, you can’t go past the Axon 7. Interestingly, these are two of the most affordable devices on our list.

Thanks for reading and be sure to check out the other categories in the Best of Android 2016 series.

Credits

Tested by: Gary Sims, Andrew Grush, Nirave Gondhia, John Velasco, Joshua Vergara, Lanh Nguyen
Series Contributors: Rob Triggs, Edgar Cervantes, Kris Carlon
Series Editors: Nirave Gondhia, Bogdan Petrovan, Andrew Grush