What is benchmarking and why does it matter? Everything you need to know

If you’ve ever read a product review here on Android Authority or elsewhere, you may have heard the term benchmarking thrown around. But what does it mean? Put simply, benchmarks allow us to measure how well a particular product or service performs in the real world. It’s almost the opposite of a manufacturer’s claims and spec sheets, which are based on their own internal testing. Benchmarks help you make an informed purchase decision so it’s important to understand how they work.

To that end, let’s explore the concept and understand what factors you should look out for when reading or performing your own benchmarks.

What is benchmarking and how does it work?

Benchmarking refers to the testing of a device’s speed, power consumption, battery life, and other factors. But collecting data only represents one aspect of benchmarking. The other involves comparing results between competitors, allowing you to gain a better understanding of how one chip performs versus the rest.

The term benchmarking comes from the world of business, where it refers to the process of comparing one business’ performance to another. Over the past few decades, the term has been adapted to fit the tech industry and is now used whenever we measure the performance of both, hardware and software.

You’ll find almost every kind of tech product can be benchmarked, often in multiple ways. One of the most common types is the CPU benchmark, which directly impacts the speed of the device. There are a few ways to perform this benchmark as well, ranging from theoretical measurements like clock speed to real-world tests like video editing and gaming performance.

For smartphones, we’ll typically benchmark the CPU, graphics package, battery life, display, and just about any other differentiating factor between models. As a consumer, reading these results can help you compare different phones before making a decision. For an example, take a look at our Google Tensor G2 benchmark results.

Reviews cover several other areas of consumer electronics too, so you’ll find dozens of different benchmark types out there. In the world of TVs, for example, benchmarks allow us to measure color gamut, brightness, contrast, and HDR performance.

What do benchmarks tell us about our computers and smartphones?

In our testing, We often find that devices with identical hardware don’t necessarily perform the same. One might have a beefier cooling setup to complement a power-hungry CPU, while another might buckle under the load. Likewise, some manufacturers may fine-tune their devices to perform better after rigorous testing. Here’s one such example:

The above charts showcase our benchmarks of the Galaxy S23 Ultra and its Snapdragon 8 Gen 2 for Galaxy chip. Two other smartphones on the graph — the OnePlus 11 and REDMAGIC 8 Pro — pack the same SoC. However, the latter two phones are configured to run at slightly different speeds from the factory.

The results make it clear that each phone in the Snapdragon 8 Gen 2 family behaves a little differently. If you’re after the most powerful phone, the results make it clear that the Galaxy S23 Ultra comes out on top. We also used multiple benchmarks, namely GeekBench 5, PCMark, and 3DMark, to ensure consistency. The third one tests the graphics performance specifically, which gives you an idea of gaming performance.

In the past, benchmarking has allowed us to pinpoint issues with devices. For example, a device posting much lower scores than the others may indicate thermal throttling or a cooling issue. Likewise, a battery benchmark can help you understand how long the device will last in day-to-day usage.

It’s worth noting that benchmarks don’t just matter to reviewers. If you’re ever buying second-hand hardware, you can run your own benchmarks and compare them against results online. This way, you can be sure that the hardware is performing as expected and that it’s free from problems and defects.

Types of benchmarks

We’ll get into some of the ways in which you can perform your own benchmarks, but it’s worth talking about the different kinds first. Here’s a list of the most common tests we perform here at Android Authority:

1. CPU benchmarks: We’ve already seen one example of a CPU benchmark and it’s usually pretty straightforward. A higher score indicates better performance. Most benchmarks will perform a fixed task, like compressing a large file or crunching large numbers in a spreadsheet.
2. GPU benchmarks: There are a few different ways to benchmark a GPU, ranging from gaming performance to video encoding. The latter tests for hardware acceleration, or the ability to quickly render large videos. In games, meanwhile, a benchmark will measure the number of frames generated by the GPU per second — a higher result indicates smoother performance.
3. Battery benchmarks: In portable devices like smartphones, having a large cell doesn’t automatically lead to longer battery life. This is because an inefficient CPU or display can lead to faster drain and worse endurance in the real world.
4. Charging benchmarks: We’ve compared a lot of different fast charging technologies on the market, including a few proprietary ones like Oppo’s SuperVOOC protocol. Charging benchmarks help us find anomalies like Google’s Pixel smartphones drawing just 23W of power from the official 30W adapter.
5. Display benchmarks: Most smartphones on the market these days use OLED displays, but that’s not the end of it. The latest flagship smartphones use newer materials that can get brighter, while also consuming less power. Benchmarks using external tools help us measure the display’s color accuracy, efficiency, and even heat output.
6. ANC benchmarks: The Bluetooth audio market has become highly competitive, but a more expensive product doesn’t always the yield best performance. One area in which only a few major manufacturers hold a lead is active noise cancelation (ANC). We measure ANC using external tools to understand a particular headsets ability to block environmental noises.
7. Modem speed tests: You may have the fastest 5G connection in your neighborhood, but it doesn’t mean much if your devices can’t keep up. Speed tests can check your device’s modem capabilities as well as signal reception. For example, our testing revealed that the Pixel 6 series exhibited worse network performance than the competition.

How trustworthy are benchmarks?

Benchmarks hold a place of prestige among manufacturers, as they can objectively prove when one device is better than another. Unfortunately, however, we’ve seen some attempts to game the system and come out ahead. This has led to quite a bit of controversy in recent years, with even big smartphone manufacturers like Samsung and OnePlus getting caught in the crossfire.

A common tactic we’ve seen over the years is that certain devices will detect when a benchmarking app is running and temporarily boost performance to achieve higher scores. Of course, this level of performance may come at the expense of higher battery drain or heat output. And in actual day-to-day use outside of benchmarking apps, the phone may restrict performance to deliver a better user experience.

The good news is that benchmarking apps usually catch on to manipulation and other nefarious practices pretty quickly. Independent media can also come up with ways to bypass manufacturer intrusion. For example, modifying benchmark apps to report a different package name to the system can help avoid detection. That way, the system treats it as a regular app and we can collect accurate benchmarking results.

So should you trust benchmarks? For the most part, yes. We’ll recommend reading more than one review of any product to make sure the results match up. And of course, pay attention to the actual benchmarks too. If you’re a gamer, for example, look at GPU benchmarks that specifically measure the device’s performance in the titles you play. A video editing benchmark may measure the GPU’s capability as well, but it doesn’t tell you much about gaming performance.

What are some popular benchmarking tools?

Now that we’ve got the basics of benchmarking out of the way, you probably want to know which tools you can use to test your own hardware. Fortunately, most benchmarking software offers a free version for personal use so getting started is quite easy. Let’s take a look at some of the most popular benchmarking tools for smartphones and computers.

Smartphone benchmarking tools

When it comes to benchmarking smartphones, here are your best options:

• Geekbench 6: We use Geekbench 6 on Android and iOS to primarily measure the CPU performance of smartphones. At the end of the test, you get two scores: one for single-core and the other for multi-core performance.
• PCMark and 3DMark: PCMark and 3DMark allow you to gauge your device’s CPU and GPU performance against other smartphones. It tests a variety of real-world applications, ranging from web browsing to photo editing. It also evaluates storage speed, battery life, and gaming performance. And since it maintains a database of scores, you can see if your particular device is underperforming versus other phones.
• CPU Throttling Test: Many smartphone SoC makers conveniently overlook the fact that their chips cannot sustain peak performance for very long. CPU Throttling Test helps you stress test your phone and take it to its thermal limits. You can record the relative performance level over time. If the score drops significantly after a minute or two as it does in the above picture, you may see stutters or other slowdowns in heavier tasks like gaming and navigation.
• Ampere: With so many wall chargers and cables out there, it’s difficult to estimate just how much power your smartphone draws from the outlet. Ampere can give you this information, letting you know if you’re using sub-optimal charging hardware.

Besides these software tools, we may also use dedicated hardware but these are often much more expensive. For example, we’ve used the X-Rite i1Display Pro to benchmark displays.

Computer benchmarking tools

The list of benchmarking tools for computers doesn’t look all too different, as we can also use Geekbench and PCMark on desktop operating systems like Windows and macOS. But enthusiasts generally also prefer the following software:

• Cinebench: When it comes to CPU testing, Cinebench is one of the most trusted and reliable benchmarking tools out there. Like Geekbench, it can provide single and multi-core results.
• CrystalDiskMark: Whether you’re looking to test a hard drive, microSD card, or SSD, CrystalDiskMark can help you measure disk performance. Besides sequential read and write speeds, it also tests random I/O, which affects real-world app loading times.
• Unigine Heaven: If you need to know how well a particular computer works for gaming, Unigine Heaven should be your go-to choice. It’s a popular GPU-focused benchmark that many use to determine
• FurMark and Prime95: If you’re looking for a benchmark that takes your computer to its thermal limits, Prime95 and FurMark do just that for your CPU and GPU respectively. If you’re buying a second-hand graphics card, running FurMark can help you understand its cooling and sustained performance.

So in conclusion, benchmarking is one of the most important aspects of testing any gadget — from smartphones to TVs. We’ve seen benchmarking tools come a long way over the years, so it’s almost always worth reading objective testing and reviews before making a purchase.