Basemark Power Assessment Tool review

I don’t know how many times I have said it, but I will say it again, “power efficiency is everything on mobile.” Normally we focus on the power efficiency of the CPUs and GPUs at the heart of our smartphones and tablets (for example when the new Cortex-A73 CPU core design was launched), however there is more to power  efficiency than just the System-on-a-Chip. Android is an eco-system, a combination of hardware, software and online services. OEMs like Samsung, LG, Sony, HTCetc make the hardware; Google, Amazon, Microsoft etc. provide the services; but the software is made by “everyone.” By that I mean that the apps we run on our smartphones can be battery friendly or battery killers.

To  help with app optimization Basemark has launched the latest version of its Power Assessment Tool (PAT). You have probably heard of Basemark in connection with its benchmarking tools like Basemark OS II or Basemark ES 3.1, however the company does more than just write benchmark apps. At its heart Basemark is a system performance and power consumption analysis company and it has a range of tools that are used by OEMs like AMD, Imagination Technologies, NVIDIA, and Qualcomm.

Unlike other tools to measure app power consumption, PAT is hardware based not software based. There is a software element for analyzing the results, however the actual measuring is performed in hardware, something that makes the PAT quite unique.

So how does it do that? Do you need to take your phone apart to use it? No. The basic principle is this: if the battery in your device is full and it is plugged into a charger then the amount of power your phone uses when performing different operations can be measured by seeing how much power is drawn from the charger.

The basic setup is this. You connect your smartphone to the PAT and the PAT to your PC and to your phone’s charger. When you run apps on your phone it will drawn power from the charger which is detected by the PAT and sent to your PC to be recorded by the PAT software.

Power measurement

Once you have all the cables connected and the software installed on your PC you need to check the battery level on your device. Even if the phone is claiming that it is 100% charged it might not actually be charged fully. When you hit the “REC” button on the software you will get a live update of the amount of power being drawn by the phone. After watching it for a few moments you can see if it is declining or steady. If it is declining it means that the phone is still charging. This initial phase needs to be done with the phone in flight mode and with the screen off.

When the baseline is hit then you can start testing. Take the phone out of flight mode, run the app you want to test, then put it back into flight mode and wait for the power level to hit the baseline again. This last part is important as there are times when the app requires more power than it is drawing from the charger. The software is able to calculate the amount of extra energy taken from the battery and add it to the overall app power usage numbers.

Obviously, the more power that an app is drawing the quicker the battery will drain. The last thing any developer wants to do is release a new app, build an audience, and then find that people abandon it because it has a reputation of draining the battery quickly. This is bad news for games, but even worse news for social media or messaging apps.

The downside of this testing process is that it can be quite long-winded. Switching the phone in and out of flight mode, waiting for the battery to reach its baseline, etc, all means that it can take maybe 30 to 45 minutes to run one test, especially if the conditions are against you.

A feature that I would like to see on the software side is the ability to mark points on the data while it is being recorded. For example, say you are running your app and then you activate a certain feature or move to a new activity, it would be useful to mark that on the data so you can refer to it afterwards when you are performing your analysis.

Benchmarking

One interesting aspect of the PAT is that it can be used for power benchmarking. The same app can be run on multiple devices and the power usage measured. This would then give us an indication of how power efficient any given device is compared to another while running the same code. It can even be used to see how much power a benchmark tool uses, allowing for investigation into how “real world” are the loads generated, and also if the device deliberately ups its power usage (which is linked to performance) when it detects a benchmark running.

As an example I installed Epic Citadel on two phones, the Samsung Galaxy Note 5 and the HUAWEI Mate 8. I used PAT to measure the power usage when running the benchmark mode of the app, first in Ultra High Quality mode and then in High Performance mode.

The average power consumption for the Note 5 during the two benchmark runs was 4.3 watts. According to the PAT, running the benchmark in Ultra High Quality mode takes significant more power than running it in High Performance mode. In the latter mode the average power consumption was around 3 watts, however for the former it was around 5.4 watts. These last two figures are estimates as they doesn’t include any surplus power that was drawn from the battery (and not the charger) during the test.

I performed the same test on the Mate 8. This time the PAT recorded an average power consumption of 3.44 watts, a little lower than the Note 5. This is a measurement for the overall system including the CPU, GPU and importantly the screen (which is of a lower resolution on the Mate 8). According to the live results, the Mate 8 used an average of 3.5 watts during the Ultra High Quality mode benchmark and 3.3 watts for the High Performance mode test run.

So it seems that the Note 5 pays a high price when running Epic Citadel in Ultra High Quality mode and is the main reason that its average power usage was higher than the Mate 8. For a more in-depth analysis I would need to test several apps and increase the number of test devices.

Wrap-up

Clearly the PAT isn’t designed for hobbyists and indie developers, it is a serious piece of equipment aimed at certain types of OEMs and software vendors. This is reflected in the pricing and the way that pricing is expressed. “The PAT is priced at 995 € + VAT for corporate customers,” is how the press release puts it.

Having said that, it is certainly a valuable tool for professional engineering teams, for “corporate customers.” Not only does it allow software engineers to test their apps for power efficiency, it also allows them to benchmark that efficiency across different devices, different versions of Android, and different releases of their software.

As for me, I will certainly be digging deeper to see how this tool can help me benchmark SoCs and new devices that come onto the market.