Best of Android: How we test audio

We here at Android Authority have a special relationship with the audio site SoundGuys, who provide a lot of muscle for our testing efforts. When we want to test smartphones for audio performance, we let them go a little wild with it, and they use the same process to determine the best smartphones for audio as they do for Best of Android.

Using a dedicated interface, our testers use a 3.5mm to 2×1/4″ TRS connector Y-cable to measure the wired output of each test phone. A free test suite called Room EQ Wizard is able to measure things like dynamic range, distortion, noise, frequency response, and more. We then save all the test result files so we can compare each result to each other. Additionally, we then input all the values into our scoring system.

Despite calls for using ultra high-def test files, most people will never ever listen to one of those. Additionally, there’s a lot of information you can glean from a simple 96kHz/24-bit test file — and that’s what we use. CD quality sound is “only” 44.1kHz/16-bit, and that’s more than sufficient to satisfy the perceptual limits of the vast majority of humans on Earth. Our tests are overkill.

Because we didn’t test the ultra-high-bitrate/high-sample rate files, there’s a certain limit on how well each unit could have performed. While there’s some debate as to what reviewers should be testing at, we go with roughly double the highest common settings that most people will use. In this case, something that would meet or exceed CD quality (44.1kHz/16-bit), because no streaming service can do that currently.

Speaker testing is fairly rudimentary, but without the availability of a truly anechoic chamber: we can’t give you speaker quality tests that will reflect your usage. By playing a pink noise sample at full volume, we can measure the gross maximum SPL with an electret microphone pointed directly at the phone. The front-facing units on the whole perform better than those on the bottom of the phone, given the sound is actually directed at the user. But trust me when I tell you, virtually all speakers tested suck out loud. You do not want them to be your primary, secondary, or even tertiary means of music consumption.

Obviously, the existence of extra features and other concerns like Bluetooth have to factor into our decisions. In this case, we’ve determined through testing that with the exception of AAC, most phones handle it acceptably well.

What we test

Simply put, when we test wired audio, we collect results on:

1. Noise floor
2. Dynamic range
3. Total harmonic distortion (THD)
4. Frequency response
5. Speaker loudness
6. Is there a headphone jack?

We could go crazy with a few more measures, but audio is pretty straightforward. Issues with bitrate will present themselves in other tests, and so on. Models without a headphone jack will not get points for that category.

What you should know

When it comes to wired listening: the lower distortion and noise are, the better the result. Similarly, the lower the deviation found in testing frequency response is, the less your audio will be altered. While some people like to artfully tune their music, any component that isn’t the headphones or the software playing back the music shouldn’t affect the signal at all. Only deviations + / – 3dB will be noticeable at all. You might even be surprised to learn that your hearing is not as good as you think.

Few phones have issues here, but it’s not unheard of for a modern phone to have some weird problems here and there. Most phones test here are all what we’d categorize as near “perceptually perfect,” given their performance meets or exceeds what your average human can hear. However, they’re not actually perfect, and users with more power-hungry headphones may run into issues.

In our testing, we noticed that the phones with dongles sometimes refuse to output sound at the specified sample rate. Why this happens we have no idea, but we’re able to reproduce our results almost exactly between several different copies of each device with three testers. The upshot is that these phones should have an easier time dispelling IM distortion, the tradeoff is that it technically isn’t performing as well.

While objectively collected data is all well and good, it doesn’t exactly tell you everything you need to know about a phone’s performance. Now that digital media’s performance has started to sail beyond the limits of human perception, test results matter less and less—while features matter more and more. Very few (if any) phones will sound much worse than another with popular streaming services.