A smartphone drove our car, and we lived to tell the tale

It’s an unusually cold morning in Barcelona and, together with two of my colleagues, I’m shivering in an empty parking lot outside the city’s legendary Camp Nou.

We’re waiting for our ride, a Porsche Panamera. Not bad from a bunch of humble Android bloggers, right? Alas, we’re not here for a joyride. We’ve actually come to see HUAWEI’s RoadReader project, probably the coolest application of AI at this year’s MWC. (Admittedly, the bar for cool AI applications was pretty low.)

With RoadReader, HUAWEI wanted to showcase the AI prowess of its Mate 10 Pro smartphone and the Kirin 970 processor inside it. Put simply, the company turned a phone into the brains of a robotic Panamera. 

There are caveats. We’re not offered an autonomous tour around the Camp Nou. We’re not even getting a ride around the parking lot. Instead, HUAWEI set up a short, straight test track designed to showcase the phone’s ability to identify and react to obstacles using AI.

A fair bit of additional hardware is required to make the magic happen. Mounted on top of the limousine, a modified bag carrier houses a high-end Sony DSLR and a Teradek Bolt 3000 wireless video transceiver (catalog price: $10,490), which act as the eyes feeding images into the Mate 10 Pro’s brain. The trunk is also packed with robotic equipment that converts the input from the Mate 10 Pro into commands for the car.

After the team solves a glitch with the custom app running RoadReader, we’re finally ready to put our lives into the virtual hands of an Android smartphone. There’s a safety driver behind the wheel, and a big red button mounted on the Panamera’s center console lets us stop the car in case things go awry. But I still feel a little anxious, and the fact that we had to sign waivers absolving HUAWEI of any liability did not help.

There are two phases to the demo. In the first one, the car slowly makes its way down the test track looking for – and learning – the objects it needs to avoid. HUAWEI tested the system with hundreds of objects, but for the demo, it settled on cardboard cutouts showing a bicycle, an oversized soccer ball, and a dog.

After the slow run, it’s time for the real deal. First, we tell the phone what evasive action it should take for each obstacle: swerve to the left, swerve to the right, or simply brake.

A couple more taps on the screen and we barrel down the track. The phone is driving.

It’s all over in a few seconds. The car, or more accurately, the phone, performed just as I instructed it to, taking a hard swerve to the right to avoid the cardboard dog attendants rolled out on the track. The dog is unscathed, the car is in one piece, and the Android bloggers inside breathe a sigh of relief.

While HUAWEI’s test may sound a bit underwhelming, it’s still a cool thing to experience. In part, it’s thanks to the physical sensation of accelerating in a powerful car. But I also appreciated what HUAWEI was able to do in just five weeks with a consumer device that 90 percent of users will only ever use for Facebook and YouTube.

The Mate 10 Pro can run complex machine learning operations and can do it fast enough to control a hunk of metal moving at more than 50 kmph. Multiple times a second, the device extracts still frames from the video feed it receives from the roof-mounted camera. It then runs image recognition algorithms on each frame, looking for potential obstacles on the road. As soon as an obstacle is detected, the phone tells the car to maneuver itself out of harm’s way.

The system harnesses the NPU inside the Kirin 970 processor, and Google’s TensorFlow library of machine learning tools. It all happens locally and on-the-fly. It’s a good example of what developers can do with these tools and other AI platforms. It’s definitely more exciting than the other AI application that HUAWEI can show for now – a live translation app developed by Microsoft.

Artificial Intelligence has become a major buzzword in the mobile industry – companies are taking advantage of the vagueness of the term to slap it on features that have nothing to do with machine learning. AI is also highly technical and, let’s face it, difficult to understand and boring for most people. It’s good to see HUAWEI attempt to demonstrate the concept with a real-world application we can all relate to.

Peter Gauden, a marketing chief with HUAWEI, was quick to clarify that what we saw in Barcelona was just a cool concept that should not be taken as a suggestion that HUAWEI is entering the red-hot autonomous driving market. But Gauden also pointed out that HUAWEI owns the full stack, from the AI-optimized silicon, to popular consumer devices, software prowess, and cutting-edge telecom infrastructure. In other words – and this is entirely speculation – if HUAWEI would ever decide to try its hand at self-driving tech or any other application of AI, it would be better positioned than most companies out there.