I wanted Google's secret AI dictation app to replace Wispr Flow, but it couldn't

Large language models have given AI the edge it has today, but one of their more obscure uses is improved voice dictation. We saw a version of it in Gboard, which lets you talk to your phone in natural language and have it convert your speech into text, complete with emoji and punctuation. I rely on a standalone tool called Wispr Flow to achieve similar results on my Mac and avoid typing out long texts and emails.

I always thought that since Google is already so good with voice dictation on Android through Gboard, a Google tool for desktop would wipe the floor with anything else on the market. But as it turns out, I was completely wrong. Google’s AI Edge Eloquent works along the same lines as Wispr Flow, with the only exception that it performs much worse — so much so that it’s mostly unusable for any kind of serious work.

What is Google AI Edge Eloquent?

Google AI Edge Eloquent is an offline voice dictation tool that works without the internet, using Gemma — Google’s class of language models that run entirely on-device. On your first installation, you will be asked to download a basic offline model with support for 2 billion parameters. If your laptop has more than 16GB of RAM, you can even download a 12-billion-parameter model for better performance. Since my Mac falls into the first category, I ran my tests using the 2-billion-parameter model. Edge Eloquent is available on iOS and macOS, but not on Android, which is understandable since Gboard’s advanced voice dictation feature already does the trick.

It’s the little things that make it frustrating

You know what? It took me at least 15 tries for Edge Eloquent to write “frustrating” in the heading above. And that’s the kind of annoyance I have to deal with every single time I use it.

This Google tool has a built-in polishing feature that cleans up your text before pasting it into the text box. More frequently than I would like, it changes the very structure of a sentence, sometimes altering the tense, omitting words, and rephrasing it into a perfect AI-written copy rather than something I would write in my own language. And when I turn the polishing feature off to avoid exactly this, Edge Eloquent keeps all the ums and ahs and just pastes everything as-is.

In either case, I am required to spend a considerable amount of time rechecking everything phrase by phrase and editing what I just said. Sadly, there is no middle ground — if there had been a slider to pick the amount of polish, I would be a much happier user of Google AI Edge Eloquent.

What further added to my editing workload was that Edge Eloquent often disregarded my commands for new lines or em dashes far more frequently than Wispr Flow. Secondly, I found that the offline Gemma model hallucinates quite a bit. There have been at least a couple of instances where it added phrases completely unrelated to the text that I hadn’t even uttered.

Google does deserve some credit, though

What I listed above are the issues that keep breaking my workflow often enough that I can’t rely on it for any serious work, like drafting this story. However, if I enunciate my words more clearly and maintain a slow dictation pace (while being okay with a tad more editing along the way), I can indeed get AI Edge Eloquent to work at a satisfactory level. It feels even more amazing when I realize that its Mac app is doing all this magic entirely offline without stressing the system too much.

I kept an eye on my Mac’s Activity Monitor while dictating a piece, and Edge Eloquent only showed brief spikes in CPU, GPU, and RAM usage — none of which severely affected any of the other apps open on my system. I’m sure the spikes would be noticeably different had I installed the 12-billion-parameter model, but so far, everything seems well-tamed. And the fact that it is completely free to use is a huge selling point in its favor.

Since Wispr Flow works in the cloud, it’s got an obvious advantage. I’ve had far fewer structural issues with it, though it’s not entirely bulletproof. It does have its own fair share of issues, albeit far fewer compared to Google AI Edge Eloquent. For instance, it gets brand names right almost every time, which Google’s alternative frequently fails to do, and I can maintain a faster dictation pace with Wispr Flow without needing to stress every word to get it right. That’s a particularly important feature for a writer like me who gets in the zone when working on a story.

I want the best of both worlds

Even though Wispr Flow is a paid product and requires a constant internet connection to work at all, I’d prefer it over Google’s AI Edge Eloquent simply because of its better reliability. When I’m focusing on writing, having to frequently go back and edit what the dictation tool got wrong is not only time-consuming, but it also breaks my flow. I just can’t afford to lose my train of thought as often as I have with Google’s product. Maybe Edge Eloquent performs better on a beefier system running the 12-billion-parameter model, but what I have didn’t cut it for me.

That doesn’t mean I’ll ditch Edge Eloquent entirely. Since it’s almost 60–70% there in terms of transcription and polish, I can still use it in situations where Wispr Flow fails. For example, it will come in handy when I’m on a train with a spotty network and need to quickly turn around a story before reaching my destination. Wispr Flow recently had an outage that resulted in slower responses and, in many cases, no transcription at all. A good thing about a local model is that it isn’t affected by cloud outages at all.

Google has done a fantastic job of finding a niche use case for its on-device LLMs. Considering how efficient it is on the hardware while running entirely offline, it works decently well. Since it’s completely free, it could be perfectly suited for personal use, whether for occasionally jotting emails or writing prompts for ChatGPT. But for more professional use, Wispr Flow remains the undisputed king for me, while AI Edge Eloquent will be my fallback tool.