Affiliate links on Android Authority may earn us a commission. Learn more.
Evidence suggests Google scaled back AI Overview search results compared to original rollout
- Third-party analytics data suggests Google’s AI Overview results in Search were significantly scaled back after the original rollout.
- Allegedly, these results may have dropped from 27% of queries to just 11% of them.
- Interestingly, this drop happened before word started to spread about the terrible results AI Overview was providing to users.
At this point, pretty much everyone reading this is likely familiar with Google’s AI Overview system built into Google Search. Unfortunately, you probably know about it for reasons Google did not anticipate, namely for hilarious examples of terrible answers provided by AI to users spreading across the internet like wildfire. Examples included encouraging people to drink urine, eat at least one small rock each day, and put glue on pizza.
Now, according to a third-party analytics firm called BrightEdge (via WIRED), we have an idea of how Google has changed its AI Overview after its initial rollout.
According to BrightEdge, right after Google I/O, AI Overview results appeared in about 27% of search queries tracked by the firm. Just a few days later, this number dropped significantly to just 11% of results. Notably, this was before word started to spread about Gemini’s terrible answers.
By the time the memes began and Google had penned a blog post apologizing for the problems, AI Overview results had stayed around that 11% mark.
We can only speculate, but this heavily suggests that Google saw that AI Overview wasn’t working properly well before word of its failures started to spread. Of course, dropping the results to a much lower percentage of Search queries didn’t stop that from becoming a significant problem. However, it is possible that Google saved itself from an even more embarrassing situation by preemptively dropping results before things got out of hand.
If nothing else, according to Jim Yu — BrightEdge’s founder and executive chairman — this shows that Google decided to put the brakes on AI Overview, at least when compared to the original post-I/O rollout. “There’s obviously some risks [Google is] trying to tightly manage,” he said.
In a statement to WIRED, Google said this:
We’re continuing to refine when and how we show AI Overviews so they’re as useful as possible, including a number of technical updates in the past week to improve response quality.
However, Google declined to share internal metrics to show if AI Overview results had, in fact, dropped by the numbers suggested by BrightEdge.
Still, AI Overview — or the idea of AI bolstering search queries — isn’t going away. “Big picture is that the AI moment in search is inevitable, and I think it’s going to get much better,” Yu said.