Google's Reddit monopoly sparks concerns for alternative search engines

Earlier this year, Google entered into an exclusive deal with Reddit to use its user-generated content for training the company’s AI apps. Fast forward to today, and the Mountain View tech giant is the sole search engine that’s able to display recent search results from Reddit. Now, this has, undoubtedly, made Google’s platform all the more capable. But what does this mean for you if you use one of the many alternative search engines, like DuckDuckGo or Bing?

Well, in a nutshell, it means searching for Reddit content using “site:reddit.com” will no longer yield results on these alternative platforms (via 404 Media). The reporting outlet found that DuckDuckGo is turning up a handful of Reddit search results for now, but there’s no further data on where the links go. Older Reddit links, from prior to last week, that is, still come up, but because alternative search engines are unable to crawl Reddit anymore, Google is your best bet for finding the most recent Reddit content.

So, why does this matter? The fact is that Google has, for some time now, been accused of monopolistic practices. By securing exclusive access to crawl Reddit, the tech giant has further consolidated its position as the dominant search engine. Additionally, given that the quality of search results on Google hasn’t been impressive in the recent past, the company’s move makes it harder for people to migrate to other platforms for a better search experience without compromising access to a good amount of current user-generated content.

404 Media connected with Colin Hayhurst, CEO of search engine Mojeek, who noted that his platform was blocked from crawling Reddit starting in early June. While Hayhurst has attempted to contact Reddit to resolve this issue, he has, unfortunately, yet to hear back.

In the past, Reddit has been vocal about how companies are scraping its platform to train their AI models. Over time, the company began introducing stringent measures to stop AI firms from being able to exploit its content without authorization. In June, the company posted, “In the next few weeks, we’ll be updating our robots.txt instructions to be as clear as possible: if you are using an automated agent to access Reddit, you need to abide by our terms and policies, and you need to talk to us.”

Reddit seems to have gone ahead with this and updated its robot.txt instructions, and the timeline aligns with the recent restrictions that alternative search engines have observed. We’ll have to wait and watch to see how much of an impact this has on smaller search engines and users who want a different search experience than what Google can offer.