I ditched Google for ChatGPT Search to find out if the grass is really greener

Whether it’s the relentless onslaught of ads or the dozens of inauthentic links occupying the top spot, using Google Search has never felt more unrewarding than in 2024. So it’s perhaps not surprising that entire AI startups like Perplexity have sprung up to threaten Google’s search business. Shortly after Google was forced to respond with its controversial AI Overviews feature, OpenAI also threw its hat into the arena with ChatGPT Search. The latter is finally rolling out to the general public, so I took it for a spin to find out if ChatGPT could potentially supplant Google for my search needs.

What is ChatGPT Search?

ChatGPT Search combines the chatbot’s excellent natural language understanding with a search engine for up-to-date information. If you’ve ever used Microsoft Copilot (formerly Bing Chat), you will feel right at home – the way they work is extremely similar, inline citations and everything. While OpenAI hasn’t revealed how it ranks online sources, the company’s VP of Engineering confirmed on Reddit that the feature relies on Bing to some extent.

OpenAI has also released a Chrome extension that allows you to set ChatGPT as your default search engine. I did just that for the past couple of days to find out if I could live with it long-term — here’s how it went.

ChatGPT Search vs Google: Which is better?

Before I can show you any side-by-side comparisons, it’s worth noting that ChatGPT Search has a few advantages over Google from the outset. It doesn’t have any kind of advertising or sponsored links, at least in its current state.

However, there are downsides too. ChatGPT Search takes a couple of seconds longer to respond than a typical search engine. And the responses themselves don’t always contain the information you’re looking for. This means typing in a prompt yet again, while Google offers a wide variety of links and at least some likely cover the topic in-depth. Finally, ChatGPT only references a handful of sources to draw its conclusions, which could lead to biased or outright false responses.

All of these drawbacks apply to Google’s AI Overviews too, which currently show at the top of select search queries. The company has broadened their scope over time, though, so we may see AI-generated summaries at the top of search results more often if ChatGPT Search threatens Google’s business.

That said, Google does still offer a traditional search experience with a list of indexed links if you scroll past the AI-generated summary. Meanwhile, ChatGPT only reveals a handful of search results in a narrow sidebar when you click on a small “Sources” button.

Prompt 1: USD to JPY and weather

As a frequent traveler, I tend to look up exchange rates between various currencies. This can be tricky for an AI to handle since the value of currencies and stocks keep changing in real time. So without a dedicated data source for this kind of information, it’s not surprising that ChatGPT Search doesn’t keep up with Google.

ChatGPT Search’s response indicated that one US dollar would be worth 153.03 Japanese Yen. It quoted a single source for this information, Wise, but clicking through the link revealed a slightly lower rate of 152.4. International markets were closed at this time, so there was no possibility of sudden volatility. Google Search managed to respond with the correct 152.4 figure, thanks to the search giant’s partnership with various financial data brokers.

I asked for Intel’s stock price next, and ChatGPT Search managed to respond with the correct price and a useful infographic. The markets were still closed at this time, so I couldn’t test its ability to update in real-time but it’s an indication of OpenAI working to solve this problem. But asking for the weather worked, even if the chatbot failed to determine my city accurately. Google has a wealth of location data via my phone, meanwhile, so it had no trouble pinpointing the exact suburb I was in.

Winner: Google Search for speed and accuracy, but ChatGPT isn’t too far behind

Prompt 2: When will Android 15 come to the Galaxy S24?

The Android 15 update has only made its way to a handful of Pixel devices so far, but we have covered how manufacturers like Samsung have started testing the update in recent days too. ChatGPT picked our coverage, and correctly said that the Galaxy S24 series will get its Android 15 update alongside One UI 7 sometime in 2025. It also corroborated this information with another source.

Google offered links to four news stories, followed by an irrelevant list of search suggestions. I could’ve clicked through to find the answer. But if I’m just looking for a quick answer, ChatGPT is the unquestionable winner.

Winner: ChatGPT Search

Prompt 3: Capital gains tax in Canada

My search activity balloons during tax season, but Commonwealth countries often announce changes during the annual budget so chatbots with hard knowledge cutoffs fail to offer accurate guidance. Unsurprisingly then, ChatGPT with its new search capabilities managed to pull information from the official website of the Government of Canada to answer my query. It also explained how to calculate the exact amount I’d have to pay in taxes.

Asking Google Search the same question yielded an article snippet that answered my query, but it didn’t offer any follow-up information.

Winner: ChatGPT Search

Prompt 4: Cheapest eSIM plan for Japan

Not too long ago, I wouldn’t trust ChatGPT with a question that would influence my purchasing decisions. But with the world’s knowledge now at the chatbot’s fingertips, could it finally rival Google Search? Only kind of — ChatGPT managed to spit out a list of eSIM apps with the cheapest plans for Japan, as requested.

However, upon further inspection, I found that ChatGPT’s very first source article was a year old and had outdated prices. The chatbot’s answer was still generally relevant, but it underscores the problems of taking AI responses at face value.

Google didn’t offer an AI-generated response for this question, and instead flooded me with ads. Scrolling down to the actual results, though, it did provide relatively recent links. I also appreciated that it placed authoritative Japan-focused blogs at the top of the search result list rather than generic travel websites looking to capitalize on a popular search term. Clearly, the old way isn’t too bad in this case if you don’t mind clicking through one or two links.

Winner: Google Search

Prompt 5: Vitamin D dosage

Google’s AI Overviews finally made an appearance in response to this prompt, offering a detailed summary from at least four government-backed sources. This is heartening to see, given that ChatGPT only referenced third-party websites like GoodRx, Mayo Clinic, and WebMD. Still, the quality of information was consistent across both search tools, so it’s technically a tie.

Winner: Tie

Should you replace Google with ChatGPT Search?

Switching to ChatGPT as your primary search engine doesn’t make sense just yet. The overall experience is still rather barebones and the chatbot has limited means to fetch real-time information. Finding follow-up information can also be quite tedious, as ChatGPT doesn’t offer related search suggestions like Google. This is an area where Perplexity excels, although it often relies on outdated and inaccurate sources too.

Having said that, AI-powered search engines do have their place. If the information you’re looking for isn’t too recent and requires an in-depth explanation, ChatGPT Search can indeed pull information from various sources to deliver a better response than Google. In two of the above examples, I preferred ChatGPT’s response as it delivered an answer quickly and at a glance. However, I still had to rely on my personal knowledge and experience to know that the responses were factually correct.

ChatGPT Search also requires a Plus subscription for now, which carries a hefty $20 per month subscription fee. Given that both Google and Perplexity offer a similar service for free, I don’t see this requirement staying long-term.

All in all, I’d say ChatGPT Search is a good starting point and a great secondary source of information. But despite what OpenAI says, it’s not a search engine replacement and I would caution against relying on it for any critical task.