Google's Search Generative Experience tested: A Bing competitor at long last

After falling behind Microsoft’s Bing in the AI race, Google has finally taken the wraps off its next-generation search product. Powered by Google’s PaLM 2 language model, Search Generative Experience is a new feature that tries to combine generative AI with conventional search results.

That’s right, Google’s in-house AI will soon automatically answer your queries and summarize articles from the web. But how well does it work in the real world? We put it to the test and compared it against the competition to find out. Here are the results.

What is Google Search Generative Experience?

Google’s Search Generative Experience (SGE) is a new experimental search page that puts AI answers and summaries front and center. It’s an optional feature that you’ll have to manually turn on — for now, anyway. Once enabled, a new AI-generated text box appears at the top of nearly every Google search result, specifically above the first external link (pictured above).

Google says that the new SGE feature aims to eliminate the guesswork of finding the right link and makes finding the right answer easier. Indeed, asking simple queries like “How much will I have to pay to share my Netflix account with family?” could become a whole lot more painless with this new feature. You can see an example of Search Generative Experience versus Google’s former approach to displaying a Featured Snippet below.

What can Google’s AI search do?

Google’s new AI-generated summary feature is far more useful when you use it for nuanced and less straightforward queries. Plenty of us regularly use chatbots like ChatGPT to plan our vacations and ask for advice, so integrating that functionality right into search would help tremendously. Here’s an example:

However, you can’t directly ask for advice or complex instruction. When I searched “Help me plan a 7 day trip to Japan”, Google simply returned a list of links. Even clicking on the new Converse button didn’t do anything besides return the same search results page. As a long-time Google Assistant user, this behavior doesn’t surprise me. The Assistant often responds with “Here are some results from the web” rather than dictating an answer.

In general, I’ve found that Google’s Search Generative Experience box only appears when you don’t address or expect it. So if you want advice or opinion, you’re still better off with Bing Chat or Google’s Bard chatbot.

Google Search Generative Experience vs. New Bing: Which one wins?

Google famously lost the AI race to Microsoft — the latter unveiled AI results in Bing several months ago. But now that Google has seemingly caught up, how do its AI responses stack up vs. Bing?

For starters, I’ve noticed that Google adds AI-generated answers to search queries more often than Bing. The two companies have also taken somewhat opposing approaches to integrating AI into their search results. Bing adds a dense answer box with numbered source links, while Google’s SGE takes up the whole page and doesn’t include any links inside the text. Here’s an example comparing the two:

I found that Bing and Google both did a reasonably good job of answering questions. However, they’re not always necessary and I wish there was a way to quickly dismiss Google’s large AI-generated text box. Bing’s approach to displaying its AI answer along the right-hand side of the page allows me to reach the actual search results faster.

Apart from that, here are a few more differences I noticed between Google’s SGE and Microsoft’s AI-powered Bing.

• Google’s AI-generated text only appears in the US, India, and Japan for now, while Bing Chat has expanded to most countries.
• The new generative experience only shows up if you initiate a search via the Chrome web browser or Google app on Android and iOS.
• You can ask Google follow-up questions but don’t expect to always get AI-generated answers. Bing, meanwhile, drops you into full-fledged Chat mode if you scroll up or expand the AI answer. In other words, don’t expect to use Google’s Bard chatbot via a search result page.

How accurate is the AI summary in Google search results?

In terms of accuracy, I’m happy to report that Google’s AI results have improved significantly since my first experience with Bard. The company’s chatbot would often hallucinate facts even when asked simple questions. That’s not as much of a problem with SGE, likely due to the company’s upgraded PaLM 2 language model under the hood.

That’s not to say Google never makes any mistakes or states confusing facts. Like, for example, when I searched for “best Android phones”, the AI-generated summary matched our recommendations almost perfectly. However, it chose to highlight the Galaxy S23 as “Good for gaming” and the Pixel 7 as “Good for business”.

How did Google’s AI arrive at these conclusions? At a glance, it’s unclear if those labels are grounded in reality, so it’s important to exercise caution. That’s especially true when you consider that product recommendations directly link you to Amazon, Best Buy, and other retailers.

In my opinion, there’s a potential conflict of interest when AI-generated search results directly encourage you to make a purchase. But perhaps it’s no different than sponsored listings that still appear on the search result page.

How to activate and use generative AI in Google Search

Google has released a flurry of experimental AI features like SGE and MusicLM, but they’re only available to a limited audience for now. The good news is that you can sign up right away. Here’s how:

1. Navigate to Google’s AI Labs page using the Chrome web browser.
2. Scroll down and click on the Learn more or Join waitlist button.
3. If you’re logged out, sign in using your primary Google account.
4. To confirm your participation, simply click the Join waitlist button.

After joining the waitlist in late May, I received the invitation to try Google’s Search Generative Experience less than a week later. However, invitations seem to roll out in batches so you may get access sooner or later than me.

It’s also worth noting that SGE is only available in the US, India, and Japan for now. However, some users have reported success with using a VPN to join the waitlist and access the feature once invited.

Once you get the email inviting you to try Google’s Search Labs features, follow these steps:

1. Navigate to the Google homepage.
2. Click on the Search Labs icon located at the top-right corner of your browser’s window.
3. Finally, activate the toggle associated with SGE.
4. You may also enable Code Tips and Add to Sheets at this point. The latter is a stepping stone to Google’s Duet AI for Workspace.

That’s it! The AI-generated text you’ve seen in screenshots throughout this article should now show up in your Google searches on both desktop and mobile.

FAQs