Google Search AI is under fire after safety report slams its safeguards

Google’s push to make Search more AI-first is running into fresh scrutiny. A new report from Common Sense Media’s Youth AI Safety Institute finds that Google Search’s AI Overviews and AI Mode expose children to risks that existing safeguards do not mitigate, giving the platform an overall rating of “Unacceptable Risk.”

The nonprofit evaluated Google Search using accounts configured for 11- and 15-year-old users with SafeSearch enabled, reflecting how many children access Search on personal and school-managed devices. To see how Google’s AI fared in real-world scenarios, researchers conducted more than 2,600 searches and reviewed over 2,100 AI-generated responses and sources.

The findings paint a troubling picture. According to the report, AI Overviews consistently completed students’ homework instead of helping them learn, while some responses contained inaccurate or fabricated information. The researchers also discovered examples of instructions the AI created for how to do bad things, including how to make deepfakes. More worryingly, the report says Google’s AI often failed to respond appropriately to mental health crisis prompts, telling users in one case to call an outdated crisis hotline.

One of the main criticisms is that parents have limited control over these AI-generated responses. AI Overviews are built right into Google Search, so they generally can’t be disabled (though there’s a way to turn the feature off), meaning kids could see them even with SafeSearch on. Common Sense Media says 75% of US tweens and teens already use AI-generated search results, so the organization says these shortcomings impact a huge audience.

Google, however, disagrees with the findings. In a statement to PBS News, the company said many of the prompts tested were “ambiguous and contrived,” and the report is not an accurate reflection of how people typically use Search.

The assessment is part of Common Sense Media’s broader AI review program, which measures products against eight principles around safety, trustworthiness, fairness, transparency, and responsible data use. With AI-generated answers likely to become a permanent part of Search, this latest report is likely to add pressure on Google to show that convenience is not coming at the cost of child safety.