More Android apps could soon save you from sending accidental nudes

Google has been working surprisingly hard at improving Android’s system-level Photo Picker experience, as we’ve spotted features in progress, such as a built-in media editor and a location data toggle that some apps have already begun to adopt. We’ve now spotted Google preparing to bring its on-device Sensitive Content Warning system to the Android Photo Picker, potentially letting third-party apps detect and warn users about nudity before media is shared.

Google has adopted the Sensitive Content Warning system in Google Messages as a setting that, when enabled, automatically detects, blurs, and warns you about incoming or outgoing images and videos containing nudity. This system is powered by the Android System SafetyCore, the underlying Google system service that provides a common infrastructure for apps to use to protect their users from unwanted content. SafetyCore runs its content classification process exclusively on-device and claims not to share the results with Google.

Further, SafetyCore only comes into play when an application integrates with SafetyCore and specifically requests that content be classified. In Google Messages, images and videos aren’t classified unless they’re sent through the app with the Sensitive Content Warnings feature turned on.

According to code discovered in Android 17 QPR2 Beta 4, Google is now working to integrate Android System SafetyCore within the Android Photo Picker.

We couldn’t piece together the complete picture for this feature in action from the available code just yet. But we can form a reasonable guess based on how SafetyCore works in Google Messages.

Presumably, once the feature is fully integrated into Photo Picker and goes live, it will protect users from accidentally sending unwanted images and videos to others. It will potentially work when a user tries to send media that contains nudity. Since a lot of apps use Android’s Photo Picker, this feature will allow these apps to request the Photo Picker to run a nudity check on the selected media before it is sent, which could potentially save you from accidentally embarrassing yourself.

Going a step further, depending on the app and how it implements age-based restrictions, apps could also stop minors and underage users from sending media that contains nudity. Currently, Google already forces the Sensitive Content Warnings feature for Supervised users in Google Messages, while unsupervised teens (13-17 years of age) can have the feature disabled through a parent, though there are no stops in place.

The obvious limitation of Photo Picker’s upcoming SafetyCore integration is that users could circumvent it by sending images and videos as files, bypassing Photo Picker itself. Of course, apps could also simply stick to warning users about the content they are about to send, like Google Messages does, rather than restricting them.

Still, SafetyCore integration within Photo Picker could save third-party apps from building their own content moderation tools while respecting your privacy and needs. We’ll keep you updated as soon as we learn more about the feature.