I fed Pixel Screenshots 1,800 screenshots and asked it questions — here's what I learned

The Google Pixel 9 series has been impressive on many fronts, especially in the many creative ways it uses on-device AI to improve your smartphone experience. Google is using these local smarts through a brand new app called Pixel Screenshots that elevates spontaneous screenshots from temporary and forgettable captures to searchable and actionable context cards.

I have been using the Pixel 9 Pro XL as my daily driver for the past few weeks, and after feeding the app with over 1,800 screenshots (of which only 300 got processed, but more on that later), I gotta admit I’m impressed with how functional it has made screenshots for me. That being said, there are rough edges that Google needs to smooth out, and there’s plenty of room to grow, too.

So, what is the Pixel Screenshots app exactly?

The Pixel Screenshots app is the quintessential answer to the question, “What if the hundreds of screenshots I take every year could actually be useful in some way?”

It is a Pixel-exclusive app that uses on-device Gemini Nano with multimodality to organize all of your screenshots and help you recall all of the information in them. Think of it as an app that puts every screenshot through an OCR process and creates a database of the embedded information, letting you ask questions around those screenshots and get answers instantly. Thanks to the app, you can take a screenshot and forget about it but still have access to its contents and the tiniest bits of data in it at a moment’s notice.

What does the Pixel Screenshots app really do?

Google says that the Pixel Screenshots app has three main functions: Save, Organize, and Recall.

Save — A brand new memory for your screenshots

When you take a screenshot on the new Pixel 9 series, it is saved to your phone’s local storage, just like on any other Android flagship.

The magic begins at this stage, as the screenshot is also fed into the Pixel Screenshots app, which automatically begins processing it. It scans through and recognizes details in the image, such as addresses, prices, contact numbers, and more, alongside other metadata, such as the app or website on which the screenshot was taken.

The Screenshots app then uses AI to generate a title and summary of the information. Going a step beyond, it also suggests further actions, such as adding dates to Google Calendar or adding reminders. I was happy to see that the AI is smart enough to recommend the right time for the reminder in some cases, as you can see for the Black Friday reminder above. I no longer need to manually type dates and times when adding reminders — just screenshot them and set a reminder, done.

Organize — Manually bundle and collect similar screenshots

The Pixel Screenshots app also gives you the perfect platform to create collections of different screenshots. If you’ve ever wanted to create different folders for your screenshots without cluttering your gallery app or affecting your ability to locate something later, this is the right place to do it. I found it handy when shopping for clothing and accessories, but over time, it can also become an awesome tool for organizing book recommendations I come across online or cool tech products.

If you’re into home design, travel, food, or anything else, these collections can almost be your own local Pinterest. Plus, you can add notes to collections to add more recall value on why you created the collection.

Recall — Find anything, anytime

The bigger highlight of the Pixel Screenshots app is that it takes the information from your screenshots and makes it all searchable. You can ask questions using natural language in the search bar within the app, and the Screenshots app will present an answer from the screenshots it has processed, along with the exact capture it came from. That way, you can tap and see the entire context of that answer.

Google says you will also be able to use Gemini anywhere on your phone to help you find a saved screenshot in the future. However, this functionality isn’t live yet on my phone (or I am just doing it wrong and couldn’t get it to work).

The Pixel Screenshots experience: Changing the way you screenshot

The Pixel Screenshots app solves a key dilemma for me by adding a long-term purpose to screenshots beyond immediate sharing.

I am someone who takes a lot of screenshots of anything and everything mindlessly. Whether it be a meme to share with my friends that I can’t bother to save the image of, spicy gossip in the group chats, a product recommendation that I’m sharing with a relative, or something that I just want to mindlessly save just in case I need to refer to it in the future, I am quite trigger happy with my volume down and power button pushing, and I do it without as much of a second thought.

Despite relying on apps like Files by Google and Google Photos to keep things tidy, I have accumulated over 1800 screenshots. These have all been fed into the Pixel Screenshots app, but the Pixel 9 Pro XL throttles the app to process only a limited number of images every day. Because of this, I could only get about 300 images to process, and it just… stopped. I hope my entire library gets processed eventually, but for now, I haven’t been able to manually force the app to analyze the rest of my huge screenshot collection. I wish there was a button to do just that.

And that is the first real problem I faced with Pixel Screenshots. If you have a large collection like I do, you may not be able to import it easily into the app. Or you might want to do it in batches. For every screenshot you take after that, though, things should be more straightforward as the app immediately processes new captures when your phone is a bit idle. Throttling shouldn’t be an issue there — at least not with normal daily screenshot use.

Screenshots could soon replace your quick notes

For these 300 images, the Pixel Screenshots app gave me a sense of objective. In the few weeks of using the Pixel 9 Pro XL, I have started screenshotting more information, knowing that I can properly search it later. It’s a bit lazy in a way but very practical, too. No need to bookmark pages, take notes, save images, etc… Just trust that the Screenshots app will handle it for me.

For instance, I was reading reviews of a local movie theatre on Reddit and took a screenshot of a post and its comments. The information at hand wasn’t useful enough to make a dedicated note in my note-taking app, but it wasn’t trivial enough either to ignore completely.

When I had to recall the experience to someone, I asked the Screenshots app about the movie theatre. It instantly presented a one-line summary and the exact screenshot that I was trying to recall. This would have otherwise taken me many minutes of scrolling through random screenshots or another Google search to locate the exact Reddit post.

When I had to tell someone the flavors of ice cream roll cakes that I had recently browsed, I didn’t need to open my food delivery app and find my way to the product entry, neither did I need to go through several unrelated screenshots to get to my information. I asked Pixel Screenshots, and it surfaced the exact screenshot I was looking for without a fuss. The same played out with some monsoon alerts I had captured.

In all the other instances I’ve used the app, I liked how most of my images have been contextualized, letting me search and group together related captures. This has made trawling through hundreds of screenshots such a breeze and made it so much easier to locate something I had captured a while ago.

You still have to verify the information for hallucination

However, AI isn’t perfect, and the Screenshots app does hallucinate with some of its responses. While its screenshot identification and contextual sorting have been amazing, the generated summaries can be dicey with some wrong answers. Google has a disclaimer in place for incorrect info, and I recommend paying heed to it.

For instance, in one of the screenshots, it told me the price of the ice cream roll cake is $10, which is absolutely incorrect (neither the number nor the currency matches any info in the screenshot). In another instance, it got the date wrong as July 7, 2024, when the social media post was made on July 8, 2024, and mentions “today.” In yet another instance, it got the date wrong for a music festival by seemingly associating a Google Photos album screenshot with an imaginary date (the screenshot had no date, and the screenshot I was looking for hadn’t been processed by then).

Over the course of my usage, I quickly became more reliant on the Pixel Screenshots app to locate screenshots and less trusting of it for spitting out correct information from those screenshots.

Verifying the information seems like an unnecessary step if I can just land on the correct screenshot and glance at the info, so I have started ignoring the AI summary much like I ignore it in my routine Google searches.

What the Pixel Screenshots app could improve

It’s evident that the Pixel Screenshots app is still in its nascent stages. While the app currently provides a good experience, it has tremendous potential to grow.

For instance, I’d love to be able to archive older screenshots. This feature may seem unnecessary given the app’s purpose of screenshotting-and-forgetting, but I’d still love to have manual control and the ability to set aside older captures. Google already duplicates basic image editing features within Screenshots, so an archive or cleanup feature would be right at home.

Beyond that, several oversights need to be ironed out. For instance, multi-selecting screenshots is a chore since there is no drag-to-mass-select functionality. There is no scroll bar for quickly scrolling through screenshots, nor is there any timeline indication, making the app feel like a long, endless sea of screenshots.

I would have also loved it if Pixel Screenshots could auto-categorize screenshots. One easy categorization could be by app, which would create item groups for apps like WhatsApp, Instagram, Chrome, etc. Another could be to borrow some of the smarts that are powering Google Photos and make smart categories like shopping lists, travel bucket lists, upcoming events, books or food or media recommendations, and so on.

Speaking of Google Photos, there is no easy way to import old screenshots into the app. If all of your screenshots are successfully migrated from your old phone as part of the Android setup, then good for you. Otherwise, you are stuck. I manually transferred my older screenshots to Pictures > Screenshots but, as I said earlier, got stuck trying to get them all processed in time for this review.

That gives me my next wishlist item for improvement: There should also be an option to force AI processing of pending screenshots. You can force-process individual screenshots, but there is no other way to bypass the throttled processing. If you’re thinking of importing a ton of older screenshots, I can’t promise you that all of them will be processed quickly enough (or ever since mine are still processing weeks into usage).

While I appreciate the on-device processing that is the highlight of the Pixel Screenshots app, I would also love to see the functionality carry over to Google Photos as an opt-in feature. I wouldn’t mind if Google opened up screenshot processing to all Android users, not just Pixel owners, but kept it locked behind a Google One paywall for them (like other premium Google Photos features). Such a move would add universality to the feature, as all the screenshots that I have already backed up on Google Photos could be cleanly indexed, categorized, and instantly searchable. Years of screenshots would no longer be tied down to a smartphone, creating your personalized AI knowledge base.

The Pixel Screenshots app is a good start, and I appreciate this use of on-device AI

You can consider Pixel Screenshots a fairly niche use case for AI, but it makes sense to me. There’s only so much text and image generation an average user needs, and the Pixel has that covered through Pixel Studio and Gemini. Apps like the Pixel Weather app and this Pixel Screenshots app are additional avenues for Google to flex its on-device AI muscles, and I am optimistic about their future. This is good and useful AI, and I like it.