Anthropic releases Claude 3.5 Sonnet, claims it outperforms GPT-4o and Gemini 1.5 Pro

Anthropic has released Claude 3.5 Sonnet, the first of three models in the AI startup’s next-generation Claude 3.5 large language model family. According to Anthropic’s official announcement, the new model “sets new industry benchmarks” across various AI benchmarks. Specifically, the company claims that Claude 3.5 Sonnet delivers more accurate responses than both GPT-4o and Gemini 1.5 Pro.

Similar to ChatGPT, Anthropic is making Claude 3.5 Sonnet available for free — albeit in a limited capacity. We ran into the rate limit after just a handful of responses, beyond which you’ll be prompted to pay $20 per month for a Pro plan or wait a few hours for the counter to reset.

Still, Claude 3.5 Sonnet boasts impressive results across various benchmarks like coding, math, and graduate-level understanding — at least according to figures shared directly by Anthropic. While self-reported benchmarks aren’t necessarily reliable, Anthropic’s previous best model keeps up with most of the competition in crowdsourced LLM leaderboards.

The new model also brings improved vision capabilities, which is most noticeable in reasoning tasks like reading and interpreting graphs. Anthropic says Claude 3.5 Sonnet can accurately transcribe text from images, once again with accuracy that rivals GPT-4o and Gemini 1.5 Pro.

Claude 3.5 Sonnet resembles Google’s latest LLM in that the middle-sized model surpasses the largest variant of its predecessor. Gemini 1.5 Pro is currently the search giant’s best model and surpasses Gemini 1.0 Ultra. Similarly, Claude 3.5 Sonnet takes the top spot for now and replaces Claude 3 Opus at a fraction of the cost. However, a larger “Opus” variant of Claude 3.5 will also be released later this year, alongside a smaller and faster “Haiku” model.

For now, Anthropic has also announced a new Artifacts feature that allows Claude to generate code snippets, documents, and even small games in a new window, separate from the chat interface. While this feature is experimental for now, the company says it’s the foundation of a “collaborative work environment.” Anthropic envisions a future where teams can work on design or code projects together in a shared space, similar to Google Docs’ real-time collaboration.

Anthropic received a whopping $2.75 billion from Amazon in its latest funding round this March. The company has also received a funding commitment of $2 billion from Google, with at least $500 million already disbursed so far. So even though Claude hasn’t become a household name like ChatGPT yet, Anthropic is one of OpenAI’s biggest rivals. However, neither Amazon nor Google has yet moved to integrate Claude into their products.