At the frontier of AI, a few major companies are continuously releasing important updates to their large language models (LLMs). One that's been turning heads lately is Anthropic's newest model: Claude 3.5 Sonnet. But can it dethrone my go-to AI companion, ChatGPT? 

I decided to put it to the test.

The basics of Claude’s new model

Anthropic's Claude 3.5 Sonnet outperforms GPT-4o (ChatGPT’s latest update, as of this writing) in many ways, particularly those involving reasoning. It makes some strides in graduate-level reasoning and text-based problem-solving, and although its scores in those areas are still relatively low, they should still give Sonnet an advantage in tackling complex prompts, solving multi-step problems, and doing nuanced analysis of text.

Like GPT-4o, Claude 3.5 Sonnet is natively multimodal. That means it can interpret and understand images, including charts and graphs, and transcribe text from images.

It also introduced Artifacts, an extremely useful feature for working with AI on standalone documents—think essays, simulations, and computer programs—that you develop collaboratively with the tool. It lets you see your work in real time and even lets you download the project when you’re done.

How to get access to Claude 3.5 Sonnet

You can sign up for Claude 3.5 Sonnet at claude.ai or through their iOS or Android apps. A free account gives you a set number of prompts per day, while the $20/month professional plan bumps that up by about five times. You can also access the model on poe.com. For my tests, I tried both platforms to get a well-rounded experience.

What I loved about Claude

Claude 3.5 Sonnet has a lot to offer. Here are my favorite features compared to ChatGPT's GPT-4o model.

Pretty good styles

I typically avoid using AI for first drafts; I prefer to write my initial messy version and then use AI to polish it. But for this test, I tried Claude and ChatGPT in two modes: one with an outline and one with just a general topic.

With just the topic, both tools struggled as expected—it's an impossible challenge for the AI when you don’t provide clear guidance. The AI has to guess what you want, and usually the results are unsatisfactory. 

But I was pleasantly surprised by how well Claude 3.5 Sonnet expanded an outline into coherent paragraphs. Its style varied quite a bit between runs, but since it didn't have access to my "voice" custom instructions, just a few generic descriptors, I cut it some slack. When I asked for a less formal tone, it did a great job making the writing more approachable and less stuffy.

Did it save time? Probably not. Did it improve the end result? I think so. Like all AI writing, it had trouble nailing that clear throughline, so I still needed to do heavy editing to get the right flow. But I liked (and kept) a lot of its phrasing suggestions.

Artifacts are intriguing, but imperfect

Artifacts are one of Claude 3.5 Sonnet's coolest new features. Basically, instead of doing everything in the main chat window, Anthropic has changed up the UX design so that you can work with certain kinds of output like documents, code, and simulations side-by-side. I can see this being especially useful for those who like to work in a "cyborg" style that blends human and AI input.

You don't have to worry too much about accidental overwrites either—you can move back and forth between drafts using the arrows at the bottom. I particularly enjoyed its ability to generate simulations based on descriptions, and then iterate on them to change and add features. I had a blast putting together a simulation of the solar system, complete with moons and dwarf planets.‎

Great brainstorming and wider analysis

Analysis of text and nuanced understanding of prompts are supposed to be some of Claude 3.5 Sonnet's biggest advantages, and after my tests, I tend to agree. 

When I asked both Sonnet and ChatGPT to summarize an article about the Ayamé-Yuval-Oliver TikTok drama, their summaries were both pretty good. (Note: Since Claude 3.5 Sonnet doesn't have browsing capabilities, I had to download the article as a PDF and upload the file.) Normally, I would advise against using double-barrel prompts like this, but I wanted to see how both models handled more complex requests.

When it came to discussing the implications for the future of entertainment, I was pleasantly surprised by the quality of ideas Sonnet generated. It not only produced more ideas (seven vs. ChatGPT's four, with three overlapping), but its responses were also denser than GPT-4's. Head-to-head with GPT-4o, I could cherry-pick the best ideas from both.

Both tools interpreted my prompts slightly differently. Consistently, ChatGPT provided more detailed information on fewer points, while Sonnet was more succinct.

For example, when I asked about architectural styles in three different cities, Claude provided a bullet-point list while ChatGPT gave a lengthy response with an example.

That context window is amazing

Every so often, I bump up against ChatGPT-4's 32K token context limit, especially when working with multiple files or particularly long documents. But Claude 3.5 Sonnet boasts a massive 200K context window, allowing it to remember things from much earlier in the conversation and work with much longer documents.

These capabilities represent what I consider truly magical about AI—the ability to do things that would otherwise be impossible. Sonnet can summarize very long documents with impressive accuracy, far outperforming GPT-4o in this area.

A word of caution, though: Very long conversations quickly eat up your token limits, so it's good practice to start a new conversation when your work with the tool gets lengthy. As a bonus, this habit will get you into the good practice of saving your project TL;DRs in your prompt library, too.

Oh my god, SEARCH!

This isn’t about the model itself, but I am thrilled about the ability to search through my chats. Since I use AI tools so frequently, I often lose track of specific conversations. Thank you, Anthropic, for including this feature.

Room for improvement

Every cutting-edge model has its drawbacks, and when you're used to a certain workflow, switching can be challenging. Here are a few features that I missed.

Hallucinations and limitations on browsing

I've found ChatGPT-4 to be pretty good at avoiding hallucinations, so switching to Claude 3.5 Sonnet was jarring because it gave me a lot of incorrect information. I usually ask ChatGPT to browse when I'm looking for factual information to minimize hallucinations and provide references to check. Unfortunately, Claude Sonnet doesn't have web browsing capabilities yet, so I couldn't use this tactic to help.

Even more frustrating, when I asked Claude to add a Kismet element to its responses—one of my favorite custom instructions for ChatGPT—I consistently got hallucinations instead of quirky follow-ups, which was extremely irritating. So, be cautious!

SVG, not diffusion-based images

I enjoy ChatGPT's ability to generate diffusion-based images based on conversation context, a feature I missed with Claude.

Although Sonnet can generate SVG images using code, like ChatGPT, they have to be relatively simple. Both were able to generate a red circle inside a blue square, but neither could draw something more complex like Koch snowflake. Here are Claude's four attempts:

Like ChatGPT, Claude can't "see" the images it creates so you have to screenshot or download them and re-upload them to the chat. Definitely something I wish both platforms would fix.

No custom instructions or memory

I have my favorite custom instructions, and I was annoyed at having to copy and paste them into the chat. And despite my struggles with setting up ChatGPT-4o memories effectively, I missed that feature too.

Claude's long context window means you can add information manually, but it requires more effort on your part to make it work.

Mixed results with advanced prompts

I tested two of my favorite advanced prompts: the Flipped Interaction Pattern and the Persona Pattern. (To summarize, the Flipped Interaction Pattern has the AI ask you questions about your request until it has enough information to complete it; the Persona Pattern has the AI take on a role in its interactions with you). 

ChatGPT tends to get confused if you take the persona pattern too far because of its smaller context window, so Sonnet definitely has an edge here. I had such a fun time chatting with an AI character from one of my stories, as played by Claude, that I ran out of prompts even on the Pro plan. What can I say? It did a great job acting like a believable AI.

But I was disappointed with Sonnet's interpretation of the Flipped Interaction Pattern. Both models asked similar questions, but ChatGPT-4 dug a lot deeper with me, while Sonnet jumped into the task after only a few questions and didn't seem to fully interpret what I was looking for. As someone who uses this pattern frequently, I was disappointed to find Claude wasn't as good at it as ChatGPT.

The best of both worlds

So is it better to use Claude or ChatGPT? My answer is no: You should use both models.

It's worth switching to Sonnet for tasks it excels at, like working with huge or numerous documents in its expansive context window, as well as generating simulations or drafts in Artifacts. Keep in mind that both models still have room for improvement when it comes to reasoning, even if they're showing progress. We'll likely need to wait for future iterations to see significant advancements in that.

I found that working with both models in tandem gave me the best results. This matches with research on using AI tools for better brainstorming: combining the best outputs from different approaches often leads to better outcomes. While Sonnet tends to provide more information, it can sometimes require more effort to flesh its points out. But this is just its default behavior—good prompting techniques work well on both tools and can help you get the most out of each.

Here are my tips for maximizing your use of both models:

1. Use Sonnet for tasks requiring a longer context or memory and for more complex text analysis
2. Switch to ChatGPT (or Perplexity) for its web-browsing capabilities when fact-checking or trying to find external sources
3. Stick with ChatGPT (or Midjourney) for image generation
4. Compare outputs from both models on creative tasks to spark new ideas
5. Experiment with advanced prompts on both to see which responds better for your specific needs

So go ahead, mix it up, and let Claude and ChatGPT team up to bring out the best in your projects. Two AI brains are better than one.