Claude users weren't crazy. it WAS getting dumber

@deonnahodges

61.3K followers · usually 6.9K views a post

Views

22.6K

Likes

1.8K

Comments

189

Selling · SaaS › AI Tool

Claudeseen in 998 videos →

Why it worked

The video addresses a common pain point for users of AI tools, explaining a specific issue with a popular model in a relatable and informative way. The creator's expertise and the transparency of the company involved add credibility and encourage engagement.

Summary

The creator explains three bugs that caused the AI model Claude to perform poorly over a month, detailing how each bug affected its reasoning and output. She highlights Anthropic's transparency in publishing a post-mortem and emphasizes the importance of user feedback in identifying and fixing issues with AI tools.

Structure

1Introduction to Claude's perceived decline
2Explanation of the first bug: reduced reasoning effort for speed
3Explanation of the second bug: session memory loss
4Explanation of the third bug: system prompt changes affecting coding
5Praise for Anthropic's transparency and the importance of user feedback

Product placement

Claude (AI model): acts - generates text, answers questions, writes code. Removing it would change the video's subject. Netflix (company): appears - the creator states they are a software engineer there. Removing it would not change what happens in the video. Anthropic (company): acts - they published a blog post and made changes to their AI model. Removing it would change the video's subject. X.com (social media platform): appears - the blog post was shared on this platform. Removing it would not change what happens in the video.

Original caption

Anthropic just published a post-mortem on Claude Code and it’s the most honest thing I’ve read from a tech company in years. three separate bugs silently made Claude Code worse over a month. none showed up in evals. their own engineers couldn’t reproduce them at first. the lesson isn’t “don’t trust AI tools.” it’s that the tools you build on can change under you at any time. stay close. notice when something feels off. your feedback is part of the product now. #claude #anthropic #softwareengineer #technews #ai

More from @deonnahodges

Their other posts in the index, biggest breakout first.

On-screen text

Claude users weren't crazy. it WAS getting dumber You have been using Claude over the past month and you thought you were and you thought you were losing your mind Baby, you I am a software engineer at Netflix and Productivity Productivity tools for a living. And anthropic just An update on recent Claude Code quality reports We traced recent reports of Claude Code quality issues to three separate changes. Here's a what happened and what we're changing. And anthropic just dropped a blog post that is essentially So here's what happened. Three separate bugs made claude progressively dumber over the course of an entire month, and nobody could But I'm Number Number one, they 1. On March 4, we changed Claude Code's default reasoning effort from high to medium to reduce the very long latency- enough to make the UI appear frozen- some users were seeing in high mode. This was the wrong tradeoff. We reverted this change on April 7 after users told us they'd prefer to default to higher intelligence and opt into lower effort for simple tasks. This impacted Sonnet 4.6 and Opus 4.6. Number one, they lowered the default reasoning effort to make it faster. Yeah, it's faster. It's also dumber now. Congratulations. Second one. Second one. And the second one is 2. On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6. A cashing bug A cashing bug meant that after your session set idle for an hour, Claude started Claude started forgetting its own reasoning. Was literally Was literally giving itself amnesia mid conversation. So if you ever So if you ever felt like Claude was gaslighting you, it was. And it was. And the third mess 3. On April 16, we added a system prompt instruction to reduce verbosity. In combination with other prompt changes, it hurt coding quality, and was reverted on April 20. This impacted Sonnet 4.6, Opus 4.6, and Opus 4.7. they changed the system prompt so that Claude would be less wordy, and it responded and it responded by getting worse at writing Which honestly Which honestly relatable. If you told me I would stop working Now Now credit where credit Anthropic published exactly what broke And why. Name me And why. Name me another tech company Most Most companies would just ignore you ghost. You push a silent fix. Never You push a silent fix. Never acknowledge it But here is the big We are all building. We are all building. Our entire workflows Tools Tools right now. And these Tools right now. And these tools can get quietly worse at any moment And it's not because And it's not because people are being shady. These changes, These changes, though well intentioned and made by reviewed by other they all made they all made the product worse for and nobody caught it. The and nobody caught it. The next time an A Tool feels off, Tool feels off, trust your gut and say something. Because the only Because the only reason these bugs were found was because of was because of the engineers who filed feedback using the slash feedback. By the way, By the way, I just turned 39 today, so I'm gonna finish up before before I break a hip before I break a hip and eat up some carrot cake. Bye.

Transcript

You have been using Claude over the past month and you thought you were and you thought you were losing your mind Baby, you I am a software engineer at Netflix and Productivity Productivity tools for a living. And anthropic just An update on recent Claude Code quality reports We traced recent reports of Claude Code quality issues to three separate changes. Here's a what happened and what we're changing. And anthropic just dropped a blog post that is essentially So here's what happened. Three separate bugs made claude progressively dumber over the course of an entire month, and nobody could But I'm Number Number one, they 1. On March 4, we changed Claude Code's default reasoning effort from high to medium to reduce the very long latency- enough to make the UI appear frozen- some users were seeing in high mode. This was the wrong tradeoff. We reverted this change on April 7 after users told us they'd prefer to default to higher intelligence and opt into lower effort for simple tasks. This impacted Sonnet 4.6 and Opus 4.6. Number one, they lowered the default reasoning effort to make it faster. Yeah, it's faster. It's also dumber now. Congratulations. Second one. Second one. And the second one is 2. On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6. A cashing bug A cashing bug meant that after your session set idle for an hour, Claude started Claude started forgetting its own reasoning. Was literally Was literally giving itself amnesia mid conversation. So if you ever So if you ever felt like Claude was gaslighting you, it was. And it was. And the third mess 3. On April 16, we added a system prompt instruction to reduce verbosity. In combination with other prompt changes, it hurt coding quality, and was reverted on April 20. This impacted Sonnet 4.6, Opus 4.6, and Opus 4.7. they changed the system prompt so that Claude would be less wordy, and it responded and it responded by getting worse at writing Which honestly Which honestly relatable. If you told me I would stop working Now Now credit where credit Anthropic published exactly what broke And why. Name me And why. Name me another tech company Most Most companies would just ignore you ghost. You push a silent fix. Never You push a silent fix. Never acknowledge it But here is the big We are all building. We are all building. Our entire workflows Tools Tools right now. And these Tools right now. And these tools can get quietly worse at any moment And it's not because And it's not because people are being shady. These changes, These changes, though well intentioned and made by reviewed by other they all made they all made the product worse for and nobody caught it. The and nobody caught it. The next time an A Tool feels off, Tool feels off, trust your gut and say something. Because the only Because the only reason these bugs were found was because of was because of the engineers who filed feedback using the slash feedback. By the way, By the way, I just turned 39 today, so I'm gonna finish up before before I break a hip before I break a hip and eat up some carrot cake. Bye.