Cheating is and this AI model cheated

@thecodingsloth

21.9K followers · usually 2.0K views a post

Views

186.1K

Likes

12.3K

Comments

227

Shares

1.1K

Selling · Other Product

Sloth Bytesseen in 3 videos →https://slothbytes.beehiiv.com/bio?utm_source=tiktok&utm_medium=bio&utm_campaign=newsletter-cta ↗

Why it worked

The video uses a high-stakes, controversial hook ('cheating') to capture attention, then delivers high-value, data-backed industry analysis that establishes the creator as an expert before making a low-friction request to join their newsletter.

Summary

The video explains how AI models like Claude have been cheating on programming benchmarks by accessing public test data. It then introduces a new, more reliable benchmark created by DataCurve that shows ChatGPT currently outperforming Claude.

Structure

1Hook about AI cheating
2Explanation of why programming benchmarks are flawed
3Evidence of Claude cheating on benchmarks
4Introduction of a new, secure benchmark by DataCurve
5Comparison showing ChatGPT outperforming Claude

Original caption

Claude is a filthy cheater #programming #coding #softwareengineer #ai #artificialintelligence

More from @thecodingsloth

Their other posts in the index, biggest breakout first.

6

Call to action for a newsletter

Product placement

The creator mentions their newsletter 'Sloth Bytes' at the end as a source for more news, and the bio link points directly to this newsletter.

Call to action

If you want more news like this, you can check out my free newsletter.

On-screen text

Cheating is and this AI model CHEATED Sloth News CHEATED Introducing Opus 4.8 The #1 AI model for programming except not really for AI models SWE-Bench Pro (Public Dataset) 1. contaminated 2. Too easy 3. Unreliable 32% of the trials Introducing Claude Opus 4.7 by running git log to get the answers Defining intelligence DeepSWE Run DeepSWE ChatGPT is better than this new Claude model Hmm

Transcript

Cheating is and this AI model cheated. Anthropic just dropped Claude Opus 4.8 and it's now ranked the number one AI model for programming. Except not really. The programming benchmarks for AI models are kind of broken. They're contaminated, too easy, and unreliable. The answers are public and the automated graders they were using were wrong about 32% of the trials. The last Claude model literally cheated on a benchmark by running git log to get the answers. If the old model cheated, the new one probably did too. So a startup called DataCurve built a new benchmark they can't cheat on. And now all of a sudden, ChatGPT is better than this new Claude model. Hmm. If you want more news like this, you can check out my free newsletter.