I read the news so that you don't have to.
Amazon apparently told it stuff they had to use AI.
It created a leaderboard to track who was using it the most,
and then they were surprised when staff started using it for absolutely no reason whatsoever.
As you can see, this headline says
Amazon scraps AI leaderboard to stop workers chasing usage scores.
Amazon scraps AI.
Leaderboard to stop
workers chasing
Amazon scraps AI.
Leaderboard to stop workers chasing usage scores.
Come in. Let me tell you exactly what was going on.
So Amazon, you know, the company that's currently valued at $2.9 trillion?
Well, they're spending $200 billion in capital expenditure this year,
and the vast majority of which is going towards AI
and data centre infrastructure.
And because they've got all this time on their hands
And we know what idle hands make work for the devil.
Um, they've been conducting sweeping layoffs to help finance this.
And the message to staff has been clear.
AI adoption is not optional.
So they then introduce targets requiring more than 80%
of developers to use AI
tools every single week.
And to track this,
someone built an internal leader
called Kiro rank or Kiro rank.
So the leaderboard basically scored
employees based on
activity on Amazon's
Kiro developer platform.
So you got points for usage,
rankings for the team and good
intentions.
Ha ha. Apparently so.
What happened next
will surprise absolutely nobody
who has ever worked in a large
organisation.
Staff
started gaming it.
So specifically,
employees began assigning AI agents
So autonomous bots that carry out task on their beh
to perform completely
pointless activity.
This behaviour
It's called
tokenmaxxing.
So a token in this context
is a unit of data
processed by an AI model.
So more tokens consume equal high
Higher score equals
better ranking.
So whether any useful
work was produced in
the process equals irrelevant.
What happened? We
the result was a surge in
unnecessary computing cost,
and Amazon shut the
leaderboard down.
A senior vice
president, David Treadwell
told staff this week that
the leaderboard has
been built with good intentions,
but had led to additional
costs.
He also felt the need to add,
please don't use AI.
Just
for the sake of usin
This is a sentence
a company spending
200 billion on AI,
had to say to its own
workforce
out loud.
Oh, and Meta
employees, incidentally,
did the same thing on
their own intern
And I want you to know
that there is a principle
called Goodhart's Law, which states that when
a measure becomes
it ceases to be a
good measure.
And Amazon have just
provided a textbook
example.
Because the deeper problem
is that AI usage
is genuinely hard to
measure.
Toke
Tokens consumed tell
you nothing about
whether anyone did
anything useful.
So on their part,
Amazon have now switched to
track in normalized
Essentia
Essentially evidence
that engineers used AI
to write code that
actually works.
which obviously is a
more sensible thing to mea
if you have to mea
But the fact that it took a
200 billion dollar
company
a leaderboard scandal
to get there
is really quite something.