LECUN LEAVES META, CALLS LLMS A DEAD END
He's Been Right About AI for 40 Years. Now He Thinks Everyone Is Wrong.
Yann LeCun invented many fundamental components of modern AI. Now he's convinced most in his field have been led astray by the siren song of large language models.
Yann LeCun is leaving Meta
after more than a decade
leading its AI research,
the entire industry
Meta's FAIR research division
in 2013.
And in recent years,
Meta's AI chief: LLMs will never
reach human-level intelligence
Sorry, Else -- AGI won't arrive next year
he's become increasingly vocal
in his belief
that large language models
will never reach superintelligence.
"We're easily fooled into thinking they are intelligent because of their fluency with language, but really, their understanding of reality is very superficial," he said.
"They're useful, there's no question about that. But on the path towards human-level intelligence, an LLM is basically an off-ramp, a distraction, a dead end."
Last year,
he called LLMs
"basically an off-ramp,
a distraction, a dead end"
with the reasoning being that
LLMs only understand text,
but true intelligence
requires understanding
how the physical world
actually works.
To illustrate current limitations, LeCun pointed to a striking comparison:
modern LLMs train on billions of tokens—an amount of text that would
take a human half a year to read. "A four-year-old child could
awake for only about 15,000 hours, processes a comparable amount of
data through visual perception alone.
A four-year-old child has seen as much data as the biggest LLM in the
form of visual perception, and for blind children, it's touch," LeCun said.
"That tells you a number of things. We're never going to get to human-
level intelligence by just training on text.
a four-year-old child
has processed vastly more data
about the physical world
through seeing
and touching things
more than any LLM
has processed through text alone.
"Never mind trying to reproduce human intelligence," LeCun said at the
summit. "We can't even reproduce cat intelligence or rat intelligence.
Any house cat can plan very highly complex actions."
And we can't even reproduce
cat or rat intelligence
because those animals
understand the physical world
and can plan
complex physical actions in turn.
for his departure here
is also quite telling.
Because this summer,
Alexandr Wang
Chief AI Officer
Meta brought in Alexandr Wang
from Scale
as Chief AI Officer
to lead a new division called
Meta Superintelligence Labs,
Meta names OpenAI's Shengjia Zhao
as chief scientist of AI
Superintelligence Lab
and they also brought on
Shengjia Zhao
as their chief scientist.
Both are LLM believers
that are focused on scaling,
and both effectively
became LeCun's superiors
in Meta's organizational structure.
Meta lays off 600 from 'bloated' AI
unit as Wang cements leadership
Last month,
Meta laid off hundreds
in its AI division,
including senior FAIR directors,
Personal Superintelligence
Over the last few months we have begun to see glimpses of our AI systems improving themselves.
The improvement is slow for now, but undeniable. Developing superintelligence is now in sight.
It seems clear that in the coming years, AI will improve all our existing systems and enable the
creative and discovery of new things that aren't imaginable today. But it is an open question what we
will direct superintelligence towards.
"superintelligence is now in sight."
So LeCun's departure
comes around the same time
as Meta consolidating
around a vision that he thinks
the wrong approach to take.
on his new startup,
which will focus on what he calls
"world models,"
which are basically AI systems
that can understand
the physical world
rather than just
processing text alone.
He thanked Meta founder and CEO Mark Zuckerberg and said that because
of his and other Meta executives' previous interest in AGI, Meta will be a
"partner in the new company". However, LeCun did not specify the nature
of the partnership.
Meta will partner
although neither party
has provided details
on what that partnership
would look like yet.