What if a cheap, a fast AI could remember everything?

@agenticamit

12.1K followers · usually 3.9K views a post

Views

3.8K

Likes

116

Comments

0

Shares

19

SoundTikTok sound · original sound - agenticamit

Promoted product

Agentic Amitagenticamit.com

Real failures. Useful lessons. Practical guides on AI agents, prompts, courses, and the research behind them.

4 videos

Why it worked

The video effectively explains a complex AI concept using clear visuals and a simple analogy, making it accessible and engaging for a broad audience interested in AI advancements.

Summary

This video explains a new AI memory model that combines the benefits of Transformers and RNNs by taking snapshots of memory and blending them. This allows for affordable, long-context memory that can be dialed in to suit specific needs.

Structure

1Introduction to the problem of AI memory cost vs. recall
2Explaining the trade-off between Transformers and RNNs
3Introducing the snapshotting and blending method
4

Original caption

transformers remember but cost a fortune; RNNs are cheap but forget. this model snapshots its memory and blends the relevant ones — recall you can afford #AIresearch #machinelearning #LLM #longcontext

More from @agenticamit

Product placement

Agentic Amit (creator's name, likely the entity behind the product/service being promoted, acts as a brand for the educational content)

Call to action

Link is in my bio.

On-screen text

one fixed memory what was it? grow the memory. the 50-second paper MEMORY CACHING RNNS THAT REMEMBER a trade-off, two ways TRANSFORMER keep every token perfect recall cost RNN one fixed state overwrites the past cheap, but forgets D(L*) D(L) PERFECT RECALL VS CHEAP COMPUTE photograph the memory at checkpoints live hidden state 1 snapshot kept 2 snapshots kept 4 snapshots kept CHECKPOINT THE HIDDEN STATE retrieve 1 of 5 retrieve 2 of 5 RETRIEVE + AGGREGATE, NOT RE-READ memory you can dial in O(L) - RNN effective memory O(L*) - Transformer more checkpoints recall cost vs length barq D(L) D(L*) <L closes the gap, keeps the budget prior RNN Memory Caching Transformer gap O(L) sub-quad NEAR-TRANSFORMER Transformer Transformer Recall, RNN Budget BEATS PRIOR RECURRENT MODELS LONG CONTEXT remember more, pay less cache the memory. one fixed cell recalls on demand growing memory PAPER'S LINKED IN BIO Agentic Amit

Transcript

What if a cheap, a fast AI could remember everything? Because right now AI memory comes with a brutal catch. The model that remembers everything, Transformers, cost a fortune. And the cheap ones are an ends, stay fast by keeping one tiny memory and then just paving over the past. For years, you had to pick one of these two. The new design refuses to pick, and the trick behind it is almost embrasingly simple. As the R&N runs, it just takes snapshots. It photographs its own memory at checkpoints along the way. Little saves it can come back to later. So when it needs something from the past, it doesn't just re-read the whole sequence. It pulls the relevant snapshots and blends them together. And the best part is that it's a dial. Effective memory now grows with the length of what you feed it, so you can slide anywhere between R&N cheap and transformer complete. The payoff here is that it closes the gap with transformers on recall and beats every older recurring model on language and long contests. So finally, memory you can actually afford to keep. Link is in my bio.