693B TOTAL PARAMETERS

@ai_vanta_ai

4.5K followers · usually 2.6K views a post

Views

683.5K

Likes

37.1K

Comments

443

Shares

6.4K

SoundTikTok sound · LET ME BE

Promoted product

colibri

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 - JustVugg/colibri

2 videos2 creators

Why it worked

The video effectively highlights a significant pain point in AI accessibility (the need for expensive GPUs) and presents a compelling solution (Colibri) that runs on existing hardware. The clear demonstration, comparison to familiar services like Netflix, and emphasis on cost savings and offline use make it highly appealing and shareable.

Summary

This video showcases 'Colibri,' a 744-billion parameter AI model that can run on a laptop without a dedicated GPU. It achieves this by streaming parts of the model from disk, similar to how Netflix streams content, making large AI models accessible offline and cost-free for token usage, thereby challenging the necessity of expensive GPUs.

Structure

1Introduction of Colibri and its no-GPU capability
2Comparison with traditional GPU-dependent AI models and NVIDIA's pitch
3

Original caption

A developer just ran a 744 billion parameter AI model called Calibri on a laptop without a GPU. By streaming parts of the model from disk like Netflix, Calibri makes massive AI accessible offline and zero-cost for tokens, challenging the need for expensive GPUs. #calibri #glm52 #offlinenlp #aitechnology #opensourceai

More from @ai_vanta_ai

Product placement

Colibri (AI model): ACTS - it is demonstrated running on a laptop without a GPU, and its functionality is explained. Removing it would change the video's topic. NVIDIA (GPU manufacturer): APPEARS - shown as an example of what is NOT needed, and a GPU chip is shown. Removing it would not change the video's topic. GLM 5.2 (AI model): ACTS - it is mentioned as the model being used by Colibri. Removing it would change the video's topic. OpenAI (AI company): APPEARS - mentioned as a comparison point for pricing. Removing it would not change the video's topic. Netflix (streaming service): APPEARS - used as an analogy for how Colibri streams data. Removing it would not change the video's topic. GitHub (code hosting platform): ACTS - the Colibri repository is shown and instructions to clone it are given. Removing it would change the video's topic.

Call to action

Check out the GitHub link in the comments for a full install guide.

On-screen text

693B TOTAL PARAMETERS NO GPU AI MODEL Safetensors Model size 744B params Files info Inference Providers NEW Text Generation AI model with no graphics card Vincenzo Fornaro JustVugg Follow Founder of Colibrì a tiny engine, immense model for affordable hardware. in Italy with GLM 5.2 that beats you pay for Nvidia's is that you need their GPUs like this and you just prove you don't called Calibri It works about ten Gigs of the model thinking at So Calibri part and streams off your hard as it needs it The same way Netflix streams instead of making you download it easier than it sounds the repo and run one set up file The whole thing builds itself The only heavy part is download 370 gigs on your drive you just type "Coli" in chat completely offline If you build apps type "Coli serve" instead into an open AI endpoint # Data Setup # Data Transformation transform = transforms.Compose([ transforms.ToTensor() ]) # Download Data train = datasets.MNIST(root='data', train=True, download=True, transform=transform) test = datasets.MNIST(root='data', train=False, download=True, transform=transform) # Data Loaders train_dataloader = DataLoader(dataset=train, batch_size=32, shuffle=True) test_dataloader = DataLoader(dataset=test, batch_size=32, shuffle=False) 100% 100% 100% 9.91M/9.91M [00:00<00:00, 17.8MB/s] 28.9K/28.9K [00:00<00:00, 480KB/s] 1.65M/1.65M [00:00<00:00, 3.84MB/s] 4.54K/4.54K [00:00<00:00, 6.77MB/s] in your code token build goes to zero ever leaves your machine with a full install guide the comments

Transcript

Someone just ran a 744 billion parameter AI model called Calibri on a laptop without a GPU. By streaming parts of the model from disk like Netflix, Calibri makes massive AI accessible offline and zero-cost for tokens, challenging the need for expensive GPUs. #calibri #glm52 #offlinenlp #aitechnology #opensourceai Someone just ran a 744 billion parameter AI model on a laptop with no graphics card. A developer in Italy pulled it off with GLM 5.2. That beats the ones that you pay for. Nvidia's pitch is that you need their GPUs like this, and you just prove you don't. The project is called Calibri. It works with GLM 5.2, that beats the ones that you pay for. Nvidia's pitch is that you need their GPUs like this, and you just prove you don't. The project is called Calibri. It works because it only uses about ten gigs of the model, thinking at any moment. So Calibri keeps that part in your RAM and streams the rest off your hard drive as it needs it. The same way Netflix streams instead of making you download it. And running it is easier than it sounds. You clone the repo and run one set up file. The whole thing builds itself. The only heavy part is download. The model download, which takes 370 gigs on your drive. After that, you just type "Coli" in chat, and you're talking to it completely offline. If you build apps, type "Coli serve" instead. And it turns into an open AI endpoint. You change one line in your code, and your token build goes to zero, because nothing ever leaves your machine. With a full install guide in the comments.