Colibrì runs a 744-billion-parameter model on a laptop with 25GB of RAM, in pure C with zero dependencies.

@github.awesome

54.5K followers · usually 8.7K views a post

Views

84.2K

Likes

3.8K

Comments

41

Shares

467

SoundTikTok sound · original sound - github.awesome

Promoted product

Colibrì

Why it worked

The video effectively demonstrates a powerful AI model running on unexpectedly modest hardware, appealing to a wide audience interested in accessible AI technology. The clear explanation and focus on practical application make it highly shareable.

Summary

This video showcases Colibrì, a 744-billion-parameter language model that can run on a consumer laptop with 25GB of RAM. Despite being slow, it's highlighted as a frontier model that can answer correctly on affordable hardware.

Structure

1Introduction to Colibrì and its capabilities
2Explanation of its size and memory requirements
3Discussion of its performance and speed
4Highlighting its ability to run on cheaper hardware
5Mention of its open-source nature on GitHub

Original caption

Colibrì runs a 744-billion-parameter model on a laptop with 25GB of RAM, in pure C with zero dependencies. A model that size normally wants hundreds of gigabytes of memory or a rack of GPUs you'll never own. The author's blunt that it's slow, roughly a tenth of a token per second on his humble box, but it's a frontier model answering correctly on hardware cheaper than an H100's fan. #github #opensource

More from @github.awesome

Product placement

Colibrì: acts - a large language model that runs on consumer hardware. Removing it would change what happens in the video. GitHub: appears - a platform where the Colibrì project is hosted. Removing it would not change what happens in the video.

On-screen text

colibrì THE IDEA A COLIBRI RUNS A 744 BILLION PARAMETER MODEL ON A LAPTOP WITH 25 GB OF RAM IN PURE C WITH ZERO DEPENDENCIES A MODEL THAT SIZE NORMALLY WANTS HUNDREDS OF GIGABYTES OF MEMORY OR A RACK OF GPUS YOU'LL NEVER OWN THE AUTHORS BLUNT THAT IT'S SLOW ROUGHLY A TENTH OF A TOKEN PER SECOND ON HIS HUMBLE BOX BUT IT'S A FRONTIER MODEL ANSWERING CORRECTLY ON HARDWARE CHEAPER THAN AN H100'S FAN

Transcript

Colibrì runs a 744-billion-parameter model on a laptop with 25GB of RAM, in pure C with zero dependencies. A model that size normally wants hundreds of gigabytes of memory or a rack of GPUs you'll never own. The author's blunt that it's slow, roughly a tenth of a token per second on his humble box, but it's a frontier model answering correctly on hardware cheaper than an H100's fan. #github #opensource