HE MADE AI FINE-TUNING ALMOST FREE

@kira.nex.tech

3.0K followers · usually 1.1K views a post

Views

28.2K

Likes

1.5K

Comments

275

SoundTikTok sound · Magic Glockenspiel

Promoted product

Souptrysoup.dev ↗

Why it worked

The video effectively addresses a significant pain point in the AI community (the high cost of fine-tuning) and presents a clear, accessible solution with a tangible demonstration of its effectiveness on accessible hardware.

Summary

A developer from Kazakhstan has created an open-source tool called Soup that significantly reduces the cost and complexity of fine-tuning AI models. Soup allows users to train large AI models on consumer-grade hardware, like gaming laptops with limited VRAM, by using a technique called layer streaming.

Structure

1Introduction to the problem of expensive AI fine-tuning
2Introduction of Alpamys and his tool, Soup
3Explanation of how Soup works (layer streaming)
4Demonstration of Soup's efficiency on low-end hardware
5Highlighting the open-source nature and benefits of Soup

Product placement

Soup: ACTS - it is a tool that fine-tunes AI models. Removing it would change the video's topic. GitHub: APPEARS - it is a platform where the code is hosted. Removing it would not change the video's topic. Product Hunt: APPEARS - it is a platform where the tool is featured. Removing it would not change the video's topic. MSI: APPEARS - it is a brand of GPU shown. Removing it would not change the video's topic. NVIDIA: APPEARS - it is a brand of GPU mentioned. Removing it would not change the video's topic. Llama: ACTS - it is an AI model that is fine-tuned. Removing it would change the video's topic.

Original caption

Alpamys from Kazakhstan created Soup, an open-source tool that makes fine-tuning large AI models affordable and easy. It uses layer streaming to train huge models on low VRAM, turning any gaming laptop into a powerful AI trainer. #soupai #alpamys #aitools #finetuning #opensourceai

More from @kira.nex.tech

On-screen text

HE MADE AI FINE-TUNING ALMOST FREE been bothering community for years models used expensive you'd rent cloud GPUs training code debug for days just gave used someone trained model his name's he built called soup here's what changed you you want to do data run soup train that's it file one Command, Fine-tune and post-train LLMs in one command. No SSH, no config hell. Website Quick Start Config Docs Commands Models Discord Product Hunt pypi v0.73.3 downloads 91k python 3.10-3.12 license Apache-2.0 tests 19175 passed passing website trysoup.dev Discord join arch DOI 10.5281/zenodo.21771064 FEATURED ON Product Hunt 103 GITHUB TRENDING #4 Repository Of The Day Soup turns the pain of LLM fine-tuning into a simpl Soup figures config, one command, done. pip install "soup-cli[train]" # soup init --template chat soup train Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B-Instruct + NF4, batch 1, seq 512 on an RTX 3050 Laptop 4 GB - 3.32 GB peak - 119.6 tok/s. Full video (90s) Why Soup? Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that. Zero SSH. Never SSH into a broken GPU box again. One config. A simple YAML file is all you need. Auto everything. Batch size, GPU detection, quantization - handled. Works locally. Train on your own GPU with QLoRA. No cloud required. What's New v0.73.3 - every pull request in this release came from someone other than the maintainer. All 24 of them, from eight people, five of whom appear here for the first time. What they found is the interesting part: four separate flags that were validated, documented, and then read by nothing. Assistant-only masking trained on zero tokens, with a normal loss curve. A tokenized return BatchEncoding which is not a dict - slipped past the guard, so the label mask was built from the mapping's key strings. No exception, no warning, a loss curve that looks like training. Found by reading the type, not by hitting the bug. LORA adapters layer verdict and nothing else. Fixing it correctly also meant not defaulting to a plain True but the breaks round-tripping for 21 of 173 shipped configs. On Windows, a process that genuinely exits with code 259 read as alive forever, because that is also STILL ACTIVE. It defeated run reconciliation and could wedge the MCP execution cap shut with no error an operator could act on. New: soup mcp serve --allow- execute runs a planned training or export behind a single-use, server- generated confirmation token - no command, no argv, no client- supplied environment - with the config snapshotted at plan time and protected paths digested by content, so a model cannot be swapped between planning and running. The measurement record for the earlier VRAM work, published as written - including the three readings withdrawn during it - is benchmarks/gate-v0.73.1-measured-vram- fit.md. is something # soup.yaml - then just 'soup tra training: stream_layers: true quantization: 4bit batch_size: 4 stream_source: auto seed: 1234 # base # NF4 # bigg # RAM # new Python 3.10-3.12 only. v0.73.0 adds the upper bound that was missing: on 3.13+, pip used to resolve untested PyTorch wheels that crash in the native extension before Soup runs at all. Previous release - v0.72.4, align on a laptop (DPO / ORPO / SimPO / KTO over layer streaming) Previous release - v0.71.40, soup reward synth (generate a reward verifier from your data) Previous release - v0.71.39, CI for weights not prompts (emit and provenance - Called layer verdict) Previous release - v0.71.38, The gate grows teeth (real leg-2 regression gate) Full history: CHANGELOG.md GitHub Releases. Quick Start normally loading a full 8 billion parameter like llama feast 16 gigs of Soup loads one layer time then swaps next layer training a Down by the river bank Deposit a check at the bank on 3.3 that's less gaming laptops just tested a 4 gig GPU 119 tokens per second free fine- tuning on hardware you already own the whole open source if you're models and of cloud costs the move

Transcript

There's a developer from Kazakhstan who just solved something that's been bothering the AI community for years. Fine-tuning models used to be expensive. You'd rent cloud GPUs, write custom training code, debug for days. Most people just gave up and used someone else's trained model. His name's Alpamys and he built a tool called Soup. Here's what changed: you describe what you want your model to do, point it at your data, run Soup train. That's it. One YAML file, one command. Soup figures config, one command, done. Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B-Instruct + NF4, batch 1, seq 512 on an RTX 3050 Laptop 4 GB - 3.32 GB peak - 119.6 tok/s. Full video (90s). Why Soup? Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that. Zero SSH. Never SSH into a broken GPU box again. One config. A simple YAML file is all you need. Auto everything. Batch size, GPU detection, quantization - handled. Works locally. Train on your own GPU with QLoRA. No cloud required. What's New? v0.73.3 - every pull request in this release came from someone other than the maintainer. All 24 of them, from eight people, five of whom appear here for the first time. What they found is the interesting part: four separate flags that were validated, documented, and then read by nothing. Assistant-only masking trained on zero tokens, with a normal loss curve. A tokenized return (BatchEncoding) which is not a dict - slipped past the guard, so the label mask was built from the mapping's key strings. No exception, no warning, a loss curve that looks like training. Found by reading the type, not by hitting the bug. LORA adapters. Layer verdict. And nothing else. Fixing it correctly also meant not defaulting to a plain True, but the breaks round-tripping for 21 of 173 shipped configs. On Windows, a process that genuinely exits with code 259 read as alive forever, because that is also STILL ACTIVE. It defeated run reconciliation and could wedge the MCP execution cap shut with no error an operator could act on. New: soup mcp serve --allow-execute runs a planned training or export behind a single-use, server-generated confirmation token - no command, no argv, no client-supplied environment - with the config snapshotted at plan time and protected paths digested by content, so a model cannot be swapped between planning and running. The measurement record for the earlier VRAM work, published as written - including the three readings withdrawn during it - is benchmarks/gate-v0.73.1-measured-vram-fit.md. The clever part is something called layer streaming. Normally loading a full 8 billion parameter model like Llama needs at least 16 gigs of VRAM. Soup loads one layer at a time, then swaps in the next layer. Training a huge model on 3.3 gigs. That's less than most gaming laptops. A developer just tested it on a 4 gig GPU and got 119 tokens per second. Free fine-tuning on hardware you already own. The whole thing is open source. If you're working with models and tired of cloud costs, the move is Soup.