HE MADE AI FINE-TUNING ALMOST FREE

@kira.nex.tech

3.0K followers · usually 1.1K views a post

Views

34.8K

Likes

2.0K

Comments

470

SoundTikTok sound · The Winner Is... - DeVotchKa Version

Promoted product

Souptrysoup.dev ↗

Why it worked

The video effectively highlights a significant pain point in the AI community (expensive fine-tuning) and presents a clear, accessible solution (Soup). By showcasing its ability to run on affordable hardware and its open-source nature, it appeals to a broad audience of developers and AI enthusiasts.

Summary

A developer from Kazakhstan created Soup, an open-source tool that makes AI fine-tuning affordable and easy. It allows users to train large AI models on low-end hardware like gaming laptops by using layer streaming to manage VRAM efficiently.

Structure

1Introduction to the problem of expensive AI fine-tuning
2Introduction of Alpamys and his tool, Soup
3Explanation of how Soup simplifies the fine-tuning process
4Demonstration of Soup's efficiency on low-end hardware
5Highlighting Soup's open-source nature and cost-effectiveness

Product placement

Soup (ACTS - fine-tunes AI models, makes AI fine-tuning affordable and easy, runs on user's hardware, trains huge models on low VRAM, turns gaming laptops into AI trainers; removing it would change what happens in the video), Air Astana (APPEARS - airline logo on a sign; removing it would not change what happens in the video), MSI (APPEARS - logo on a GPU and laptop; removing it would not change what happens in the video), Product Hunt (APPEARS - platform logo and mention; removing it would not change what happens in the video), GitHub (APPEARS - platform logo and mention; removing it would not change what happens in the video)

Original caption

Alpamys from Kazakhstan created Soup, an open-source tool that makes fine-tuning large AI models affordable and easy. It uses layer streaming to train huge models on low VRAM, turning any gaming laptop into a powerful AI trainer. #soupai #alpamys #aitools #finetuning #opensourceai

More from @kira.nex.tech

On-screen text

HE MADE AI FINE-TUNING ALMOST FREE been bothering community for years models used expensive you'd rent cloud GPUs training code debug for days just gave used someone trained model his name's he built called soup here's what changed you you want to do data run soup train that's it file Soup Fine-tune and post-train LLMs in one command. No SSH, no config hell. Website Quick Start Config Docs Commands Models Discord Product Hunt pypi v0.73.3 downloads 91k python 3.10-3.12 license Apache-2.0 tests 19175 passed passing website trysoup.dev Discord join DOI 10.5281/zenodo.21771064 FEATURED ON Product Hunt 103 4 REPOSITORY OF THE DAY #4 Repository Of The Day Soup turns the pain of LLM fine-tuning into a simplSOUP figures config, one command, done. pip install "soup-cli[train]" soup init --template chat soup train Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B-Instruct-NF4, batch 1, seq 512 on an RTX 3050 Laptop 4 GB - 3.32 GB peak - 119.6 tok/s. Full video (90s) Why Soup? Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that. Zero SSH. Never SSH into a broken GPU box again. One config. A simple YAML file is all you need. Auto everything. Batch size, GPU detection, quantization - handled. Works locally. Train on your own GPU with QLoRA. No cloud required. What's New v0.73.3 - every pull request in this release came from someone other than the maintainer. All 24 of them, from eight people, five of whom appear here for the first time. What they found is the interesting part: four separate flags that were validated, documented, and then read by nothing. Assistant-only masking trained on zero tokens, with a normal loss curve. A tokenized return BatchEncoding which is not a dict - slipped past the guard, so the label mask was built from the mapping's key strings. No exception, no warning, a loss curve that looks like training. Found by reading the type, not by hitting the bug. LORA adapters lora adapters decision is now explicit and testable instead of hidden inside a 900-line function. soup --no-reexec training: stream_layers: true quantization: 4bit batch_size: 4 stream_source: auto seed: 1234 printed a launch command with your own flags missing - follow it literally and you trained without --fsdp, and the run succeeded, so nothing pointed back at the hint. Two hand-maintained copies of "what the user typed"; the printed one is deleted, and the hint now derives from the argv that actually launches the run. training.bnb_4bit_use_double_quant was read by nothing. Every 4-bit path hardcoded True, so setting it to false changed your config fingerprint and nothing else. Fixing it correctly also meant not defaulting to a plain True but the breaks round-tripping for 21 of 173 shipped configs. On Windows, a process that genuinely exits with code 259 read as alive forever, because that is also STILL ACTIVE. It defeated run reconciliation and could wedge the MCP execution cap shut with no error an operator could act on. New: soup mcp serve --allow- execute runs a planned training or export behind a single-use, server- generated confirmation token - no command, no argv, no client- supplied environment - with the config snapshothed at plan time and protected paths digested by content, so a model cannot be swapped between planning and running. The measurement record for the earlier VRAM work, published as written - including the three readings withdrawn during it - is benchmarks/gate-v0.73.1-measured-vram- fit.md. is something # base # NF4 # bigg # RAM # new Python 3.10-3.12 only. v0.73.0 adds the upper bound that was missing: on 3.13+, pip used to resolve untested PyTorch wheels that crash in the native extension before Soup runs at all. Previous release - v0.72.4, align on a laptop (DPO / ORPO / SimPO / KTO over layer streaming) Previous release - v0.71.40, soup reward synth (generate a reward verifier from your data) Previous release - v0.71.39, CI for weights not prompts (emit and provenance - Called layer verdict) Previous release - v0.71.38, The gate grows teeth (real leg-2 regression gate) Full history: CHANGELOG.md GitHub Releases. Quick Start normally loading a full 8 billion parameter like llama feast 16 gigs of Soup loads one layer time then swaps next layer training a huge model on 3.3 that's less gaming laptops just tested a 4 gig GPU 119 tokens free line tuning on hardware you the whole open source if you're models and of cloud costs the move