QWEN 3.8 27B ON A 3090 4090 OR 5090 IS ALREADY FRONTIER IF YOU DROP THE DEFAULTS

@larpu_acc

5.8K followers · usually 3.8K views a post

Views

18.6K

Likes

1.2K

Comments

94

Shares

77

SoundTikTok sound · original sound - unhappy_505

Promoted product

Qwen 3.8-27B

2 videos2 creators

Why it worked

The video likely resonated with users interested in running large AI models locally by providing specific, actionable technical advice and optimization parameters that are not widely known or easily discoverable. The use of niche terminology and direct instructions appeals to a dedicated community seeking to push the boundaries of their hardware for AI tasks.

Summary

The video provides technical configuration tips for running the Qwen 3.8 27B AI model locally on high-end GPUs. It highlights specific flags and parameters like '-spec-type draft-mtp', '-cache-type-k q8_0', '-cache-type-v q8_0', '-np 1', and '-kv-cache-dtype bfloat16' to optimize performance and fix issues with long context and chat templates.

Structure

1Introduction of Qwen 3.8 27B on high-end GPUs
2Key flag: -spec-type draft-mtp
3Cache and memory pool settings for 24GB VRAM
4

Original caption

#larpochka #qwen #localai #claude #machinelearning

More from @larpu_acc

Product placement

Qwen 3.8 27B (acts: a large language model that can be run locally, used for text generation and other AI tasks; removing it would change the video's topic) Nvidia 3090, 4090, 5090 (appears: computer hardware components, likely used for running AI models; removing them would not change the video's topic) Claude (appears: likely referring to Anthropic's Claude AI model, mentioned as a comparison or alternative; removing it would not change the video's topic) LocalAI (appears: a tool for running AI models locally; removing it would not change the video's topic)

On-screen text

QWEN 3.8 27B ON A 3090 4090 OR 5090 IS ALREADY FRONTIER IF YOU DROP THE DEFAULTS • most important flag is -spec-type draft-mtp the draft head is already baked into the weights keep -spec-draft-n-max at 2 anything higher kills the single mtp layer • for 24gb you need -cache-type-k q8_0 -cache-type-v q8_0 and the same for the draft add -np 1 so it does not reserve a second memory pool • long context gets fixed with -kv-cache-dtype bfloat16 • and never forget -jinja or the chat template breaks and the quant looks broken SAVE, TWIN <3