claude-real-video lets Claude actually watch a video instead of just reading its transcript.

claude-real-video lets Claude actually watch a video instead of just reading its transcript. Paste a YouTube link into most tools and the model never sees a single frame; Gemini samples at a fixed one-per-second and misses fast cuts entirely. This detects real scene changes, so a ten-minute static slide collapses to one frame while a rapid-cut reel keeps every beat. It dedupes repeated shots, transcribes the audio, and hands you a folder any model can read. #github #opensource Same 58-second clip: fixed 1 fps sampling = 58 frames. crv keeps the 26 that actually differ -- and --grid packs them into 3 contact sheets. Fewer tokens, nothing missed. This free version lets your AI see the video. crv Pro lets it understand it — how it was shot (cut rhythm, camera moves) plus a timestamped timeline of what frames can't show: gestures, expressions, voice pitch shifts, emotion, sound events. One-time founder price. Most AI tools don't really see a video. Paste a YouTube link into ChatGPT and it reads the transcript, not the picture. Claude won't take a video file at all. Even Gemini, which can read video natively, has to send it up to Google and samples frames at a fixed interval (1 fps by default), so fast cuts slip past. claude-real-video does it differently, and locally: point it at a URL or a file, and it pulls the frames that actually matter (every scene change, not a fixed quota), throws away the near-duplicates, transcribes the audio, and hands you a clean folder any LLM can read. All the processing happens on your own machine — what gets sent anywhere is only the frames/text you choose to paste into an LLM afterwards. cry "https://www.youtube.com/watch?v=..." # crv-out/frames/*.jpg crv-out/transcript GEMINI SAMPLES AT A FIXED 1 PER Then drop the frames MANIFEST.txt into Claude / ChatGPT / Gemini and ask away. Want to eyeball what the model will see first? Add --viewer -- it writes a local viewer.html (video + keyframe grid + transcript) you can double-click open. No network, no extra installs. Not doing LLM work? It also works as a general-purpose video keyframe extractor — scene-change detection + dedup, no ML models to download. Using Claude Code? Install it as a skill so Claude watches videos on its own: Why not just sample frames? Most "let an LLM watch a video" scripts (and Gemini's own pipeline) grab frames at a fixed interval — e.g. one per second. That over-samples a static screencast and under-samples a fast-cut reel. claude-real-video is smarter: | fixed-interval sampling | claude-real-video | |---|---| | Frame selection | every N seconds | scene-change detection + density floor | | Repeated shots (A-B-A cuts) | sent again every time | sliding-window dedup sends each shot once | | Static slide (10 min) | ~600 near-frames | collapses to 1 (dedup) | | Fast-cut reel | misses frames between samples | catches each visual change | | Audio | often ignored | Whisper transcript w/ language detect | | Where the processing happens | often in someone's cloud | on your machine (you choose what to share with an LLM afterwards) | THIS DETECTS REAL SCENE CHANGES What --grid output looks like One contact sheet = nine consecutive keyframes, in order, filenames on each cell — the model reads a sequence, not scattered stills. How Perseverance Drives on Mars How Perseverance Drives on Mars Use it from Python SO A 10 MINUTE STATIC SLIDE from claude_real-video import process r = process("https://youtu.be/", "out", lang= print(r.frame_count, r.transcript_path) COLLAPSES TO ONE FRAME WHILE A RAPID CUT REEL KEEPS EVERY BEAT How it works 1. Fetch - yt-dlp for URLs (optional cookies), or copy a local file. 2. Extract - one chronological ffmpeg select pass grabs every scene change plus a density floor (at least one frame every --fps-floor seconds), so fast cuts and slow screencasts are both covered. 3. Dedup - real pixel difference (downscaled RGB, not a perceptual hash hashes go blind on flat colours and equal-luma hue changes) against a sliding window of the last --dedup-window kept frames, so an A-B-A cutaway doesn't re-send a shot the model has already seen. --report writes report.html showing every keep/drop decision with its diff %, for tuning. 4. Text if the video already has subtitles (a sidecar .srt/.vtt next to a local file, or an embedded subtitle track), those are used as the transcript - faster and more accurate than re-transcribing. Only when there are no subtitles does it fall back to Whisper on the audio (skipped cleanly if there's no audio). IT DEDUPS REPEATED SHOTS 5. Audio (optional, --keep-audio) - save the full original soundtrack (audio.m4a: music + speech + effects, copied losslessly from the source). The transcript only has the words; the audio file lets a model that can listen (Gemini, GPT-4o, ...) actually hear the music and tone. 6. Manifest MANIFEST.txt summarises everything for the model Notes Only download content you have the right to. The --cookies option is for your own, authorised access - don't ship credentials in a repo. Re-running overwrites the output directory. cry Pro — understand how a video was shot The free version tells your AI what's on screen. crv Pro tells it how it was shot — and why it works. Camera moves, editing rhythm, action bursts, plus a one-flag breakdown report: hook analysis, pacing curve, camera language, Reels-algorithm lens, and a rubric your own LLM completes into a full video teardown. This free tool tells an LLM what is on screen. A stack of keyframes can't tell it how the video moves — the camera work and the pacing. cry Pro adds everything the free version can't hear or feel: Camera-move classification - every shot labelled static / pan / tilt / zoom / handheld (verified against ground-truth footage) Editing rhythm - shot list, cuts per minute, and how pacing shifts across the video Perception timeline - the subtle things frames can't show: gestures and expressions (a smile, a hand raised, pointing), voice pitch rises and pauses, speaker emotion, and non-speech sound events — all timestamped TRANSCRIBES THE AUDIO AND HANDS YOU A FOLDER ANY MODEL CAN READ