Previously, when AI processed videos, it basically relied entirely on subtitle text. Even tools that claimed to support video processing just randomly sampled a hundred or so frames, and completely missed charts, demos, and interactive effects in the footage. It was basically half-blind.

Recently, an open-source project called `claude-video`, created by Brad Bonanno, has gone viral. It lets Claude Code actually "see" videos — you just pass it a YouTube link, Zoom recording, or local video file, and it handles downloading, frame extraction, and subtitle generation in the background, so you can keep working on your code without interruption.


## What Makes It Smart

The core idea can be summed up in one sentence: **Only capture frames where the visual content actually changes**.

It uses FFmpeg for scene-change detection, capturing a frame only when the on-screen content changes. Unlike the old uniform frame sampling method, this saves tokens and is more accurate.

It also handles subtitles differently based on the source: YouTube uses the free official subtitles directly; for recordings without built-in subtitles like Zoom and Loom, it generates subtitles via Whisper. No unnecessary resource waste.

After processing, it will also ask if you want to save the analysis to your knowledge base for quick access later.

## What Can It Actually Do?

User Ray summarized three common use cases perfectly:

1. **From "describing images" to "understanding interactions"**: Previously, you could only send screenshots to teach AI how to write frontend code, but now it can understand dynamic effects like animations, hover states, and transitions.

2. **Remote debug screen recordings**: Users can record a video of the error they encounter, send it to Claude, and the AI will automatically extract key frames and locate the problem by cross-referencing it with your codebase.

3. **Smart frame extraction + Whisper collaboration**: It doesn't bloat your context window — it only sends key change frames and timelines to Claude, maximizing cost-efficiency.

People are also using it to analyze hooks in competitor videos, summarize long videos, and turn meeting recordings into action items.

## How to Use It

Installation is straightforward, just run two commands in Claude Code:

```

/plugin marketplace add bradautomates/claude-video

/plugin install watch@claude-video

```

Then just use `/watch` followed by your link or local file path:

```

/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?

/watch ~/Movies/screen-recording.mp4 when does the UI break?

```

It supports the `--start` and `--end` flags to specify a time range, and `--detail` to control frame extraction density (options: transcript / efficient / balanced / token-burner).

## Additional Details

The project already has over 3000 stars and is released under the MIT license. It relies on yt-dlp and FFmpeg under the hood, and uses Groq or OpenAI for Whisper inference.

Some users have raised concerns about accuracy with long videos: the creator notes that the default setting uses sparse scanning for videos over 10 minutes, and recommends using `--detail token-burner` or specifying a time range for longer content. Others have asked about Whisper accuracy for PPT page changes in Zoom meetings — this still needs to be tested on a case-by-case basis.

One developer already plans to feed a full week of consulting call recordings into the tool, so action items can be整理好 before the meeting wrap-up even finishes.

This project actually solves a very painful problem of "non-text multimodal context loss". Previously, AI video processing either burned through tons of tokens to process the entire video, or only relied on subtitles. Now, with the coordinated workflow of scene detection + speech-to-text, it packs the most useful information into a limited context window.

This isn't flashy showboating — it simply frees people from the tedious work of re-watching videos repeatedly.