video-tools-mcp

video-tools-mcp

An MCP server that gives AI agents local video tools. Claude, Cursor, and other MCP clients can read video metadata, find shots and cuts, extract frames to look at, and check keyframes. The tools run FFmpeg on your machine. You do not need an API key, and the server does not upload your video anywhere.

Tool	What it does
video_metadata	Reads the duration, container, codecs, resolution, frame rate, rotation, bitrate, and audio facts.
detect_shots	Finds the camera cuts and returns each shot with a start and an end time.
extract_frames	Extracts JPEG frames at given times, or spread evenly, and returns the images to the agent.
check_keyframes	Lists the keyframe times and checks the interval for seeking, trimming, and HLS or DASH streaming.
video_context	Builds one JSON document with the media facts, shots, and keyframes in the open Video Context schema.

Each tool accepts a local file path or an http(s) URL.

Requirements

• Node.js 20 or later.
• FFmpeg and ffprobe on your PATH.
  • macOS: brew install ffmpeg
  • Ubuntu or Debian: sudo apt install ffmpeg
  • Windows: winget install ffmpeg

Install

Claude Code

claude mcp add video-tools -- npx -y github:video-context/video-tools-mcp

Claude Desktop, Cursor, Windsurf, and other MCP clients

Add this to the MCP configuration file of your client:

{
  "mcpServers": {
    "video-tools": {
      "command": "npx",
      "args": ["-y", "github:video-context/video-tools-mcp"]
    }
  }
}

For Claude Desktop, the file is claude_desktop_config.json. For Cursor, the file is .cursor/mcp.json.

Example prompts

• "Look at ~/Movies/demo.mp4 and tell me what happens in it."
• "Find the cuts in launch.mp4 and list each shot with its length."
• "Get 8 frames from this video and write alt text for each one."
• "Will ad.mp4 stream well with HLS? Check the keyframes."
• "Make a Video Context JSON file for every video in this folder."

Example output

video_context returns a document like this one (illustrative):

{
  "schema_version": "0.1",
  "video_id": "launch.mp4",
  "duration": 15,
  "media": {
    "container": "mov,mp4,m4a,3gp,3g2,mj2",
    "video": { "codec": "h264", "width": 1280, "height": 720, "fps": 30, "pixel_format": "yuv420p" }
  },
  "shots": [
    { "start": 0, "end": 3 },
    { "start": 3, "end": 6 }
  ],
  "keyframes": [0, 3, 6],
  "provenance": [{ "source": "ffmpeg", "tracks": ["shots", "keyframes"] }]
}

How it works

• video_metadata runs ffprobe -show_format -show_streams.
• detect_shots runs the FFmpeg select='gt(scene,0.3)' filter on a small copy of each frame. Change threshold to find more or fewer cuts.
• check_keyframes reads the packet flags with ffprobe. It does not decode the video, so it is fast.
• extract_frames seeks to each time and saves one JPEG. The files go to your temporary folder.

These tools find technical facts. They do not understand the content. To search many videos by meaning, or to get transcripts, on-screen text, objects, and summaries, see Video Context.

Use the tools in code

import { buildContext, detectShots, extractFrames } from 'video-tools-mcp'

const { shots } = await detectShots('launch.mp4', { threshold: 0.25 })
const frames = await extractFrames('launch.mp4', { times: shots.map((s) => s.start) })

Develop

npm install
npm test

The tests make a short video with FFmpeg, so FFmpeg must be installed.

Related

• video-schema: the open JSON schema for video data, with converters from Google Video Intelligence, Amazon Rekognition, and Amazon Transcribe.
• awesome-video-understanding: a list of video understanding APIs, models, tools, and datasets.
• Free video tools in the browser: scene detection, frame extraction, a keyframe checker, and a metadata viewer.

License

MIT