skill

claude-sound-ear

Give your Claude eyes and ears on audio. An installable Claude skill that turns any audio or video soundtrack into things an AI can actually reason over — a readable spectrogram, objective loudness/ti

thetoyosibello/claude-sound-earcommunitylistedunreviewed

Use skill

Set up

Skills are installed by placing their folder where your agent loads skills.

Download ZIP from GitHub Open the repository

The download comes from github.com/thetoyosibello/claude-sound-ear itself, not from our server, so what you get is exactly what its author published.

Install instructions come from the README

Skills do not use an MCP config. Follow the steps in the README below.

README

synced Oct 23, 2026

sound-ear

Give Claude eyes and ears on audio. An installable Claude skill that turns any audio or video soundtrack into things an AI can actually reason over — a readable spectrogram, objective loudness/timing facts, a semantic map of every sound effect, offline speech transcription, and sound-driven cymatic visuals (including seamless long-form loops for sleep/ambient channels).

Claude can't natively hear a .wav, but it can read images and structured text. sound-ear converts sound into exactly those, then Claude reasons over the result.

What it does

Command	Purpose	Needs
`hear.sh see <media>`	Spectrogram + waveform + loudness/silence/scene-cut report	ffmpeg
`hear.sh map <media>`	Semantic sound map: every SFX with timestamp, label, description, mix flags	Vertex or CLAP
`hear.sh transcribe <media>`	Offline speech-to-text	faster-whisper
`hear.sh cymatics <media>`	Sound-driven visuals (any aspect ratio)	numpy + PIL + ffmpeg
`hear.sh loop <media> --minutes N`	Seamless long-form loop (sleep/ambient) rendered from a short seed	numpy + PIL + ffmpeg
`hear.sh doctor`	Device-compatibility report + engine routing	—

Timing always comes from ffmpeg / spectral-flux onset detection, never from the language model (audio LLMs describe sound well but localize it poorly), so timestamps stay accurate.

Device-aware by design

Run hear.sh doctor and the skill reports what your machine supports and picks the best available engine for map:

vertex — Gemini on your own Google Cloud (best quality). Needs google-genai + ADC.
clap — offline zero-shot labeling. Needs PyTorch + transformers.

If neither is present it falls back to see (visual) and transcribe (speech), which run anywhere with ffmpeg.

Install

git clone https://github.com/tysitv/claude-sound-ear.git ~/.claude/skills/sound-ear
cd ~/.claude/skills/sound-ear
./setup.sh            # base + offline speech (light)
./setup.sh --vertex   # + cloud map engine (google-genai)
./setup.sh --clap     # + offline SFX labeling (pulls PyTorch, ~2GB)

System python3 needs numpy and pillow for the visual/onset tools. On Intel macOS the installer pins PyTorch 2.2.2 / transformers 4.44.2 / numpy<2 (the last combo with Intel-mac wheels); Apple Silicon and Linux get current versions.

Notes

Long-form performance: on a CPU, cymatics renders slower than realtime. loop renders a short seed and tiles it, so a one-hour video costs seconds-of-seed render time, not hours.
Privacy: analysis (see, transcribe, cymatics, loop, and the CLAP engine) is fully local. The vertex engine sends audio to your own Google Cloud project.