prosody-lens

Prosody Lens

Prosody Lens analyzes voice memos and speech recordings, then produces a readable delivery report plus an interactive HTML timeline.

It is local-first: no API key, cloud transcription, or paid service is required for the bundled analyzer.

What It Does

• Converts an input audio file into analysis-ready mono WAV.
• Measures pause structure, rough pitch movement, loudness/energy, possible acoustic peaks, and opening/middle/closing progression.
• Optionally uses Praat/Parselmouth for higher-fidelity pitch and intensity.
• Auto-focuses long quiet leading/trailing audio so charts do not collapse into the far left when a selected file contains dead air, even when the spoken portion is only a small fraction of the file.
• Surfaces candidate prosodic contour patterns and loose repeat families.
• Saves analyst-approved pattern exemplars into a JSON library.
• Matches future clips against approved pattern-library examples with correlation, DTW, or hybrid scoring.
• Adds a static Pattern Review Workbench inside report.html for approving, rejecting, renaming, annotating, and exporting candidate review JSON.
• Adds Visual only, Map/Card/Library visual layouts, sparse transcript word callouts, and Download image for low-text stakeholder sharing.
• Smooths visual-only pitch contours for shareable images while preserving raw pitch data for metrics and JSON review.
• Uses collision-aware word callout placement so labels stay readable in dense visual-only cards.
• Generates prosody.json, report.md, and an interactive report.html.
• Embeds browser-friendly MP3 playback in the HTML by default.
• Supports repeated-run trend records with --history.
• Keeps clinical and emotional interpretation out of scope.

Requirements

Required:

• Python 3.9+
• numpy
• ffmpeg
• ffprobe

Optional:

• Praat/Parselmouth for higher-fidelity pitch and intensity analysis.
• WhisperX or Montreal Forced Aligner for transcript/word alignment.
• openSMILE for standardized acoustic feature extraction.
• librosa or Plotly for richer future visualizations.

No API key is required. Groq Whisper or other hosted transcription tools are optional and should only be used with explicit approval for the specific file.

Install

From a published GitHub repo:

npx skills add jeremyknows/prosody-lens

Manual local install:

git clone https://github.com/jeremyknows/prosody-lens.git ~/.codex/skills/prosody-lens
cd ~/.codex/skills/prosody-lens
python3 -m pip install -r requirements.txt

Optional Praat/Parselmouth install:

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -r requirements.txt -r requirements-praat.txt

Setup Check

Run these before the first analysis:

python3 --version
python3 - <<'PY'
import numpy
print("numpy", numpy.__version__)
PY
ffmpeg -version | head -1
ffprobe -version | head -1
python3 scripts/prosody_analyze.py --help

If ffmpeg is missing on macOS:

brew install ffmpeg

Quick Start

python3 scripts/prosody_analyze.py /absolute/path/to/audio.ogg \
  --out-dir ./analysis/prosody/sample \
  --pitch-method auto \
  --speaker speaker-1 \
  --goal clarity \
  --take-label baseline \
  --history ./analysis/prosody/prosody-history.jsonl

open ./analysis/prosody/sample/report.html

Known pattern exemplar:

python3 scripts/prosody_analyze.py /absolute/path/to/pattern.ogg \
  --out-dir ./analysis/prosody/known-pattern \
  --pattern-label "analyst supplied label" \
  --pattern-notes "why this clip matters"

Build a reusable pattern library:

cp references/pattern-library-starter.json ./analysis/prosody/pattern-library.json

python3 scripts/prosody_analyze.py /absolute/path/to/pattern.ogg \
  --out-dir ./analysis/prosody/pattern-exemplar \
  --pattern-library ./analysis/prosody/pattern-library.json \
  --save-pattern-label "analyst-approved contour label" \
  --save-pattern-rank 1 \
  --save-pattern-notes "Accepted after listening to candidate #1"

python3 scripts/prosody_analyze.py /absolute/path/to/new-clip.ogg \
  --out-dir ./analysis/prosody/new-clip \
  --pattern-library ./analysis/prosody/pattern-library.json \
  --library-match-method hybrid \
  --library-match-threshold 0.62

High-fidelity local run when Praat/Parselmouth is installed:

python3 scripts/prosody_analyze.py /absolute/path/to/audio.ogg \
  --out-dir ./analysis/prosody/praat-run \
  --pitch-method praat \
  --pitch-floor 75 \
  --pitch-ceiling 400

With a transcript:

python3 scripts/prosody_analyze.py /absolute/path/to/audio.ogg \
  --transcript /absolute/path/to/transcript.txt \
  --out-dir ./analysis/prosody/sample-with-transcript

Share-safe report without embedded audio:

python3 scripts/prosody_analyze.py /absolute/path/to/audio.ogg \
  --out-dir ./analysis/prosody/share-safe \
  --share-safe

Outputs

If MP3 encoding is unavailable in the local ffmpeg build, the HTML falls back to embedded WAV playback.

Natural Language Usage

Ask your agent:

• "Analyze this voice memo for prosody and give me practical delivery feedback."
• "Find candidate prosodic patterns in this clip."
• "Visualize this known prosodic pattern exemplar."
• "Save this as an approved pattern and use it to match future clips."
• "Open the report, approve candidate #2, and export the review JSON."
• "Show me the visual-only view and download just the image."
• "Switch the visual snapshot to Card and download that image."
• "Visualize the cadence and pauses in this narration take."
• "Compare these two takes for clarity and pacing."
• "Show progression over time across these recordings."

When audio is available, the agent should run the analyzer rather than relying on transcript-only impressions.

Interface Design Direction

Prosody Lens includes explicit HTML/CSS direction for generated artifacts in references/interface-design.md. Agents should use it when creating or editing report.html: warm cream paper, red accent, deep teal structure, tactile controls, selectable Map/Card/Library visual snapshots, click-to-seek charts, transcript word callouts, active-audio focus disclosure, tabular metrics, mobile-safe wrapping, and reduced-motion support.

Pattern Library Workflow

The library workflow is the serious path for speech-pattern work:

1. Start from references/pattern-library-starter.json.
2. Run a report on a known or suspected pattern clip.
3. Listen to the top candidate cards in report.html.
4. Use Pattern Review Workbench to approve/reject/rename the candidate and copy or download the JSON payload.
5. Save approved candidates with the payload's suggested_cli, or manually use --save-pattern-label and --save-pattern-rank.
6. Run future clips with the same --pattern-library.
7. Review Pattern Library Matches in the HTML report and prosody.json.
8. Use --library-match-method dtw or hybrid when contour timing varies across examples.

Seed patterns are vocabulary only. They become matchable only after approved examples are saved. This keeps the analyzer honest: it reports contour similarity to known examples, not unsupported accent diagnoses.

The review workbench is intentionally static. It does not write to disk from the browser; it exports a JSON handoff with the selected rank, label, pattern ID, notes, normalized contour signature, DTW sequence, and suggested CLI command.

For stakeholder review, use Visual only in report.html to hide the word-heavy sections, choose Map, Card, or Library, then use Download image to export that active PNG visual snapshot. The image is rendered locally from the embedded SVG, using the actual waveform, pitch, loudness, pause bands, and top pattern contours. The shareable pitch line is smoothed and downsampled as a presentation contour so the phrase shape is readable instead of dominated by raw frame-level pitch jitter. If a transcript was supplied, sparse transcript words render as dot-on-arc, connector-line, translucent-card callouts. Dense labels are placed in separate lanes or skipped if there is no clean slot; use forced alignment for exact word timing.

If a selected file has a long quiet head or tail, Prosody Lens focuses the analysis/playback copy on the active audio window and records the original duration, focused duration, and trimmed leading/trailing seconds in active_audio. The detector uses both noise-floor and peak-relative energy so short speech at the front of a mostly silent file still gets focused.

Privacy And Sharing

Default report.html embeds a playable copy of the raw voice audio. Treat it as private audio, not just a metrics report. Use --share-safe when the report may be shared without the underlying voice recording.

The bundled analyzer runs locally. Do not upload private audio to third-party APIs unless the user explicitly approves the specific file, cost posture, and data-handling tradeoff.

Limitations

• Praat/Parselmouth is optional. Without it, the analyzer uses a practical fallback pitch tracker.
• Pattern labels are descriptive contour sketches, not accent diagnoses.
• Pattern-library matching is contour similarity against approved examples. DTW helps compare variable-speed contours, but it is not a final phonological classifier.
• Possible acoustic peaks are ranked places to listen first, not confirmed emphasized words.
• Word-level emphasis and speaking rate need transcript/word alignment.
• Noise, music, multiple speakers, heavy compression, and echo can distort metrics.
• This skill is descriptive only. It must not diagnose medical conditions, personality, honesty, emotion, or intent.

Troubleshooting

Verification

Before publishing or sharing a new build:

npx skills-ref validate .
python3 scripts/prosody_analyze.py --help
python3 scripts/prosody_analyze.py /absolute/path/to/test-audio.ogg \
  --out-dir /tmp/prosody-lens-smoke
python3 scripts/prosody_analyze.py /absolute/path/to/test-audio.ogg \
  --out-dir /tmp/prosody-lens-library-smoke \
  --pattern-library /tmp/prosody-lens-pattern-library.json \
  --save-pattern-label "smoke test contour" \
  --library-match-method hybrid
test -f /tmp/prosody-lens-smoke/report.html
test -f /tmp/prosody-lens-smoke/prosody.json
test -f /tmp/prosody-lens-pattern-library.json

For UI changes, open report.html and verify audio playback, chart click-to-seek, loop duration controls, Pattern Review Workbench JSON export, Visual only mode, Map/Card/Library layout switching, transcript word callouts, active-layout Download image PNG export, active-audio focus disclosure on dead-air fixtures, mobile layout, and no console errors.