Skip to content

sprezzature-audio

Audio: captions

Local artificial intelligence (AI) captions, speaker diarization and speaker identification for video / audio. WebVTT / SubRip subtitle format (SRT) from a local whisper.cpp, who-spoke-when via NeMo, who-is-who via embeddings or a transcript pass. The audio never leaves the machine.

Work in progress: the caption backend is still settling and accuracy baselines are being collected, so captions are drafts to review.

SKILL.md (the agent spec) ⭐ on GitHub

Makes

  • World Wide Web Consortium (W3C) WebVTT / SRT / plain-text captions from a local whisper.cpp build
  • "Who spoke when" via NVIDIA NeMo Sortformer diarization
  • "Who is who" via TitaNet embeddings or a transcript-mined pass
  • Speaker-labelled WebVTT caption format (VTT) with named voice cues; bilingual EN/FR default
  • The same modality read backwards: a narration from text, landing as a WAV and a WebVTT off one timeline, so a page can follow the reading sentence by sentence
  • Translated subtitles, through a local model

Audits

  • Reading speed, line width, cue duration and overlap, against the thresholds Netflix publishes: 17 characters a second, 42 to a line, 5/6 s at the shortest, 7 s at the longest, two lines at most
  • Flags rather than laws, because a children’s programme and a dense talk justify different numbers
  • Project-vocab biasing on the caption path

Say one of these

The skill triggers itself on phrasing like this, no command to memorize.

captions transcribe video / audio WebVTT SRT diarization who spoke when identify speakers translate captions are these subtitles readable narrate this text

Reference library

The knowledge the skill draws on: read them straight from the repo.

Captions with AI Diarization

Frequently asked questions

How do I install sprezzature-audio?

Run pip install sprezzature-audio (Python 3.10 to 3.13): captions, diarization, speaker identification, translation, narration and the HTTP and MCP surfaces all come with it, with no extra to choose. The install includes torch, but import sprezzature_audio loads no model until a model-backed function is called.

What does sprezzature-audio need besides pip?

sprezzature-audio needs ffmpeg and espeak-ng (for the Kokoro voice) on the system, plus model weights fetched on first use or ahead of time with sprezzature-audio-install (Whisper), sprezzature-audio-install-diarize --no-install (NeMo Sortformer and TitaNet) and sprezzature-audio-install-voice (the voices). Translation and the optional name-guessing pass also need an Ollama server serving qwen3-vl:8b. The caption audit, the speaker merge and the WER and DER metrics need none of this.

Does sprezzature-audio send my audio to a cloud service?

No: sprezzature-audio transcribes with whisper.cpp, finds and names speakers with NeMo models and synthesises voices on your own machine, with no API key. Translation and name guessing call the Ollama server named by --url, on localhost by default, and the only downloads are the model weights.

Which voices and languages does sprezzature-audio support?

sprezzature-audio narrates with three engines from speaker-helper: Kokoro (54 built-in voices, in English, Spanish, French, Hindi, Italian, Japanese, Portuguese and Mandarin), Chatterbox, and OpenVoice, which clones a voice from a short sample. With no --voice, the language is detected from the text and a voice that speaks it is picked; a page narrated by sprezzature-publish gets the engine speaker-helper's router chooses, Chatterbox for an offline run. Captions cover the languages Whisper recognises, detected when --lang is empty.

Can I use sprezzature-audio without Claude Code?

Yes: sprezzature-audio is also a Python library, fifteen commands behind the front door sprezzature-audio, an HTTP API with a Docker image, and an MCP server, sprezzature-audio-mcp, serving the API and /mcp on one port. The API and MCP tools cover the fast, model-free half (caption conversion, speaker merge, WER, DER); transcription and narration run as commands. The skill itself comes from pip install sprezzature then sprezzature-skills --install.

How is sprezzature-audio different from the Whisper API, pyannote or ElevenLabs?

sprezzature-audio runs locally and works both ways, speech to timed text and text to timed speech, each landing beside a WebVTT, and it then audits the caption track for reading speed, width, duration and overlap. Hosted services such as ElevenLabs or the OpenAI audio API bill per use and receive your audio; pyannote has no ceiling on the number of speakers, whereas the Sortformer checkpoint used here is built for up to four.

Can I run sprezzature-audio in CI?

Yes: sprezzature-audio-audit talk.vtt checks a caption track against the Netflix Timed Text Style Guide defaults (17 characters a second, 42 a line, two lines), needs only the Python standard library, prints JSON with --format json and exits 1 on any finding. Transcription, diarization and narration also run headless, but they download large model weights, so a CI job usually audits tracks rather than producing them.

Is sprezzature-audio free?

Yes: sprezzature-audio is open source under the BSD 3-Clause licence, with no account, API key or paid tier, and its source is on GitHub. The models it runs are open weights downloaded to your machine, each under its own licence.