Skip to content

fix(cli): Parakeet decodes in 30 s windows, not 60 s - #6038

Open
Vitozhu04 wants to merge 1 commit into
mainfrom
wenbo/parakeet-30s-windows
Open

Vitozhu04 wants to merge 1 commit into
mainfrom
wenbo/parakeet-30s-windows

Conversation

@Vitozhu04

Copy link
Copy Markdown
Collaborator

What

silenceCuts defaults to 30 s windows instead of 60 s. Cuts still go to the quietest 100 ms within ±5 s of each target, and the worker's dropped-gap splice/retry is unchanged.

Why

When the speaker changes every few seconds, Parakeet drops whole stretches of a long window. The dropped-gap retry recovers part of that, not all of it. Shorter windows lose less and cost nothing elsewhere.

WER %, measured through the worker's own loop (silenceCuts → decode → droppedSpeechGaps → spliceGap/retry → mergeWindowsToWords, imported from this package), same sherpa-onnx-node 1.13.8 and recognizer config as the CLI, stock int8 model:

set 60 s (today) 30 s
speaker-switch mashups (6 × ~2 min) 33.3 27.6
same, loudness-levelled 39.0 28.3
earnings calls, multi-speaker (6 × ~2 min) 14.8 12.9
real videos (14) 3.5 3.3
one-speaker monologues (6 × ~2 min) 5.4 5.8

The driver reproduces today's CLI output exactly at 60 s (33.3 / 14.8 match hyperframes transcribe -e parakeet). Decode speed is unchanged (RTF ≈ 0.03–0.06). 20 s was no better than 30 s (levelled mashups 14.3 vs 10.8 with the fp32-accuracy model in the companion PR). Word-timing error vs MFA/Buckeye references, and Studio Clean-up filler/dead-air cuts, show no difference between 30 s and 60 s beyond noise.

Related work

Companion change (separate PR, pending model hosting): replace the int8 model with weight-only int8 (fp32 accuracy at int8 size). With that model the mashup numbers are 17.5 at 60 s → 12.4 at 30 s.

How

One default (windowSeconds = 60 → 30) plus its doc comment. The worker calls silenceCuts without a window argument, so nothing else changes.

Test plan

  • Unit tests updated: the 60 s test passes 60 explicitly; new test for the 30 s default; "shorter than one window" now uses 28 s
  • vitest run src/whisper/ passes, except manager.install.test.ts › never installs when the runtime vanishes…, which fails on main too on machines with a system whisper-cli in /opt/homebrew/bin
  • Eval kit run (Earnings-22 / URGENT / LibriSpeech / Buckeye / real videos), numbers above
  • Comments say why

🤖 Generated with Claude Code

Parakeet drops whole stretches of a long window when the speaker changes
every few seconds. Through the worker's own loop (dropped-gap splice and
retry included), stock int8 model, WER %, 60 s -> 30 s:
speaker-switch mashups 33.3 -> 27.6 (loudness-levelled 39.0 -> 28.3),
earnings calls 14.8 -> 12.9, real videos 3.5 -> 3.3, monologues 5.4 -> 5.8.
Decode speed is unchanged. 20 s was no better than 30 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this, the evaluation behind it is more rigorous than most tuning changes get, and routing it through the worker's own loop was the right way to measure.

The Comments check is red at this head: the comment ratchet refuses the two lines added to the silenceCuts doc comment, since a file's comment share may only go down. The rationale already lives in the PR description, so trimming the doc comment back should clear it.

The larger question is the default itself. Your table shows one speaker monologues going from 5.4 to 5.8 while everything else improves, and single speaker audio is probably the common case. Could the window be exposed as an option with the default left at 60 for now, until the owner of transcription quality weighs in? Minor: halving the window roughly doubles the chance of a very short final window. A 30.05 s file ends with a 0.05 s slice going to the recognizer on its own, so a small floor that folds a tiny tail into the previous window would help.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants