Zum Hauptinhalt springen

Whistle: Speech-to-Text in 16.9 MB — and the Benchmark Bars Nobody Labelled

Fortgeschritten
What you'll learn
  • Understand what is actually inside 16.9 MB — and which part of the 9x size headline is architecture versus a quantization choice the comparison does not hold constant
  • Read the vendor's own WER chart including the competitor bars it never labels: Whistle wins 5 of 8 head-to-heads and loses the three that look most like real life
  • Know why 30 seconds is a wall in the encoder rather than a setting, and build the VAD segmentation it forces on you
  • Find the streaming API the launch discussion concluded did not exist, and understand why its real latency is not the 11.1 ms time-to-first-token
  • Use keyword biasing as the primary accuracy lever, not a nicety — it is the difference between 41% and 96% on a closed vocabulary
  • Decide between Whistle, Whisper, Parakeet and a cloud API for your specific audio

On 8 October 2026 a blog post titled "Whistle: Speech to Text in 16.9 MB" reached 768 points on Hacker News. The number in the title is the whole pitch, and it is true: the entire multilingual speech recognition model is a single 16.9 MB file that runs on a CPU with no GPU and no dependencies.

The discussion that followed split cleanly in two. People who read the benchmark chart were impressed. People who pointed a microphone at it were not — one tester reported 70 correct out of 170 utterances. Both groups were right, and the reason they were both right is printed in the vendor's own chart. It is just in the bars that carry no number.

The one-sentence version​

Whistle buys its size by quantizing a small encoder–decoder ASR model to 2–4 bits and band-limiting its front end to 250–3500 Hz, which costs it almost nothing on close-microphone single-speaker speech — where it genuinely beats Whisper base — and costs it a great deal on far-field, overlapping or accented speech, where it is the worst of the three models in its own comparison.

What is actually in the 16.9 MB​

ComponentDetail
Front end25 ms window, 10 ms hop, 80 log-mel bins, band-limited 250–3500 Hz
Stem128-channel convolutional, downsamples to one frame per 80 ms
Encoder8 Simple Attention blocks, Monarch Hadamard MLP
Decoder8 laddered Simple Attention blocks, width 512, 8 query heads to 2 KV heads
Vocabulary8,192 text pieces plus 7 language tokens
Output ceiling320 tokens per clip
Input ceiling480,000 samples — 30 s at 16 kHz mono
Container.cact, Cactus Quants, 2–4 bit
LicenceApache-2.0

Three of those rows do more work than the rest, and two of them are the ones people skip.

The 9x size headline does not hold precision constant​

The comparison everyone quotes is 16.9 MB against Whisper base's 145.3 MB. That is one ninth the size — but the vendor's own footnote says Whistle is measured at 2 to 4 bit while Whisper is fp32 in memory on the CPU. Those are 8–16x apart in bits per weight before a single architectural decision is made.

The honest apples-to-apples row in the same chart is Moonshine tiny v2 at 41.9 MB in int8. Whistle is 2.5x smaller than that, not 9x. The remaining 2.5x is real and is where the architecture earns its keep — but if you repeat the 9x number you are mostly repeating a quantization choice.

This is the same trap covered in how to read an AI benchmark without getting fooled: the number is not wrong, the comparison is not held constant.

Band-limiting to 3500 Hz is a design decision with a predictable error class​

The front end keeps 250–3500 Hz and throws away everything else. That is roughly telephone band, and it is a sensible choice for a model meant to run on a wearable — most speech energy lives below it.

Most, not all. The acoustic-phonetics literature is specific here: 3–4 kHz is enough bandwidth for intelligible vowels but not for most voiceless fricatives, whose spectra are concentrated above 4 kHz. English s has its greatest energy concentration in roughly the 5–10 kHz range, peaking near 8 kHz — more than twice Whistle's cutoff — and sh centres near 4 kHz with energy extending to 8.

Cutting at 3500 Hz does not make those sounds inaudible. It removes the band that separates them from each other most cleanly.

So before you run a single test you can predict the error class: confusions and dropped word-endings on fricatives, worst on speakers whose rhythm or accent gives those consonants less duration to compensate with. The field reports in the launch thread — Spanish output with "non-existent words and terrible typos", Mexican and Venezuelan accents not detected, heavy accents degrading sharply — are that prediction arriving on schedule.

The benchmark bars nobody labelled​

Here is the part worth the page. Whistle's chart plots word error rate across nine test sets for three models. It prints a number on every Whistle bar and on none of the others.

The bars are still there, drawn to a linear scale. Whistle's printed labels calibrate that scale exactly — 6.867 pixels per WER point, consistent to ±0.01 across all nine — so the competitor values are recoverable from bar height. Doing that produces the table the chart stops just short of giving you.

Test setWhat it isWhistleWhisper base ≈Moonshine tiny v2 ≈
LibriSpeech test-cleanRead audiobooks, clean4.314.894.51
LibriSpeech test-otherRead audiobooks, harder10.4911.0011.71
SPGISpeechFinancial calls7.65—7.70
Earnings-22Earnings calls, accented19.01—21.25
AMIMeetings, far-field, overlapping26.0721.5022.78
AMI cleanedSame, cleaned refs22.87——
TED-LIUMConference talks7.614.995.64
FLEURSMultilingual read speech21.3924.50—
MLSMultilingual audiobooks24.9023.10—

Lower is better; bold marks the winner of each row. Whistle's column is the vendor's printed figures. The other two columns are read off the chart's bars — the vendor states they are Whisper's and Moonshine's own published numbers, and declines to print them.

Whistle wins five of the eight rows where a comparison exists. For a model 2.5x smaller than its nearest-precision rival that is a genuine result, and the FLEURS row — 3.1 points better than Whisper base on multilingual read speech — is the most impressive thing in the chart.

Now look at what it loses:

  • AMI, by 4.57 points to Whisper base. Far-field microphones, multiple speakers, overlap, interruption. Whistle is not merely behind here, it is the worst of the three, losing to a model 8.6x its size.
  • TED-LIUM, by 2.62 points. Single speaker, but a large room, a stage mic and unscripted delivery.
  • MLS, by 1.80 points. Multilingual long-form, averaged over six non-English languages.

The pattern is not subtle once the numbers are side by side. Every row Whistle wins is close-microphone and effectively single-speaker. Every row it loses has either distance between the mouth and the microphone or more than one person in the room. Read-audiobook and dictation audio is exactly the profile a 16 kHz band-limited front end with a tiny encoder handles well. Reverberation and overlapping talkers are exactly what it cannot afford the capacity to disentangle.

That single sentence reconciles the two halves of the launch thread. Everyone who tested dictation into a phone held near their face liked it. Everyone who pointed it at a film clip, a meeting or a room disliked it. Nobody was measuring wrong.

The number to carry away

4.31 and 26.07 are the same model on the same weights. If you only remember one WER figure for Whistle, remember the spread, not the headline.

A note on the 70/170 report​

The most-quoted field result in the thread — 70 correct out of 170, against 168/170 for a much larger Qwen ASR model — is a real measurement, but it is a measurement of stock settings on a closed vocabulary. The same tester reported 164/170 after tuning. What changed is the subject of the next section, and it is the most under-sold feature on the model card.

Keyword biasing is the primary accuracy lever​

Whistle decodes with five beams scored by length-normalised log probability, and it will accept a list of words you expect to hear and favour them during that search, matched with an Aho-Corasick automaton.

In a general-purpose ASR model this is a convenience. In a 16.9 MB model with an 8,192-piece vocabulary it is closer to load-bearing. The model has very little capacity to spend on an implicit language model, so proper nouns, product names, place names and domain jargon are precisely what it gets wrong — and precisely what a keyword list fixes. Going from 70/170 to 164/170 is 41% to 96% on the same weights.

Transcribe with the two arguments that matter

import needle

r = needle.transcribe(
  "clip.wav",
  keywords=["Kubernetes", "Grafana", "Prometheus", "Aoife", "Nguyen"],
  word_timestamps=True,
  language="en",          # skip detection when you already know
)

print(r["text"])
for w in r["words"]:
  print(w["word"], w["start"], w["end"], w["probability"])

Three things in that snippet are worth stating plainly:

  1. keywords is not a post-processing filter. It changes the beam search, so it can recover a word the decoder would otherwise never have placed.
  2. language="en" removes a failure mode, not just latency. Automatic detection across seven languages on a short or noisy clip is one more thing that can go wrong. If your app knows the language, say so.
  3. Every word carries its own probability. That is your confidence gate — see the handoff pattern below.
Watch out
  • A keyword list that is too long dilutes the bias and starts pulling the decoder toward words nobody said. Keep it to the vocabulary your users actually produce in this context — the current screen, the current contact list, the current document — not your entire product glossary.

The 30-second wall is in the encoder, not in a config file​

30 seconds at 16 kHz is 480,000 samples. After the convolutional stem that is one frame per 80 ms, so 375 encoder frames, and the decoder tops out at 320 output tokens. Those are fixed shapes in a 16.9 MB tensor file. There is no flag that raises them, and a launch-thread tester who recorded past the limit got a runtime error rather than a truncated transcript.

So any clip longer than 30 seconds means you own the segmentation — and segmentation is where long-form ASR accuracy actually lives, because a cut through the middle of a word or a clause is an error the model cannot recover from.

Guided walkthrough1 of 5
  1. 16 kHz mono is not a recommendation, it is the input contract. A 44.1 kHz stereo file needs converting before it reaches the model; the base install handles 16 kHz WAV and raw samples, and the [mic] extra adds resampling and microphone capture.

Silence is handled; low-SNR speech is not​

The model card makes a specific and welcome claim: silence and steady noise return an empty transcript rather than an invented sentence, because silence detection runs before the beam search. That is a direct fix for Whisper's most notorious failure — confidently hallucinating subtitles over nothing.

Read the claim precisely. It covers silence and steady noise. It does not cover quiet, distant or overlapping speech, and the thread contains exactly that failure: a tester reported the model emitting "Thank you." repeatedly for over a minute while real dialogue was playing, and another found a negation flipped — "can handle" for "can't handle" — plus a word invented outright.

Those are the classic artefacts of training on web video captions, where "Thanks for watching" occupies an enormous share of the text, and they survive the silence gate because the gate was never aimed at them. If a dropped n't would change what your application does, a per-word probability check is not optional.

Streaming exists — the launch thread concluded it did not​

A recurring complaint in the discussion was that Whistle cannot do live transcription, because the demo showed no partial text while speaking. The demo did not; the library does. needle.stream(chunks) takes live microphone input and commits words that consecutive overlapping passes agree on.

That policy — decode an overlapping window repeatedly, emit only the prefix two passes agree on — is the LocalAgreement family of streaming ASR, and knowing that tells you the thing the spec sheet will not:

Your perceived latency is not the 11.1 ms time-to-first-token. It is the agreement delay: how long a word has to wait before a second pass confirms it. TTFT is a decode-speed measurement on a complete 10-second clip. In a streaming loop the user-visible number is set by your window stride and how long you are willing to wait for stability, and it will be hundreds of milliseconds, not eleven.

Fast is still fast — 11.1 ms against Whisper base's 73.2 ms and Moonshine's 22.8 ms, and 1,319 tokens/s decode against 266 and 262, measured on 10 s of audio on an Apple M4 Pro CPU. Just do not promise a product manager 11 ms of live captioning.

Two features that are quietly more interesting than the size​

The decoder is laddered, so one file is a family​

Every depth from two layers up is a deployable model, selected at load time with --audio-depth. You download one 16.9 MB blob and choose your accuracy/latency point on the device, which is the right place to choose it — a watch and a phone can run the same artifact at different depths. For scale, the sibling Needle text model decodes at roughly 4,000 tokens/s at two layers and 400 at twenty on a Raspberry Pi 5.

embed() gives you speech matching with no transcript at all​

The encoder output is available directly: one row per 80 ms frame, via needle.Whistle().embed(audio). That is wake-word spotting, command matching, audio deduplication and "find where in this recording someone said this" — without decoding text, without a language model, and without the error classes discussed above, because you never ask it to spell anything.

For a lot of embedded jobs ("did the user say one of these eleven things?") this is the correct tool and transcription is the detour.

Run it from the CLI, including the built-in three-way comparison

pip install cactus-needle

# microphone playground
needle whistle playground

# put Whistle next to Whisper and Moonshine on YOUR audio
needle whistle compare my-clip.wav

# air-gapped / embedded: fetch a platform engine and the weights
needle download macos-arm64
needle download whistle
./macos-arm64/needle --model whistle.cact --audio clip.wav --audio-word-timestamps

The compare subcommand deserves a mention: the vendor ships the tool that checks its own claims against your audio. Given how sharply the WER spread above depends on recording conditions, running it on a representative clip of your audio is worth more than any table on this page.

One reproducibility property worth copying​

The engine reads no environment variables. Every behaviour is a compiled default or an explicit flag. That means the same blob plus the same flags scores the same on any machine, which is a property most inference stacks cannot claim and which matters enormously when you are trying to work out whether last week's eval regressed or last week's shell did.

Choosing: Whistle, Whisper, Parakeet, or a cloud API​

Your situationPickWhy
Dictation on a phone or wearable, close mic, offline, 7 supported languagesWhistleThis is the design centre. 16.9 MB, no GPU, word timestamps, and it beats Whisper base on exactly this audio profile
Wake words or fixed command sets on a microcontrollerWhistle embed()Skip transcription entirely; match encoder frames
Meeting recordings, multiple speakers, room micsNot WhistleIts own chart puts it last on AMI. Use a full-size Whisper, Parakeet, or a cloud ASR with diarization
Any language outside the sevenNot WhistleEnglish, German, French, Spanish, Italian, Dutch, Polish only. No Chinese, despite being the most-requested addition in the launch thread
Best open-weight accuracy-per-second on a GPU, EnglishParakeetNVIDIA's parakeet-tdt-0.6b-v2 was the practitioner favourite in the thread; CC-BY-4.0
Audio you cannot send anywhere, and accuracy matters more than sizeWhisper large, localStill the accuracy reference. You are trading ~100x the footprint for it
Medical, legal or financial transcripts of recordCloud ASR with a human in the loopThe negation-flip failure mode above is disqualifying on its own

A hybrid is usually the right answer, and it is the same shape as every other local/remote split on this site: run the small model on everything, use the per-word probabilities as the gate, escalate the uncertain fraction. See Claude plus local models for the general pattern and a private local AI stack for where a transcription step belongs in it.

What this tells you about the on-device wave generally​

Whistle is a good instance of a pattern now showing up across small models: the capability is real and the operating envelope is narrow, and only the first half fits in a headline. The 2-bit quantization is not a cheat — the model genuinely works — but it buys its size by spending the capacity that would have handled hard audio, and the vendor's chart shows exactly where that bill lands while labelling only the rows where it does not.

The practical rule generalises past ASR: when a small model publishes a wide benchmark spread, the spread is the specification. Find the row where it does worst, work out what that test set has that the others do not, and check whether your real input looks like that row. For Whistle the answer is distance and overlap. For a quantized language model it is usually long context — see what "98.2% retention" hides.

Check yourself

0/6
  1. Whistle is 16.9 MB against Whisper base's 145.3 MB. What makes that 9x comparison misleading?
  2. Whistle scores 4.31 WER on LibriSpeech test-clean and 26.07 on AMI. What distinguishes the test sets it wins from the ones it loses?
  3. Your app must transcribe a 90-second voice note. What does Whistle's 30-second limit require?
  4. A tester went from 70/170 to 164/170 correct utterances on the same Whistle weights. What is the most likely lever?
  5. Whistle reports 11.1 ms to first token. What will a user see in a live-captioning loop?
  6. The model card says silence returns an empty transcript rather than an invented sentence. What does that NOT protect you from?
Drücke Enter oder die Leertaste, um die Karte umzudrehen. Nutze die Pfeiltasten links und rechts, um zwischen den Karten zu wechseln.Begriff angezeigt.
1 / 12

Sources & further reading​

Next​