Shipping soonEst. August 2026benchmarkvoicepublication

whisper-vs-groq

Public benchmark: which speech-to-text provider is fastest AND most accurate on UK phone audio?

§ 01 · What it is

A rigorous head-to-head benchmark of speech-to-text providers (OpenAI Whisper, Groq Whisper Turbo, Deepgram Nova-3, AssemblyAI Universal) on the audio conditions that actually matter for UK voice agents: 8kHz phone-quality audio, UK regional accents, Welsh place names. Full methodology, code, anonymised audio samples, and a reproducible result set.

§ 02 · Money it saves

£3,000-£12,000/yr per voice agent

for anyone building or specifying a UK voice agent — choosing the wrong STT provider costs money and latency for years

The math: OpenAI Whisper direct costs ~5× more per minute than Groq Whisper Turbo AND is ~4× slower. On a mid-volume voice agent (2,000 calls/mo, avg 3 min each), that's £250/mo in provider costs alone, plus the compound cost of every call feeling sluggish (drop-off, missed bookings). Multi-year decision, one afternoon of research.

§ 03 · How it works

01
Test corpus: 200 anonymised UK phone calls, hand-transcribed by native English + Welsh speakers. Covers hospitality bookings, tradespeople quote requests, legal enquiries. Balanced across regional accents (Cardiff, Newport, Swansea, English Midlands, London, RP).
02
Each provider is called with identical audio input via their production API. Response latency measured end-to-end (audio-in to text-out). Word Error Rate calculated against the human ground truth.
03
Additional metrics: cost per minute, latency variance (p50/p95/p99), handling of Welsh place names, handling of alphanumeric strings (booking references, phone numbers).
04
All code, prompts, and (anonymised) audio published. Anyone can re-run. Anyone can add new providers via a pull request.

Stack

PythonJupyterProvider SDKs (openai, groq, deepgram, assemblyai)Published methodology PDF

§ 04 · Why it beats what's out there

Every existing STT benchmark tests clean studio audio in American English. The one thing UK voice-agent builders need — real UK phone-quality audio with UK accents — has no rigorous public data.

What's out there	The gap it leaves
Vendor-published STT benchmarks	Tested on clean studio audio in American English. There's a massive gap between those numbers and what actually happens on a real UK phone call — 8kHz, compression, background noise, regional accents.
Vendor marketing pages and demo reels	Cherry-picked audio. Not reproducible. Not UK-focused. Effectively marketing, not data.
Nothing else public for UK phone audio	This is the wedge. First rigorous public benchmark of STT on UK phone-quality audio = permanent citation asset every UK voice-agent builder finds when they search.

Want this in your business?

I'm building this now. If it fits your problem, let's talk before it ships.

Book a reality-check call →

Follow along

Build notes and progress on this system will be posted in the Notes. Or check GitHub when the repo goes public.