Addis Scribe on Hausa
Hausa speech recognition on WAXAL, FLEURS and Common Voice against Google Gemini Flash and Google Chirp 3: about level with Gemini on WAXAL, behind Chirp on FLEURS, ahead of both on Common Voice.
Read the benchmarkBenchmarksHow our models measure up,
How our models measure up,
with the workings shown.
We publish the test sets, the scoring rules, every system's raw output and the limitations for every run, including the results where another system comes out ahead. If you want to check us, everything you need is on these pages or in the archive.
Published runs
Addis Scribe on Sidama
Live and uploaded Sidama speech recognition on 150 WAXAL test clips against Ethio-ASR and Google Gemini. Ethio-ASR Sidaama is slightly ahead.
Read the benchmarkAddis Scribe on Wolaytta
Live and uploaded Wolaytta speech recognition on 103 WAXAL test clips against Ethio-ASR and Google Gemini, with a narrow lead on a small set.
Read the benchmarkAddis Scribe on Swahili
Live Swahili speech recognition on African Voices, FLEURS and Common Voice against Google and four open models.
Read the benchmarkAddis Scribe on Afaan Oromo
Live and uploaded Afaan Oromo speech recognition against Google Chirp 2, Google Gemini, Ethio-ASR and the WAXAL benchmark models on 150 WAXAL test clips, with accuracy and live latency.
Read the benchmarkAddis Scribe on Amharic
Realtime Amharic speech recognition against Google Chirp 3, Google Gemini, Hohe and Shook on 100 held-out WAXAL clips and FLEURS, with accuracy and live latency.
Read the benchmarkAddis Voice 2 on Amharic
Addis Voice 2 against Azure, Gemini, OpenAI, ElevenLabs and Meta MMS on 100 Amharic prompts from three open datasets, with audio, judge transcripts and confidence intervals.
Read the benchmarkHow we publish
What our runs include
Fixed inputs
Public held-out test sets, drawn once with a fixed seed, with the exact rows published so the input set can be rebuilt independently and checked against ours.
Full results
Error rates for every system and mode, latency where we measured it, and the rows where a competitor leads, highlighted for them rather than left out.
Named conditions
Systems without documented support for the language, or that cannot run in the mode tested, are labelled as such, and the model and settings used for every system are published.
The Addis Voice 2 Amharic run publishes its inputs, every system's outputs and the scorer, so the numbers can be rebuilt, at addisai/amharic-tts-benchmark. Both Addis Scribe runs, Amharic and Afaan Oromo, can be reproduced with your own keys from addisai/addis-scribe-benchmark: the clips, human references, every system's transcripts, the scorer, a runner per system and a local app that puts your numbers next to ours. The Hausa, Sidama, Wolaytta and Swahili runs use public test sets and name every system they show; their per-clip transcripts are not published yet.