Addis Scribe transcribes Swahili, live and from uploads. We compared it with Google Gemini Flash, Google Chirp 3 and four open Swahili models, on three public test sets, with the same scorer for every system. Given the whole clip, Google's systems are ahead on all three sets. Live against Google's own live transcription, Addis Scribe makes fewer word and character errors on African Voices and Common Voice.
Live against Google Chirp 3 live, Addis Scribe makes fewer word and character errors on African Voices and Common Voice, and about half the character errors on FLEURS. When Google Gemini Flash and Google Chirp 3 get the whole clip, they are ahead on all three sets.
With both systems streaming, Addis Scribe records 7.9% WER and 2.4% CER on African Voices Swahili, against 8.9% and 3.4% for Google Chirp 3 live, and 19.6% WER and 7.0% CER on Common Voice, against 21.4% and 11.5%. On FLEURS the word error rates are level (14.6% and 14.4%), and Addis Scribe makes about half the character errors (4.5% against 8.1%).
Read the full accuracy tablesStreaming in 1.12 s steps, Addis Scribe records 7.9% WER on the African Voices Swahili test, where Google Gemini Flash (6.3%) and Google Chirp 3 (7.2%) are ahead. On Common Voice it records 19.6%, against 15.8% for Gemini Flash and 18.7% for Chirp 3, but the fewest character errors of the three: 7.0% CER, against 7.4% and 9.1%. Both Google systems were given the whole clip.
Read the full accuracy tablesOn FLEURS Swahili, Addis Scribe records 14.6% WER. Google Gemini Flash and Google Chirp 3 each record 8.8%.
Read the full accuracy tablesWord error rate and character error rate on each set. Addis Scribe and the Tonykip streaming model were scored streaming with 1.12 s chunks, and Google Chirp 3 both ways: on the whole clip and live (streaming). Every other system was given the whole clip.
1,999 test clips, word error rate, streaming systems, lower is better
1,999 test clips, character error rate, streaming systems, lower is better
2,000 clips, word error rate, streaming systems, lower is better
2,000 clips, character error rate, streaming systems, lower is better
487 test clips, word error rate, streaming systems, lower is better
487 test clips, character error rate, streaming systems, lower is better
| System | Mode | African Voices | FLEURS | Common Voice | |||
|---|---|---|---|---|---|---|---|
| WER | CER | WER | CER | WER | CER | ||
| Addis Scribe Swahili | Live, 1.12 s chunks | 7.9% | 2.4% | 14.6% | 4.5% | 19.6% | 7.0% |
| Google Gemini Flash | Whole file | 6.3% | 2.0% | 8.8% | 4.2% | 15.8% | 7.4% |
| Google Chirp 3 | Whole file | 7.2% | 2.3% | 8.8% | 3.1% | 18.7% | 9.1% |
| Google Chirp 3 | Live, streaming | 8.9% | 3.4% | 14.4% | 8.1% | 21.4% | 11.5% |
| MMS-1B-all | Whole file | 14.3% | 3.5% | 14.5% | 3.5% | 20.8% | 6.1% |
| BuzzASR Swahili | Whole file | 13.8% | 4.2% | 25.5% | 10.0% | 16.5% | 5.9% |
| Tonykip Swahili streaming ASR | Live, 1.12 s chunks | 12.6% | 3.6% | 30.4% | 10.9% | 39.4% | 12.8% |
| Whisper large-v3 | Whole file | 39.6% | 9.6% | 34.7% | 8.8% | 64.1% | 26.2% |
Bold marks the best value in each column.
Addis Scribe streams live audio. The other systems were given the whole recording, so they are scored for accuracy only.
| System | Live audio | How it was run |
|---|---|---|
| Addis Scribe Swahili | Native streaming | Streams in 1.12 s chunks through the Addis Scribe API, so text appears while people speak. Scored live only. |
| Google Gemini Flash | Offline only | Given the whole clip with a transcription prompt. No live mode, so scored for accuracy. |
| Google Chirp 3 | Native (cloud API) | Google Cloud speech recognition (sw-KE), scored two ways: given the whole clip, and live, with the audio streamed in 0.2 s frames. |
| MMS-1B-all | Offline only | Meta's open multilingual model with the Swahili adapter, given the whole clip. |
| BuzzASR Swahili | Offline only | Open Swahili model, given the whole clip. |
| Tonykip Swahili streaming ASR | Native streaming | Open Swahili streaming model, run through the same streaming runtime in 1.12 s chunks. |
| Whisper large-v3 | Offline only | OpenAI's open multilingual model, given the whole clip. |
The test sets are public. None of these test clips were used to train Addis Scribe.
| Test set | Where to find it | Clips | What it measures |
|---|---|---|---|
| African Voices | African Voices Swahili release | Swahili test split, 2,000 clips | Swahili speech with human transcripts |
| FLEURS | google/fleurs | sw_ke test split, all 487 clips | Read sentences with human transcripts |
| Common Voice | Mozilla Common Voice | Swahili, 2,000 clips | Read sentences recorded by volunteers |
Every system on a set is scored with the same scorer and text normaliser, against the same human transcripts. WER is word error rate and CER is character error rate, both lower is better.
Add language=sw to a Scribe upload, live session or long-file job. Uploads, live streaming, word timestamps, captions all work in Swahili, at the same per-character price as Amharic. Swahili text comes back with capital letters, punctuation and numbers as digits. Speaker labels are not available in Swahili.