Transcribing multilingual and code-switched speech
The hardest audio isn’t noisy — it’s bilingual. Here’s why, and what to do about it.
· 6 min read
A Spanish-English intake interview. A Berlin standup that drifts between German and English every few sentences. A grandmother telling a family story in two languages at once. This is how a large share of the world actually talks — and it is the audio most transcription tools handle worst.
Understanding why helps you pick tools and set expectations honestly.
Code-switching is not translation
Code-switching is moving between languages within one conversation — often within one sentence. The speaker isn’t confused and doesn’t want translation: a faithful transcript keeps each word in the language it was said in.
That is the standard to hold a tool to. A transcript that silently translates the Spanish half of a sentence into English has destroyed information, even if every word is “accurate”.
Why single-language models fall over
Most speech recognisers decide on a language early — sometimes from the first few seconds — and then interpret everything through that choice. When the speaker switches, the model keeps listening in the wrong language and produces confident nonsense: real words from the wrong vocabulary that happen to sound similar.
These errors are worse than gaps, because nothing flags them. The sentence reads as fluent text and is simply wrong.
What newer multilingual models do differently
Models trained across many languages at once can follow a switch mid-stream, because they are not locked to one vocabulary. In practice the difference shows up exactly at the switch points: names, greetings, and the one-word borrowings that pepper bilingual speech.
Some systems also accept language hints — telling the model up front which languages to expect. If your tool supports this, use it: narrowing the space from “any language” to “Spanish and English” measurably reduces wrong-vocabulary errors.
Preparation that actually moves the needle
The generic audio advice — close microphone, quiet room — matters double here, because the model has less margin. Beyond that:
- Tell the tool the expected languages in advance, if it lets you
- Supply proper nouns from both languages — names are the first casualties
- Keep speakers to one microphone each where possible; crosstalk plus code-switching compounds
- Test with five minutes of representative audio before committing to a long session
Reviewing a multilingual transcript
Review differently than a monolingual one. Errors cluster at the switches, so scan for language boundaries and check the first few words after each. If a passage suddenly reads as oddly fluent but slightly off-topic, suspect a wrong-language stretch — the signature failure.
For anything published or quoted, have a speaker of both languages read the transcript against the audio. Twenty minutes of native review catches what no tool will.
Setting expectations
Even the best current systems make more errors on code-switched speech than on either language alone. Plan for that: budget review time, verify names and numbers, and treat the transcript as a strong draft rather than a finished record.
The gap is closing fast — but honest workflow beats optimistic tooling every time.