Realtime Translation

Real-time translation: get translated text alongside transcription while speech is happening.

Usage

Configure translation

Set the source language and translation target via session.update:

{
  "type": "session.update",
  "language": "zh",
  "translate_to": "en"
}

• language: source language of the audio
• translate_to: target language for translation

Translation output

When enabled, translation results are returned alongside transcription in the conversation.item.input_audio_transcription.completed event.

Source and translated text correspondence

Source and translation for each utterance arrive in the same conversation.item.input_audio_transcription.completed event, ensuring correspondence. No client-side matching needed.

Switching languages

Change language at any time during the session:

{ "type": "session.update", "language": "en" }

New utterances after the switch use the new language. In-flight utterances complete with the previous language.

Latency

Translation adds additional latency:

Translation latency depends on audio length, context, and service load. Measure end-to-end latency in your integration and tune session settings for your use case.

Multilingual / code-switching

• Auto-detect: set language: "auto"
• Mixed language: upstream supports code-switching, accuracy depends on model and audio quality
• Chinese aliases: zh-cn, zh-tw, mandarin etc. are normalized to zh

UI strategy

1. Receive conversation.item.input_audio_transcription.completed event
2. Display source text on first line
3. Display translation on second line
4. Each utterance is independent, no replacement needed

See Build Live Translation for a detailed guide.

Related

• Session Configuration — session.update details
• Live Translation — translation concepts
• Build Live Translation — UI guide