Realtime Overview

Stable

What the Realtime API does, input, output, and architecture.

Realtime Overview

Realtime is a Voice Context Layer mode for live speech. Recorded is the other mode, for full-file processing. See Voice Context Layer for the umbrella model.

The Realtime API provides streaming speech transcription over WebSocket. You send PCM16LE audio frames, the server returns transcription events.

What it does

Live captions: text appears while the person is still speaking
Real-time translation: source and translated text together (coming soon for external sessions)
Multi-language: Whisper-compatible language support
Server-side VAD: automatic speech segment detection, no client-side splitting needed

Input

Audio format: PCM16LE / 16kHz / mono
Transport: JSON text frames (base64) or binary frames (raw PCM)
Frame size: recommended 100ms frames (~3200 bytes), max 1 MiB

Output

Server pushes JSON events. The primary output is:

conversation.item.input_audio_transcription.completed — a speech segment has been transcribed

Supporting events:

input_audio_buffer.speech_started / speech_stopped — speech segment boundaries
session.created — connection established
error — error events

Architecture

Your app                   LansonAI Gateway                   Upstream STT
  │                            │                                  │
  ├── WS connect (Bearer) ───→ session.created                   │
  │                            │                                  │
  ├── Send PCM16 frames ──────→ gateway relay (binary PCM) ─────→ VAD + STT
  │                            │                                  │
   │←── input_audio_buffer.speech_started ──────│←── input_audio_buffer.speech_started
   │                            │                                  │
   │←── conversation.item.input_audio_transcription.completed ─│←── conversation.item.input_audio_transcription.completed
  │                            │                                  │
  ├── Close ─────────────────→ meter flush → R2 ledger            │

vs. Recorded Speech

Dimension	Realtime WS	Recorded HTTP
Transport	WebSocket full-duplex	HTTP async
Latency	Milliseconds	Minutes
Input	PCM16LE audio stream	Audio URL
Output	Event stream	Complete result JSON
VAD	Server-side	Not needed
Best for	Live, real-time	Post-processing, batch

Next steps

Realtime Quickstart — runnable example
Connection Lifecycle — connect, timeout, close
Audio Input — audio format reference

Choose an API

Realtime Voice Context, Recorded Audio, and Voice Agent — when to use which.

Realtime Quickstart

Minimal runnable WebSocket examples.