Calls

A call is one live conversation. Your server decides who may start one and what the character knows; the client renders it and reports five states. Those are the only two surfaces you touch.

Character stories and learning experiences

Yes. Define an original or appropriately licensed character, its speaking style and the context your server supplies for each call. Your application owns the story, source material, permissions and consequences of a user's choices. A character-driven lesson or story is an application design, not a guarantee of historical accuracy, learning outcomes or professional advice. Start with the Studio to evaluate a character, then use the SDK to integrate the experience.

Continuing a character conversation

Keep the character's behavior in instructions and supply relevant prior context when your server starts the next call. Your application decides what to save, what the user can inspect or delete, and which parts of a story belong in the next interaction. A new session does not automatically inherit persistent relationship memory. Use the Calls and Tool calling documentation to connect context and optional transcript delivery without treating diagnostic logs as your application's memory store.

What your server decides

The session policy on your connect endpoint. Every field here is authoritative — whatever the browser sent for these is discarded, and a field you do not set is absent rather than inherited from the caller. The Field column is the SDK name; the Wire column is the key a hand-rolled request sends — snake_case and strict, see Calling the API directly below.

The complete session creation request must fit within 64 KiB of JSON. This byte budget includes instructions, context, and all other fields; non-ASCII text can use multiple bytes per character.

There is no standalone voice_style field. Put persona, wording, and reply style in instructions. Sessions speak with Fish voices only. A voice with any other provider (Cartesia, Breeze Blue) is not an error: the call speaks with the character's own Fish voice, or with a Fish house voice matched to the character, and keeps that override's language and speed. A Fish voice takes speed, emotion, and language. The platform carries the session instructions and voice configuration separately: the model writes the reply under the session instructions, then TTS renders that text under the voice configuration.

Two modes

Same character, same policy — only mode changes. A voice call draws on separate capacity, so it never competes with a video call.

Recordings

Your backend can request a recording on POST /realtime/livekit/session. Recording defaults to off, and browser requests cannot override your server policy. Use audio for a mixed recording of published user and avatar audio, video for silent video, or audio_video for both in one MP4.

# Use your authenticated server-side HTTP client; see the API reference.
response = client.post("/realtime/livekit/session", json={
    "avatar_id": avatar_id,
    "recording": "audio_video",
})
response.raise_for_status()
grant = response.json()
recording_id = grant["recording"]["recordingId"]

# Later, after the call has ended:
recording = client.get(f"/recordings/{recording_id}").json()
if recording["status"] == "ready":
    access = client.get(f"/recordings/{recording_id}/access").json()
    # Return access["url"] only to someone allowed to view this call.

The recording ID stays the same through pending, recording, processing and ready. A failure is failed; retention expiry is expired. Ending the call starts finalization, so the file may become ready afterward. Recording reads require a server API key with recordings:read and are limited to your workspace. The canonical recording schema defines the artifact and playback-access response.

RTA stores the media privately for 30 days. Keep recordingId in your database. Playback URLs last at most one hour and support seeking; request a new URL when one expires. A URL's expiresAt is separate from the recording's retainedUntil. Renewing playback access does not extend retention.

Recording captures authorized tracks already published in the call. It does not open a camera, request screen capture, or grant additional publishing rights. Transcript delivery remains a separate option; use sessionId and timestamps to join transcript turns to the recording.

If recording is unavailable, the mint returns recording_unsupported before starting the call. A capture startup failure returns recording_failed. Raw API callers must let RTA create a fresh room for each recorded session; reusing room_name or setting create_room: false is refused. Endpoint details are in the API reference.

Connection history

Enable connectionHistory: true in your server session policy when you want the SDK to retain connection changes with the call. On the wire this isconnection_history: true. Collection defaults to off and requires an SDK release with connection-history support. Your application keeps using the existingonConnectionDetailsChange callback for live behavior; it does not write an uploader.

The SDK uses existing LiveKit connection events and batches their changes to RTA with a short-lived capability that can only write to that session. It adds no receiver polling. Read the history through GET /sessions/{sessionId}/connection-historyusing a server key with usage:read. RTA retains at most 240 received changes for 30 days. Missing data remains unknown. Client timestamps are approximate; connection quality is not proof of successful device playback or an exact recording timeline.

What the client reports

Five states. Write copy for the ones you care about; the SDK ships none.

Network facts and call diagnostics

Use RTA's call status, actions and end reasons for ordinary product flows. Internally the SDK reads LiveKit's connection and track facts through its public APIs. A connected room or an available track does not prove that the user saw a frame or heard audio. Keep playback observations and business rules separate from transport diagnostics. The React integration shows the SDK's existing call-level callbacks.

Keep the RTA session_id together with your own call or segment ID. Server-authored metadata on startCall becomes client_metadata on the wire and carries application correlation independently of transcript storage. The granted room name and the platform's session timestamps provide the starting point for a LiveKit investigation. Room names can be reused; a room SID identifies a specific room lifetime.

Platform operators can investigate detailed historical network information in LiveKit Cloud. Its Analytics API lists sessions and participant details on Scale plans or higher; use the returned LiveKit session ID for detail requests, not the RTA session ID. Project management credentials remain with the operator; an RTA API key does not grant access to the shared LiveKit project. A developer diagnostic endpoint would need to authorize the RTA session first and expose only permitted fields. The documented API does not establish whether a browser actually presented each frame.

What you can do mid-call

call.say(text, { steer })  // say something to her; `steer` shapes THIS reply only
call.sayAndEnd(text)       // speak one exact line, verbatim, then close
call.keepAlive()           // the user is still here — postpone the idle disconnect
call.end()                 // end now
call.secondsRemaining      // to the hard stop, or null before the clock lands
call.queuePosition         // place in line while waiting, else null

# These are CLIENT-side actions — they act on a live call, which lives in the
# browser or native app. A Python backend owns the policy, not the live turn.
#
# What Python does own — and there is no Python SDK, so this half is plain HTTP:
#   POST /realtime/livekit/session           mint a call — body is snake_case and strict:
#                                            avatar_id, mode, instructions, initial_context,
#                                            max_session_seconds, stt_mode, capabilities,
#                                            transcript_webhook, client_metadata (the Wire
#                                            column above, not the SDK field names)
#   POST /realtime/livekit/session/release   free the slot early
#   GET  /credits/balance                    what the next call must fit inside

say(text, { steer }) is the hook for tool results — steer applies to one reply and then evaporates, so a lookup result cannot leak into every later turn. See Tool calling.

Ending well

A call that stops mid-sentence feels broken no matter how good the render was. When time is nearly up you get a callback and a handle; whatever you pass to sayAndEnd is spoken exactly as written — never rewritten by the model, never interrupted — and only then does the call close.

<AvatarCall
  client={client}                                  // createProxyClient({ proxyUrl })
  avatarId={avatarId}
  balanceMs={creditRemainingMs}

  onEnding={async ({ secondsLeft, call }) => {
    if (secondsLeft > 15) return;                 // wait for the last window
    call.sayAndEnd(await writeGoodbye(character));
  }}
  onQuiet={({ secondsLeft }) => toast(`Still there? (${secondsLeft}s)`)}
  onLowBalance={({ secondsLeft }) => openTopUp(secondsLeft)}
  onEnded={({ reason }) => showEndScreen(reason)}
/>

onEnded always carries one of user_ended · session_cap · idle · disconnected · out_of_credits · agent_ended · failed. Key your end screen off it — "talk again?" and "you're out of minutes" are different screens with different outcomes.

The idle timer is real and enforced: when it fires the client disconnects and capacity is freed. That is deliberate. An abandoned tab holding a live character is the most expensive thing that can happen in this product, and it would be on your bill.

You can also release a call from your server. rta.endCall(sessionId, { reason }) frees the slot immediately — best-effort and idempotent: true when acknowledged, false for anything else, never a throw, so calling it twice (or on a call that already ended) is safe. Reach for it when your backend learns the call is over before the client does: a webhook, an admin action, a superseded reconnect. (Wire: POST /realtime/livekit/session/release, realtime:write scope.)

From the client, call.end() is the same thing and is what you normally want. The SDK also sends a release on tab-close via releaseLiveKitSessionBeacon(), which survives the page going away where a normal fetch would not. Release early and often: a held slot is capacity nobody else can use, and the queue is shared.

Calling the API directly

Not on TypeScript? The same call is one authenticated POST — the Python tab on Quickstart has a working endpoint. Two things to know before hand-rolling it:

• Every endpoint is strict, and casing is per endpoint. An unknown or mis-cased key is a rejected call, not a dropped field — and which casing is right depends on where you are writing. The realtime routes on this page (/realtime/livekit/session and its /release) are snake_case; every REST resource endpoint — avatars, clips, keys, assets — is camelCase. The preview /scenes routes answer in snake_case, like the mint they feed. Measured against the published spec: 2 of the 10 request bodies are snake_case at the top level and 8 are camelCase — with one exception worth knowing, because the strictness above turns it into a 422: POST /avatars is camelCase outside and snake_case inside voice (auto_description, voice_id). The SDK translates its camelCase policy for you; a hand-rolled request has no such help — send the Wire column from the policy table above (initial_context, max_session_seconds, transcript_webhook, client_metadata…), never the SDK names.
• Pass the connection payload to your client byte-for-byte. It is validated strictly — add one key and the client rejects the whole thing, so the call never opens and nothing points at the cause.

Full endpoint list, scopes, and error codes: API reference.