Docs

Calls

What your server decides, what the client reports, and how to end a call without cutting the character off mid-sentence.

A call is one live conversation. Your server decides who may start one and what the character knows; the client renders it and reports five states. Those are the only two surfaces you touch.

What your server decides

The session policy on your connect endpoint. Every field here is authoritative — whatever the browser sent for these is discarded, and a field you do not set is absent rather than inherited from the caller. The Field column is the SDK name; the Wire column is the key a hand-rolled request sends — snake_case and strict, see Calling the API directly below.

FieldWireWhat it does
instructionsinstructionsHer behavior contract — who she is and how she speaks. Up to 4,000 characters. The highest-leverage field on the whole surface.
contextinitial_contextUp to 32 prior messages, replayed as memory. This is what makes a call continue a story instead of starting cold. On the wire it is initial_context, and each entry is { role: "system" | "user" | "assistant", content } with content 1–4000 characters — the SDK types this for you, raw HTTP does not.
maxSecondsmax_session_secondsHard stop, up to 1800. Enforced on our side, so it holds even if your client stops reporting. Compute it from the balance you just admitted.
listenstt_mode: "server" | "off"Speech recognition. Default true — she hears the user the whole call. Set false only if you drive every turn yourself — see Tool calling.
voicevoiceOverride her stored voice for this call.
videorender_backend: "generative", or omit itHow she is rendered: omit it for the generated loop + state map, or { mode: "generative" } to synthesize the video live — see Creating an avatar.
clientToolscapabilities: ["client_tools"]true opts the call into the client tool plane — functions declared in your page that she can call mid-conversation. See Tool calling.
transcripttranscript_webhook: { url, secret } — https URL, secret 16–200 chars{ url, secret } — get the two-sided transcript back, signed, after the call ends. On the wire it is transcript_webhook, and secret is 16–200 characters — the SDK types this for you, raw HTTP does not. See Tool calling.
metadataclient_metadataUp to 16 string pairs, echoed verbatim on that transcript so you can attribute it without a lookup.

Two modes

ModeWhat the user getsUse it for
avatar (default)She is on screen — audio and videoThe full call
voiceAudio onlyA cheaper on-ramp, or a fallback when bandwidth will not carry video

Same character, same policy — only mode changes. A voice call draws on separate capacity, so it never competes with a video call.

What the client reports

Five states. Write copy for the ones you care about; the SDK ships none.

statusMeaning
connectingGetting her ready
waitingEvery slot is busy and you are holding a place in line — call.queuePosition has the number. Not an error; the SDK retries for you. Render the position, not a failure.
liveShe is there
recoveringA blip; reconnecting automatically with backoff
endedOver — onEnded already told you why

What you can do mid-call

call.say(text, { steer })  // say something to her; `steer` shapes THIS reply only
call.sayAndEnd(text)       // speak one exact line, verbatim, then close
call.keepAlive()           // the user is still here — postpone the idle disconnect
call.end()                 // end now
call.secondsRemaining      // to the hard stop, or null before the clock lands
call.queuePosition         // place in line while waiting, else null
# These are CLIENT-side actions — they act on a live call, which lives in the
# browser or native app. A Python backend owns the policy, not the live turn.
#
# What Python does own — and there is no Python SDK, so this half is plain HTTP:
#   POST /realtime/livekit/session           mint a call — body is snake_case and strict:
#                                            avatar_id, mode, instructions, initial_context,
#                                            max_session_seconds, stt_mode, capabilities,
#                                            transcript_webhook, client_metadata (the Wire
#                                            column above, not the SDK field names)
#   POST /realtime/livekit/session/release   free the slot early
#   GET  /credits/balance                    what the next call must fit inside

See Tool calling for the Python side of the conversation — the signed transcript you receive when a call ends.

say(text, { steer }) is the hook for tool results — steer applies to one reply and then evaporates, so a lookup result cannot leak into every later turn. See Tool calling.

Ending well

A call that stops mid-sentence feels broken no matter how good the render was. When time is nearly up you get a callback and a handle; whatever you pass to sayAndEnd is spoken exactly as written — never rewritten by the model, never interrupted — and only then does the call close.

<AvatarCall
  client={client}                                  // createProxyClient({ proxyUrl })
  avatarId={avatarId}
  balanceMs={creditRemainingMs}

  onEnding={async ({ secondsLeft, call }) => {
    if (secondsLeft > 15) return;                 // wait for the last window
    call.sayAndEnd(await writeGoodbye(character));
  }}
  onQuiet={({ secondsLeft }) => toast(`Still there? (${secondsLeft}s)`)}
  onLowBalance={({ secondsLeft }) => openTopUp(secondsLeft)}
  onEnded={({ reason }) => showEndScreen(reason)}
/>

onEnded always carries one of user_ended · session_cap · idle · disconnected · out_of_credits · agent_ended · failed. Key your end screen off it — "talk again?" and "you're out of minutes" are different screens with different outcomes.

The idle timer is real and enforced: when it fires the client disconnects and capacity is freed. That is deliberate. An abandoned tab holding a live character is the most expensive thing that can happen in this product, and it would be on your bill.

You can also release a call from your server. rta.endCall(sessionId, { reason }) frees the slot immediately — best-effort and idempotent: true when acknowledged, false for anything else, never a throw, so calling it twice (or on a call that already ended) is safe. Reach for it when your backend learns the call is over before the client does: a webhook, an admin action, a superseded reconnect. (Wire: POST /realtime/livekit/session/release, realtime:write scope.)

From the client, call.end() is the same thing and is what you normally want. The SDK also sends a release on tab-close via releaseLiveKitSessionBeacon(), which survives the page going away where a normal fetch would not. Release early and often: a held slot is capacity nobody else can use, and the queue is shared.

Calling the API directly

Not on TypeScript? The same call is one authenticated POST — the Python tab on Quickstart has a working endpoint. Two things to know before hand-rolling it:

  • Every endpoint is strict, and casing is per endpoint. An unknown or mis-cased key is a rejected call, not a dropped field — and which casing is right depends on where you are writing. The realtime routes on this page (/realtime/livekit/session and its /release) are snake_case; every REST resource endpoint — avatars, clips, keys, assets — is camelCase. Measured against the published spec: 2 of the 9 request bodies are snake_case at the top level and 7 are camelCase — with one exception worth knowing, because the strictness above turns it into a 422: POST /avatars is camelCase outside and snake_case inside voice (auto_description, voice_id). The SDK translates its camelCase policy for you; a hand-rolled request has no such help — send the Wire column from the policy table above (initial_context, max_session_seconds, transcript_webhook, client_metadata…), never the SDK names.
  • Pass the connection payload to your client byte-for-byte. It is validated strictly — add one key and the client rejects the whole thing, so the call never opens and nothing points at the cause.

Full endpoint list, scopes, and error codes: API reference.