# Calls

> What your server decides, what the client reports, and how to end a call without cutting the character off mid-sentence.

- Canonical: https://realtimeavatar.ai/docs/sessions
- Updated: 2026-10-12

A call is one live conversation. Your server decides who may start one and what the character knows; the client renders it and reports five states. Those are the only two surfaces you touch.

## Character stories and learning experiences

Yes. Define an original or appropriately licensed character, its speaking style and the context your server supplies for each call. Your application owns the story, source material, permissions and consequences of a user's choices. A character-driven lesson or story is an application design, not a guarantee of historical accuracy, learning outcomes or professional advice. Start with the Studio to evaluate a character, then use the SDK to integrate the experience.

## Continuing a character conversation

Keep the character's behavior in instructions and supply relevant prior context when your server starts the next call. Your application decides what to save, what the user can inspect or delete, and which parts of a story belong in the next interaction. A new session does not automatically inherit persistent relationship memory. Use the Calls and Tool calling documentation to connect context and optional transcript delivery without treating diagnostic logs as your application's memory store.

## What your server decides

The `session` policy on your connect endpoint. Every field here is authoritative — whatever the browser sent for these is discarded, and a field you do not set is absent rather than inherited from the caller. The Field column is the SDK name; the Wire column is the key a hand-rolled request sends — snake_case and strict, see Calling the API directly below.

| Field | Wire | What it does |
| --- | --- | --- |
| `instructions` | `instructions` | Her behavior contract — who she is and how she speaks. Up to 16,000 characters. The highest-leverage field on the whole surface. |
| `context` | `initial_context` | Up to 32 prior messages, replayed as memory. This is what makes a call continue a story instead of starting cold. On the wire it is `initial_context`, and each entry is `{ role: "system" \| "user" \| "assistant", content }` with content 1–8000 characters — the SDK types this for you, raw HTTP does not. |
| `maxSeconds` | `max_session_seconds` | Hard stop, up to 1800. Enforced on our side, so it holds even if your client stops reporting. Compute it from the balance you just admitted. |
| `listen` | `stt_mode: "server" \| "off"` | Speech recognition. Default true — she hears the user the whole call. Set `false` only if you drive every turn yourself — see [Tool calling](https://realtimeavatar.ai/docs/tool-calling). |
| `voice` | `voice` | Override her stored voice for this call. Must be a Fish voice; see below. |
| `video` | `render_backend: "generative"`, or omit it | How she is rendered: omit it for the generated loop + state map, or `{ mode: "generative" }` to synthesize the video live — see [Creating an avatar](https://realtimeavatar.ai/docs/video). |
| `camera` | `camera` | Optional boolean. Your server authorizes camera sharing with true; capture remains off until the user enables it. Works in voice and avatar calls. Use `useAvatarCamera` to toggle within the same session without interrupting the microphone. See [React and Native](https://realtimeavatar.ai/docs/react). User camera pixels are excluded from server video recordings. |
| `clientTools` | `capabilities: ["client_tools"]` | `true` opts the call into the client tool plane — functions declared in your page that she can call mid-conversation. See [Tool calling](https://realtimeavatar.ai/docs/tool-calling). |
| `transcript` | `transcript_webhook: { url, secret }` — https URL, secret 16–200 chars | `{ url, secret }` — get the two-sided transcript back, signed, after the call ends. On the wire it is `transcript_webhook`, and `secret` is 16–200 characters — the SDK types this for you, raw HTTP does not. See [Tool calling](https://realtimeavatar.ai/docs/tool-calling). |
| `metadata` | `client_metadata` | Up to 16 string pairs, echoed verbatim on that transcript so you can attribute it without a lookup. |
| `recording` | `recording` | `off` (default), `audio`, `video`, or `audio_video`. Your server decides whether to record after obtaining consent. Audio includes published user and avatar audio. |

The complete session creation request must fit within 64 KiB of JSON. This byte budget includes instructions, context, and all other fields; non-ASCII text can use multiple bytes per character.

There is no standalone `voice_style` field. Put persona, wording, and reply style in `instructions`. Sessions speak with Fish voices only. A `voice` with any other provider (Cartesia, Breeze Blue) is not an error: the call speaks with the character's own Fish voice, or with a Fish house voice matched to the character, and keeps that override's `language` and `speed`. A Fish `voice` takes `speed`, `emotion`, and `language`. The platform carries the session instructions and voice configuration separately: the model writes the reply under the session instructions, then TTS renders that text under the voice configuration.

## Two modes

| Mode | What the user gets | Use it for |
| --- | --- | --- |
| `avatar` (default) | She is on screen — audio and video | The full call |
| `voice` | Audio only | A cheaper on-ramp, or a fallback when bandwidth will not carry video |

Same character, same policy — only `mode` changes. A voice call draws on separate capacity, so it never competes with a video call.

## Recordings

Your backend can request a recording on `POST /realtime/livekit/session`. Recording defaults to `off`, and browser requests cannot override your server policy. Use `audio` for a mixed recording of published user and avatar audio, `video` for silent video, or `audio_video` for both in one MP4.

```python
# Use your authenticated server-side HTTP client; see the API reference.
response = client.post("/realtime/livekit/session", json={
    "avatar_id": avatar_id,
    "recording": "audio_video",
})
response.raise_for_status()
grant = response.json()
recording_id = grant["recording"]["recordingId"]

# Later, after the call has ended:
recording = client.get(f"/recordings/{recording_id}").json()
if recording["status"] == "ready":
    access = client.get(f"/recordings/{recording_id}/access").json()
    # Return access["url"] only to someone allowed to view this call.
```

The recording ID stays the same through `pending`, `recording`, `processing` and `ready`. A failure is `failed`; retention expiry is `expired`. Ending the call starts finalization, so the file may become ready afterward. Recording reads require a server API key with `recordings:read` and are limited to your workspace. The [canonical recording schema](https://realtimeavatar.ai/recording.contract.ts) defines the artifact and playback-access response.

RTA stores the media privately for 30 days. Keep `recordingId` in your database. Playback URLs last at most one hour and support seeking; request a new URL when one expires. A URL's `expiresAt` is separate from the recording's `retainedUntil`. Renewing playback access does not extend retention.

Recording captures authorized tracks already published in the call. It does not open a camera, request screen capture, or grant additional publishing rights. Transcript delivery remains a separate option; use `sessionId` and timestamps to join transcript turns to the recording.

If recording is unavailable, the mint returns `recording_unsupported` before starting the call. A capture startup failure returns `recording_failed`. Raw API callers must let RTA create a fresh room for each recorded session; reusing `room_name` or setting `create_room: false` is refused. Endpoint details are in the [API reference](https://realtimeavatar.ai/docs/api-reference).

## Connection history

Enable `connectionHistory: true` in your server session policy when you want the SDK to retain connection changes with the call. On the wire this is`connection_history: true`. Collection defaults to off and requires an SDK release with connection-history support. Your application keeps using the existing`onConnectionDetailsChange` callback for live behavior; it does not write an uploader.

The SDK uses existing LiveKit connection events and batches their changes to RTA with a short-lived capability that can only write to that session. It adds no receiver polling. Read the history through `GET /sessions/{sessionId}/connection-history`using a server key with `usage:read`. RTA retains at most 240 received changes for 30 days. Missing data remains unknown. Client timestamps are approximate; connection quality is not proof of successful device playback or an exact recording timeline.

## What the client reports

Five states. Write copy for the ones you care about; the SDK ships none.

| `status` | Meaning |
| --- | --- |
| `connecting` | Getting her ready |
| `waiting` | Every slot is busy and you are holding a place in line — `call.queuePosition` has the number. **Not an error**; the SDK retries for you. Render the position, not a failure. |
| `live` | She is there |
| `recovering` | A blip; reconnecting automatically with backoff |
| `ended` | Over — `onEnded` already told you why |

## Network facts and call diagnostics

Use RTA's call status, actions and end reasons for ordinary product flows. Internally the SDK reads LiveKit's connection and track facts through its public APIs. A connected room or an available track does not prove that the user saw a frame or heard audio. Keep playback observations and business rules separate from transport diagnostics. The [React integration](https://realtimeavatar.ai/docs/react) shows the SDK's existing call-level callbacks.

Keep the RTA `session_id` together with your own call or segment ID. Server-authored `metadata` on `startCall` becomes `client_metadata` on the wire and carries application correlation independently of transcript storage. The granted room name and the platform's session timestamps provide the starting point for a LiveKit investigation. Room names can be reused; a room SID identifies a specific room lifetime.

Platform operators can investigate detailed historical network information in LiveKit Cloud. Its [Analytics API](https://docs.livekit.io/deploy/admin/analytics-api/) lists sessions and participant details on Scale plans or higher; use the returned LiveKit session ID for detail requests, not the RTA session ID. Project management credentials remain with the operator; an RTA API key does not grant access to the shared LiveKit project. A developer diagnostic endpoint would need to authorize the RTA session first and expose only permitted fields. The documented API does not establish whether a browser actually presented each frame.

## What you can do mid-call

**TypeScript**

```ts
call.say(text, { steer })  // say something to her; `steer` shapes THIS reply only
call.sayAndEnd(text)       // speak one exact line, verbatim, then close
call.keepAlive()           // the user is still here — postpone the idle disconnect
call.end()                 // end now
call.secondsRemaining      // to the hard stop, or null before the clock lands
call.queuePosition         // place in line while waiting, else null
```

**Python**

```python
# These are CLIENT-side actions — they act on a live call, which lives in the
# browser or native app. A Python backend owns the policy, not the live turn.
#
# What Python does own — and there is no Python SDK, so this half is plain HTTP:
#   POST /realtime/livekit/session           mint a call — body is snake_case and strict:
#                                            avatar_id, mode, instructions, initial_context,
#                                            max_session_seconds, stt_mode, capabilities,
#                                            transcript_webhook, client_metadata (the Wire
#                                            column above, not the SDK field names)
#   POST /realtime/livekit/session/release   free the slot early
#   GET  /credits/balance                    what the next call must fit inside
```

See [Tool calling](https://realtimeavatar.ai/docs/tool-calling) for the Python side of the conversation — the signed transcript you receive when a call ends.

`say(text, { steer })` is the hook for tool results — `steer` applies to one reply and then evaporates, so a lookup result cannot leak into every later turn. See [Tool calling](https://realtimeavatar.ai/docs/tool-calling).

## Ending well

A call that stops mid-sentence feels broken no matter how good the render was. When time is nearly up you get a callback and a handle; whatever you pass to `sayAndEnd` is spoken **exactly as written** — never rewritten by the model, never interrupted — and only then does the call close.

```tsx
<AvatarCall
  client={client}                                  // createProxyClient({ proxyUrl })
  avatarId={avatarId}
  balanceMs={creditRemainingMs}

  onEnding={async ({ secondsLeft, call }) => {
    if (secondsLeft > 15) return;                 // wait for the last window
    call.sayAndEnd(await writeGoodbye(character));
  }}
  onQuiet={({ secondsLeft }) => toast(`Still there? (${secondsLeft}s)`)}
  onLowBalance={({ secondsLeft }) => openTopUp(secondsLeft)}
  onEnded={({ reason }) => showEndScreen(reason)}
/>
```

`onEnded` always carries one of `user_ended` · `session_cap` · `idle` · `disconnected` · `out_of_credits` · `agent_ended` · `failed`. Key your end screen off it — "talk again?" and "you're out of minutes" are different screens with different outcomes.

The idle timer is real and enforced: when it fires the client disconnects and capacity is freed. That is deliberate. An abandoned tab holding a live character is the most expensive thing that can happen in this product, and it would be on your bill.

**You can also release a call from your server.** `rta.endCall(sessionId, { reason })` frees the slot immediately — best-effort and idempotent: `true` when acknowledged, `false` for anything else, never a throw, so calling it twice (or on a call that already ended) is safe. Reach for it when your backend learns the call is over before the client does: a webhook, an admin action, a superseded reconnect. (Wire: `POST /realtime/livekit/session/release`, `realtime:write` scope.)

From the client, `call.end()` is the same thing and is what you normally want. The SDK also sends a release on tab-close via `releaseLiveKitSessionBeacon()`, which survives the page going away where a normal `fetch` would not. Release early and often: a held slot is capacity nobody else can use, and the queue is shared.

## Calling the API directly

Not on TypeScript? The same call is one authenticated POST — the Python tab on [Quickstart](https://realtimeavatar.ai/docs/quickstart) has a working endpoint. Two things to know before hand-rolling it:

- **Every endpoint is strict, and casing is per endpoint.** An unknown or mis-cased key is a rejected call, not a dropped field — and which casing is right depends on where you are writing. The realtime routes on this page (`/realtime/livekit/session` and its `/release`) are **snake_case**; every REST resource endpoint — avatars, clips, keys, assets — is **camelCase**. The preview `/scenes` routes answer in snake_case, like the mint they feed. Measured against the published spec: 2 of the 10 request bodies are snake_case at the top level and 8 are camelCase — with one exception worth knowing, because the strictness above turns it into a 422: `POST /avatars` is camelCase outside and **snake_case inside** `voice` (`auto_description`, `voice_id`). The SDK translates its camelCase policy for you; a hand-rolled request has no such help — send the Wire column from the policy table above (`initial_context`, `max_session_seconds`, `transcript_webhook`, `client_metadata`…), never the SDK names.
- **Pass the connection payload to your client byte-for-byte.** It is validated strictly — add one key and the client rejects the whole thing, so the call never opens and nothing points at the cause.

Full endpoint list, scopes, and error codes: [API reference](https://realtimeavatar.ai/docs/api-reference).

---

Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt