Realtime Avatar API

Voice or video, on one meter

<AvatarCall client={client} avatarId={avatarId} />                 // she is on screen
<AvatarCall client={client} avatarId={avatarId} mode="voice" />   // audio only

body = {"avatar_id": avatar_id, "mode": "avatar"}   # she is on screen
body = {"avatar_id": avatar_id, "mode": "voice"}    # audio only

It is priced to leave on. Live-call self-serve overage meters at about $5 an hour — $4.80–$5.70 depending on plan — with a free tier to build against. That is the difference between a demo you show and a feature you ship: a companion app can afford to let someone talk.

How she is rendered is in Creating an avatar.

She listens while she talks

Most voice AI takes turns like a walkie-talkie: you talk, you stop, it answers. This one is full-duplex — she is hearing you the whole time she is speaking — and in practice that shows up as four things you do not have to build:

• Interrupt her and she stops, mid-sentence, and can acknowledge it in character rather than snapping to silence.
• A cough or an "mm-hm" does not derail her. A backchannel is not an interruption, and being cut off by one is what makes a system feel brittle.
• A pause is not the end of your turn. Whether you are finished is judged by what you said, not by how long you have been quiet — so she neither talks over you nor leaves a gap.
• Silence and language switches are handled. Going quiet is a signal she can act on, and she follows a language change inside a single sentence.

Nothing to configure — it is how every call behaves. The one limit worth knowing: she will not talk over you with a new sentence while you are speaking. That is a deliberate trade for the voice and model choices this platform is built on.

Your tools, wired into the conversation

A character that can only talk is a demo. The useful version books the appointment, checks the order, writes the code and keeps talking while it runs.

You declare a tool the same way you would brief a colleague: what it is called, and when to reach for it. The description is the whole teaching signal — she reads it and decides.

import type { AvatarTool } from "realtime-avatar/tools";

export const checkOrder: AvatarTool<{ order_id: string }> = {
  description:
    "Look up the status of a customer's order. Call this whenever they ask " +
    "where something is, or when it will arrive.",
  parameters: {
    type: "object",
    properties: { order_id: { type: "string" } },
    required: ["order_id"],
  },
  execute: async ({ order_id }, { signal }) => {
    const order = await api.order(order_id, { signal });
    return `${order.status}, arriving ${order.eta}.`;
  },
};

# Tools run in the page that renders the call — there is no hosted execution,
# so there is nothing to declare in Python. Your backend's part is the GRANT:
body["capabilities"] = ["client_tools"]

# The mint schema is strict: a tools[] field on the request itself is a 422.
# The page registers the manifest over RPC once the room is connected.

Then your server grants the tool plane at mint, and the page registers the tools once the room is connected. Omitting execute is a compile error rather than a timeout you find in production — and a tool has 2.5 seconds to answer, so anything slow acknowledges fast and delivers the real result out of band:

// server — the session policy grants the client tool plane for this call
session: async ({ avatarId }) => ({ instructions, clientTools: true })

// client — register over RPC after connect; the record key is the tool's name
import { attachAvatarTools } from "realtime-avatar/tools";

const { accepted, rejected } = await attachAvatarTools(room, {
  check_order: checkOrder,
});

# server — grant the capability on the mint. That is the whole backend part:
body["capabilities"] = ["client_tools"]

# The worker only exposes tool registration to a session minted with the
# capability. If the page reports the registration method is missing, the fix
# is HERE, not in the page.

Full pattern: Tool calling.

Everything described here is realtime-avatar@0.10.0. On an older install the exports differ and the compiler will say so — update the package rather than working around it.

Where to go next

• Quickstart — key to live call in three steps.
• Calls — the policy your server decides, the five states, ending gracefully.
• Creating an avatar — one image in; the platform generates the loop and a map of states it switches between, or synthesizes the video live.
• Tool calling — running your own agent loop against a live character.
• API reference — every endpoint, scope, and error.