Creating an avatar

Between sentences your character is not speaking — and that is most of a conversation. A still portrait reads as a dead connection. You do not have to fix that yourself any more: creation takes one image and generates the motion for you.

One image in, a moving character out

Upload a portrait, create the avatar, and the platform renders — in the background — everything a live call needs:

• The looping idle video: a ~10s closed arc whose first and last frame are both the portrait, so it loops seamlessly by construction. This becomes the avatar's source video — the state she rests in.
• A multi-clip motion library: a motion director looks at the portrait and the persona and designs short closed-arc clips — a signature idle variant, a listening state, a gesture — each ending exactly where it began, so the realtime worker can swap between them at invisible seams.

The create call returns immediately with status: "preprocessing" (the wire also reports the loop's own idleVideoStatus); poll getAvatar(id) (GET /avatars/{id} on the wire) until ready, and read the library with GET /avatars/{id}/clips. Render calls inherit the generated library automatically; a mint that sends an explicit empty clip_library: [] opts that call out of clips entirely. A motion prompt at creation art-directs the loop (wire motionPrompt, or the Avatars page's Resting motion direction); a clip that fails to render fails alone — the avatar still ships with the loop and the clips that made it.

Creation is image-only: one portrait is the entire input, and every piece of video an avatar plays is generated by the platform. There is no custom video upload. Everything below this point is the session-time layer — how rendering works, what the generated state map does for you, and the generation gadgets the creation pipeline itself runs, still addressable directly when you want to art-direct a clip.

Two modes

session: async ({ avatarId }) => ({
  video: { mode: "generative" },   // that is the entire switch
})

body = {
    "avatar_id": avatar_id,
    "mode": "avatar",
    "render_backend": "generative",   # that is the entire switch
}

Generative ignores clips by construction — the type makes passing them impossible rather than silently dropping them. Everything below is looping.

A live call needs a video source — which every avatar created from an image now grows on its own. Before the generated loop attaches, a mint is refused with a clear error, never a black or broken call; once it attaches, calls can connect while the realtime cache still warms (the first call starts slower). Poll status to ready for the fully-warm contract; a failed status carries the reason in error.

Looping: the resting loop

The simplest thing that works — and you do not configure it. The avatar's source video is the generated idle loop, set once at creation, so the default session body is already the right one:

session: async ({ avatarId }) => ({
  // no video config: the avatar rests in its generated idle loop
})

body = {
    "avatar_id": avatar_id,
    "mode": "avatar",
    # no video fields: the avatar rests in its generated idle loop
}

Looping: the generated state map

One loop is presence. A map is performance — and an image-created avatar already has one. At creation, the motion director designs a small library of states from the portrait and the persona: an idle variant she occasionally drifts into, a listening state that fires when the user starts speaking, a gesture with a plain-English whenHint ("when feeling a bit shy or embarrassed"). The library rides every call automatically; there is nothing to send.

The hint is read by the character, not by a rules engine you have to write. She knows what is happening in the conversation — she is the one having it — so she picks the state that fits, and the worker switches at a seam where it will not show. Inspect the library any time:

const { data } = await rta.listClips(avatarId);
// [{ clipId: "idle_special", role: "idle",    status: "ready", url: "…", whenHint: null },
//  { clipId: "listen",       role: "listen",  status: "ready", url: "…", whenHint: null },
//  { clipId: "gesture_shy_tuck_hair", role: "gesture", status: "ready", url: "…",
//    whenHint: "when feeling a bit shy or embarrassed" }]

library = client.get(f"/avatars/{avatar_id}/clips").json()
revision = library["revision"]   # 0 = creation-owned; bumps on every accepted PUT
clips = library["data"]
# [{"clipId": "idle_special", "role": "idle", "status": "ready", "url": "…", "whenHint": None},
#  {"clipId": "listen", "role": "listen", "status": "ready", "url": "…", "whenHint": None},
#  {"clipId": "gesture_shy_tuck_hair", "role": "gesture", "status": "ready", "url": "…",
#   "whenHint": "when feeling a bit shy or embarrassed"}]

A mint that sends an explicit empty clip_library: [] opts that call out of clips entirely — she stays in the resting loop. That is the whole session-time control surface: the states themselves come from the platform.

Every clip is a sentence, and you can change it

The starter library is a starting point, not a fixed set. Declare the library you want and the platform reconciles: a clip whose description is unchanged is kept and keeps serving, a new or revised one is queued to render, and one you leave out is retired. The 202 is acceptance, not readiness. revision is the top-level field of the GET envelope beside data — 0 while the creation pipeline owns the library; each accepted declare increments it, and the 202 body carries the new value.

In the dashboard, open Avatars → Clips to inspect and declare generated prompt clips without code. The editor preserves the API's full desired-set semantics: removing a row retires it, and every save carries the revision you loaded. Starter clips created before editable directions were stored ask for a new motion direction before the dashboard can take ownership of them. Uploaded clip rows remain read-only there until the upload asset contract has one dashboard SSOT.

const { plan } = await rta.setClipLibrary(avatarId, {
  expectedRevision: revision,          // compare-and-set; a concurrent writer 409s
  clips: [
    { clipId: "idle_soft",   role: "idle",
      source: { motionPrompt: "breathing gently, a slow blink" } },
    { clipId: "listen_lean", role: "listen",
      source: { motionPrompt: "leans in a little, attentive, small nod" } },
    { clipId: "gesture_wave", role: "gesture", whenHint: "when greeting someone",
      source: { motionPrompt: "raises a hand and waves warmly, then lowers it" } },
  ],
});
// plan → { kept: [...], queued: [...], retired: [...] }

await rta.waitForClips(avatarId);      // settles when nothing is still RENDERING

r = client.put(f"/avatars/{avatar_id}/clips", json={
    "expectedRevision": revision,   # compare-and-set; a concurrent writer 409s
    "clips": [
        {"clipId": "idle_soft", "role": "idle",
         "source": {"motionPrompt": "breathing gently, a slow blink"}},
        {"clipId": "listen_lean", "role": "listen",
         "source": {"motionPrompt": "leans in a little, attentive, small nod"}},
        {"clipId": "gesture_wave", "role": "gesture",
         "whenHint": "when greeting someone",
         "source": {"motionPrompt": "raises a hand and waves warmly, then lowers it"}},
    ],
})
# httpx does not raise on 4xx: 403 clip_library_not_enabled, 409 revision_conflict
# (body carries the current revision), 422 clip_declaration_rejected
r.raise_for_status()
plan = r.json()["plan"]

Declaring is a per-tenant rollout: until it is enabled for your tenant the PUT answers 403 clip_library_not_enabled and avatars keep their creation-generated library. The other refusals — 409 revision_conflict, 422 clip_declaration_rejected — are in Refusals worth telling apart.

Per clip: motionPrompt (the description the platform renders from), whenHint (plain English the character reads to decide when it fits), durationSeconds (4–8, default 5), and reroll: true to force a new take of an otherwise-unchanged clip. At most twelve clips: up to six idle, up to two listen, the rest gestures.

Six idles is a rotation, not six alternatives to one. Each is a variant she can drift into while resting, so a character with several reads as alive rather than looped; one is perfectly normal.

You can also upload a clip — source: { assetId } instead of a prompt — but it is pose-validated before it can serve: its first and last frames have to land on the avatar's rest pose, or that clip settles failed with the verdict in poseCheck and the rest of the library is untouched. That validation is the whole reason an upload is allowed at all; without it a supplied video makes every switch visible.

The one route that takes clip URLs, POST /avatars/{id}/clips, is deprecated and validates nothing: it reconciles an externally hosted library to the video cache for tenants who served their own clips before generated libraries existed, and it cannot add a clip to a generated one.

The resting loop is not a clip, and it is also a sentence

The loop she plays when nothing else is happening is the avatar's source, not a library entry — a clip with role: "idle" is a variant spliced over it, and declaring one never changes what she rests in. Direct the loop at creation with motionPrompt, and re-direct it any time afterwards:

const { servingUrl } = await rta.setLoop(avatarId, {
  motionPrompt: "tilts her head, a small amused smile, settles back to centre",
});
// servingUrl = the loop she is playing RIGHT NOW, for the whole render

await rta.waitForLoop(avatarId);   // throws if the render gave up

accepted = client.put(f"/avatars/{avatar_id}/loop", json={
    "motionPrompt": "tilts her head, a small amused smile, settles back to centre",
}).json()
# accepted["servingUrl"] is the PREVIOUS loop (None if she had none yet) — she plays it for the whole render.
# Poll GET /avatars/{id} until idleVideoStatus is "ready" (or "failed").

She stays ready and keeps serving the previous loop for the entire render, then the swap publishes in one step. Your clip library is untouched — clips render against the portrait, not against the loop, so a re-direct re-queues nothing and does not move the library's revision. Describe a closed arc: it has to end where it began, or the loop snaps every time it wraps.

And keep it small. Because a separate model animates her mouth during speech, every loop description is held to a nearly-still envelope — blinks, a soft closed-lip smile, a shift in weight, a few degrees of head drift. A laugh, a head turn, or an open mouth comes back inside that envelope anyway: a new render, a new charge, and a loop that looks like the one you already had. It is the only outcome here that carries no error code, which is exactly what makes it worth knowing before you write the sentence.

What makes a switch invisible

Every clip the director renders starts and ends on the same rest pose — the portrait itself. That shared frame is what turns a switch into a splice instead of a jump, and it is why each clip is generated independently from the one anchor image rather than chained off the previous clip's last frame, which would compound drift with every hop. Clips stay short (~5s closed arcs): the lever for natural idle is how many states × how long she dwells, not clip length.

Editing the loop

Preview — not yet enabled on the public fleet. The field is accepted and forwarded today, and currently resolves to the unedited loop. Build against it if you want to be ready; do not ship a feature to your users that depends on it yet.

A state map switches between clips. Editing rewrites the clip itself: the generated loop is re-rendered under a sentence you write, and the result is lip-synced by the same path as an unedited one. One loop becomes every season, every set, every outfit — without ever generating a new one.

session: async ({ avatarId }) => ({
  video: {
    edits: { instruction: "a snowy cabin at night, warm lamplight" },
  },
})

body = {
    "avatar_id": avatar_id,
    "mode": "avatar",
    "support_edits": {"instruction": "a snowy cabin at night, warm lamplight"},
}

Editing runs upstream of the lips, on the closed-mouth plate, so it costs nothing at the start of a call — she is on screen and talking from the first word, she simply has not changed clothes yet. The first lap through the loop is live and expensive; every lap after replays what that one banked, so a fixed instruction converges on the cost of plain looping.

It needs the avatar's video source — the generated loop — so it is refused at the mint while the loop is still rendering, rather than falling back. It also cannot be combined with generative — a synthesized body has no clip to open — and the type makes that pairing unrepresentable rather than a runtime error. A voice call is refused for the same reason: no video track, nothing to rewrite.

Editing is per call, and off unless you ask. Nothing stored on the character turns it on, so a call that omits edits is byte-identical to one made before this option existed — which matters, because the first edited lap holds a scarce GPU editor and can wait minutes on a cold one. If you want every call for a character edited, say so in your session policy; that is the same place you already decide the persona and the ceiling.

Letting the look follow the conversation

Experimental — "runtime edit". The shape of live may still change, and the re-dress does not reach every capacity tier yet, so a call may run to the end on its opening set. See Experimental features for what that word promises. The trust boundary below is not experimental — it is enforced on every mint.

Add live and the set stops holding still: the look follows the conversation, within the bounds you set. Those rules are yours, not your user's — they are the app's policy about what a conversation is allowed to do to the picture, which is why they are server-owned like instructions. A browser that could set them could redress your character into anything.

edits: {
  instruction: "her apartment at golden hour",
  live: {
    rules: "only change the room and the light, never her face or clothes; " +
           "only when they ask to go somewhere",
    cooldownSeconds: 60,
  },
}

body["support_edits"] = {
    "instruction": "her apartment at golden hour",
    "live_edit": {
        "rules": ("only change the room and the light, never her face or clothes; "
                  "only when they ask to go somewhere"),
        "cooldown_seconds": 60,
    },
}

Write the rules narrowly. Every change of direction buys a fresh lap on a scarce editor and a visible style pop — the editor has no mid-stream prompt update, so a look changes at a seam rather than easing. cooldownSeconds is the floor that stops an eager director spending a GPU per user turn; it is a bound, not a preference, and the default is deliberately conservative.

Generation pricing

The generated library itself is currently included with creation — the loop's generation is metered per-model exactly like a resting-loop update on the Avatars page, and the extra library clips are not billed separately while pricing settles. The rates below apply when you drive the generators directly:

Generation is billed at the model vendor's own list price. We add no margin — what you pay is what MiniMax and ByteDance charge, so the table below moves when their rate cards move. Prices are per generated clip and charged on success; failed generations are not charged.

Prices are for a 10-second clip at 24fps — H3 renders 768P portrait, the Seedance tiers 720p 16:9 — the shape worth generating for a resting loop. The Seedance rates come from ByteDance's published Seedance rate card ($0.15/sec at 720p, $0.12 Fast, $0.08 Mini); H3 is $0.08/s at 768P. Credits bill at $1 ≈ 720 credit-seconds, so seedance-2.0 is 1,080 credits and the default minimax-h3 is 576.

1080p costs materially more on every model — Seedance 2.0 is $0.37/sec there, roughly 2.5× the 720p rate. For a clip that spends its life as a background loop behind a speaking character, 720p is the right default.

Uploading the portrait

The one asset you bring is the image. Upload the bytes, or hand us a URL and we fetch it server-side — the second is what you want for anything large (the origin must answer with Content-Length; the same 8 MB image cap as an upload applies):

// bytes you already hold — createAvatar takes the asset id from here
const asset = await rta.uploadAsset(file, { kind: "image" });

// or a URL we fetch for you (no multipart through your runtime)
const asset = await rta.createRemoteAsset({
  kind: "image",
  remoteUrl: "https://your-bucket.example.com/rin/portrait.png",
});

asset.url; // https://realtimeavatar.ai/api/assets/… — public, ready to use

# client: the httpx.Client from the quickstart (base_url https://realtimeavatar.ai/api/v1, bearer API key)

# bytes you already hold
with open("portrait.png", "rb") as fh:
    asset = client.post("/assets", files={"file": fh}, data={"kind": "image"}).json()
# raw bytes without a filename: files={"file": ("portrait.png", data, "image/png")} —
# the server keys on the part's Content-Type (jpeg/png/webp)

# or a URL we fetch for you
asset = client.post("/assets/remote", json={
    "kind": "image",
    "remoteUrl": "https://your-bucket.example.com/rin/portrait.png",
}).json()

asset["publicUrl"]  # https://realtimeavatar.ai/api/assets/… — public, ready to use
# (the SDK surfaces this same field as asset.url; on the wire it is publicUrl)

The returned URL is publicly readable — no auth, no signing. Treat these URLs as unguessable, not private: the key is random and nothing indexes it, but anyone holding the link can fetch it. Do not upload anything whose exposure would matter.

Uploading costs nothing — you are charged for generation, not storage. Accepted kinds are image (portraits), audio (voice material), and video for an uploaded motion clip declared with source: { assetId }. Video upload does not make the asset an Avatar source: new Avatars are still portrait-only, and an uploaded action clip must pass the rest-pose check before it can serve.