# Realtime Avatar > A realtime AI avatar API and TypeScript SDK. Give a character a face and a voice, put them > on a live call in a browser or a native app, and let them use your tools. One mint endpoint > on your server decides who may call and what the character knows; one component renders it. The sections between generated markers below are rebuilt by `bun scripts/content/sync-content.ts` — do not hand-edit them. ## Start here - [Realtime Avatar](https://realtimeavatar.ai/): the product itself — what it is, what it replaces, and a prompt you can hand to a coding agent. Also as markdown at https://realtimeavatar.ai/index.md, which inlines the whole FAQ. - [Pricing](https://realtimeavatar.ai/pricing): every plan and what a live minute costs. Also as markdown at https://realtimeavatar.ai/pricing.md - [Overview](https://realtimeavatar.ai/docs): what the product is and what a call costs. - [Quickstart](https://realtimeavatar.ai/docs/quickstart): key → app on an example avatar → your own character. - [Authentication](https://realtimeavatar.ai/docs/authentication): what belongs on your server and why. ## Agent tools (MCP) If you are a coding agent, you do not have to guess at this account. There is an MCP server — `realtime-avatar-mcp` on npm, stdio, no build step — that tells you which avatars actually exist, what a call costs, and what the balance is, before you write any code. Avatar ids cannot be guessed; call `list_avatars` first. { "mcpServers": { "realtime-avatar": { "command": "npx", "args": ["-y", "realtime-avatar-mcp"], "env": { "REALTIME_AVATAR_API_KEY": "YOUR_API_KEY" } } } } Read tools (`list_avatars`, `get_avatar`, `credit_balance`, `list_sessions`, `list_clips`) are on by default. Set `REALTIME_AVATAR_ALLOW_WRITES=1` to also get `create_avatar_from_image`, `create_avatar_from_video`, `upload_asset`, `create_remote_asset`, `start_call`, `set_loop`, `set_clip_library` and `sync_clips`. Full page: https://realtimeavatar.ai/docs/mcp.md The server is a BUILD-TIME tool. Applications should still mint sessions from their own server route — that endpoint is where the caller is authorised and the character briefed. ## Machine-readable surfaces - MCP server for agents: `npx -y realtime-avatar-mcp` — see above, and https://realtimeavatar.ai/docs/mcp.md - Full docs as one file: https://realtimeavatar.ai/llms-full.txt - Every docs page as markdown: append `.md` to its URL. - Or ask any page for markdown at its own URL: send `Accept: text/markdown`. Pages with a published mirror answer `200 text/markdown` and name it in `Content-Location`; pages without one answer HTML if you will take it, and `406` if you said you would not. - The landing page as markdown, FAQ inlined: https://realtimeavatar.ai/index.md - Pricing as markdown: https://realtimeavatar.ai/pricing.md - OpenAPI 3.1 for the whole API: https://realtimeavatar.ai/openapi.json — 24 operations, request and response schemas, and the scope each one needs, machine-readably in `x-required-scopes` on every operation and `x-scopes` on the security scheme. Generate a client from it rather than reading the endpoint table. - API authentication, as RFC 9728 protected-resource metadata: https://realtimeavatar.ai/.well-known/oauth-protected-resource/api — the bearer scheme and every scope a key can carry. There is no OAuth authorization server: keys are minted in the dashboard or with `POST /api/v1/api-keys`, and a 401 or 403 says so in `WWW-Authenticate`. - A wrong path answers a real `404` everywhere, with the site map in the body — as markdown or JSON if you asked for either, and always as JSON under `/api`. ## Pricing One meter: seconds on air. Speech recognition, the LLM, the voice and rendering are included. Published self-serve overage is $0.08–$0.095 per minute ($4.80–$5.70 per hour). Your own hosting and external tools remain separate. | Plan | Monthly | Included realtime | Overage | Concurrent streams | | --- | --- | --- | --- | --- | | Sandbox | Free | 8 min | — | 2 | | Starter | $9/mo | 120 min | $5.70/hour | 3 | | Developer | $24/mo | 600 min | $5.10/hour | 10 | | Studio | $119/mo | 3,000 min | $4.80/hour | 20 | | Scale | Quoted | Your volume | Quoted | Custom | Creating a character is billed separately from talking to it: $3 per avatar beyond the number your plan includes, its generated resting loop included. Credits are one pot: the same balance pays for generating a character and for the minutes it then talks. Scale is priced by conversation rather than published, so no figure for it appears on this site. Full table: https://realtimeavatar.ai/pricing · as markdown: https://realtimeavatar.ai/pricing.md Cost calculator: https://realtimeavatar.ai/tools/avatar-cost-calculator · worked scenarios: https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing ## Documentation (raw markdown) Every docs page is mirrored as plain markdown — append `.md` to any docs URL. - [Realtime Avatar API](https://realtimeavatar.ai/docs.md): Put a live character in your product, on a voice or video call, with your own tools wired into the conversation. - [Quickstart](https://realtimeavatar.ai/docs/quickstart.md): From an API key to a live avatar call in three steps: get a key, build the app on an example avatar, then swap in a character of your own. - [Authentication](https://realtimeavatar.ai/docs/authentication.md): Get a key, keep it on your server, and decide who may start a call. - [Build with Lovable, Bolt, v0 and Replit](https://realtimeavatar.ai/docs/ai-app-builders.md): Add a live AI avatar call to an app built in Lovable, Bolt.new, v0 or Replit: a copy-paste prompt per builder, where the API key goes, and a starter repo that opens in one click. - [Build a realtime AI avatar in Next.js](https://realtimeavatar.ai/docs/nextjs.md): Build a working Next.js avatar app with a server-side session proxy, React video UI, a complete local starter, and troubleshooting. - [TanStack Start](https://realtimeavatar.ai/docs/tanstack-start.md): Mount the realtime avatar proxy in a TanStack Start app: a splat server route on the server, one component in the browser. - [Express](https://realtimeavatar.ai/docs/express.md): Mount the realtime avatar proxy as Express middleware — Express 4 or 5 — with the same authorize and session policy every adapter takes. - [Hono, Workers, Bun and Deno](https://realtimeavatar.ai/docs/hono.md): Mount the realtime avatar proxy on any Fetch-handler runtime — Hono, Cloudflare Workers, Bun or Deno — including the apiKey factory Workers require. - [Realtime AI avatars with React and React Native](https://realtimeavatar.ai/docs/react.md): Embed live AI avatar calls in React or React Native with AvatarCall, useAvatarCall, connection states, and a server-side session proxy. - [Build avatar apps with the Realtime Avatar MCP server](https://realtimeavatar.ai/docs/mcp.md): Connect a coding agent to Realtime Avatar: configure the MCP server, inspect available avatars, and build a working app with the TypeScript SDK. - [Creating an avatar](https://realtimeavatar.ai/docs/video.md): One image in, a moving character out: generated motion, idle variations and requestable actions, full-library declarations, loop editing, and generation prices. - [Calls](https://realtimeavatar.ai/docs/sessions.md): What your server decides, what the client reports, and how to end a call without cutting the character off mid-sentence. - [Editing a character](https://realtimeavatar.ai/docs/editing.md): Change an avatar's clips and resting loop after creation: how to declare them, how each change settles, and the refusals worth telling apart. - [Tool calling](https://realtimeavatar.ai/docs/tool-calling.md): The platform never executes your tools. Here is the boundary, the client tool plane, and the server loop that turns a call into a working agent with a face. - [API reference](https://realtimeavatar.ai/docs/api-reference.md): Every public endpoint on https://realtimeavatar.ai/api/v1 — what it takes, what it returns, which scope it needs, and how it fails. - [Experimental features](https://realtimeavatar.ai/docs/experimental.md): What is experimental right now, what that word promises, and how to turn each one on. Included with Studio and above. ## Blog (raw markdown) Published posts as markdown; append `.md` to any post URL. - [Build an Avatar Assistant People Can Use Without Speaking or Hearing](https://realtimeavatar.ai/blog/accessible-avatar-assistant-without-voice.md): Build a text\-first appointment assistant with optional avatar calls, verified confirmations, keyboard controls, and clear timeout recovery\. - [Build an AI Avatar Fantasy Adventure with Choices Beyond a Menu](https://realtimeavatar.ai/blog/ai-avatar-fantasy-adventure-story-game.md): Build a fantasy adventure where players invent solutions, cross a floating market with an avatar guide, and reach an ending shaped by their ideas\. - [Build an AI Avatar Horror Hotline Players Can Hang Up On](https://realtimeavatar.ai/blog/ai-avatar-horror-hotline-game.md): Build a playable AI avatar horror hotline with a stranded caller, a fictional island map, three consequential choices, and endings players can replay\. - [Build an AI Avatar Interrogation Mystery Players Can Solve](https://realtimeavatar.ai/blog/ai-avatar-interrogation-mystery-game.md): Build a playable AI avatar mystery with three suspects, cross\-checkable clues, a fixed solution, and an accusation that rewards evidence over endless chat\. - [Build an AI Avatar Party Game with an Improv Host](https://realtimeavatar.ai/blog/ai-avatar-party-game-improv-host.md): Build a three\-round AI avatar party game where friends pitch absurd inventions, survive a twist, and vote for a winner, with a host who keeps it moving\. - [The $1M ARR AI Companion Playbook for 2027](https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027.md): Build toward a $1M ARR AI companion business: choose a niche, rent realtime infrastructure, cap call costs, and test retention and distribution\. - [Build an AI Customer Support Avatar That Knows When to Hand Off](https://realtimeavatar.ai/blog/ai-customer-support-avatar-human-handoff.md): Build a support avatar with authenticated order lookup, consent\-based summaries, durable tickets, and honest human\-handoff states using Realtime Avatar\. - [Build an AI Email Assistant That Shows the Evidence Behind Its Briefing](https://realtimeavatar.ai/blog/ai-email-assistant-source-linked-briefings.md): Build a read\-only AI email briefing with source\-linked exceptions, visible sync limits, thread comparisons, and optional voice discussion before a trip\. - [Build an AI Game NPC That Respects Quest State and Spoilers](https://realtimeavatar.ai/blog/ai-game-npc-dialogue-quest-state.md): Build an AI game NPC with game\-owned quest state, spoiler\-safe context, retry\-safe rewards, save recovery, and optional browser avatar conversations\. - [Build an AI Museum Avatar Guide That Can Show Its Sources](https://realtimeavatar.ai/blog/ai-museum-avatar-guide-collection-sources.md): Build an AI museum avatar guide with official collection records, visible sources, careful attribution, object\-level image rights, and honest unknowns\. - [Build AI Roleplay Chat with Edits, Rewinds and Separate Story Branches](https://realtimeavatar.ai/blog/ai-roleplay-chat-edit-rewind-branches.md): Build AI roleplay chat with distinct edit, regenerate, rewind and branch actions, consistent checkpoints, and guards against canceled or stale replies\. - [Build an AI Shopping Avatar That Compares Products Before Checkout](https://realtimeavatar.ai/blog/ai-shopping-avatar-product-comparison.md): Build a read\-only shopping avatar with exact Shopify variants, source\-backed comparisons, honest stock labels, and price rechecks before checkout\. - [Build an AI Story Generator That Can Stop, Save and Continue Cleanly](https://realtimeavatar.ai/blog/ai-story-generator-streaming-save-cancel.md): Build a streaming AI story generator with separate drafts, explicit acceptance, reliable stop controls, revision checks and honest recovery after disconnects\. - [AI Story Generator or Character Roleplay? Choose How You Want to Play](https://realtimeavatar.ai/blog/ai-story-generator-vs-character-roleplay.md): Choose an AI story generator, character roleplay or branching fiction\. Try three cozy prompts, fix unwanted narration and keep your story going\. - [How to Build an AI Bible Study Chatbot with Voice and an Avatar](https://realtimeavatar.ai/blog/build-ai-bible-study-avatar.md): Build a voice\-and\-avatar Bible study guide with source\-linked passages, clear AI disclosure, translation labels, and application\-owned conversation context\. - [Build AI Character Video Keepsakes: Turn One Great Scene Into a Takeaway](https://realtimeavatar.ai/blog/build-ai-character-video-keepsakes.md): Build AI character video keepsakes from player\-approved moments, with editable prompts, clear costs, private previews and a small paid\-demand test\. - [Build an AI Companion App: Make the First Conversation Worth Returning To](https://realtimeavatar.ai/blog/build-ai-companion-app-first-session.md): Build an AI companion app's first session with character choice, a useful opening, optional calls, explicit guest saves, and honest credit exits\. - [Build an AI Roleplay Comic Generator: Give Players a Story Worth Sharing](https://realtimeavatar.ai/blog/build-ai-roleplay-comic-generator.md): Turn an original roleplay scene into a four\-panel comic with editable storyboards, consistent characters, panel retries, and a paid\-issue experiment\. - [How to Build an AI Tarot Chatbot with Voice and a Live Avatar](https://realtimeavatar.ai/blog/build-ai-tarot-chatbot-avatar.md): Build an AI tarot chatbot with stable card draws, tool\-backed context, voice and avatar sessions, clear pricing, and a separate astrology calculation layer\. - [Character\.AI Alternatives for Builders: Consumer App, Agent Framework or Avatar API?](https://realtimeavatar.ai/blog/character-ai-alternative-api-build-vs-buy.md): Choose a Character\.AI alternative for your own app by separating character authoring, agent infrastructure and managed avatar calls, with a concrete test\. - [Free AI Character Chat: Check the Limits Before Starting a Story](https://realtimeavatar.ai/blog/free-ai-character-chat-voice-video-limits.md): Find free AI character chat that fits your routine\. Check text limits, premium models, ads, voice minutes, video credits, and what happens after a trial\. - [Build an Interactive Romance Short Drama Players Can Step Into](https://realtimeavatar.ai/blog/interactive-romance-avatar-short-drama.md): Build a replayable romance episode with an AI avatar: a station reunion, meaningful choices, satisfying endings, and a practical live\-call implementation\. - [MCP, A2A, PAP and Agent Payments: What Each Layer Actually Does](https://realtimeavatar.ai/blog/mcp-a2a-pap-agent-payment-protocols.md): Compare MCP, A2A, PAP, ACP and AP2 by responsibility and maturity, with a shopping\-agent example separating access, approval, payment and verified results\. - [Keep Personal Agent Tasks Running After the Call Ends](https://realtimeavatar.ai/blog/personal-agent-background-tasks-after-call.md): Keep personal agent tasks running after a voice call with app\-owned jobs, honest status, safe retries, and recoverable context in Realtime Avatar\. - [Designing a Personal Agent Users Can Understand, Correct and Approve](https://realtimeavatar.ai/blog/personal-agent-trust-approval-interface.md): Design a personal agent approval and correction interface that separates proposals, permission and verified results, with practical Realtime Avatar boundaries\. - [Build a Personal AI Assistant for Daily Planning with Voice and an Avatar](https://realtimeavatar.ai/blog/personal-ai-assistant-daily-planning-avatar.md): Build a daily\-planning personal AI assistant with calendar context, voice and an avatar\. Keep schedules, memory and confirmed actions in your application\. - [Personal AI Assistant Memory: Corrections, Provenance and Migration](https://realtimeavatar.ai/blog/personal-ai-assistant-memory-correction-migration.md): Build personal AI assistant memory that preserves corrections, tracks provenance, imports only supported context, and stays accurate across new calls\. - [Build a Personal Assistant That Continues from Phone to Desktop](https://realtimeavatar.ai/blog/personal-assistant-phone-to-desktop-handoff.md): Design a phone\-to\-desktop assistant handoff with confirmed trip context, authenticated resume, fresh calls, and safeguards against stale device commands\. - [Build a Talking Virtual Pet Players Want to Visit, Not Worry About](https://realtimeavatar.ai/blog/talking-virtual-pet-avatar-game.md): Build a talking virtual pet around tiny expeditions, meaningful choices and a personal scrapbook, with satisfying visits and no missed\-day guilt\. - [Building Interactive Avatar Training Courses with Realtime AI](https://realtimeavatar.ai/blog/building-interactive-avatar-training-courses-with-realtime-ai.md): Explore the engineering of dynamic, real\-time AI avatar training courses, moving beyond static video to interactive, adaptive learning experiences\. - [Your personal agent deserves a face](https://realtimeavatar.ai/blog/give-your-personal-agent-a-face.md): Instinct, Muse, and Tab are making personal agents part of everyday life\. Realtime Avatar helps developers give their own agent a face people can talk to\. - [From Portrait to Presence: Realtime AI Avatars for Dynamic Training](https://realtimeavatar.ai/blog/from-portrait-to-presence-ai-avatars-for-realtime-training.md): Explore how to build interactive AI training experiences using realtime avatars, created from a single image and deployed for live, dynamic learning\. - [Field Notes: Realtime AI Avatars for Engaging and Scalable Training](https://realtimeavatar.ai/blog/field-notes-realtime-ai-avatars-for-engaging-and-scalable-training.md): Explore how realtime AI avatars transform corporate training, offering scalable, consistent, and interactive learning experiences\. Understand creation\. - [Field Notes: Navigating Live Avatar API Alternatives for Developers](https://realtimeavatar.ai/blog/live-avatar-api-alternatives-for-developer-architectures.md): An engineer's guide to Live Avatar API alternatives, focusing on real\-time conversational AI, single\-image avatar creation, and integration flexibility\. - [Choosing a Realtime AI Avatar API for Live Web Conversations](https://realtimeavatar.ai/blog/choosing-realtime-ai-avatar-api-web-conversations.md): Navigate API options for live AI avatars in web apps\. Understand the division of labor, latency, customization, and integration of leading platforms\. - [Which Talking Avatar APIs Include Speech Recognition, the Language Model, and Text\-to\-Speech?](https://realtimeavatar.ai/blog/full-stack-talking-avatar-apis-stt-llm-tts-integration.md): Explore how Realtime Avatar APIs orchestrate speech recognition, language models, and text\-to\-speech for interactive digital human experiences\. - [Realtime AI avatar APIs compared \(October 2026\): price per minute, free tiers, SDKs, tools](https://realtimeavatar.ai/blog/realtime-ai-avatar-apis-compared-2026.md): Tavus, LiveAvatar, Anam, Simli, Beyond Presence, D\-ID, bitHuman and Realtime Avatar compared on price per minute, free tier, SDKs and tool calling\. - [How to put a real\-time AI avatar in a React Native or Expo app](https://realtimeavatar.ai/blog/realtime-ai-avatar-react-native-expo.md): Add a live, interruptible AI avatar video call to React Native or Expo: install, dev build, server route, native call screen and idle video, with real SDK code\. - [Realtime AI Avatars: Navigating GitHub for Open\-Source Components and Managed SDKs](https://realtimeavatar.ai/blog/realtime-ai-avatars-github-components-vs-managed-sdk.md): Explore the landscape of realtime AI avatars on GitHub, from assembling open\-source components to leveraging production\-ready SDKs like the \`realtime\-avatar\`\. - [How much does Tavus CVI cost per minute? Tavus pricing and the per\-hour math \(October 2026\)](https://realtimeavatar.ai/blog/tavus-pricing-per-minute.md): Tavus conversational video costs $0\.26 to $0\.35 per minute in overage, $15\.60 to $21 an hour\. Plan\-by\-plan math, the 30\-second minimum, and comparisons\. - [Create Your Own AI Character: Personality, Voice and a Face](https://realtimeavatar.ai/blog/generating-realtime-ai-avatars-from-portrait-to-live-interaction.md): Create an original AI character with a practical personality brief, portrait, voice and greeting\. Learn what to customize, test and check before paying\. - [The Engineering of Realtime AI Avatar Chatbots: Bridging Talk and Action](https://realtimeavatar.ai/blog/the-engineering-of-realtime-ai-avatar-chatbots.md): Explore the core architecture of realtime AI avatar chatbots, from full\-duplex conversation to tool integration, for dynamic and actionable interactions\. - [Architecting Real\-Time AI Video Avatars: From Image to Live Presence](https://realtimeavatar.ai/blog/architecting-realtime-ai-video-avatars-from-image-to-live-presence.md): Explore the architecture behind real\-time AI video avatars, from single portrait image creation to live, interactive conversational experiences\. - [Realtime Avatar: A Tavus Alternative for Live AI Characters](https://realtimeavatar.ai/blog/tavus-alternative-realtime-avatar-griffin.md): Looking for a Tavus alternative after Griffin's October 1 launch? Discover Realtime Avatar for custom AI characters and live conversations, then try it free\. - [AI Voice Chat or Video Call? What Changes When a Character Talks Back](https://realtimeavatar.ai/blog/field-notes-on-realtime-ai-avatar-video-from-portrait-to-live-interaction.md): Choose between AI voice messages, live voice chat, avatar video calls and generated clips, with a practical checklist for quality, controls and cost\. - [Realtime AI Avatars: Engineering Presence, Informed by Developer Insights](https://realtimeavatar.ai/blog/realtime-ai-avatars-engineering-presence-developer-insights.md): Explore the technical challenges and solutions for building real\-time AI avatars, drawing insights from developer communities and practical implementation\. - [The Production Divide: Open Source vs\. Managed APIs for Realtime AI Avatars](https://realtimeavatar.ai/blog/the-production-divide-open-source-vs-managed-apis-for-realtime-ai-avatars.md): Building real\-time AI avatars from open\-source components presents significant engineering challenges\. We explore the divide and practical solutions\. - [How to Build a Live AI Avatar Host for Streaming](https://realtimeavatar.ai/blog/real-time-ai-avatars-for-streaming-architects-of-live-digital-presence.md): Build a live AI avatar host with server\-owned sessions, moderated audience questions and a separate broadcast path\. Compare supported interfaces and costs\. - [AI Character Chat and Roleplay: Choose the Conversation You Want](https://realtimeavatar.ai/blog/building-engaging-realtime-ai-avatar-chat-experiences.md): Choose AI character chat for casual banter or longer roleplay\. Try practical prompts, check repetition and story control, and compare limits before paying\. - [Open Source Realtime AI Avatars: Assembling Production\-Ready Experiences](https://realtimeavatar.ai/blog/open-source-realtime-ai-avatars-assembling-production-ready-experiences.md): Exploring the landscape of open source components for realtime AI avatars and the significant engineering effort required to bring them to production\. - [Meta's Muse Realtime Avatar and Realtime Avatar: What's the difference?](https://realtimeavatar.ai/blog/meta-muse-realtime-avatar-vs-realtime-avatar.md): Meta introduced Muse Realtime Avatar on September 23, 2026\. Here's how the Muse experience compares with creating a live character for your own product\. - [The Architecture of Live AI Avatar Video: Realtime Presence, Not Just Playback](https://realtimeavatar.ai/blog/the-architecture-of-live-ai-avatar-video-realtime-presence-not-just-playback.md): Explore the engineering behind live AI avatar video\. Understand how to build dynamic, interactive digital characters for your applications using a realtime\. - [A Developer's Field Guide to Realtime Avatar APIs: Beyond Simple Video Streaming](https://realtimeavatar.ai/blog/a-developers-field-guide-realtime-avatar-apis-beyond-simple-video-streaming.md): For developers evaluating HeyGen API alternatives, understand the critical difference between streaming video and building truly interactive, full\-duplex AI\. - [Field Notes on Real\-Time Avatars: The Emergence of Live Digital Presence](https://realtimeavatar.ai/blog/field-notes-on-real-time-avatars-the-emergence-of-live-digital-presence.md): Explore the engineering behind real\-time AI avatars, digital characters that respond instantly to live input, enabling interactive experiences\. - [Architecting Avatars That Talk Back: Realtime Interaction Principles](https://realtimeavatar.ai/blog/architecting-avatars-that-talk-back-realtime-interaction-principles.md): Explore the engineering behind interactive AI avatars that talk back, focusing on the TIC Realtime Avatar platform's full\-duplex communication and visual\. - [Field Notes on Open\-Source Interactive AI Avatars: Realtime Challenges and Solutions](https://realtimeavatar.ai/blog/open-source-interactive-ai-avatars-realtime-challenges.md): Explore the landscape of open\-source interactive AI avatars, dissecting the real\-time challenges and practical steps for building embodied conversational\. - [Building Production\-Ready Interactive AI Avatars with a TypeScript SDK](https://realtimeavatar.ai/blog/building-production-ready-interactive-ai-avatars.md): Explore how to move from open\-source concepts to robust, real\-time AI avatar applications using a powerful TypeScript SDK\. - [Realtime Live Avatars: Decoding API Pricing for Production](https://realtimeavatar.ai/blog/realtime-live-avatars-decoding-api-pricing-for-production.md): Unpack the economics of live avatar APIs\. Explore TIC Realtime Avatar's usage\-based pricing, its "face layer" architecture, and how to build and ship\. - [Designing Interactive AI Avatar Chatbots: Beyond Conversation](https://realtimeavatar.ai/blog/designing-interactive-ai-avatar-chatbots.md): Field notes on building AI avatar chatbots that engage in natural, full\-duplex conversations and execute actions through integrated tools\. - [Architecting Realtime Interactive AI Avatar Chat Experiences](https://realtimeavatar.ai/blog/realtime-interactive-ai-avatar-chat-architecture.md): Building interactive AI avatar chats requires a blend of real\-time audio, expressive video, and intelligent agent logic\. This guide covers the essential\. - [Building Interactive AI Avatar Applications: Real\-Time Presence in Practice](https://realtimeavatar.ai/blog/building-interactive-ai-avatar-apps.md): Explore the architecture and implementation of interactive AI avatar applications\. Learn how to create real\-time, two\-way conversational experiences with\. - [How to test barge\-in in a realtime avatar app](https://realtimeavatar.ai/blog/testing-barge-in-in-a-realtime-avatar-app.md): Test realtime avatar interruptions with repeatable voice scenarios, recordings, and clear pass criteria before choosing an API or shipping your app\. - [Build a realtime avatar language\-practice app in Next\.js](https://realtimeavatar.ai/blog/build-a-realtime-avatar-language-practice-app-nextjs.md): Run a complete Next\.js avatar starter, turn it into a café roleplay partner, and test the conversation with a concrete rubric\. Includes source download and server\-side session policy\. - [How to benchmark realtime avatar startup and turn latency](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency.md): A reproducible measurement protocol for avatar startup, first audible reply, interruptions and failures, with downloadable Next\.js instrumentation and reporting guidance\. - [Evaluating a LiveAvatar or Tavus alternative: a developer checklist](https://realtimeavatar.ai/blog/liveavatar-tavus-alternative-developer-checklist.md): Compare realtime avatar APIs by pipeline ownership, startup, conversational behavior, integration effort and total call cost\. Includes a runnable Next\.js evaluation app\. - [Kling AI Avatar API: video jobs and live conversations](https://realtimeavatar.ai/blog/navigating-the-kling-ai-avatar-api-landscape.md): What Kling Avatar v2's fal endpoint accepts and returns, how it differs from a live avatar session, and what to measure before choosing an integration\. - [Avatar Renderer vs Full\-Stack API: A Developer Integration Guide](https://realtimeavatar.ai/blog/renderer-or-full-stack-realtime-avatar-apis-compared.md): Choose an avatar renderer or a managed conversation API for your voice agent\. Compare audio inputs, tools, latency measurements and complete session costs\. - [Generated video that answers back: making a live avatar interactive, not just alive](https://realtimeavatar.ai/blog/generated-video-that-answers-back.md): A live avatar that performs on a loop is easy\. One that changes course when you ask is a scheduling problem — and per\-second pricing is what makes the fix affordable\. - [Integrating the AI Talking Avatar API: Principles of Realtime Embodiment](https://realtimeavatar.ai/blog/integrating-the-ai-talking-avatar-api-principles-of-realtime-embodiment.md): Explore the technical foundations and practical integration of an AI talking avatar API\. Understand the session architecture, timing measurements and usage costs behind live conversations\. - [Decoding Realtime Presence: An Engineer's Guide to the AI Avatar Video API](https://realtimeavatar.ai/blog/decoding-realtime-presence-an-engineers-guide-to-the-ai-avatar-video-api.md): Explore the core mechanics and practical applications of an AI avatar video API\. Understand how real\-time, interactive video streams empower dynamic user experiences, from customer support to immersive gaming\. - [Live avatars in 2026: a field guide to the research](https://realtimeavatar.ai/blog/live-avatars-2026-research-field-guide.md): How real\-time avatars became possible: the distillation recipe behind streaming video generation, the listening\-head turn, full\-duplex voice, and what the 2026 literature says is still hard\. - [The Core Mechanics of a Realtime AI Avatar Generator API: From Image to Embodiment](https://realtimeavatar.ai/blog/the-core-mechanics-of-a-realtime-ai-avatar-generator-api.md): Explore how a realtime AI avatar generator API turns a portrait into a conversational character, with session integration, timing measurements and usage costs\. - [LiveAvatar API billing: credits, conversation modes, and total cost](https://realtimeavatar.ai/blog/demystifying-heygen-live-avatar-api-pricing.md): Understand LiveAvatar FULL versus LITE credit units, then compare subscription allowances and complete conversation costs with Realtime Avatar\. - [Realtime avatar API pricing: included minutes, overage, and total cost](https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing.md): Calculate live\-avatar API costs from monthly minutes and concurrency\. Catalog\-derived plan prices, worked examples, and an interactive calculator with explicit assumptions\. - [Agentic localization: how this blog stays multilingual without a translation team](https://realtimeavatar.ai/blog/agentic-localization-pipeline.md): Every post on this blog can ship in English, Chinese, and Japanese\. The translators are LLM agents; the editors are pull requests\. Here's the pipeline\. (also [zh](https://realtimeavatar.ai/blog/agentic-localization-pipeline/zh.md)) - [Looking for a HeyGen alternative? Decide what you're actually replacing first](https://realtimeavatar.ai/blog/heygen-alternative-for-realtime-avatars.md): A developer's guide to choosing a HeyGen alternative in 2026: video\-generation platforms vs realtime avatar APIs, what each costs per minute, and when each one wins\. - [HeyGen API vs LiveAvatar Pricing: What a Live Conversation Costs](https://realtimeavatar.ai/blog/heygen-api-pricing-explained.md): Compare HeyGen video generation with LiveAvatar's live conversation plans, credit conversion and RTA pricing using current official sources and clear limits\. - [Payload CMS vs\. a git\-based blog: what we chose for an agent\-written site](https://realtimeavatar.ai/blog/payload-cms-vs-git-based-blog.md): We researched running Payload 3 on Cloudflare Workers \+ D1, weighed it against keeping content in the repo, and picked the stack AI agents are best at maintaining\. - [Synthesia pricing in 2026 — and the alternatives when you need more than rendered video](https://realtimeavatar.ai/blog/synthesia-pricing-and-alternatives.md): Synthesia's plans, what the API actually includes, and the alternative paths — D\-ID, HeyGen, and realtime avatar APIs — when your use case is interactive rather than rendered\. - [The Architecture of Presence: Integrating a Realtime AI Avatar API](https://realtimeavatar.ai/blog/the-architecture-of-presence-integrating-a-realtime-ai-avatar-api.md): How to integrate a realtime AI avatar API: the architecture behind sub\-second, audio\-clocked conversational avatars, and what it takes to ship one in your app\. - [AI Boyfriend or Girlfriend: Choosing Text, Voice and Video Companionship](https://realtimeavatar.ai/blog/ai-companion-apps-with-a-face.md): Choose an AI boyfriend, girlfriend or companion by comparing text, voice and video, checking conversation continuity, and budgeting for regular use\. (also [zh](https://realtimeavatar.ai/blog/ai-companion-apps-with-a-face/zh.md)) - [Roleplay at scale: AI avatars for sales and corporate training](https://realtimeavatar.ai/blog/ai-avatars-for-corporate-training.md): The best training is rehearsal against a difficult human\. Realtime avatars make that rehearsal infinitely available, repeatable, and measurable\. - [What is a realtime avatar API? A builder's guide](https://realtimeavatar.ai/blog/what-is-a-realtime-avatar-api.md): From a single portrait to a character that answers in live, audio\-synced video: the moving parts of a realtime avatar API and what to evaluate before you build on one\. --- # Documentation — full pages # Realtime Avatar API > Put a live character in your product, on a voice or video call, with your own tools wired into the conversation. - Canonical: https://realtimeavatar.ai/docs - Updated: 2026-09-14 ## Voice or video, on one meter **TypeScript** ```tsx // she is on screen // audio only ``` **Python** ```python body = {"avatar_id": avatar_id, "mode": "avatar"} # she is on screen body = {"avatar_id": avatar_id, "mode": "voice"} # audio only ``` **It is priced to leave on.** Live-call self-serve overage meters at **about $5 an hour** — $4.80–$5.70 depending on plan — with a free tier to build against. That is the difference between a demo you show and a feature you ship: a companion app can afford to let someone talk. How she is rendered is in [Creating an avatar](https://realtimeavatar.ai/docs/video). ## She listens while she talks Most voice AI takes turns like a walkie-talkie: you talk, you stop, it answers. This one is full-duplex — she is hearing you the whole time she is speaking — and in practice that shows up as four things you do not have to build: - **Interrupt her and she stops**, mid-sentence, and can acknowledge it in character rather than snapping to silence. - **A cough or an "mm-hm" does not derail her.** A backchannel is not an interruption, and being cut off by one is what makes a system feel brittle. - **A pause is not the end of your turn.** Whether you are finished is judged by what you said, not by how long you have been quiet — so she neither talks over you nor leaves a gap. - **Silence and language switches are handled.** Going quiet is a signal she can act on, and she follows a language change inside a single sentence. Nothing to configure — it is how every call behaves. The one limit worth knowing: she will not talk over you with a new sentence while you are speaking. That is a deliberate trade for the voice and model choices this platform is built on. ## Your tools, wired into the conversation A character that can only talk is a demo. The useful version books the appointment, checks the order, writes the code and keeps talking while it runs. You declare a tool the same way you would brief a colleague: what it is called, and *when to reach for it*. The description is the whole teaching signal — she reads it and decides. **TypeScript** ```ts import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` **Python** ```python # Tools run in the page that renders the call — there is no hosted execution, # so there is nothing to declare in Python. Your backend's part is the GRANT: body["capabilities"] = ["client_tools"] # The mint schema is strict: a tools[] field on the request itself is a 422. # The page registers the manifest over RPC once the room is connected. ``` Then your server grants the tool plane at mint, and the page registers the tools once the room is connected. Omitting `execute` is a compile error rather than a timeout you find in production — and a tool has **2.5 seconds** to answer, so anything slow acknowledges fast and delivers the real result out of band: **TypeScript** ```ts // server — the session policy grants the client tool plane for this call session: async ({ avatarId }) => ({ instructions, clientTools: true }) // client — register over RPC after connect; the record key is the tool's name import { attachAvatarTools } from "realtime-avatar/tools"; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` **Python** ```python # server — grant the capability on the mint. That is the whole backend part: body["capabilities"] = ["client_tools"] # The worker only exposes tool registration to a session minted with the # capability. If the page reports the registration method is missing, the fix # is HERE, not in the page. ``` Full pattern: [Tool calling](https://realtimeavatar.ai/docs/tool-calling). > These examples target `realtime-avatar@0.16.0`. On an older install the exports differ and the compiler will say so — update the package rather than working around it. ## Where to go next - [Quickstart](https://realtimeavatar.ai/docs/quickstart) — key to live call in three steps. - [Calls](https://realtimeavatar.ai/docs/sessions) — the policy your server decides, the five states, ending gracefully. - [Creating an avatar](https://realtimeavatar.ai/docs/video) — one image in; the platform generates the loop and a map of states it switches between, or synthesizes the video live. - [Tool calling](https://realtimeavatar.ai/docs/tool-calling) — running your own agent loop against a live character. - [API reference](https://realtimeavatar.ai/docs/api-reference) — every endpoint, scope, and error. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Quickstart > From an API key to a live avatar call in three steps: get a key, build the app on an example avatar, then swap in a character of your own. - Canonical: https://realtimeavatar.ai/docs/quickstart - Updated: 2026-09-14 Three steps. Your first call runs on a public example avatar, so nothing is blocked on creating one — and the app is one step, not two: a server half that holds the key and a client half that renders. The SDK ships both. ## 1. Get a key and install the SDK Open [workspace Settings → API keys](https://realtimeavatar.ai/platform/settings#api-keys), give your key a name, and select **Create API key**. Copy the full key and save it as `REALTIME_AVATAR_API_KEY` in your server’s environment. The default permissions work with this quickstart; advanced settings are optional. The list stays masked. Workspace owners and admins can return to Settings and select **Copy** to copy newly created keys again while they are active and unexpired. Legacy keys stored as hashes only cannot be recovered if lost and need a replacement. Revoked or expired keys cannot be copied. See [Authentication](https://realtimeavatar.ai/docs/authentication) for details. The SDK is on npm — no registry configuration, no auth token: **TypeScript** ```bash npm install --save-exact realtime-avatar@0.16.0 ``` Self-contained: the contracts package is bundled into the build, so there is no private dependency to resolve. Pin an exact version for reproducible CI. **Python** ```bash pip install httpx ``` There is no Python SDK — the live call is rendered by a browser or native client, so Python's half is plain HTTP and one dependency. ## 2. Build the app on an example avatar An app is two halves and the SDK ships both: a connect endpoint on your server, and one component in the browser. Hand this whole step to a coding agent if you would rather not type it — the prompt is on your key in [settings](https://realtimeavatar.ai/platform/api-keys). ### The server half — the connect endpoint Your API key must never reach a browser, so the browser talks to your app and your app talks to us. One function is that endpoint — it decides **who** may start a call and **what** the character knows when they do: **Next.js** ```tsx // app/api/realtime-avatar/[...path]/route.ts import { createRealtimeAvatarRoute } from "realtime-avatar/nextjs"; export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, // Who may do this. "connect" is the only operation that costs money to start, // so that is where the wallet check belongs; the reads stay cheap. authorize: async ({ request, operation }) => { const user = await currentUser(request); if (!user) return new Response("Unauthorized", { status: 401 }); if (operation === "connect" && !(await hasCredits(user))) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, // What the character knows for THIS call. Whatever the browser sent for these // concerns is discarded — this callback is the only source. session: async ({ request, avatarId, mode }) => { const user = await currentUser(request); const character = await db.character(avatarId); return { instructions: character.prompt, context: await db.recentTurns(user.id, avatarId), maxSeconds: secondsTheBalanceAffords(user, mode), }; }, }); ``` The catch-all segment matters — the route answers four paths under one prefix (`POST /connect`, `POST /end`, `GET /avatars`, `GET /credits`), so a single `[...path]` segment serves all of them. **TanStack Start** ```tsx // src/routes/api/realtime-avatar/$.ts import { createFileRoute } from "@tanstack/react-router"; import { realtimeAvatarServerRoute } from "realtime-avatar/tanstack-start"; const handlers = realtimeAvatarServerRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, authorize: async ({ request, operation }) => { const user = await requireUser(request); if (!user) return new Response("Unauthorized", { status: 401 }); if (operation === "connect" && !(await hasCredits(user))) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, session: async ({ request, avatarId, mode }) => { const user = await requireUser(request); const character = await loadCharacter(avatarId); return { instructions: buildPrompt(character), context: await loadRecentTurns(user.id, avatarId), maxSeconds: secondsTheBalanceAffords(user, mode), }; }, }); export const Route = createFileRoute("/api/realtime-avatar/$")({ server: { handlers }, }); ``` The trailing `$` is Start's splat segment — without it the handler only ever sees the mount path and answers 404. Build the handlers once at module scope. On Cloudflare Workers pass a factory — `apiKey: () => getEnv().KEY` — since there is no `process.env`. **Python** ```python # Your backend decides; the browser renders. Same split, no SDK needed. import os, httpx from fastapi import Depends, FastAPI from fastapi.responses import JSONResponse from pydantic import BaseModel BASE = "https://realtimeavatar.ai/api/v1" client = httpx.Client(base_url=BASE, timeout=30.0, headers={ "Authorization": f"Bearer {os.environ['REALTIME_AVATAR_API_KEY']}"}) app = FastAPI() class StartCallRequest(BaseModel): avatar_id: str mode: str = "avatar" # "avatar" | "voice" queue_ticket_id: str | None = None # a client retrying after a 429 hands its ticket back @app.post("/api/calls") def start_call(req: StartCallRequest, user=Depends(current_user)): if not has_credits(user): return JSONResponse({"code": "insufficient_credits"}, 402) character = load_character(req.avatar_id) seconds = int(seconds_the_balance_affords(user, req.mode)) # wire: int, 1..1800 body = { "avatar_id": character.avatar_id, "mode": req.mode, "stt_mode": "server", # Decided HERE. The client sends none of this. "instructions": build_prompt(character), # ≤32 of exactly {"role": "system" | "user" | "assistant", "content": 1..8000 chars} — # strict: any other key is a 422 "initial_context": [{"role": t.role, "content": t.text} for t in recent_turns(user.id, character.avatar_id)][-32:], "max_session_seconds": max(1, min(seconds, 1800)), # Optional: a signed POST of the transcript when the call ends (receiver: Tool calling). "transcript_webhook": {"url": "https://your.app/api/rta/transcript", # https only "secret": os.environ["TRANSCRIPT_SECRET"]}, # 16..200 chars "client_metadata": {"user_id": str(user.id), # ≤16 string pairs, "character_id": character.avatar_id}, # echoed back to you } if req.queue_ticket_id: body["queue_ticket_id"] = req.queue_ticket_id r = client.post("/realtime/livekit/session", json=body) data = r.json() if r.status_code == 429 and "queue_ticket_id" in data: # Every slot busy — the queue, not an error. Relay it VERBATIM: the busy body # carries queue_ticket_id (the only handle that RELEASES the slot when the # caller gives up) and recommended_retry_ms (when to come back). The client # retries WITH the ticket; reshaping this strands it for its whole TTL. return JSONResponse(data, 429) if not r.is_success: # 402 / 409 / 422 — and the 429s that are NOT the queue (plan concurrency # ceiling, rate limit). Forward {error, code?}, but not at 429: an SDK # client reads every 429 as the queue. return JSONResponse(data, 503 if r.status_code == 429 else r.status_code) return data # return this VERBATIM ``` Two rules when hand-rolling. First, casing is **per endpoint**, and every endpoint is strict — an unknown or mis-cased key is a rejected call. The two realtime routes shown here (`/realtime/livekit/session` and its `/release`) are **snake_case**; every REST resource endpoint — avatars, clips, keys, assets — is **camelCase**. Measured against the published spec: 2 of the 10 request bodies are snake_case at the top level and the other 8 are camelCase — except `POST /avatars`, which is camelCase outside and snake_case inside `voice` (`auto_description`, `voice_id`), as the sample above shows. And the SDK's policy names are not the wire names: `context` is `initial_context`, `maxSeconds` is `max_session_seconds`, `transcript` is `transcript_webhook`, `metadata` is `client_metadata`. Second, if your client uses this SDK the response must reach it **byte-for-byte** — the SDK hands the body to the room unvalidated, so a wrapped grant never connects: the token and URL read as undefined and the call sits in `connecting` with no error. A client driving the room directly reads only the fields it needs and tolerates extras, but relaying verbatim keeps both paths working. The same goes for a busy `429`: it is the queue only when the body carries `queue_ticket_id`, and a retry that does not present that ticket mints a fresh one every poll — its position never advances. Everything in `session` is authoritative: whatever the browser sends for the persona, the memory, the voice, or the time limit is discarded before the request leaves your server. A field your policy does not set is absent rather than inherited, so an omission fails closed rather than open. `maxSeconds` is the one to get right on day one — it is what stops a call your balance cannot cover. See [Authentication](https://realtimeavatar.ai/docs/authentication) for the full list of what belongs on the server and why. ### The client half — render the call Build the client once, then render one component. It handles the queue, reconnects, the idle timer, and the video surface for you. Pass a public example avatar id to`avatarId` — `seed-rin-ashfall` is one — and this call is live before you have created anything; step 3 swaps in your own: An example avatar serves its own still and idle loop, so the surface has something to show before the first frame arrives. A catalog row with no `idleVideoUrl` is a stream-only host published for one of the live channels; a call to one is refused with 409, so call an id whose row carries an idle loop. The two below are `seed-rin-ashfall`’s. **Both props are optional and the filenames are not a scheme** — they differ from character to character, so copy the pair that belongs to the avatar you are using rather than deriving them from its name. Point `poster` and `idleVideoUrl` at them and the page is never empty: she is on screen while the call is still connecting. **React** ```tsx import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); export function Call({ avatarId }: { avatarId: string }) { return ( showEndScreen(reason)} > {(call) => call.status === "waiting" ? In line: {call.queuePosition} : null } ); } ``` Live media needs the DOM. In Next.js mark the file `"use client"` and import it with `{ ssr: false }`; in TanStack Start gate it on a mounted flag or load it lazily. **React Native** ```tsx import { AvatarVideoSurface, RealtimeAvatarLiveKitRoom, SessionLifecycleRoomBridge, createProxyClient, registerGlobals, useSessionLifecycle, } from "realtime-avatar/react-native"; registerGlobals(); // once, at app startup // Native has no page origin — the proxy URL must be ABSOLUTE. const client = createProxyClient({ proxyUrl: "https://your-app.example.com/api/realtime-avatar", }); export function Call({ avatarId }: { avatarId: string }) { const lifecycle = useSessionLifecycle({ client, session: { avatarId } }); if (!lifecycle.grant) return null; // map lifecycle.phase to your UI return ( — hand the idle clip to YOUR player, looped and muted. renderIdleVideo={({ url, style }) => } /> ); } ``` **Native is the lower-level surface today.** `` is web-only — the session brain is the same module, but you compose the native room and video surface yourself, and you pick the video player (`expo-video`, `react-native-video`). Live media does not run in Expo Go; you need a dev client or an EAS build. `status` is five values — `connecting` · `waiting` · `live` · `recovering` · `ended` — and you write the copy for the ones you care about. The SDK ships no strings. The `children` render prop hands you a `call` handle: `say()`, `sayAndEnd()`, `keepAlive()`, `end()`, and `secondsRemaining`. Want your own layout around the video rather than on top of it? `useAvatarCall()` returns `{ call, view }` so you can place the view anywhere. **React Native** runs the same session brain with native media — `useSessionLifecycle`, the grant and queue handling, and the quality governor are the identical modules; only the room bridge and the video surface are native twins. Import from `/react-native`, call `registerGlobals()` at startup, and pass an absolute `proxyUrl` (native has no page origin). The one-component `` wrapper is web-only for now. ## 3. Make the character yours The example avatars get you a working call; your own character is one create call away, and nothing above changes but the id. An avatar is one portrait image plus a voice. Upload the image and the platform does the rest **in the background**: it generates the looping idle video and a small motion library — an idle variant, a listening state, a gesture — from that single frame. From a trusted server: **TypeScript** ```ts import { RealtimeAvatar } from "realtime-avatar"; const rta = new RealtimeAvatar({ apiKey: process.env.REALTIME_AVATAR_API_KEY! }); // Upload the portrait, then create the avatar from it. const asset = await rta.uploadAsset(portraitBlob, { kind: "image", filename: "rin.png" }); const avatar = await rta.createAvatar({ displayName: "Rin", sourceKind: "image", // one frontal portrait: jpeg/png/webp, up to 8MB sourceAssetId: asset.id, voice: { auto_description: "Warm, clear, mid-pitch — natural and conversational." }, }); console.log(avatar.id); // ava_… console.log(avatar.status); // "preprocessing" — generation has started ``` **Python** ```python import os, time, httpx client = httpx.Client( base_url="https://realtimeavatar.ai/api/v1", timeout=60.0, headers={"Authorization": f"Bearer {os.environ['REALTIME_AVATAR_API_KEY']}"}) # Register the portrait, then create the avatar from it. asset = client.post("/assets/remote", json={ "kind": "image", "remoteUrl": "https://cdn.example.com/rin/portrait.png", }).json() avatar = client.post("/avatars", json={ "displayName": "Rin", "sourceAssetId": asset["id"], "voice": {"auto_description": "Warm, clear, mid-pitch — natural and conversational."}, }).json() print(avatar["id"]) # ava_… print(avatar["status"]) # "preprocessing" — generation has started # ("failed" + avatar["error"] if it could not be queued) # Poll until the loop and clips have rendered — a minute or two. while avatar["status"] == "preprocessing": time.sleep(15) avatar = client.get(f"/avatars/{avatar['id']}").json() print(avatar["status"]) # "ready" — mint calls; "failed" — avatar["error"] says why ``` > **Creation is asynchronous.** The create call returns in milliseconds with `status: "preprocessing"`; the idle loop and the clip library render in the background over the next minute or two. Poll `getAvatar(id)` (`GET /avatars/{id}` on the wire) until `status` is `ready` before minting a call. Before the loop attaches a mint is refused with a clear error — never a broken call. **Why a `failed` status happened is not on the SDK object.** `getAvatar()` returns the five fields an integration branches on — `id`, `displayName`, `sourceKind`, `status`, `defaultVoiceId` — and nothing else. The wire's `GET /avatars/{id}` carries the diagnostics on top of those: `error` (the reason a `failed` avatar failed) and `idleVideoStatus` (how far the loop got). Read those with a plain authenticated fetch when you are showing a human what went wrong. And do not branch on `sourceKind` to tell a finished avatar from a fresh one: once the generated loop attaches it reads `video`, and `sourceAssetId` names the loop now serving rather than the portrait you uploaded (that is `anchor.url` on `GET /avatars/{id}/clips`). Gate on `status` — or `idleVideoStatus` — never on `sourceKind == "image"`. `motionPrompt` optionally art-directs the generated loop ("soft cafe light, gentle idle energy") — set it as **Resting motion direction** on the [Avatars page](https://realtimeavatar.ai/platform/avatars) or on the wire's `POST /avatars`; omit it for the house default. Once `ready`, `GET /avatars/{id}/clips` lists the generated motion library — render calls inherit it automatically. The Avatars page's **Clips** action provides the same full-library declaration flow for generated prompt clips when the workspace rollout is enabled. `voice: { auto_description }` lets the platform pick the best-matching Fish Audio voice from your description; pass `voice: { voice: { provider, voice_id } }` to choose one explicitly — an explicit voice is nested under `voice.voice` (in Python, `"voice": {"voice": {"provider": "fish", "voice_id": "…"}}`); a top-level `voice: { provider, voice_id }` is a `422`. Omit `voice` and one is cast the same way from the avatar's name and persona, so every avatar is created with a voice. You can also do all of this on the [Avatars page](https://realtimeavatar.ai/platform/avatars) with no code — see [Creating an avatar](https://realtimeavatar.ai/docs/video). Either way you end up with an `ava_…` id, which is the only thing your client needs to know. **Build from an image.** One portrait in, every piece of video generated by the platform — that is the path these docs describe. An image source queues the motion generation above: the avatar sits in `preprocessing` until the idle loop and the clip library are rendered, and only then does it mint calls. **There is no other lane.** `sourceKind: "video"` — registering a looping clip you host — is closed to new callers and answers `422`. (That is the field you send on create; a ready image-built avatar reads `sourceKind: "video"` too, because the loop the platform generated is now its source.) An avatar is built from one portrait and the platform renders the idle loop and every motion clip from it; that shared rest pose is what makes a state switch a splice instead of a jump, and a supplied video cannot honour it. Tenants already creating from video keep working, and their existing avatars are untouched — the door is shut for new integrations, not behind anyone. > **Your editor may still suggest `createAvatarFromVideo()`.** It is on its way out and its doc comment is older still — it warns that an image-sourced avatar publishes a black track, which stopped being true when image-only creation shipped. Use `createAvatarFromImage()`. ## That is the whole surface Two calls: `createRealtimeAvatarRoute` on the server, `AvatarCall` on the client. You never touch the transport, the reconnect logic, or a wire format — and because you never touch them, we can change them underneath you without breaking your app. If you do want that control — your own media policy, your own transport handling — every lower-level primitive is still exported and still supported. Most teams never need to. ## What to read next - [Calls](https://realtimeavatar.ai/docs/sessions) — the policy your server decides, the five states, and how to end a call without cutting her off mid-sentence. - [Tool calling](https://realtimeavatar.ai/docs/tool-calling) — if your character needs to actually do things. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Authentication > Get a key, keep it on your server, and decide who may start a call. - Canonical: https://realtimeavatar.ai/docs/authentication - Updated: 2026-09-10 ## Keys Create a key in [workspace Settings → API keys](https://realtimeavatar.ai/platform/settings#api-keys) and save the full key securely on your server. The list keeps keys masked; the masked text cannot be used for API calls. Workspace owners and admins can use the **Copy** action to copy newly created keys again while they are active and unexpired, including after leaving Settings and returning later. Legacy keys created before repeat copying was available were stored as hashes only and cannot be recovered. If you lose one, create a replacement and update your integration. Revoked or expired keys cannot be copied; replace them to resume API access. Use the full key in a bearer header: ```text Authorization: Bearer YOUR_API_KEY ``` New keys use one format: `tic__`. Existing `tic_live_` and `tic_test_` keys remain valid without any changes. Use names to distinguish integrations. Each key can carry its own spend limit, which is useful when handing a key to a subsystem you would rather cap. The cap is enforced against the wallet, so it does nothing on a workspace billed as **unlimited**: that account never consults a balance, and the per-key check sits behind the one it skips. The limit is still accepted and echoed back, so if you are on an unlimited plan, treat a per-key cap as a note to yourself rather than a control. **There is no separate test or sandbox mode.** All keys call real services and use the same balance in their workspace. The legacy prefixes do not change permissions or billing. To keep development usage separate, use a separate workspace, or a per-key `spendLimitCreditMicros` low enough to fail closed. ### Scopes You pick these at creation. A key made in the dashboard starts with every scope enabled except `*`, so it works straight away — untick whatever this key has no business doing, particularly `api_keys:write`. Creating one over the API instead and omitting `scopes` gives you the narrower `realtime:write`, `avatars:read`, `credits:read`. | Scope | Grants | | --- | --- | | `realtime:write` | Start and end calls | | `avatars:read` | List and fetch avatars and assets | | `avatars:write` | Create and update avatars, upload assets, sync clips | | `credits:read` | Read the balance | | `usage:read` · `usage:write` | Usage reporting | | `api_keys:write` | Mint further keys | | `*` | Everything — avoid outside trusted back-office jobs | ## The one rule: the key stays on your server A browser holding the key could start unlimited calls on your account, so it never gets one. Your client talks to your app; your app talks to us. That is what the route adapters are — `createRealtimeAvatarRoute` and its siblings — mount one and `authorize` is your gate: **Next.js** ```tsx import { createRealtimeAvatarRoute } from "realtime-avatar/nextjs"; export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, // never NEXT_PUBLIC_ prefixed authorize: async ({ request, operation }) => { const user = await currentUser(request); if (!user) return new Response("Unauthorized", { status: 401 }); // "connect" is the only operation that costs money to START. if (operation === "connect" && !(await hasCredits(user))) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, }); ``` **TanStack Start** ```tsx import { realtimeAvatarServerRoute } from "realtime-avatar/tanstack-start"; const handlers = realtimeAvatarServerRoute({ apiKey: () => getEnv().REALTIME_AVATAR_API_KEY, // a factory works on Workers authorize: async ({ request, operation }) => { const user = await requireUser(request); if (!user) return new Response("Unauthorized", { status: 401 }); if (operation === "connect" && !(await hasCredits(user))) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, }); ``` **Python** ```python # No handler to mount — your own endpoint is the gate. from fastapi import Depends, HTTPException from fastapi.responses import JSONResponse @app.post("/api/calls") def start_call(req: StartCallRequest, user=Depends(current_user)): if not user: raise HTTPException(401) if not has_credits(user): # starting a call is the costly one # HTTPException would nest this under "detail"; keep code top-level. return JSONResponse({"code": "insufficient_credits"}, 402) ... ``` Return a `Response` to refuse, or nothing to allow. There are four operations: `connect`, `end`, `avatars`, and `credits`. Put your wallet check on `connect` — it is the only one that costs money to start — and leave the reads cheap. The route exposes no avatar mutation at all: creating or deleting a character is a server-side job for the `RealtimeAvatar` class, never something a browser reaches through this handler — so there is nothing to forget to gate. Every operation still passes `authorize`, so this callback is the gate, not a second line behind one. A key with a `NEXT_PUBLIC_` or `VITE_` prefix is inlined into the client bundle at build time. That is the one mistake that actually leaks a key, and the prefix is the only warning you get. ## What the client is not allowed to decide The persona, the memory, the voice, and the time limit are all yours. The handler strips them from whatever the browser sent and uses your `session` policy instead — and a field your policy does not set is **absent** rather than inherited, so forgetting one fails closed. The one worth getting right on day one is `maxSeconds`: it is what stops a call your balance cannot cover. See [Calls](https://realtimeavatar.ai/docs/sessions) for the full policy. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Build with Lovable, Bolt, v0 and Replit > Add a live AI avatar call to an app built in Lovable, Bolt.new, v0 or Replit: a copy-paste prompt per builder, where the API key goes, and a starter repo that opens in one click. - Canonical: https://realtimeavatar.ai/docs/ai-app-builders - Updated: 2026-10-05 AI app builders write the code from a prompt, so the fastest path is a prompt that already knows the one rule they tend to get wrong: **the API key stays on a server**. Every builder below can run server code (a Supabase Edge Function, a TanStack Start or Next.js route, or a Node server), and the SDK ships the route that goes there. The browser only renders the call. Each prompt uses the public example avatar `seed-rin-ashfall`, so the first call works on a fresh key before you have created a character. Create a key in [Settings, API keys](https://realtimeavatar.ai/platform/settings#api-keys) first. The free Sandbox plan includes 8 minutes of realtime, and the prompts cap every call at 120 seconds. | Builder | Fastest start | Server code runs in | Where the key goes | | --- | --- | --- | --- | | Lovable | [The Lovable prompt below](#lovable) | TanStack Start route (new projects) or Supabase Edge Function (Vite projects) | Secrets: the Add secret form, or More, Cloud, Secrets | | Bolt.new | [Open the starter in Bolt](https://bolt.new/~/github.com/theinfluencecompany/realtime-avatar-starter-vite) | The starter's Node route, or a Supabase Edge Function | `.env` file, or Bolt Secrets | | v0 | [The Next.js prompt below](#v0) | Next.js route handler | Vars panel (Vercel environment variables) | | Replit | [Import the starter to Replit](https://replit.com/github.com/theinfluencecompany/realtime-avatar-starter-vite) | The starter's Node server | Secrets tool | ## The starter repo [realtime-avatar-starter-vite](https://github.com/theinfluencecompany/realtime-avatar-starter-vite) is the shape Lovable and Bolt generate: Vite, React, TypeScript and Tailwind, with one page that renders a live call through `AvatarCall`. It carries the server half twice, sharing one policy file, so it runs wherever you put it: - `supabase/functions/realtime-avatar/index.ts`, a Supabase Edge Function, for Lovable Cloud and Bolt with Supabase. - `server/app.ts`, a Hono app that runs inside the Vite dev server (one process, one port) and behind `npm start` in production, for Bolt, Replit and any Node host. The page picks the route on its own: the Supabase function when `VITE_SUPABASE_URL` is set, the same-origin `/api/realtime-avatar` otherwise. It is MIT licensed; copy any part of it. ## Lovable Lovable builds from prompts and cannot import a GitHub repo, so start from the prompt. Projects created since May 2026 use TanStack Start, and older ones are React and Vite with a Supabase backend (Lovable Cloud); the prompt covers both and lets Lovable check which one it is in. 1. [Open the prompt in Lovable](https://lovable.dev/#prompt=Add%20a%20live%20AI%20avatar%20video%20call%20to%20this%20app%20with%20the%20Realtime%20Avatar%20SDK.%0A%0APackages%3A%20realtime-avatar%20(pin%20exactly%200.26.0)%2C%20livekit-client%2C%20%40livekit%2Fcomponents-react%2C%20zod.%0A%0ARules%2C%20do%20not%20break%20them%3A%0A1.%20The%20API%20key%20is%20a%20server%20secret%20named%20REALTIME_AVATAR_API_KEY.%20Never%20put%20it%20in%20browser%20code%20and%20never%20in%20a%20VITE_%20or%20NEXT_PUBLIC_%20variable.%20Ask%20me%20for%20it%20as%20a%20secret.%0A2.%20The%20browser%20never%20calls%20Realtime%20Avatar%20directly.%20It%20calls%20ONE%20server%20route%20of%20ours%20that%20holds%20the%20key.%20The%20SDK%20ships%20that%20route%3B%20it%20answers%20POST%20%3Cbase%3E%2Fconnect%20and%20POST%20%3Cbase%3E%2Fend%20and%20relays%20the%20platform's%20answer%20unchanged.%20Do%20not%20wrap%20or%20reshape%20what%20it%20returns.%0A3.%20The%20server%20decides%20the%20call%3A%20instructions%2C%20maxSeconds%20and%20which%20avatar%20ids%20are%20allowed.%20Ignore%20anything%20the%20browser%20sends%20for%20those.%0A4.%20Use%20the%20public%20example%20avatar%20%22seed-rin-ashfall%22%20and%20allow%20only%20that%20id%20for%20now.%0AReference%3A%20https%3A%2F%2Frealtimeavatar.ai%2Fllms.txt%20and%20https%3A%2F%2Frealtimeavatar.ai%2Fdocs%2Fquickstart.md%0A%0AHow%20to%20build%20the%20server%20route%20depends%20on%20this%20project's%20stack.%20Check%20which%20one%20it%20is.%0A%0AIF%20this%20project%20uses%20TanStack%20Start%20(new%20Lovable%20projects%20do)%3A%0AServer%3A%20create%20the%20file%20src%2Froutes%2Fapi%2Frealtime-avatar%2F%24.ts%0A%0A%20%20%20%20import%20%7B%20createFileRoute%20%7D%20from%20%22%40tanstack%2Freact-router%22%3B%0A%20%20%20%20import%20%7B%20realtimeAvatarServerRoute%20%7D%20from%20%22realtime-avatar%2Ftanstack-start%22%3B%0A%0A%20%20%20%20const%20handlers%20%3D%20realtimeAvatarServerRoute(%7B%0A%20%20%20%20%20%20%2F%2F%20A%20factory%2C%20read%20per%20request.%20Read%20the%20secret%20the%20way%20this%20project%20reads%20other%20server%20secrets.%0A%20%20%20%20%20%20apiKey%3A%20()%20%3D%3E%20process.env.REALTIME_AVATAR_API_KEY%20%3F%3F%20%22%22%2C%0A%20%20%20%20%20%20authorize%3A%20(%7B%20operation%20%7D)%20%3D%3E%0A%20%20%20%20%20%20%20%20operation%20%3D%3D%3D%20%22connect%22%20%7C%7C%20operation%20%3D%3D%3D%20%22end%22%20%3F%20undefined%20%3A%20new%20Response(%22Not%20found%22%2C%20%7B%20status%3A%20404%20%7D)%2C%0A%20%20%20%20%20%20session%3A%20(%7B%20avatarId%20%7D)%20%3D%3E%0A%20%20%20%20%20%20%20%20avatarId%20%3D%3D%3D%20%22seed-rin-ashfall%22%0A%20%20%20%20%20%20%20%20%20%20%3F%20%7B%20instructions%3A%20%22You%20are%20Rin%2C%20a%20warm%2C%20curious%20guide.%20Short%20spoken%20sentences%2C%20one%20question%20at%20a%20time.%22%2C%20maxSeconds%3A%20120%20%7D%0A%20%20%20%20%20%20%20%20%20%20%3A%20new%20Response(%22Avatar%20not%20allowed%22%2C%20%7B%20status%3A%20403%20%7D)%2C%0A%20%20%20%20%7D)%3B%0A%0A%20%20%20%20export%20const%20Route%20%3D%20createFileRoute(%22%2Fapi%2Frealtime-avatar%2F%24%22)(%7B%20server%3A%20%7B%20handlers%20%7D%20%7D)%3B%0A%0AThe%20trailing%20%24%20in%20the%20filename%20matters%3A%20it%20is%20the%20splat%20that%20lets%20one%20file%20answer%20%2Fconnect%20and%20%2Fend.%0A%0AClient%3A%20a%20page%20with%20a%20%22Call%20Rin%22%20button.%20Clicking%20it%20mounts%20the%20call%20component%20below%20(render%20it%20only%20in%20the%20browser%2C%20after%20mount%2C%20never%20during%20server%20rendering).%20When%20the%20call%20ends%2C%20show%20the%20button%20again.%0A%0A%20%20%20%20import%20%7B%20AvatarCall%2C%20createProxyClient%20%7D%20from%20%22realtime-avatar%2Freact%22%3B%0A%0A%20%20%20%20const%20client%20%3D%20createProxyClient(%7B%20proxyUrl%3A%20%22%2Fapi%2Frealtime-avatar%22%20%7D)%3B%0A%0A%20%20%20%20%3CAvatarCall%0A%20%20%20%20%20%20client%3D%7Bclient%7D%0A%20%20%20%20%20%20avatarId%3D%22seed-rin-ashfall%22%0A%20%20%20%20%20%20poster%3D%22https%3A%2F%2Frealtimeavatar.ai%2Fapi%2Fassets%2Fpublic%2Fcharacters%2Frin-ashfall%2Fportrait.png%22%0A%20%20%20%20%20%20idleVideoUrl%3D%22https%3A%2F%2Frealtimeavatar.ai%2Fapi%2Fassets%2Fpublic%2Fcharacters%2Frin-ashfall%2Fidle-10s.mp4%22%0A%20%20%20%20%20%20style%3D%7B%7B%20width%3A%20%22100%25%22%2C%20maxWidth%3A%20420%2C%20aspectRatio%3A%20%223%20%2F%204%22%20%7D%7D%0A%20%20%20%20%20%20onEnded%3D%7B()%20%3D%3E%20setInCall(false)%7D%0A%20%20%20%20%3E%0A%20%20%20%20%20%20%7B(call)%20%3D%3E%20(%0A%20%20%20%20%20%20%20%20%3Cdiv%20style%3D%7B%7B%20position%3A%20%22absolute%22%2C%20bottom%3A%2016%2C%20left%3A%200%2C%20right%3A%200%2C%20textAlign%3A%20%22center%22%20%7D%7D%3E%0A%20%20%20%20%20%20%20%20%20%20%7Bcall.status%20%3D%3D%3D%20%22waiting%22%20%3F%20%3Cp%3EIn%20line%3A%20%7Bcall.queuePosition%7D%3C%2Fp%3E%20%3A%20null%7D%0A%20%20%20%20%20%20%20%20%20%20%3Cbutton%20onClick%3D%7Bcall.end%7D%3EEnd%20call%3C%2Fbutton%3E%0A%20%20%20%20%20%20%20%20%3C%2Fdiv%3E%0A%20%20%20%20%20%20)%7D%0A%20%20%20%20%3C%2FAvatarCall%3E%0A%0AThe%20microphone%20only%20works%20on%20https%20or%20http%3A%2F%2Flocalhost.%0A%0AOTHERWISE%20(a%20React%20%2B%20Vite%20project%20with%20Lovable%20Cloud)%3A%0AServer%3A%20a%20Supabase%20Edge%20Function%20named%20%22realtime-avatar%22.%0A%0Asupabase%2Ffunctions%2Frealtime-avatar%2Findex.ts%3A%0A%0A%20%20%20%20import%20%7B%20realtimeAvatarHono%20%7D%20from%20%22npm%3Arealtime-avatar%400.26.0%2Fhono%22%3B%0A%0A%20%20%20%20const%20cors%20%3D%20%7B%0A%20%20%20%20%20%20%22Access-Control-Allow-Origin%22%3A%20%22*%22%2C%0A%20%20%20%20%20%20%22Access-Control-Allow-Headers%22%3A%20%22authorization%2C%20x-client-info%2C%20apikey%2C%20content-type%22%2C%0A%20%20%20%20%20%20%22Access-Control-Allow-Methods%22%3A%20%22GET%2C%20POST%2C%20OPTIONS%22%2C%0A%20%20%20%20%7D%3B%0A%0A%20%20%20%20const%20route%20%3D%20realtimeAvatarHono(%7B%0A%20%20%20%20%20%20apiKey%3A%20()%20%3D%3E%20Deno.env.get(%22REALTIME_AVATAR_API_KEY%22)%20%3F%3F%20%22%22%2C%0A%20%20%20%20%20%20authorize%3A%20(%7B%20operation%20%7D)%20%3D%3E%0A%20%20%20%20%20%20%20%20operation%20%3D%3D%3D%20%22connect%22%20%7C%7C%20operation%20%3D%3D%3D%20%22end%22%20%3F%20undefined%20%3A%20new%20Response(%22Not%20found%22%2C%20%7B%20status%3A%20404%20%7D)%2C%0A%20%20%20%20%20%20session%3A%20(%7B%20avatarId%20%7D)%20%3D%3E%0A%20%20%20%20%20%20%20%20avatarId%20%3D%3D%3D%20%22seed-rin-ashfall%22%0A%20%20%20%20%20%20%20%20%20%20%3F%20%7B%20instructions%3A%20%22You%20are%20Rin%2C%20a%20warm%2C%20curious%20guide.%20Short%20spoken%20sentences%2C%20one%20question%20at%20a%20time.%22%2C%20maxSeconds%3A%20120%20%7D%0A%20%20%20%20%20%20%20%20%20%20%3A%20new%20Response(%22Avatar%20not%20allowed%22%2C%20%7B%20status%3A%20403%20%7D)%2C%0A%20%20%20%20%7D)%3B%0A%0A%20%20%20%20Deno.serve(async%20(req)%20%3D%3E%20%7B%0A%20%20%20%20%20%20if%20(req.method%20%3D%3D%3D%20%22OPTIONS%22)%20return%20new%20Response(null%2C%20%7B%20status%3A%20204%2C%20headers%3A%20cors%20%7D)%3B%0A%20%20%20%20%20%20const%20res%20%3D%20await%20route(%7B%20req%3A%20%7B%20raw%3A%20req%20%7D%20%7D)%3B%0A%20%20%20%20%20%20const%20headers%20%3D%20new%20Headers(res.headers)%3B%0A%20%20%20%20%20%20for%20(const%20%5Bk%2C%20v%5D%20of%20Object.entries(cors))%20headers.set(k%2C%20v)%3B%0A%20%20%20%20%20%20return%20new%20Response(res.body%2C%20%7B%20status%3A%20res.status%2C%20headers%20%7D)%3B%0A%20%20%20%20%7D)%3B%0A%0AIn%20supabase%2Fconfig.toml%20add%3A%0A%0A%20%20%20%20%5Bfunctions.realtime-avatar%5D%0A%20%20%20%20verify_jwt%20%3D%20false%0A%0A(The%20gateway's%20JWT%20check%20rejects%20the%20browser's%20CORS%20preflight%3B%20the%20function%20does%20its%20own%20checks.)%0AStore%20REALTIME_AVATAR_API_KEY%20as%20an%20Edge%20Function%20secret.%0A%0AClient%3A%20a%20page%20with%20a%20%22Call%20Rin%22%20button.%20Clicking%20it%20mounts%20the%20call%20component%20below%20(it%20only%20runs%20in%20the%20browser).%20When%20the%20call%20ends%2C%20show%20the%20button%20again.%0A%0A%20%20%20%20import%20%7B%20AvatarCall%2C%20createProxyClient%20%7D%20from%20%22realtime-avatar%2Freact%22%3B%0A%0A%20%20%20%20const%20base%20%3D%20import.meta.env.VITE_SUPABASE_URL%20%2B%20%22%2Ffunctions%2Fv1%2Frealtime-avatar%22%3B%0A%20%20%20%20const%20key%20%3D%20import.meta.env.VITE_SUPABASE_PUBLISHABLE_KEY%20%3F%3F%20import.meta.env.VITE_SUPABASE_ANON_KEY%3B%0A%20%20%20%20const%20client%20%3D%20createProxyClient(%7B%0A%20%20%20%20%20%20proxyUrl%3A%20base%2C%0A%20%20%20%20%20%20%2F%2F%20Supabase's%20gateway%20wants%20the%20project's%20public%20key%20on%20every%20request.%0A%20%20%20%20%20%20fetch%3A%20(input%2C%20init)%20%3D%3E%20%7B%0A%20%20%20%20%20%20%20%20const%20headers%20%3D%20new%20Headers(init%3F.headers)%3B%0A%20%20%20%20%20%20%20%20headers.set(%22apikey%22%2C%20key)%3B%0A%20%20%20%20%20%20%20%20headers.set(%22authorization%22%2C%20%22Bearer%20%22%20%2B%20key)%3B%0A%20%20%20%20%20%20%20%20return%20fetch(input%2C%20%7B%20...init%2C%20headers%20%7D)%3B%0A%20%20%20%20%20%20%7D%2C%0A%20%20%20%20%7D)%3B%0A%0A%20%20%20%20%3CAvatarCall%0A%20%20%20%20%20%20client%3D%7Bclient%7D%0A%20%20%20%20%20%20avatarId%3D%22seed-rin-ashfall%22%0A%20%20%20%20%20%20poster%3D%22https%3A%2F%2Frealtimeavatar.ai%2Fapi%2Fassets%2Fpublic%2Fcharacters%2Frin-ashfall%2Fportrait.png%22%0A%20%20%20%20%20%20idleVideoUrl%3D%22https%3A%2F%2Frealtimeavatar.ai%2Fapi%2Fassets%2Fpublic%2Fcharacters%2Frin-ashfall%2Fidle-10s.mp4%22%0A%20%20%20%20%20%20style%3D%7B%7B%20width%3A%20%22100%25%22%2C%20maxWidth%3A%20420%2C%20aspectRatio%3A%20%223%20%2F%204%22%20%7D%7D%0A%20%20%20%20%20%20onEnded%3D%7B()%20%3D%3E%20setInCall(false)%7D%0A%20%20%20%20%3E%0A%20%20%20%20%20%20%7B(call)%20%3D%3E%20(%0A%20%20%20%20%20%20%20%20%3Cdiv%20style%3D%7B%7B%20position%3A%20%22absolute%22%2C%20bottom%3A%2016%2C%20left%3A%200%2C%20right%3A%200%2C%20textAlign%3A%20%22center%22%20%7D%7D%3E%0A%20%20%20%20%20%20%20%20%20%20%7Bcall.status%20%3D%3D%3D%20%22waiting%22%20%3F%20%3Cp%3EIn%20line%3A%20%7Bcall.queuePosition%7D%3C%2Fp%3E%20%3A%20null%7D%0A%20%20%20%20%20%20%20%20%20%20%3Cbutton%20onClick%3D%7Bcall.end%7D%3EEnd%20call%3C%2Fbutton%3E%0A%20%20%20%20%20%20%20%20%3C%2Fdiv%3E%0A%20%20%20%20%20%20)%7D%0A%20%20%20%20%3C%2FAvatarCall%3E%0A%0AThe%20microphone%20only%20works%20on%20https%20or%20http%3A%2F%2Flocalhost.) (it fills the prompt box; you press send), or paste it into an existing project. 2. When Lovable asks for `REALTIME_AVATAR_API_KEY`, paste your key into the secure secret field. To add it yourself: More, Cloud, Secrets, Add secret. 3. Open the preview, press Call Rin, and allow the microphone. **Lovable prompt** ```text Add a live AI avatar video call to this app with the Realtime Avatar SDK. Packages: realtime-avatar (pin exactly 0.26.0), livekit-client, @livekit/components-react, zod. Rules, do not break them: 1. The API key is a server secret named REALTIME_AVATAR_API_KEY. Never put it in browser code and never in a VITE_ or NEXT_PUBLIC_ variable. Ask me for it as a secret. 2. The browser never calls Realtime Avatar directly. It calls ONE server route of ours that holds the key. The SDK ships that route; it answers POST /connect and POST /end and relays the platform's answer unchanged. Do not wrap or reshape what it returns. 3. The server decides the call: instructions, maxSeconds and which avatar ids are allowed. Ignore anything the browser sends for those. 4. Use the public example avatar "seed-rin-ashfall" and allow only that id for now. Reference: https://realtimeavatar.ai/llms.txt and https://realtimeavatar.ai/docs/quickstart.md How to build the server route depends on this project's stack. Check which one it is. IF this project uses TanStack Start (new Lovable projects do): Server: create the file src/routes/api/realtime-avatar/$.ts import { createFileRoute } from "@tanstack/react-router"; import { realtimeAvatarServerRoute } from "realtime-avatar/tanstack-start"; const handlers = realtimeAvatarServerRoute({ // A factory, read per request. Read the secret the way this project reads other server secrets. apiKey: () => process.env.REALTIME_AVATAR_API_KEY ?? "", authorize: ({ operation }) => operation === "connect" || operation === "end" ? undefined : new Response("Not found", { status: 404 }), session: ({ avatarId }) => avatarId === "seed-rin-ashfall" ? { instructions: "You are Rin, a warm, curious guide. Short spoken sentences, one question at a time.", maxSeconds: 120 } : new Response("Avatar not allowed", { status: 403 }), }); export const Route = createFileRoute("/api/realtime-avatar/$")({ server: { handlers } }); The trailing $ in the filename matters: it is the splat that lets one file answer /connect and /end. Client: a page with a "Call Rin" button. Clicking it mounts the call component below (render it only in the browser, after mount, never during server rendering). When the call ends, show the button again. import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); setInCall(false)} > {(call) => (
{call.status === "waiting" ?

In line: {call.queuePosition}

: null}
)} The microphone only works on https or http://localhost. OTHERWISE (a React + Vite project with Lovable Cloud): Server: a Supabase Edge Function named "realtime-avatar". supabase/functions/realtime-avatar/index.ts: import { realtimeAvatarHono } from "npm:realtime-avatar@0.26.0/hono"; const cors = { "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Headers": "authorization, x-client-info, apikey, content-type", "Access-Control-Allow-Methods": "GET, POST, OPTIONS", }; const route = realtimeAvatarHono({ apiKey: () => Deno.env.get("REALTIME_AVATAR_API_KEY") ?? "", authorize: ({ operation }) => operation === "connect" || operation === "end" ? undefined : new Response("Not found", { status: 404 }), session: ({ avatarId }) => avatarId === "seed-rin-ashfall" ? { instructions: "You are Rin, a warm, curious guide. Short spoken sentences, one question at a time.", maxSeconds: 120 } : new Response("Avatar not allowed", { status: 403 }), }); Deno.serve(async (req) => { if (req.method === "OPTIONS") return new Response(null, { status: 204, headers: cors }); const res = await route({ req: { raw: req } }); const headers = new Headers(res.headers); for (const [k, v] of Object.entries(cors)) headers.set(k, v); return new Response(res.body, { status: res.status, headers }); }); In supabase/config.toml add: [functions.realtime-avatar] verify_jwt = false (The gateway's JWT check rejects the browser's CORS preflight; the function does its own checks.) Store REALTIME_AVATAR_API_KEY as an Edge Function secret. Client: a page with a "Call Rin" button. Clicking it mounts the call component below (it only runs in the browser). When the call ends, show the button again. import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const base = import.meta.env.VITE_SUPABASE_URL + "/functions/v1/realtime-avatar"; const key = import.meta.env.VITE_SUPABASE_PUBLISHABLE_KEY ?? import.meta.env.VITE_SUPABASE_ANON_KEY; const client = createProxyClient({ proxyUrl: base, // Supabase's gateway wants the project's public key on every request. fetch: (input, init) => { const headers = new Headers(init?.headers); headers.set("apikey", key); headers.set("authorization", "Bearer " + key); return fetch(input, { ...init, headers }); }, }); setInCall(false)} > {(call) => (
{call.status === "waiting" ?

In line: {call.queuePosition}

: null}
)} The microphone only works on https or http://localhost. ``` Lovable does not redeploy an Edge Function from a commit synced from GitHub; ask it in chat to deploy the function after you change it. ## Bolt [Open the starter in Bolt](https://bolt.new/~/github.com/theinfluencecompany/realtime-avatar-starter-vite). Bolt opens the repo in its in-browser environment, where `npm run dev` serves the page and the API route on one port. 1. Create a file named `.env` with `REALTIME_AVATAR_API_KEY=your_key` and restart the dev server. 2. Press Call Rin in the preview and allow the microphone. Bolt runs Node inside your browser tab, where an outbound request from that server can be blocked by the browser. If Call fails with a network error in the preview, connect Supabase from Bolt's integrations and use the Edge Function instead; the starter already contains it. In an existing Bolt project, paste this prompt: **Bolt prompt** ```text Add a live AI avatar video call to this app with the Realtime Avatar SDK. Packages: realtime-avatar (pin exactly 0.26.0), livekit-client, @livekit/components-react, zod. Rules, do not break them: 1. The API key is a server secret named REALTIME_AVATAR_API_KEY. Never put it in browser code and never in a VITE_ or NEXT_PUBLIC_ variable. Ask me for it as a secret. 2. The browser never calls Realtime Avatar directly. It calls ONE server route of ours that holds the key. The SDK ships that route; it answers POST /connect and POST /end and relays the platform's answer unchanged. Do not wrap or reshape what it returns. 3. The server decides the call: instructions, maxSeconds and which avatar ids are allowed. Ignore anything the browser sends for those. 4. Use the public example avatar "seed-rin-ashfall" and allow only that id for now. Reference: https://realtimeavatar.ai/llms.txt and https://realtimeavatar.ai/docs/quickstart.md This project uses Supabase (connect it from Bolt's Supabase integration first). Server: a Supabase Edge Function named "realtime-avatar". supabase/functions/realtime-avatar/index.ts: import { realtimeAvatarHono } from "npm:realtime-avatar@0.26.0/hono"; const cors = { "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Headers": "authorization, x-client-info, apikey, content-type", "Access-Control-Allow-Methods": "GET, POST, OPTIONS", }; const route = realtimeAvatarHono({ apiKey: () => Deno.env.get("REALTIME_AVATAR_API_KEY") ?? "", authorize: ({ operation }) => operation === "connect" || operation === "end" ? undefined : new Response("Not found", { status: 404 }), session: ({ avatarId }) => avatarId === "seed-rin-ashfall" ? { instructions: "You are Rin, a warm, curious guide. Short spoken sentences, one question at a time.", maxSeconds: 120 } : new Response("Avatar not allowed", { status: 403 }), }); Deno.serve(async (req) => { if (req.method === "OPTIONS") return new Response(null, { status: 204, headers: cors }); const res = await route({ req: { raw: req } }); const headers = new Headers(res.headers); for (const [k, v] of Object.entries(cors)) headers.set(k, v); return new Response(res.body, { status: res.status, headers }); }); In supabase/config.toml add: [functions.realtime-avatar] verify_jwt = false (The gateway's JWT check rejects the browser's CORS preflight; the function does its own checks.) Store REALTIME_AVATAR_API_KEY as an Edge Function secret. Client: a page with a "Call Rin" button. Clicking it mounts the call component below (it only runs in the browser). When the call ends, show the button again. import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const base = import.meta.env.VITE_SUPABASE_URL + "/functions/v1/realtime-avatar"; const key = import.meta.env.VITE_SUPABASE_PUBLISHABLE_KEY ?? import.meta.env.VITE_SUPABASE_ANON_KEY; const client = createProxyClient({ proxyUrl: base, // Supabase's gateway wants the project's public key on every request. fetch: (input, init) => { const headers = new Headers(init?.headers); headers.set("apikey", key); headers.set("authorization", "Bearer " + key); return fetch(input, { ...init, headers }); }, }); setInCall(false)} > {(call) => (
{call.status === "waiting" ?

In line: {call.queuePosition}

: null}
)} The microphone only works on https or http://localhost. ``` ## v0 v0 generates Next.js App Router apps, where the SDK's [Next.js route](https://realtimeavatar.ai/docs/nextjs) is one file. 1. Paste the prompt into a new or existing v0 chat. 2. Add `REALTIME_AVATAR_API_KEY` in the Vars panel (or Project settings, Environment Variables). The preview reads the Development value. 3. Press Call Rin in the preview and allow the microphone. **v0 prompt** ```text Add a live AI avatar video call to this app with the Realtime Avatar SDK. Packages: realtime-avatar (pin exactly 0.26.0), livekit-client, @livekit/components-react, zod. Rules, do not break them: 1. The API key is a server secret named REALTIME_AVATAR_API_KEY. Never put it in browser code and never in a VITE_ or NEXT_PUBLIC_ variable. Ask me for it as a secret. 2. The browser never calls Realtime Avatar directly. It calls ONE server route of ours that holds the key. The SDK ships that route; it answers POST /connect and POST /end and relays the platform's answer unchanged. Do not wrap or reshape what it returns. 3. The server decides the call: instructions, maxSeconds and which avatar ids are allowed. Ignore anything the browser sends for those. 4. Use the public example avatar "seed-rin-ashfall" and allow only that id for now. Reference: https://realtimeavatar.ai/llms.txt and https://realtimeavatar.ai/docs/quickstart.md This is a Next.js App Router project. Server: create app/api/realtime-avatar/[...path]/route.ts import { createRealtimeAvatarRoute } from "realtime-avatar/nextjs"; export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: () => process.env.REALTIME_AVATAR_API_KEY ?? "", authorize: ({ operation }) => operation === "connect" || operation === "end" ? undefined : new Response("Not found", { status: 404 }), session: ({ avatarId }) => avatarId === "seed-rin-ashfall" ? { instructions: "You are Rin, a warm, curious guide. Short spoken sentences, one question at a time.", maxSeconds: 120 } : new Response("Avatar not allowed", { status: 403 }), }); The [...path] catch-all matters: one file answers /connect and /end. Read REALTIME_AVATAR_API_KEY from the environment (set in the Vars panel). Never NEXT_PUBLIC_. Client: a page with a "Call Rin" button. Clicking it mounts the call component below (put it in a "use client" component and load that with next/dynamic and ssr: false). When the call ends, show the button again. import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); setInCall(false)} > {(call) => (
{call.status === "waiting" ?

In line: {call.queuePosition}

: null}
)} The microphone only works on https or http://localhost. ``` ## Replit [Import the starter to Replit](https://replit.com/github.com/theinfluencecompany/realtime-avatar-starter-vite). Its `.replit` runs `npm run dev` on one port and deploys with `npm run build` and `npm start`. 1. Open the Secrets tool and add `REALTIME_AVATAR_API_KEY`. 2. Press Run, then Call Rin in the webview, and allow the microphone. 3. To publish, deploy it, and check the deployment has the same secret. To have Replit Agent build it into an existing app instead, give it this prompt: **Replit Agent prompt** ```text Add a live AI avatar video call to this app with the Realtime Avatar SDK. Packages: realtime-avatar (pin exactly 0.26.0), livekit-client, @livekit/components-react, zod. Rules, do not break them: 1. The API key is a server secret named REALTIME_AVATAR_API_KEY. Never put it in browser code and never in a VITE_ or NEXT_PUBLIC_ variable. Ask me for it as a secret. 2. The browser never calls Realtime Avatar directly. It calls ONE server route of ours that holds the key. The SDK ships that route; it answers POST /connect and POST /end and relays the platform's answer unchanged. Do not wrap or reshape what it returns. 3. The server decides the call: instructions, maxSeconds and which avatar ids are allowed. Ignore anything the browser sends for those. 4. Use the public example avatar "seed-rin-ashfall" and allow only that id for now. Reference: https://realtimeavatar.ai/llms.txt and https://realtimeavatar.ai/docs/quickstart.md Build it as a Vite + React + TypeScript app with a small Node server (Hono) on the same port. Server: mount the SDK's route on Hono: import { Hono } from "hono"; import { realtimeAvatarHono } from "realtime-avatar/hono"; const app = new Hono(); app.all( "/api/realtime-avatar/*", realtimeAvatarHono({ apiKey: () => process.env.REALTIME_AVATAR_API_KEY ?? "", authorize: ({ operation }) => operation === "connect" || operation === "end" ? undefined : new Response("Not found", { status: 404 }), session: ({ avatarId }) => avatarId === "seed-rin-ashfall" ? { instructions: "You are Rin, a warm, curious guide. Short spoken sentences, one question at a time.", maxSeconds: 120 } : new Response("Avatar not allowed", { status: 403 }), }), ); Serve the built page from the same server, listen on 0.0.0.0, and read REALTIME_AVATAR_API_KEY from Replit Secrets. Client: a page with a "Call Rin" button. Clicking it mounts the call component below (it only runs in the browser). When the call ends, show the button again. import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); setInCall(false)} > {(call) => (
{call.status === "waiting" ?

In line: {call.queuePosition}

: null}
)} The microphone only works on https or http://localhost. ``` ## What every prompt makes the builder do - **Key on the server.** A browser holding the key can start unlimited calls on your account. The route holds it; the page gets a session grant for one call. - **Grant relayed unchanged.** The browser SDK validates the grant strictly, so a route that wraps it produces a call that never connects. - **Policy on the server.** The route sets the instructions, the 120 second cap and the avatar allowlist; whatever the page sends for them is discarded. ## Before you share the link As generated, anyone who can open the app can start a call, and every call spends your balance. Before going public, ask the builder to require a signed-in user in the route's `authorize` hook (return a 401 when there is none), and set `maxSeconds` to what you will pay for per call. See [Authentication](https://realtimeavatar.ai/docs/authentication) for what `authorize` gates, and [pricing](https://realtimeavatar.ai/pricing) for what a minute costs. ## Your own character Create an avatar from one portrait on the [Avatars page](https://realtimeavatar.ai/platform/avatars), then ask the builder to replace `seed-rin-ashfall` with its `ava_…` id in both the allowlist and the page, and to drop or replace the `poster` and `idleVideoUrl`, which belong to the example avatar. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Build a realtime AI avatar in Next.js > Build a working Next.js avatar app with a server-side session proxy, React video UI, a complete local starter, and troubleshooting. - Canonical: https://realtimeavatar.ai/docs/nextjs - Updated: 2026-09-14 Your API key must never reach a browser, so the browser talks to your app and your app talks to us. The Next.js App Router adapter handles that boundary. ## Run the complete starter [Download the Next.js starter](https://realtimeavatar.ai/downloads/nextjs-avatar-starter.zip) for a complete local app with a ready avatar, start and end controls, optional microphone, text messages, and downloadable startup observations. It pins Next.js 16.3.4, React 19.3.0 and `realtime-avatar` 0.16.0. Use Node.js 22 or newer. **Terminal** ```bash curl -fLO https://realtimeavatar.ai/downloads/nextjs-avatar-starter.zip unzip nextjs-avatar-starter.zip cd nextjs-avatar-starter npm ci cp .env.example .env.local # Set REALTIME_AVATAR_API_KEY in .env.local, then: npm run dev ``` Get a session-capable key from [API keys](https://realtimeavatar.ai/platform/api-keys), then open `http://127.0.0.1:3000`. Click **Start call**, wait for `live`, and send a message. The example uses `seed-rin-ashfall`, so no avatar creation is needed. Each call uses credits and is capped at 60 seconds. Loading the page alone does not start a call. The starter permits local development only. Before hosting it, replace its authorization callback with your own sign-in, balance and rate checks, and back `ownsSession` with session-to-user records. The SDK's default in-process ownership tracking fits a single local process; it does not span serverless instances. The snippets below show where your application policy belongs. ## The server half App Router, mounted at a catch-all so every operation reaches the handler. The `[...path]` segment is not decoration — without it the route only ever matches the mount path itself and answers 404 for `connect`, `end`, `avatars` and `credits` alike. **Next.js** ```tsx // app/api/realtime-avatar/[...path]/route.ts import { createRealtimeAvatarRoute } from "realtime-avatar/nextjs"; export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, // Who may do this. Return a Response to refuse, or nothing to allow. // "connect" is the only operation that costs money to start, so that is // where the wallet check belongs; the reads stay cheap. authorize: async ({ request, operation }) => { const user = await currentUser(request); if (!user) return new Response("Sign in", { status: 401 }); if (operation === "connect" && !user.credits) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, // What the character knows. Decided HERE — the client sends none of it. session: async ({ avatarId }) => ({ instructions: promptFor(avatarId), }), }); ``` Name the variable **without** `NEXT_PUBLIC_`. That prefix is what inlines a value into the client bundle, so a key wearing it ships to every visitor — the one mistake this whole split exists to prevent. ## The client half One component, pointed at the route you just mounted. It is the same component on every framework — see [React](https://realtimeavatar.ai/docs/react) for the hook underneath it and for controlling the call yourself. **Next.js** ```tsx "use client"; import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); export function Call({ avatarId }: { avatarId: string }) { return ; } ``` `"use client"` is required: the call holds a WebRTC room and browser media, none of which exist in a server component. ## Record a call on the server Recording is a server policy. Add `recording` to the session returned by the proxy; the browser cannot enable or disable it. RTA runs LiveKit Egress and returns a typed artifact after the call finishes. Use `audio_video` when you need one timeline for transcript and lip-sync analysis. **Next.js** ```tsx // app/api/realtime-avatar/[...path]/route.ts export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, session: async ({ avatarId }) => ({ instructions: promptFor(avatarId), recording: "audio_video", // "off" | "audio" | "video" | "audio_video" }), }); // server-only follow-up, after the call: const artifact = await rta.getRecording(recordingId); if (artifact.status === "ready") { const playback = await rta.getRecordingAccess(artifact.recordingId); // Keep recordingId; playback.url expires and can be renewed. } ``` Recording reads require the `recordings:read` scope and stay tenant-scoped. Keep only `recordingId` in your database; never persist the temporary playback URL. See [Calls](https://realtimeavatar.ai/docs/sessions) for retention, access and transcript correlation details. ## Troubleshooting the first call | Symptom | Check | | --- | --- | | 404 on connect | Keep the catch-all `[...path]` directory and match the client's proxy URL. | | Missing key or 503 in the starter | Set the server-only variable in `.env.local` and restart the development server. | | 401 or 403 | Check API key scopes and your authorization callback. The starter refuses production calls until you implement application authorization. | | 402 | Check the workspace's available credits, entitlement and active reservations. | | Waiting in queue | Keep the queue state visible. Include queue time when measuring the user's startup experience. | | No microphone audio | Use localhost or HTTPS and grant permission. Try a text message to test the rest of the path. | The starter's JSON distinguishes SDK readiness from the first displayed remote video frame. Read the [benchmark methodology](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) before comparing those values with spoken-turn latency. ## Next - [Authentication](https://realtimeavatar.ai/docs/authentication) — keys, scopes, and what `authorize` is really gating. - [Calls](https://realtimeavatar.ai/docs/sessions) — everything the `session` policy can decide. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # TanStack Start > Mount the realtime avatar proxy in a TanStack Start app: a splat server route on the server, one component in the browser. - Canonical: https://realtimeavatar.ai/docs/tanstack-start - Updated: 2026-09-02 Your API key must never reach a browser, so the browser talks to your app and your app talks to us. In Start that is one server route. ## The server half Mount at `routes/api/realtime-avatar/$.ts`. The trailing `$` is Start's splat segment and it is **load-bearing**: without it the handler only ever sees the mount path itself and answers 404 for every operation — the same trap Next.js's `[...path]` exists to avoid. **TanStack Start** ```tsx // routes/api/realtime-avatar/$.ts import { createFileRoute } from "@tanstack/react-router"; import { realtimeAvatarServerRoute } from "realtime-avatar/tanstack-start"; export const Route = createFileRoute("/api/realtime-avatar/$")({ server: { handlers: realtimeAvatarServerRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, authorize: async ({ request, operation }) => { const user = await currentUser(request); if (!user) return new Response("Sign in", { status: 401 }); if (operation === "connect" && !user.credits) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, session: async ({ avatarId }) => ({ instructions: promptFor(avatarId), }), }), }, }); ``` On Cloudflare Workers there is no `process.env`, so pass `apiKey` as a **factory** — `apiKey: () => getEnv().REALTIME_AVATAR_API_KEY` — and it is read per request instead of at module scope. ## The client half **TanStack Start** ```tsx import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); export function Call({ avatarId }: { avatarId: string }) { return ; } ``` ## Next - [Authentication](https://realtimeavatar.ai/docs/authentication) — keys, scopes, and what `authorize` is really gating. - [Calls](https://realtimeavatar.ai/docs/sessions) — everything the `session` policy can decide. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Express > Mount the realtime avatar proxy as Express middleware — Express 4 or 5 — with the same authorize and session policy every adapter takes. - Canonical: https://realtimeavatar.ai/docs/express - Updated: 2026-09-02 Your API key must never reach a browser, so the browser talks to your app and your app talks to us. In Express that is one middleware. ## The server half Express 4 and 5. Mount it on a path prefix, with a JSON body parser in front — the adapter reads a parsed body rather than a stream. **Express** ```ts import express from "express"; import { realtimeAvatarExpress } from "realtime-avatar/express"; const app = express(); app.use( "/api/realtime-avatar", express.json(), realtimeAvatarExpress({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, authorize: async ({ request, operation }) => { const user = await currentUser(request); if (!user) return new Response("Sign in", { status: 401 }); if (operation === "connect" && !user.credits) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, session: async ({ avatarId }) => ({ instructions: promptFor(avatarId), }), }), ); ``` `express.json()` is not optional here. Express hands the adapter an already-parsed body, which it re-serializes to forward — fine for this route, whose payloads are small, but it is the reason the Fetch-based adapters ([Hono](https://realtimeavatar.ai/docs/hono), [Next.js](https://realtimeavatar.ai/docs/nextjs), [TanStack Start](https://realtimeavatar.ai/docs/tanstack-start)) are the better path when you have the choice. ## The client half Express serves your API; the browser half is the same component as everywhere else, pointed at the prefix you mounted. **React** ```tsx import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); ``` If your React app is served from a different origin than the Express API, give `proxyUrl` the absolute URL and allow credentials in your CORS policy — the proxy is your endpoint, so its auth is your session, not ours. ## Next - [Authentication](https://realtimeavatar.ai/docs/authentication) — keys, scopes, and what `authorize` is really gating. - [Calls](https://realtimeavatar.ai/docs/sessions) — everything the `session` policy can decide. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Hono, Workers, Bun and Deno > Mount the realtime avatar proxy on any Fetch-handler runtime — Hono, Cloudflare Workers, Bun or Deno — including the apiKey factory Workers require. - Canonical: https://realtimeavatar.ai/docs/hono - Updated: 2026-09-02 This adapter is for Hono and for anything else built on Fetch handlers — Cloudflare Workers, Bun, Deno. It takes a `Request` and returns a `Response`, which is why one adapter covers all four. ## The server half Mount on a wildcard so every operation reaches the handler, not just the mount path. **Hono** ```ts import { Hono } from "hono"; import { realtimeAvatarHono } from "realtime-avatar/hono"; const app = new Hono<{ Bindings: Env }>(); app.all( "/api/realtime-avatar/*", realtimeAvatarHono({ // A FACTORY, not a value. On Workers there is no process.env, and a // module-scope read runs before any binding exists. apiKey: () => env.REALTIME_AVATAR_API_KEY, authorize: async ({ request, operation }) => { const user = await currentUser(request); if (!user) return new Response("Sign in", { status: 401 }); if (operation === "connect" && !user.credits) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, session: async ({ avatarId }) => ({ instructions: promptFor(avatarId), }), }), ); export default app; ``` On Node or Bun, where `process.env` exists, a plain string is fine: `apiKey: process.env.REALTIME_AVATAR_API_KEY!`. The factory form is what makes the Workers case work, and it is harmless everywhere else — the value is simply read per request instead of once at module scope. ## The client half **React** ```tsx import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); ``` ## Next - [Authentication](https://realtimeavatar.ai/docs/authentication) — keys, scopes, and what `authorize` is really gating. - [Calls](https://realtimeavatar.ai/docs/sessions) — everything the `session` policy can decide. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Realtime AI avatars with React and React Native > Embed live AI avatar calls in React or React Native with AvatarCall, useAvatarCall, connection states, and a server-side session proxy. - Canonical: https://realtimeavatar.ai/docs/react - Updated: 2026-09-18 Every server adapter mounts the same proxy, so the client half is the same everywhere: one component pointed at your route. ## The component For an existing browser React app, install the SDK and its browser media peers. The [downloadable Next.js starter](https://realtimeavatar.ai/docs/nextjs) already declares all three. **Terminal** ```bash npm install --save-exact realtime-avatar@0.20.0 zod@4.4.3 @livekit/components-react@2.9.24 livekit-client@2.22.3 ``` **React** ```tsx import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); // Audio only — no video track is requested, so nothing renders and // nothing decodes. ``` ## Build product UI from call state Use the SDK's call status, actions and end reasons for your interface. The standard call flow does not require Room, Participant, Track or WebRTC objects. **React** ```tsx console.log("Call status:", status)} onEnded={({ reason }) => console.log("Call ended:", reason)} > {(call) => } ``` Call state, transport quality and actual media presentation answer different questions. A connected room or an available track does not prove that a frame appeared on the user's screen. Keep detailed diagnostics out of the default product flow. LiveKit owns transport recovery. RTA owns session preparation, capacity and fresh grants. Your product decides which notice or retry control to show. RTA's optional quality policy callback describes its policy result; it is distinct from the provider's native connection quality. See [Calls](https://realtimeavatar.ai/docs/sessions)for correlation and historical diagnostics. ## Optional connection details SDK `0.16.0` adds `onConnectionDetailsChange` when your product needs current connection facts. It receives a snapshot and subsequent changes;`null` clears a retired source. Keep its state scoped to one call. Omitting the callback adds no reporting listeners, timer or stats polling. **React** ```tsx import { useState } from "react"; import { AvatarCall, type AvatarCallProps, type AvatarConnectionDetails } from "realtime-avatar/react"; function CallWithDetails(props: Pick) { const [details, setDetails] = useState(null); return <> {details && Local connection: {details.localQuality}} ; } ``` The snapshot contains `connectionState`, `localQuality`, and separate `audio`/`video` publisher quality and stream state. The publisher fields come from the selected tracks, which may have different publishers. An absent source is null; an existing source can report unknown quality. These facts do not establish rendered video or audible audio, and the SDK does not upload them. Your product decides which notices or user actions they warrant. Native and custom integrations pass this same callback to their existing `SessionLifecycleRoomBridge`. Both client entries export the same `AvatarConnectionDetails` type. There is no extra wrapper or separate native observer. Continue using call status and end reasons for the call lifecycle. ## The hook underneath it `AvatarCall` is a default surface over `useAvatarCall`. Reach for the hook when you want the call's state but your own layout — a custom control bar, your own video element, a call that lives inside something else. **React** ```tsx import { createProxyClient, useAvatarCall } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); function MyCall({ avatarId }: { avatarId: string }) { const { call, view } = useAvatarCall({ client, avatarId }); return <> {view}

{call.status}

; } ``` Mounting the call component starts connection. Mount it only after the user clicks a start button, and unmount it after `onEnded`. Keep connection, queue, recovery and end states visible. The [complete Next.js starter](https://realtimeavatar.ai/docs/nextjs) demonstrates that lifecycle with text messages and an optional microphone. ## Let the character see your camera Your server must authorize `camera: true` when starting the call. This grants permission; capture stays off until the user chooses to share. Mount one `useAvatarCamera` control inside the existing room, with`allowed` from the grant and `active` from your live-call and foreground state. Do not also drive the room's `video` prop. **React** ```tsx import { useAvatarCamera } from "realtime-avatar/react"; function CameraControl({ allowed, active }: { allowed: boolean; active: boolean }) { const camera = useAvatarCamera({ allowed, active }); return <>

Share camera images with AI to let the character see what you show.

{camera.error ? "Camera unavailable. Check camera permissions and try again." : ""}

; } ``` Repeated on/off toggles keep the same session and microphone. Calling`setEnabled(false)` also cancels a pending permission request in the SDK; any track returned later is stopped without publishing. Stop sharing on`visibilitychange` when hidden and on `pagehide`. Returning to the page must not automatically enable the camera. For a local preview, pass the returned `participant` and`publication` to LiveKit's `VideoTrack`, with camera source and muted playback. The default capture is front-facing, 640×360 at 5 frames per second. Visual understanding samples at most once every two seconds using Gemini 2.5 Flash-Lite; this is occasional image understanding, not continuous motion recognition. Camera images are excluded from server call recordings, although spoken descriptions can remain in recorded audio or conversation history. ## React Native A separate entry point, because the media stack is not the browser's: it wraps LiveKit's React Native room and needs an audio session the OS has granted. **React Native** ```tsx import { RealtimeAvatarLiveKitRoom, AvatarVideoSurface, registerGlobals, } from "realtime-avatar/react-native"; registerGlobals(); ``` `registerGlobals` from the same entry point must run once at app start — it installs the WebRTC globals React Native does not ship. `RealtimeAvatarLiveKitRoom` owns the OS audio session by default. Only call `useRealtimeAvatarAudioSession(connected)` yourself when the room has`manageAudioSession={false}`; keep one owner. Import `useAvatarCamera` from `realtime-avatar/react-native`for the same camera API. Use a native Pressable and Text for the control and LiveKit's native VideoTrack for the preview. **React Native** ```tsx import { useEffect, useState } from "react"; import { AppState, Pressable, Text, View } from "react-native"; import { useAvatarCamera } from "realtime-avatar/react-native"; function NativeCameraControl({ allowed, active }: { allowed: boolean; active: boolean }) { const [foreground, setForeground] = useState(AppState.currentState === "active"); const camera = useAvatarCamera({ allowed, active: active && foreground }); const { setEnabled } = camera; useEffect(() => { const listener = AppState.addEventListener("change", (state) => { if (state === "active") setForeground(true); if (state === "background") { setForeground(false); void setEnabled(false); } }); return () => listener.remove(); }, [setEnabled]); return Share camera images with AI to let the character see what you show. void camera.toggle()}> {camera.pending ? "Cancel camera request" : camera.enabled ? "Stop sharing camera" : "Share camera"} {camera.error ? "Camera unavailable. Check camera permissions." : ""} ; } ``` Treat iOS's transient `inactive` during a permission prompt differently from `background`. Stop capture in the background and require a new user action to share again. Pass false for `active`when the call ends or is held. Camera sharing reuses the existing LiveKit React Native and WebRTC native modules; it does not require expo-camera. The native binary must already contain those modules and camera permission declarations (iOS camera usage description and Android CAMERA permission). Otherwise ship a new native build. A JavaScript-only update may use OTA only when its runtime fingerprint matches the installed build and store review/release policy permits it. Update your privacy disclosure for camera images sent to an AI provider; an OTA cannot add native permissions. SDK 0.20.0 also requires a compatible Zod 4 peer, which is JavaScript-only. ## Next - Mount the server half: [Next.js](https://realtimeavatar.ai/docs/nextjs), [TanStack Start](https://realtimeavatar.ai/docs/tanstack-start), [Express](https://realtimeavatar.ai/docs/express), or [Hono and Workers](https://realtimeavatar.ai/docs/hono). - [Calls](https://realtimeavatar.ai/docs/sessions) — what the server decides on every connect. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Build avatar apps with the Realtime Avatar MCP server > Connect a coding agent to Realtime Avatar: configure the MCP server, inspect available avatars, and build a working app with the TypeScript SDK. - Canonical: https://realtimeavatar.ai/docs/mcp - Updated: 2026-09-10 **An agent cannot guess an avatar id.** That is the whole reason this exists. Hand a coding agent the docs and it will write plausible code against a character that does not exist, a plan you are not on, and a balance it cannot see — and you will find out at the first 404. The MCP server closes that gap by letting the agent read your actual account before it writes a line. ## Install For a concrete first task, ask your coding agent to inspect available avatars and credits, then adapt the [Next.js starter](https://realtimeavatar.ai/downloads/nextjs-avatar-starter.zip) using a returned avatar ID. Keep the key on the server and start a paid call only after a user action. The [integration guide](https://realtimeavatar.ai/docs/nextjs) covers the complete browser-to-server path. Published on npm as `realtime-avatar-mcp`. There is nothing to build and nothing to host: it runs over stdio, straight from `npx`. Add this to your agent's MCP config — `.mcp.json` for Claude Code, `.cursor/mcp.json` for Cursor, `claude_desktop_config.json` for Claude Desktop. **Read-only (default)** ```json { "mcpServers": { "realtime-avatar": { "command": "npx", "args": ["-y", "realtime-avatar-mcp"], "env": { "REALTIME_AVATAR_API_KEY": "YOUR_API_KEY" } } } } ``` **With writes** ```json { "mcpServers": { "realtime-avatar": { "command": "npx", "args": ["-y", "realtime-avatar-mcp"], "env": { "REALTIME_AVATAR_API_KEY": "YOUR_API_KEY", "REALTIME_AVATAR_ALLOW_WRITES": "1" } } } } ``` Create a key in [workspace Settings → API keys](https://realtimeavatar.ai/platform/settings#api-keys) and replace `YOUR_API_KEY` with the full key. The list stays masked; workspace owners and admins can use **Copy** to copy newly created keys again while they are active and unexpired. Lost legacy keys stored as hashes only need a replacement, and revoked or expired keys cannot be copied. See [Authentication](https://realtimeavatar.ai/docs/authentication) for details. Creating a key is free; calls use workspace credits. There is no separate free sandbox mode. ## What the agent gets **Read tools are on by default. Write tools are not.** An agent that can read your account cannot cost you anything; an agent that can start a call and create a character can, so that half is behind `REALTIME_AVATAR_ALLOW_WRITES=1` and you opt in deliberately. ### Always available - `list_avatars` — every avatar on the account, with the id you pass to a session mint. This is the one an agent should call before writing any code. - `get_avatar` — full detail for one avatar, including its clip set and voice. - `credit_balance` — the balance, and how much is reserved by calls in flight. - `list_sessions` — the itemised bill: when each session ran, how long it was billable for, and what it cost. - `list_clips` — the declared clip library with each clip's render status. ### Behind `REALTIME_AVATAR_ALLOW_WRITES=1` - `create_avatar_from_image` and `create_avatar_from_video` — a character from one photo. Video-source creation is restricted; a tool being listed does not grant access to that lane. - `upload_asset` and `create_remote_asset` — get the source material in. - `start_call` — mint a live session and return the grant. - `set_loop` and `set_clip_library` — redirect the resting loop and re-declare the clip set. **Creating a character spends credits and takes minutes.** Both create tools are asynchronous: the avatar comes back `preprocessing` and is not callable until it reaches `ready`. An agent that mints a session against a preprocessing avatar gets a `409` with `code: "avatar_not_ready"` — poll `get_avatar` instead of retrying the call. ## What it is not The MCP server is a *build-time* tool. It is how your agent learns the account while it writes your integration; it is not how your application talks to the platform at runtime. Your app should keep minting sessions from your own server, because that endpoint is where you decide who may call and what the character knows — see [Authentication](https://realtimeavatar.ai/docs/authentication). Never commit any API key in an agent config, regardless of its prefix. ## Try it in one line With the server configured, this prompt is enough to get a working page, because the agent can now look up a real avatar instead of inventing one: ```text Use the realtime-avatar MCP server. List my avatars, pick the first ready one, then build a one-button page that opens a live video call with her: she speaks first, two-way voice, her portrait while she connects, and a hang-up button. Mint the session from a server route, never from the browser. Follow https://realtimeavatar.ai/llms.txt for the SDK shapes. ``` --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Creating an avatar > One image in, a moving character out: generated motion, idle variations and requestable actions, full-library declarations, loop editing, and generation prices. - Canonical: https://realtimeavatar.ai/docs/video - Updated: 2026-10-16 Between sentences your character is not speaking — and that is most of a conversation. A still portrait reads as a dead connection. You do not have to fix that yourself any more: creation takes one image and generates the motion for you. ## One image in, a moving character out Upload a portrait, create the avatar, and the platform renders — in the background — everything a live call needs: - **The looping idle video**: a ~10s closed arc whose first and last frame are both the portrait, so it loops seamlessly by construction. This becomes the avatar's source video — the state she rests in. - **A multi-clip motion library**: a motion director looks at the portrait and the persona and designs short closed-arc clips — a signature idle variant, a listening state, a gesture — each ending exactly where it began, so the realtime worker can swap between them at invisible seams. The create call returns immediately with `status: "preprocessing"` (the wire also reports the loop's own `idleVideoStatus`); poll `getAvatar(id)` (`GET /avatars/{id}` on the wire) until `ready`, and read the library with `GET /avatars/{id}/clips`. Render calls inherit the generated library automatically; a mint that sends an explicit empty `clip_library: []` opts that call out of clips entirely. A motion prompt at creation art-directs the loop (wire `motionPrompt`, or the Avatars page's **Resting motion direction**); a clip that fails to render fails alone — the avatar still ships with the loop and the clips that made it. Creation is **image-only**: one portrait is the entire input, and the platform generates the initial videos. Afterwards, library clips can also use pose-compatible uploaded video assets. Below are the session-time behavior, the full-library declaration, and the generation tools you can use to art-direct motion. ## Two modes | Mode | How it works | Choose it when | | --- | --- | --- | | `looping` (default) | Speech is animated onto the looping video the platform generated from the portrait. Her face, her room, her wardrobe — pixel-identical every call. | Identity has to be exact and repeatable. Almost every companion, brand, or character product. | | `generative` | The video is synthesized as she speaks. No clips to supply, none to maintain. | You have no footage, or you want motion no loop could cover. | **TypeScript** ```ts session: async ({ avatarId }) => ({ video: { mode: "generative" }, // that is the entire switch }) ``` **Python** ```python body = { "avatar_id": avatar_id, "mode": "avatar", "render_backend": "generative", # that is the entire switch } ``` Generative ignores clips by construction — the type makes passing them impossible rather than silently dropping them. Everything below is `looping`. > A live call needs a **video** source — which every avatar created from an image now grows on its own. Before the generated loop attaches, a mint is refused with a clear error, never a black or broken call; once it attaches, calls can connect while the realtime cache still warms (the first call starts slower). Poll `status` to `ready` for the fully-warm contract; a `failed` status carries the reason in `error`. ## Looping: the resting loop You do not need a session override. The avatar's source video is the generated resting loop, and calls inherit its clip library and behavior. Resting in that loop needs no declaration at all: **TypeScript** ```ts session: async ({ avatarId }) => ({ // no video config: the avatar rests in its generated idle loop }) ``` **Python** ```python body = { "avatar_id": avatar_id, "mode": "avatar", # no video fields: the avatar rests in its generated idle loop } ``` ## Looping: shared clips, event-driven behavior A clip is media; behavior decides when to play it. The same clip can serve the `idle` variations, the `on.userSpeechStarted.clips` list, and any action under `actions` at once. There are no per-clip roles. The platform library and its behavior ride calls automatically. `userSpeechStarted` plays a listening reaction once when the user starts speaking; it is not a state held for their whole utterance. `actions` is the requestable catalogue: the language model reads each action's `description` and asks for one when it fits its reply — declaring an action never plays it by itself. Write that description for the language model, not as a generation prompt: the renderer reads `source.motionPrompt` instead. Speaker lip-sync stays on the current playback cursor during full-duplex speech; speaking does not reset the body to a separate talking loop. GET separates media and job rows in `data` from playback rules in `behavior`. Rows have no row-level `role`, `whenHint`, or `weight`; the one weight lives on the idle group. Inspect both together: **TypeScript** ```ts const library = await rta.listClips(avatarId); const { data, revision, anchor, defaultSourceAssetId, behavior } = library; // data: media/job rows; url is the serving take, status is the current job. // behavior.idle: { clips: ["idle_soft", "nod"], weight: 2 } // behavior.on?.userSpeechStarted: { clips: ["nod"] } // behavior.actions: { // agree: { description: "When agreeing with the user", clips: ["nod"] } // } ``` **Python** ```python response = client.get(f"/avatars/{avatar_id}/clips") response.raise_for_status() library = response.json() revision = library["revision"] # 0 = creation-owned; bumps on every accepted PUT clips = library["data"] # media/job rows, not behavior declarations anchor = library["anchor"] default_source_asset_id = library["defaultSourceAssetId"] behavior = library["behavior"] # behavior["idle"] = {"clips": ["idle_soft", "nod"], "weight": 2} # behavior["on"]["userSpeechStarted"] = {"clips": ["nod"]} # behavior["actions"]["agree"] = { # "description": "When agreeing with the user", "clips": ["nod"]} # idle, on and actions are optional; inspect their presence before indexing. ``` A mint that sends an explicit empty `clip_library: []` opts that call out of clips entirely — she stays in the resting loop. For persistent behavior changes, declare the avatar library rather than building a second client-side state machine. ### One weighted idle pool The stored source is addressed as `primary` and participates in the same weighted idle pool as the authored clips. Use `idle.weights` to weight any member, including the source; zero disables it. Every member avoids immediately repeating when another is available, while a sole available member loops completely. The id `primary` stays reserved for source addressing. **JSON** ```json { "idle": { "clips": ["sway", "stretch"], "weights": { "primary": 1, "sway": 2, "stretch": 1 } } } ``` Each member weight is finite, from 0–100, and defaults to 1. At least one member must have positive weight. Unavailable members are removed before one weighted draw; the remaining weights are renormalized. These are selection probabilities, not screen-time proportions. Use `weights` or the legacy aggregate`weight`, never both. Legacy `idle.weight` defaults to 1 and is divided equally among listed clips, with source weight 1. No media is re-rendered by this migration. If a disabled source is the only registered material, session admission returns a retryable `clip_library_not_ready`. ### Every clip is a sentence, and you can change it The starter library is a starting point, not a fixed set. Declare the library you want and the platform reconciles: a clip whose description is unchanged is **kept** and keeps serving, a new or revised one is **queued** to render, and one you leave out is **retired**. The 202 is acceptance, not readiness. `revision` is the top-level field of the GET envelope beside `data` — 0 while the creation pipeline owns the library; each accepted declare increments it, and the 202 body carries the new value. This is a full desired set: removing a record retires that clip, and omitted behavior is removed too. `expectedRevision` is required. Read the library first, keep the sources you still want, and send the revision you read; a stale writer receives `409 revision_conflict`. For older generated rows whose editable motion direction is null, supply a new direction before declaring them as generated sources. **TypeScript** ```ts const { revision } = await rta.listClips(avatarId); const { plan } = await rta.setClipLibrary(avatarId, { expectedRevision: revision, // required compare-and-set clips: { idle_soft: { source: { motionPrompt: "breathing gently, a slow blink" } }, nod: { source: { motionPrompt: "a small attentive nod, returns to rest", durationSeconds: 5 } }, wave: { source: { motionPrompt: "raises a hand and waves warmly, then lowers it" } }, }, idle: { clips: ["idle_soft", "nod"], weight: 2 }, on: { userSpeechStarted: { clips: ["nod"] }, }, actions: { agree: { description: "When agreeing with the user", clips: ["nod"] }, greet: { description: "When greeting the user", clips: ["wave"] }, }, }); // plan → { kept: [...], queued: [...], retired: [...] } await rta.waitForClips(avatarId); // settles when nothing is still RENDERING ``` **Python** ```python response = client.get(f"/avatars/{avatar_id}/clips") response.raise_for_status() revision = response.json()["revision"] r = client.put(f"/avatars/{avatar_id}/clips", json={ "expectedRevision": revision, # required compare-and-set "clips": { "idle_soft": {"source": {"motionPrompt": "breathing gently, a slow blink"}}, "nod": {"source": {"motionPrompt": "a small attentive nod, returns to rest", "durationSeconds": 5}}, "wave": {"source": {"motionPrompt": "raises a hand and waves warmly, then lowers it"}}, }, "idle": {"clips": ["idle_soft", "nod"], "weight": 2}, "on": { "userSpeechStarted": {"clips": ["nod"]}, }, "actions": { "agree": {"description": "When agreeing with the user", "clips": ["nod"]}, "greet": {"description": "When greeting the user", "clips": ["wave"]}, }, }) # httpx does not raise on 4xx: 403 clip_library_not_enabled, 409 revision_conflict # (body carries the current revision), 422 clip_declaration_rejected r.raise_for_status() plan = r.json()["plan"] ``` Declaring is a per-tenant rollout: until it is enabled for your tenant the PUT answers `403 clip_library_not_enabled` and avatars keep their creation-generated library. The other refusals — `409 revision_conflict`, `422 clip_declaration_rejected` — are in [Refusals worth telling apart](https://realtimeavatar.ai/docs/editing#refusals-worth-telling-apart). Each record contains only `source`: either `{ assetId }` or `{ motionPrompt, durationSeconds? }`. Generated `source.durationSeconds` is an integer from 4–8 seconds, default 5; it does not belong beside `source`. Changing source content queues media work; changing selection membership, the idle weight, or action descriptions is metadata-only and does not re-render unchanged media. An updated clip keeps its previous serving take while the new job runs, and after a failed replacement. Listening and action lists draw uniformly. Idle uses the weighted pool above: each `idle.weights` value is finite in 0–100 and defaults to 1. Legacy`idle.weight` remains supported without `weights`. Each list is nonempty with unique references to records in `clips`; the source is addressed separately as `primary` in the weight map. Selection uses only eligible serving material. Ten idle variations are valid; there is no role-count or total-clip authoring cap. Request bytes remain bounded by transport limits, and worker admission is independent: a valid declaration is not a promise of unlimited resident media or GPU capacity. Clip and action ids use 1–64 letters, digits, underscores, or hyphens; reserved keys `__proto__`, `constructor`, and `prototype` are refused. Motion prompts are nonempty after trimming and at most 1,200 characters; action descriptions are nonempty after trimming and at most 512 characters. **You can also upload a clip** — `source: { assetId }` instead of a prompt — but it is *pose-validated* before it can serve: its first and last frames have to land on the avatar's rest pose, or that clip settles `failed` with the verdict in `poseCheck` and the rest of the library is untouched. That validation is the whole reason an upload is allowed at all; without it a supplied video makes every switch visible. An action has its own id: `greet` is what the model asks for, and `wave` is only the media it plays — so several actions can share one clip, and one action can offer several for variety. ### The resting loop is not a clip, and it is also a sentence The avatar's **source** is the resting loop, and declaring idle variations does not replace it — resting is the other half of every idle draw, always available, and it is the one thing that may play twice running. So there is nothing to preserve and nothing to re-declare: the id `primary` is reserved, and a declaration that uses it is refused. **Do not declare the source asset as a variation.** That adds a *second* copy of it: the avatar rests on it AND draws it as a variation, so it plays roughly twice as often as you intended and can follow itself back-to-back through the variation path — exactly the stutter the no-immediate-repeat rule exists to prevent. `defaultSourceAssetId` is on the `listClips` envelope to tell you which asset the loop is, not to be declared. Direct the source loop at creation with `motionPrompt`, and re-direct it afterwards — that is how the resting loop changes, not through the clip library: **TypeScript** ```ts const { servingUrl } = await rta.setLoop(avatarId, { motionPrompt: "tilts her head, a small amused smile, settles back to centre", }); // servingUrl = the loop she is playing RIGHT NOW, for the whole render await rta.waitForLoop(avatarId); // throws if the render gave up ``` **Python** ```python accepted = client.put(f"/avatars/{avatar_id}/loop", json={ "motionPrompt": "tilts her head, a small amused smile, settles back to centre", }).json() # accepted["servingUrl"] is the PREVIOUS loop (None if she had none yet) — she plays it for the whole render. # Poll GET /avatars/{id} until idleVideoStatus is "ready" (or "failed"). ``` She stays `ready` and keeps serving the previous loop for the entire render, then the swap publishes in one step. **Your clip library is untouched** — clips render against the portrait, not against the loop, so a re-direct re-queues nothing and does not move the library's `revision`. Describe a *closed arc*: it has to end where it began, or the loop snaps every time it wraps. And keep it **small**. Because a separate model animates her mouth during speech, every loop description is held to a nearly-still envelope — blinks, a soft closed-lip smile, a shift in weight, a few degrees of head drift. A laugh, a head turn, or an open mouth comes back inside that envelope anyway: a new render, a new charge, and a loop that looks like the one you already had. It is the only outcome here that carries no error code, which is exactly what makes it worth knowing before you write the sentence. ### What makes a switch invisible Every clip the director renders starts and ends on the **same rest pose** — the portrait itself. That shared frame is what turns a switch into a splice instead of a jump, and it is why each clip is generated independently from the one anchor image rather than chained off the previous clip's last frame, which would compound drift with every hop. Clips stay short (~5s closed arcs): the lever for natural idle is **how many states × how long she dwells**, not clip length. ## Editing the loop > **Preview — not yet enabled on the public fleet.** The field is accepted and forwarded today, and currently resolves to the unedited loop. Build against it if you want to be ready; do not ship a feature to your users that depends on it yet. A state map switches between clips. *Editing* rewrites the clip itself: the generated loop is re-rendered under a sentence you write, and the result is lip-synced by the same path as an unedited one. One loop becomes every season, every set, every outfit — without ever generating a new one. **TypeScript** ```ts session: async ({ avatarId }) => ({ video: { edits: { instruction: "a snowy cabin at night, warm lamplight" }, }, }) ``` **Python** ```python body = { "avatar_id": avatar_id, "mode": "avatar", "support_edits": {"instruction": "a snowy cabin at night, warm lamplight"}, } ``` Editing runs **upstream of the lips**, on the closed-mouth plate, so it costs nothing at the start of a call — she is on screen and talking from the first word, she simply has not changed clothes yet. The first lap through the loop is live and expensive; every lap after replays what that one banked, so a fixed instruction converges on the cost of plain looping. It needs the avatar's **video** source — the generated loop — so it is refused at the mint while the loop is still rendering, rather than falling back. It also cannot be combined with `generative` — a synthesized body has no clip to open — and the type makes that pairing unrepresentable rather than a runtime error. A `voice` call is refused for the same reason: no video track, nothing to rewrite. Editing is **per call, and off unless you ask**. Nothing stored on the character turns it on, so a call that omits `edits` is byte-identical to one made before this option existed — which matters, because the first edited lap holds a scarce GPU editor and can wait minutes on a cold one. If you want every call for a character edited, say so in your `session` policy; that is the same place you already decide the persona and the ceiling. ### Letting the look follow the conversation > **Experimental — "runtime edit".** The shape of `live` may still change, and the re-dress does not reach every capacity tier yet, so a call may run to the end on its opening set. See [Experimental features](https://realtimeavatar.ai/docs/experimental) for what that word promises. The *trust boundary* below is not experimental — it is enforced on every mint. Add `live` and the set stops holding still: the look follows the conversation, within the bounds you set. Those rules are **yours, not your user's** — they are the app's policy about what a conversation is allowed to do to the picture, which is why they are server-owned like `instructions`. A browser that could set them could redress your character into anything. **TypeScript** ```ts edits: { instruction: "her apartment at golden hour", live: { rules: "only change the room and the light, never her face or clothes; " + "only when they ask to go somewhere", cooldownSeconds: 60, }, } ``` **Python** ```python body["support_edits"] = { "instruction": "her apartment at golden hour", "live_edit": { "rules": ("only change the room and the light, never her face or clothes; " "only when they ask to go somewhere"), "cooldown_seconds": 60, }, } ``` Write the rules narrowly. Every change of direction buys a fresh lap on a scarce editor *and* a visible style pop — the editor has no mid-stream prompt update, so a look changes at a seam rather than easing. `cooldownSeconds` is the floor that stops an eager director spending a GPU per user turn; it is a bound, not a preference, and the default is deliberately conservative. ## Generation pricing The generated library itself is currently **included with creation** — the loop's generation is metered per-model exactly like a resting-loop update on the Avatars page, and the extra library clips are not billed separately while pricing settles. The rates below apply when you drive the generators *directly*: **Generation is billed at the model vendor's own list price.** We add no margin — what you pay is what MiniMax and ByteDance charge, so the table below moves when their rate cards move. Prices are per generated clip and charged on success; failed generations are not charged. | Gadget | Model | Best for | Price | | --- | --- | --- | --- | | `minimax-h3` | MiniMax H3 · first frame → last frame · 768P | **The default.** Closed loops from any portrait — both ends pinned to the same frame, so the clip closes on itself by construction. | $0.40 | | `seedance-2.0-mini` | Seedance 2.0 Mini · 720p | Cheap iteration while you are still finding the look | $0.80 | | `seedance-2.0-fast` | Seedance 2.0 Fast · 720p | The everyday choice — clearly better motion than mini | $1.20 | | `seedance-2.0` | Seedance 2.0 · 720p | The hero clip: the one she rests in most of the time | $1.50 | | `seedance-2.5` | Seedance 2.5 · 720p | The newest tier; priced as 2.0 until its own rate card publishes | $1.50 | Prices are for a 10-second clip at 24fps: H3 at 768P and the Seedance tiers at 720p. The output follows the prepared source image; the model name does not select a landscape frame. New landscape uploads preserve their complete composition. The resting-loop editor changes motion while retaining the avatar's source framing; it has no model or aspect-ratio selector. An older avatar whose loop was already cropped needs its source framing repaired before regeneration. The Seedance rates come from [ByteDance's published Seedance rate card](https://www.byteplus.com/en/topic/578667) ($0.15/sec at 720p, $0.12 Fast, $0.08 Mini). Credits bill at $1 ≈ 720 credit-seconds, so `seedance-2.0` is 1,080 credits and the default `minimax-h3` is 288 (a 10-second clip at $0.04/sec). 1080p costs materially more on every model — Seedance 2.0 is $0.37/sec there, roughly 2.5× the 720p rate. For a clip that spends its life as a background loop behind a speaking character, 720p is the right default. ## Uploading the portrait The one asset you bring is the image. Upload the bytes, or hand us a URL and we fetch it server-side — the second is what you want for anything large (the origin must answer with `Content-Length`; the same 8 MB image cap as an upload applies): **TypeScript** ```ts // bytes you already hold — createAvatar takes the asset id from here const asset = await rta.uploadAsset(file, { kind: "image" }); // or a URL we fetch for you (no multipart through your runtime) const asset = await rta.createRemoteAsset({ kind: "image", remoteUrl: "https://your-bucket.example.com/rin/portrait.png", }); asset.url; // https://realtimeavatar.ai/api/assets/… — public, ready to use ``` **Python** ```python # client: the httpx.Client from the quickstart (base_url https://realtimeavatar.ai/api/v1, bearer API key) # bytes you already hold with open("portrait.png", "rb") as fh: asset = client.post("/assets", files={"file": fh}, data={"kind": "image"}).json() # raw bytes without a filename: files={"file": ("portrait.png", data, "image/png")} — # the server keys on the part's Content-Type (jpeg/png/webp) # or a URL we fetch for you asset = client.post("/assets/remote", json={ "kind": "image", "remoteUrl": "https://your-bucket.example.com/rin/portrait.png", }).json() asset["publicUrl"] # https://realtimeavatar.ai/api/assets/… — public, ready to use # (the SDK surfaces this same field as asset.url; on the wire it is publicUrl) ``` The returned URL is **publicly readable** — no auth, no signing. Treat these URLs as **unguessable, not private**: the key is random and nothing indexes it, but anyone holding the link can fetch it. Do not upload anything whose exposure would matter. Uploading costs nothing — you are charged for generation, not storage. Accepted kinds are `image` (portraits), `audio` (voice material), and `video` for an uploaded motion clip declared with `source: { assetId }`. Video upload does not make the asset an Avatar source: new Avatars are still portrait-only, and an uploaded action clip must pass the rest-pose check before it can serve. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Calls > What your server decides, what the client reports, and how to end a call without cutting the character off mid-sentence. - Canonical: https://realtimeavatar.ai/docs/sessions - Updated: 2026-10-12 A call is one live conversation. Your server decides who may start one and what the character knows; the client renders it and reports five states. Those are the only two surfaces you touch. ## Character stories and learning experiences Yes. Define an original or appropriately licensed character, its speaking style and the context your server supplies for each call. Your application owns the story, source material, permissions and consequences of a user's choices. A character-driven lesson or story is an application design, not a guarantee of historical accuracy, learning outcomes or professional advice. Start with the Studio to evaluate a character, then use the SDK to integrate the experience. ## Continuing a character conversation Keep the character's behavior in instructions and supply relevant prior context when your server starts the next call. Your application decides what to save, what the user can inspect or delete, and which parts of a story belong in the next interaction. A new session does not automatically inherit persistent relationship memory. Use the Calls and Tool calling documentation to connect context and optional transcript delivery without treating diagnostic logs as your application's memory store. ## What your server decides The `session` policy on your connect endpoint. Every field here is authoritative — whatever the browser sent for these is discarded, and a field you do not set is absent rather than inherited from the caller. The Field column is the SDK name; the Wire column is the key a hand-rolled request sends — snake_case and strict, see Calling the API directly below. | Field | Wire | What it does | | --- | --- | --- | | `instructions` | `instructions` | Her behavior contract — who she is and how she speaks. Up to 16,000 characters. The highest-leverage field on the whole surface. | | `context` | `initial_context` | Up to 32 prior messages, replayed as memory. This is what makes a call continue a story instead of starting cold. On the wire it is `initial_context`, and each entry is `{ role: "system" \| "user" \| "assistant", content }` with content 1–8000 characters — the SDK types this for you, raw HTTP does not. | | `maxSeconds` | `max_session_seconds` | Hard stop, up to 1800. Enforced on our side, so it holds even if your client stops reporting. Compute it from the balance you just admitted. | | `listen` | `stt_mode: "server" \| "off"` | Speech recognition. Default true — she hears the user the whole call. Set `false` only if you drive every turn yourself — see [Tool calling](https://realtimeavatar.ai/docs/tool-calling). | | `voice` | `voice` | Override her stored voice for this call. Must be a Fish voice; see below. | | `video` | `render_backend: "generative"`, or omit it | How she is rendered: omit it for the generated loop + state map, or `{ mode: "generative" }` to synthesize the video live — see [Creating an avatar](https://realtimeavatar.ai/docs/video). | | `camera` | `camera` | Optional boolean. Your server authorizes camera sharing with true; capture remains off until the user enables it. Works in voice and avatar calls. Use `useAvatarCamera` to toggle within the same session without interrupting the microphone. See [React and Native](https://realtimeavatar.ai/docs/react). User camera pixels are excluded from server video recordings. | | `clientTools` | `capabilities: ["client_tools"]` | `true` opts the call into the client tool plane — functions declared in your page that she can call mid-conversation. See [Tool calling](https://realtimeavatar.ai/docs/tool-calling). | | `transcript` | `transcript_webhook: { url, secret }` — https URL, secret 16–200 chars | `{ url, secret }` — get the two-sided transcript back, signed, after the call ends. On the wire it is `transcript_webhook`, and `secret` is 16–200 characters — the SDK types this for you, raw HTTP does not. See [Tool calling](https://realtimeavatar.ai/docs/tool-calling). | | `metadata` | `client_metadata` | Up to 16 string pairs, echoed verbatim on that transcript so you can attribute it without a lookup. | | `recording` | `recording` | `off` (default), `audio`, `video`, or `audio_video`. Your server decides whether to record after obtaining consent. Audio includes published user and avatar audio. | The complete session creation request must fit within 64 KiB of JSON. This byte budget includes instructions, context, and all other fields; non-ASCII text can use multiple bytes per character. There is no standalone `voice_style` field. Put persona, wording, and reply style in `instructions`. Sessions speak with Fish voices only. A `voice` with any other provider (Cartesia, Breeze Blue) is not an error: the call speaks with the character's own Fish voice, or with a Fish house voice matched to the character, and keeps that override's `language` and `speed`. A Fish `voice` takes `speed`, `emotion`, and `language`. The platform carries the session instructions and voice configuration separately: the model writes the reply under the session instructions, then TTS renders that text under the voice configuration. ## Two modes | Mode | What the user gets | Use it for | | --- | --- | --- | | `avatar` (default) | She is on screen — audio and video | The full call | | `voice` | Audio only | A cheaper on-ramp, or a fallback when bandwidth will not carry video | Same character, same policy — only `mode` changes. A voice call draws on separate capacity, so it never competes with a video call. ## Recordings Your backend can request a recording on `POST /realtime/livekit/session`. Recording defaults to `off`, and browser requests cannot override your server policy. Use `audio` for a mixed recording of published user and avatar audio, `video` for silent video, or `audio_video` for both in one MP4. ```python # Use your authenticated server-side HTTP client; see the API reference. response = client.post("/realtime/livekit/session", json={ "avatar_id": avatar_id, "recording": "audio_video", }) response.raise_for_status() grant = response.json() recording_id = grant["recording"]["recordingId"] # Later, after the call has ended: recording = client.get(f"/recordings/{recording_id}").json() if recording["status"] == "ready": access = client.get(f"/recordings/{recording_id}/access").json() # Return access["url"] only to someone allowed to view this call. ``` The recording ID stays the same through `pending`, `recording`, `processing` and `ready`. A failure is `failed`; retention expiry is `expired`. Ending the call starts finalization, so the file may become ready afterward. Recording reads require a server API key with `recordings:read` and are limited to your workspace. The [canonical recording schema](https://realtimeavatar.ai/recording.contract.ts) defines the artifact and playback-access response. RTA stores the media privately for 30 days. Keep `recordingId` in your database. Playback URLs last at most one hour and support seeking; request a new URL when one expires. A URL's `expiresAt` is separate from the recording's `retainedUntil`. Renewing playback access does not extend retention. Recording captures authorized tracks already published in the call. It does not open a camera, request screen capture, or grant additional publishing rights. Transcript delivery remains a separate option; use `sessionId` and timestamps to join transcript turns to the recording. If recording is unavailable, the mint returns `recording_unsupported` before starting the call. A capture startup failure returns `recording_failed`. Raw API callers must let RTA create a fresh room for each recorded session; reusing `room_name` or setting `create_room: false` is refused. Endpoint details are in the [API reference](https://realtimeavatar.ai/docs/api-reference). ## Connection history Enable `connectionHistory: true` in your server session policy when you want the SDK to retain connection changes with the call. On the wire this is`connection_history: true`. Collection defaults to off and requires an SDK release with connection-history support. Your application keeps using the existing`onConnectionDetailsChange` callback for live behavior; it does not write an uploader. The SDK uses existing LiveKit connection events and batches their changes to RTA with a short-lived capability that can only write to that session. It adds no receiver polling. Read the history through `GET /sessions/{sessionId}/connection-history`using a server key with `usage:read`. RTA retains at most 240 received changes for 30 days. Missing data remains unknown. Client timestamps are approximate; connection quality is not proof of successful device playback or an exact recording timeline. ## What the client reports Five states. Write copy for the ones you care about; the SDK ships none. | `status` | Meaning | | --- | --- | | `connecting` | Getting her ready | | `waiting` | Every slot is busy and you are holding a place in line — `call.queuePosition` has the number. **Not an error**; the SDK retries for you. Render the position, not a failure. | | `live` | She is there | | `recovering` | A blip; reconnecting automatically with backoff | | `ended` | Over — `onEnded` already told you why | ## Network facts and call diagnostics Use RTA's call status, actions and end reasons for ordinary product flows. Internally the SDK reads LiveKit's connection and track facts through its public APIs. A connected room or an available track does not prove that the user saw a frame or heard audio. Keep playback observations and business rules separate from transport diagnostics. The [React integration](https://realtimeavatar.ai/docs/react) shows the SDK's existing call-level callbacks. Keep the RTA `session_id` together with your own call or segment ID. Server-authored `metadata` on `startCall` becomes `client_metadata` on the wire and carries application correlation independently of transcript storage. The granted room name and the platform's session timestamps provide the starting point for a LiveKit investigation. Room names can be reused; a room SID identifies a specific room lifetime. Platform operators can investigate detailed historical network information in LiveKit Cloud. Its [Analytics API](https://docs.livekit.io/deploy/admin/analytics-api/) lists sessions and participant details on Scale plans or higher; use the returned LiveKit session ID for detail requests, not the RTA session ID. Project management credentials remain with the operator; an RTA API key does not grant access to the shared LiveKit project. A developer diagnostic endpoint would need to authorize the RTA session first and expose only permitted fields. The documented API does not establish whether a browser actually presented each frame. ## What you can do mid-call **TypeScript** ```ts call.say(text, { steer }) // say something to her; `steer` shapes THIS reply only call.sayAndEnd(text) // speak one exact line, verbatim, then close call.keepAlive() // the user is still here — postpone the idle disconnect call.end() // end now call.secondsRemaining // to the hard stop, or null before the clock lands call.queuePosition // place in line while waiting, else null ``` **Python** ```python # These are CLIENT-side actions — they act on a live call, which lives in the # browser or native app. A Python backend owns the policy, not the live turn. # # What Python does own — and there is no Python SDK, so this half is plain HTTP: # POST /realtime/livekit/session mint a call — body is snake_case and strict: # avatar_id, mode, instructions, initial_context, # max_session_seconds, stt_mode, capabilities, # transcript_webhook, client_metadata (the Wire # column above, not the SDK field names) # POST /realtime/livekit/session/release free the slot early # GET /credits/balance what the next call must fit inside ``` See [Tool calling](https://realtimeavatar.ai/docs/tool-calling) for the Python side of the conversation — the signed transcript you receive when a call ends. `say(text, { steer })` is the hook for tool results — `steer` applies to one reply and then evaporates, so a lookup result cannot leak into every later turn. See [Tool calling](https://realtimeavatar.ai/docs/tool-calling). ## Ending well A call that stops mid-sentence feels broken no matter how good the render was. When time is nearly up you get a callback and a handle; whatever you pass to `sayAndEnd` is spoken **exactly as written** — never rewritten by the model, never interrupted — and only then does the call close. ```tsx { if (secondsLeft > 15) return; // wait for the last window call.sayAndEnd(await writeGoodbye(character)); }} onQuiet={({ secondsLeft }) => toast(`Still there? (${secondsLeft}s)`)} onLowBalance={({ secondsLeft }) => openTopUp(secondsLeft)} onEnded={({ reason }) => showEndScreen(reason)} /> ``` `onEnded` always carries one of `user_ended` · `session_cap` · `idle` · `disconnected` · `out_of_credits` · `agent_ended` · `failed`. Key your end screen off it — "talk again?" and "you're out of minutes" are different screens with different outcomes. The idle timer is real and enforced: when it fires the client disconnects and capacity is freed. That is deliberate. An abandoned tab holding a live character is the most expensive thing that can happen in this product, and it would be on your bill. **You can also release a call from your server.** `rta.endCall(sessionId, { reason })` frees the slot immediately — best-effort and idempotent: `true` when acknowledged, `false` for anything else, never a throw, so calling it twice (or on a call that already ended) is safe. Reach for it when your backend learns the call is over before the client does: a webhook, an admin action, a superseded reconnect. (Wire: `POST /realtime/livekit/session/release`, `realtime:write` scope.) From the client, `call.end()` is the same thing and is what you normally want. The SDK also sends a release on tab-close via `releaseLiveKitSessionBeacon()`, which survives the page going away where a normal `fetch` would not. Release early and often: a held slot is capacity nobody else can use, and the queue is shared. ## Calling the API directly Not on TypeScript? The same call is one authenticated POST — the Python tab on [Quickstart](https://realtimeavatar.ai/docs/quickstart) has a working endpoint. Two things to know before hand-rolling it: - **Every endpoint is strict, and casing is per endpoint.** An unknown or mis-cased key is a rejected call, not a dropped field — and which casing is right depends on where you are writing. The realtime routes on this page (`/realtime/livekit/session` and its `/release`) are **snake_case**; every REST resource endpoint — avatars, clips, keys, assets — is **camelCase**. The preview `/scenes` routes answer in snake_case, like the mint they feed. Measured against the published spec: 2 of the 10 request bodies are snake_case at the top level and 8 are camelCase — with one exception worth knowing, because the strictness above turns it into a 422: `POST /avatars` is camelCase outside and **snake_case inside** `voice` (`auto_description`, `voice_id`). The SDK translates its camelCase policy for you; a hand-rolled request has no such help — send the Wire column from the policy table above (`initial_context`, `max_session_seconds`, `transcript_webhook`, `client_metadata`…), never the SDK names. - **Pass the connection payload to your client byte-for-byte.** It is validated strictly — add one key and the client rejects the whole thing, so the call never opens and nothing points at the cause. Full endpoint list, scopes, and error codes: [API reference](https://realtimeavatar.ai/docs/api-reference). --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Editing a character > Change an avatar's clips and resting loop after creation: how to declare them, how each change settles, and the refusals worth telling apart. - Canonical: https://realtimeavatar.ai/docs/editing - Updated: 2026-09-08 Her clips and her resting loop are both editable after creation. The resting loop is re-directed in words; each library clip can either be rendered from a motion description or use the `assetId` of a pose-compatible video you uploaded. Her id, her voice and her live calls survive every change on this page. Media changes are accepted before their background work finishes, and **she keeps serving the previous take** until its replacement is ready. Behavior-only edits do not render media. Keep the API key and declaration policy on your server; never forward arbitrary browser-authored motion or action descriptions as trusted character policy. ## Her clips Send the library you want, in full: `clips` is a record keyed by clip id; each value contains only `source`. Playback rules live separately in `idle`, `on` and `actions`. The platform reconciles source content: an unchanged clip is **kept**, new or revised media is **queued**, and a record you leave out is **retired**. The stored source joins the runtime idle pool as `primary`. That id is reserved for source addressing, so do not duplicate it in the media record. **TypeScript** ```ts const { revision } = await rta.listClips(avatarId); const { plan } = await rta.setClipLibrary(avatarId, { expectedRevision: revision, // required compare-and-set clips: { sway: { source: { motionPrompt: "sways gently, settles back to rest" } }, nod: { source: { motionPrompt: "a small attentive nod, returns to rest", durationSeconds: 5 } }, }, idle: { clips: ["sway", "nod"], weights: { primary: 1, sway: 2, nod: 1 } }, on: { userSpeechStarted: { clips: ["nod"] }, }, actions: { agree: { description: "When agreeing with the user", clips: ["nod"] }, }, }); // plan → { kept: [...], queued: [...], retired: [...] } await rta.waitForClips(avatarId); ``` **Python** ```python response = client.get(f"/avatars/{avatar_id}/clips") response.raise_for_status() library = response.json() revision = library["revision"] # top-level; 0 while creation still owns the library r = client.put(f"/avatars/{avatar_id}/clips", json={ "expectedRevision": revision, "clips": { "sway": {"source": {"motionPrompt": "sways gently, settles back to rest"}}, "nod": {"source": {"motionPrompt": "a small attentive nod, returns to rest", "durationSeconds": 5}}, }, "idle": {"clips": ["sway", "nod"], "weights": {"primary": 1, "sway": 2, "nod": 1}}, "on": { "userSpeechStarted": {"clips": ["nod"]}, }, "actions": { "agree": {"description": "When agreeing with the user", "clips": ["nod"]}, }, }) # httpx does not raise on 4xx. Refusals come back as {error, status, code} with no "plan": # 403 clip_library_not_enabled, 409 revision_conflict (body "revision" = current), 422 clip_declaration_rejected r.raise_for_status() plan = r.json()["plan"] # the 202 body also carries the new "revision" ``` Generated `source.durationSeconds` is an optional integer from 4–8 seconds, default 5. Uploaded sources contain only `assetId`; the GET envelope's `defaultSourceAssetId` names the source member of the idle pool. Selection membership, the idle weight, and action descriptions are metadata-only: changing them does not re-render unchanged sources. Motion prompts and action descriptions are screened before the declaration can change anything, including on metadata-only updates. Every selection is a plain list of clip ids — nonempty, no duplicates within one list, every reference declared in `clips` — and candidates in one list draw uniformly for listening and actions. Idle uses one weighted idle pool:`idle.weights` gives each member, including `primary`, a finite weight from 0–100. Each omitted member defaults to 1; zero disables it. At least one member must remain positive. Unavailable members are removed and weights are renormalized. The legacy aggregate `idle.weight` is still supported, but cannot be combined with `weights`. Ten idle variations are valid: there are no idle, listening, or total-clip authoring caps beyond 32 per list. Request bytes remain bounded by transport, and worker admission is independent of declaration validity, not unlimited GPU capacity. `on.userSpeechStarted` chooses a listening clip to play once, not a loop held throughout the user's speech — a triggered reaction repeats freely and never goes silent because it just played. `actions` is the requestable menu for the language model: each `description` explains when that action fits, its `clips` list selects the take, and declaring one never plays it by itself. The same clip can serve idle, listening and any action. Every idle member, including the source, avoids immediately repeating when another member is available. A sole available member loops completely. Speaker lip-sync continues on the current playback cursor during full-duplex speech; it does not select a separate talking state. **Omission is deletion.** This is the whole desired library, not a patch — a clip missing from the record is retired, and omitted `idle`, `on` or `actions` rules are removed. `expectedRevision` is required: pass the `revision` you last read and a stale writer gets a `409` instead of silently erasing another change. Re-read and reconcile after a conflict; do not blindly overwrite the revision. You read it from the top level of `GET /avatars/{id}/clips`, beside `data` — it is `0` while the creation pipeline still owns the library, every accepted declaration bumps it, and the `202` body carries the new value. For an uploaded take, upload or remotely register a video asset first, then use `source: { assetId }`. Uploaded clips are pose-validated against the avatar's rest-pose anchor before they can serve; the first and last frame must both return to that pose. The dashboard's **Clips** editor performs the same asset registration and full-library declaration without exposing a developer API key to the browser. ## Her resting loop The source loop is what `motionPrompt` described at creation. Without an authored idle group it forms a one-member idle pool. With authored idle clips, the source participates as an ordinary member of the same weighted pool. It has no separate rest draw or repeat privilege. Changing the source loop does not rewrite a library entry that references the previous source asset. **TypeScript** ```ts const { servingUrl } = await rta.setLoop(avatarId, { motionPrompt: "tilts her head, a small amused smile, settles back to centre", }); // servingUrl = the loop she is playing RIGHT NOW, for the whole render await rta.waitForLoop(avatarId); // throws if the render gave up ``` **Python** ```python resp = client.put(f"/avatars/{avatar_id}/loop", json={ "motionPrompt": "tilts her head, a small amused smile, settles back to centre", }) resp.raise_for_status() # 409 loop_pending, 422 loop_prompt_rejected / loop_not_generatable accepted = resp.json() # accepted["servingUrl"] = the loop she plays for the whole render (None if she had none yet) # Poll GET /avatars/{id} until idleVideoStatus is "ready" (or "failed"). ``` **Your clip library is untouched.** Clips render against the portrait, not against the loop, so re-describing the loop re-queues nothing and does not move the library's `revision`. Describe a *closed arc* — it has to end where it began, or the loop snaps every time it wraps. **Keep it small.** The loop is her resting state and a separate model animates her mouth, so every description is held to a nearly-still envelope: blinks, a soft closed-lip smile, a shift in weight, a few degrees of head drift. Ask for a laugh, a head turn, or an open mouth and the render comes back inside the envelope anyway — accepted, charged, and looking very much like the loop you already had. That is the one failure here with no error code attached to it, so aim inside the envelope rather than at the edge of it. ## Waiting, and how each one ends Both return on **acceptance**, not completion, so both need a wait — and the intuitive version of each is wrong: - `waitForClips` settles when nothing is still *rendering*. Waiting for every clip to reach `ready` hangs forever: a clip rejected by pose validation settles `failed`, which is terminal. - `waitForLoop` settles on `idleVideoStatus` and throws if the render gave up. A failed re-direct leaves her `ready` — she is still serving the old loop — and `idleVideoStatus` is the only status field guaranteed to move; `error` may or may not carry the reason. Poll `idleVideoStatus`, never `status` or `error`: a loop watching `status` waits for a change that never comes. ## Refusals worth telling apart | Code | Means | Do | | --- | --- | --- | | `403 clip_library_not_enabled` | a per-tenant rollout gate | ask us; nothing about the body helps | | `409 revision_conflict` | someone declared first | re-read, re-decide, re-declare | | `409 loop_pending` | a re-direct is already in flight | wait, then retry | | `422 loop_not_generatable` | no portrait to re-animate | **terminal** — do not retry | | `422 clip_declaration_rejected` | a motion prompt or action description was refused (clips route) | rewrite it | | `422 loop_prompt_rejected` | the loop description was refused (loop route) | rewrite it | | `409 avatar_not_ready` | she is not `ready` (either route) | poll `status` until `ready`, then retry; the loop route also refuses a creation that settled `failed` | A `503` with `retryable: true` (`clip_screen_unavailable`, `loop_workflow_unavailable`, `loop_workflow_queue_failed`) is transient — back off and send the same body again. Each refusal above is JSON `{ error, status, code }` — a malformed body (`422`) or an unknown avatar (`404`) carries no `code` — so switch on `code`, not on the status alone. A rejected *upload* is not an error response at all: the declaration is accepted and that one clip settles `failed` with the verdict in `poseCheck`. The rest of the library is untouched. ## What you cannot change **Her portrait.** It is the rest pose every clip is rendered to start and end on — the shared frame that makes a state switch a splice instead of a jump — so replacing it would invalidate the whole library at once. Create a new avatar instead. Everything else about her is a sentence away. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Tool calling > The platform never executes your tools. Here is the boundary, the client tool plane, and the server loop that turns a call into a working agent with a face. - Canonical: https://realtimeavatar.ai/docs/tool-calling - Updated: 2026-10-12 **Your tools never run on the platform.** There is no hosted executor — a tool executes in your page or on your server, with your credentials, under your authorization rules. That is a deliberate boundary, not a gap we forgot to fill. The reason is trust. A hosted tool runner would need your credentials, your authorization rules, and a say in when a side effect fires. Your backend already has all three and already knows which user is on the call. So execution stays where the trust already is, and you get two lanes to wire it in: **client tools**, where she calls a function you declared in your page mid-conversation, and **server steering**, where your backend runs the agent loop and hands her the result to say. ## Client tools: functions she can call in your page Opt in at mint with `clientTools: true` (wire `capabilities: ["client_tools"]`) — part of the call policy, so it is your server's decision like everything else — then declare the tools in the page after the room connects. The manifest is registered over RPC; nothing about a tool ever appears in the mint request, and the tool name is the record key. **TypeScript** ```ts // Server — the connect route opts the call in. Policy, like everything else. session: async () => ({ instructions, clientTools: true }), // Page — after room.connect(). The name she calls is the record key. import { attachAvatarTools, type AvatarTool } from "realtime-avatar/tools"; const check_order: AvatarTool<{ order_id: string }> = { description: "Look up the status of an order by id", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }) => await fetch("/api/orders/" + order_id).then((r) => r.json()), }; const { accepted, rejected } = await attachAvatarTools(room, { check_order }); ``` **A tool has 2.5 seconds.** That is a conversational floor, not a tunable — past a couple of seconds of dead air the call stops feeling live. The platform abandons the call after the deadline and tells her it failed, and the abort is *cooperative*: every `execute` receives an `AbortSignal`, but a handler that ignores it still runs to completion and can commit a side effect — only its *result* is discarded. Make anything slow idempotent, or check `signal.aborted` before you write. A tool that calls a model does not fit: return an acknowledgement inside the deadline and deliver the real answer on screen when it lands. And read the `{ accepted, rejected }` return — a rejected schema is a tool she simply does not have. ## Server steering: the two primitives | Call | What happens | Use for | | --- | --- | --- | | `sendTurn(text, { instructions })` | Interrupts stale speech and generates a reply to `text`, with `instructions` applied to *this turn only* — passed as generation instructions, not concatenated into the user message. | Injecting a tool result and letting the character deliver it in her own voice | | `sendClosingTurn(text)` | Speaks `text` **verbatim** — never routed through the model — uninterruptibly, then ends the session. | The one line that must be exact: a goodbye, a legal disclosure | Per-turn `instructions` is the tool-result channel. It steers wording, register, and content for one reply and then evaporates — it does not mutate the session prompt, so a tool result cannot leak into every later turn. ## The loop Run your agent where your data is: on your server. The call is the interface, not the brain. **TypeScript** ```ts // 1. You observe the user's turn. 2. Your agent decides a tool is needed // and runs it. 3. You hand the RESULT back and let her say it. const result = await tools.checkOrderStatus({ userId, orderId }); await session.sendTurn(userText, { instructions: [ "Answer using ONLY these facts:", JSON.stringify(result), "One or two spoken sentences. Do not read the JSON aloud.", ].join("\n"), }); ``` **Python** ```python import json from fastapi import Depends # The live turn is client-side, so Python's role is to DECIDE and to receive. # Your client calls your endpoint; you run the tool and return the steer text. @app.post("/api/turn") def turn(req: TurnRequest, user=Depends(current_user)): result = check_order_status(user.id, req.order_id) return { "text": req.text, "steer": ( "Answer using ONLY these facts: " f"{json.dumps(result)}. " "One or two spoken sentences. Do not read the JSON aloud." ), } ``` The client passes that straight into `sendTurn(text, { instructions: steer })`. Keeping the decision on your server is the point — the credentials and the authorization rules already live there. For a deterministic string — a confirmation number, a price, a required disclosure — do not route it through the model at all. Use the verbatim path so what she says is exactly what you wrote. ### Latency A tool call between the user finishing a sentence and the character starting to speak is dead air, and dead air on a video call is louder than on a chat. Two things help: - **Speak first, then resolve.** Send a short acknowledging turn immediately ("let me look"), run the tool, then send the answer turn. - **Pre-fetch at mint.** Anything you can know before the call starts belongs in the policy's `context` (wire `initial_context`), not in a tool call three seconds in. ## Getting the conversation back Your agent needs to know what was said — and what she did. Register a transcript webhook at mint time (wire `transcript_webhook`: an https URL and a 16–200 character secret; tag the session with `client_metadata`) and you get a signed POST after the call ends with the full two-sided transcript (`segments`) and the tool calls the model acted on (`tool_calls`: name, arguments, result or error, and duration). It is sent after capacity is released, so it never delays the next caller. Delivery is at-least-once: the worker waits 5 seconds for your answer, retries once after ~2 seconds on a 5xx or a timeout with the identical body and signature, and takes any status under 500 as delivered. A session with no committed turns and no tool calls sends nothing: **TypeScript** ```ts // Registered on your connect endpoint — server-side only. session: async ({ avatarId }) => ({ instructions, transcript: { url: "https://your.app/api/rta/transcript", secret: TRANSCRIPT_SECRET }, // Echoed verbatim, so the keys are yours — except user_id, which // GET /usage/sessions?endUserId= filters on. metadata: { user_id: user.id, characterId: avatarId, mode: "video" }, }) ``` **Python** ```python import hashlib, hmac, json, time from fastapi import BackgroundTasks, HTTPException, Request # Mint half — in your connect route, next to avatar_id. Wire names, not the # SDK's transcript / metadata. body["transcript_webhook"] = {"url": "https://your.app/api/rta/transcript", # https only "secret": TRANSCRIPT_SECRET} # 16..200 chars body["client_metadata"] = {"user_id": str(user.id), "characterId": character.avatar_id, "mode": req.mode} # up to 16 string pairs # Receiver half. MAX_SKEW_SECONDS = 300 def verify(body: bytes, signature: str, timestamp: str, secret: str) -> bool: if not timestamp.isdecimal() or abs(time.time() - int(timestamp)) > MAX_SKEW_SECONDS: return False # missing, malformed, or stale: replay-bound signed = f"{timestamp}.{body.decode()}".encode() expected = hmac.new(secret.encode(), signed, hashlib.sha256).hexdigest() return hmac.compare_digest(f"v1={expected}", signature) @app.post("/api/rta/transcript") async def transcript(request: Request, tasks: BackgroundTasks): raw = await request.body() # RAW bytes, before any parsing if not verify(raw, request.headers.get("x-rta-signature", ""), request.headers.get("x-rta-timestamp", ""), TRANSCRIPT_SECRET): raise HTTPException(401) payload = json.loads(raw) if already_saved(payload["session_id"]): # at-least-once: dedupe on session_id return {"ok": True} user_id = (payload.get("client_metadata") or {}).get("user_id") # None when the mint set none tasks.add_task(save_turns, user_id, payload["segments"]) tasks.add_task(save_tool_calls, user_id, payload.get("tool_calls", [])) # absent when none ran return {"ok": True} # answer inside 5 s; save after ``` Sign over the **raw request bytes**. Parsing to a dict and re-serializing changes the whitespace, and the signature will never match. `TRANSCRIPT_SECRET` is the same secret you passed as `transcript_webhook.secret` at mint. The URL must be https — the mint accepts http, but the worker silently drops it and nothing is ever delivered. **Registering it without the SDK.** The TypeScript tab above turns the webhook on; the Python tab only verifies what arrives. If your server mints over raw HTTP, the field on the mint body is `transcript_webhook` — not `transcript`, which is the SDK's name for it — and the mint is strict, so the wrong key is a 422 rather than a dropped field and no webhook is ever sent. `secret` must be 16–200 characters: it is the HMAC key the handler below verifies against, so generate it rather than typing one. The POST carries `x-rta-signature: v1=` and `x-rta-timestamp`; the signed payload is `"."` under HMAC-SHA256 with your shared secret. Verify it before trusting anything, and reject stale timestamps. `client_metadata` comes back verbatim (`{}` when the mint sent none), so you can attribute the transcript without a session lookup, and `session_id` is the key to dedupe a retry on. `tool_calls` is present only when the session ran at least one tool. Each entry is `{name, call_id, arguments, ts, ok, result | error, duration_ms}`; an entry with no `ok` field means the call produced nothing the model saw. Store it next to the segments — a reply grounded in a lookup is only auditable if the lookup itself is in the history. Arguments and results are truncated to 2,000 characters each and at most 200 calls are recorded (`tool_calls_truncated: true` marks an overflow); this is a history, not a replay. One known gap: a tool round belonging to a turn the user barged in over is not reported by the runtime, so its calls are absent even though they ran. A signed transcript webhook optionally delivers the conversation to your application after a call. That delivery setting is separate from operational diagnostics: private session logs can contain conversation text and are subject to a 30-day storage lifecycle. Disabling the webhook does not establish zero retention. Request diagnostics, billing records and provider processing have separate retention rules; review the privacy policy before sending sensitive data. ## Bringing your own model An external language-model endpoint requires workspace approval and a host allowlist; it is not a self-service voice-provider switch. The platform still handles speech recognition, turn taking, speech synthesis and video. Sending text to a character is not live audio passthrough: the public SDK does not provide an input for arbitrary audio from your own text-to-speech service. Confirm the supported integration and its data flow before choosing a provider. For avatar-mode calls, `listen: false` (wire `stt_mode: "off"`) disables the platform's speech recognition so your client can submit text through`sendTurn`. The platform still generates the response and speech: this is text steering, not an arbitrary audio-to-avatar renderer or a guarantee of verbatim playback. Voice-only calls keep server speech recognition enabled. Keep your tools, application authorization and saved conversation context on your own server. Confirm external-model admission and fallback behavior before relying on a customer-hosted endpoint in production. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # API reference > Every public endpoint on https://realtimeavatar.ai/api/v1 — what it takes, what it returns, which scope it needs, and how it fails. - Canonical: https://realtimeavatar.ai/docs/api-reference - Updated: 2026-09-14 Base URL `https://realtimeavatar.ai/api/v1`. Every request carries `Authorization: Bearer tic_…`. A machine-readable spec for the realtime surface lives at [/openapi.json](https://realtimeavatar.ai/openapi.json). **curl** ```bash curl https://realtimeavatar.ai/api/v1/credits/balance \ -H "Authorization: Bearer $REALTIME_AVATAR_API_KEY" ``` **TypeScript** ```ts import { RealtimeAvatar } from "realtime-avatar"; const rta = new RealtimeAvatar({ apiKey: process.env.REALTIME_AVATAR_API_KEY! }); await rta.creditBalance(); ``` **Python** ```python import os, httpx client = httpx.Client( base_url="https://realtimeavatar.ai/api/v1", headers={"Authorization": f"Bearer {os.environ['REALTIME_AVATAR_API_KEY']}"}) client.get("/credits/balance").json() ``` ## Realtime | Endpoint | Scope | Notes | | --- | --- | --- | | `POST /realtime/livekit/session` | `realtime:write` | Mint a session. Returns a grant, or **429** with the queue contract when capacity is full. See [Calls](https://realtimeavatar.ai/docs/sessions). | | `POST /realtime/livekit/session/release` | `realtime:write` | Free the call's slot. Free of charge and idempotent, so calling it twice — or on a call that already ended — is safe. `{ session_id \| queue_ticket_id, reason?, capacity_pool? }` — one of the two ids is required. `reason` is a closed set: `page_hide · disconnected · superseded · unmount · manual · idle_timeout`. Anything else is a **422**, and a rejected release frees nothing — the slot stays held until it expires on its own, so send one of these or omit the field. The SDK sends it for you. | | `GET /realtime/livekit/capacity` | `realtime:write` | Snapshot of **one** pool — the orchestrator default, named by `capacity_pool` in the body (`primary`). Mints fail over across pools and voice calls are not capacity-gated, so this does not predict whether a mint will be granted. Do not gate a call on it: mint, and treat the **429** as the queue. | ## Scenes (preview) Enabled per workspace; other workspaces get `403 scene_sessions_not_enabled`. A scene is a clip you supply, prepared once and then played as a call's picture: mint with `scene: { video_cache_id }` and the avatar keeps her voice and persona. You can read back and mint only scenes your workspace prepared; any other id is `404 scene_not_found`. A mint also needs the scene to be `ready`, otherwise it answers `409 scene_not_ready` (retryable while it is still building). Prepared scenes are kept for a limited time: a mint of one that has expired answers `404 scene_not_found`, and preparing it again rebuilds it. A workspace holds at most 2 scene calls at once unless support raises it; another answers `429 scene_concurrency_limit_reached`, retryable once one ends. | Endpoint | Scope | Notes | | --- | --- | --- | | `POST /scenes` | `realtime:write` | `{ source: { url, md5?, bytes? }, continuation?, profile: "scene-v0" }`. The id comes from the clip's content, not its URL, so the same bytes always get the same id, and a clip already prepared answers `ready` at once when you send its `md5` and `bytes`. Answers `202` while building and `200` once `ready` or `failed`. A `failed` state names why in `error`: `source_unreachable` (the clip could not be downloaded; retry a few times with backoff), `source_unusable` (use a different clip), `source_mismatch` (the bytes differ from your `md5` or `bytes`) or `build_failed`. Its `scene_id` is `"unresolved"` when the download failed and you sent no `md5` and `bytes`. A source the preparer refuses is `422 scene_source_rejected`; `503 scene_builder_unavailable` means preparation is not offered on this deployment. | | `GET /scenes/{sceneId}` | `realtime:write` | The build state of a scene your workspace prepared. `ready` carries `video_cache_id` and the frame counts a mint may pass through. | ## Connection history | Endpoint | Scope | Notes | | --- | --- | --- | | `GET /sessions/{sessionId}/connection-history` | `usage:read` | Read received client connection observations for a session in your workspace. Empty data means no observations were received. Enable collection in server session policy; the SDK owns uploads. | ## Recordings | Endpoint | Scope | Notes | | --- | --- | --- | | `GET /recordings` | `recordings:read` | List your workspace's recording metadata. Optional `sessionId`, `limit`, and `cursor` filters. | | `GET /recordings/{recordingId}` | `recordings:read` | Get the recording's current status and retention deadline. Media can finish processing after the call ends. | | `GET /recordings/{recordingId}/access` | `recordings:read` | Get a private playback URL for a ready recording. The URL supports seeking, lasts at most one hour, and can be renewed within retention. | ## Avatars | Endpoint | Scope | Notes | | --- | --- | --- | | `GET /avatars` | `avatars:read` | List the workspace's avatars (most recently updated first, capped at 100) as `{ data: [...] }`. | | `POST /avatars` | `avatars:write` | `{ displayName, sourceAssetId?, motionPrompt?, voice?, defaultVoiceId?, llm?, settings?, metadata? }` → 201 (`preprocessing` with a portrait attached; `draft` without one). Creation is **image-only**: `sourceAssetId` names an uploaded portrait and the platform generates the looping idle video plus a multi-clip motion library in the background — poll until `status` is `ready`. `motionPrompt` optionally art-directs the generated loop. Custom video upload is not supported. | | `GET /avatars/{id}` | `avatars:read` | Includes `status`: `draft · preprocessing · ready · failed · disabled · deleted` and `idleVideoStatus`: `none · queued · generating · ready · failed` (the background motion-generation progress). Mint only against `ready`; on `failed`, `error` says why. | | `PATCH /avatars/{id}` | `avatars:write` | Partial update; at least one field required. Profile fields allow 160 characters for the name, 4,000 for personality, 8,000 for background, 2,500 for art direction, and 1,000 for reply style. The merged stored profile, including custom metadata and the system template, must fit 16,000 characters of session instructions; exceeding that total returns **422** before the update is saved. Limits use JavaScript string length (UTF-16 code units). `portraitUrl` is in the schema but is **dashboard-only** — over this API it is always **400**. To change the portrait, register the image with `POST /assets` or `POST /assets/remote` and PATCH the returned id as `sourceAssetId`, which is its own swap lane: it consumes `anchorTimeMs` as the new source's frame and is exclusive of every other field except `sourceReview`. `sourceReview: { reason }` records that a person watched the new loop and accepts it: the automated source checks still run and are audited, but they no longer refuse the swap. | | `DELETE /avatars/{id}` | `avatars:write` | Soft delete. | | `GET /avatars/{id}/clips` | `avatars:read` | The envelope is `{ data, avatarId, revision, anchorVersion, anchor, clipLibraryEligible, defaultSourceAssetId, behavior }`. Each media/job row in `data` contains `clipId, status, url, source, uploadAssetId, motionPrompt, durationSeconds, anchorVersion, poseCheck, error, createdAt, updatedAt`; there is no row-level `role`, `whenHint`, or `weight`. `behavior` contains optional `idle` and `on` rules. The job's `status` is separate from the serving `url`, which can remain non-null during a replacement or after its failure. Map uploaded rows back to `source: { assetId: uploadAssetId }` and generated rows to `source: { motionPrompt, durationSeconds }` for PUT (omit a null duration); older generated rows with null direction require a new prompt. Behavior moves from the GET envelope to PUT's top level. Sessions always inherit the avatar's serving clips and declared behavior. Session HTTP inputs reject `clip_library`, `clip_behavior`, and `choreography`, including an empty clip library. | | `PUT /avatars/{id}/loop` | `avatars:write` | Re-direct the **source loop** from a new `motionPrompt`. This is the default resting video without an explicit idle pool; it is not privileged in idle selection. To reference a previous source asset separately, declare the `defaultSourceAssetId` returned by `listClips(avatarId)` under a new clip record id using `source: { assetId }` and reference it in `idle`. That record keeps the asset it references even after a source-loop update. Answers `202`; the render takes minutes, during which she stays `ready` and keeps serving her previous loop (returned as `servingUrl`), then the swap publishes at once. Your clip library is untouched. Billed as one generation. Requires a portrait to re-animate — `422 loop_not_generatable` (terminal) when she has none: a draft created without `sourceAssetId`, or a grandfathered avatar built from a supplied video. Do not gate on `sourceKind`: it reads `video` for every avatar once her generated loop attaches. `409 loop_pending` means one is already in flight. The description is screened like a clip's (`422 loop_prompt_rejected`). | | `POST /avatars/{id}/clips/review` | `avatars:write` | Human quality acceptance for one uploaded clip. Send `clipId`, `uploadAssetId`,`expectedRevision`, `expectedAnchorVersion` and a `reason` (5–500 characters). Accepts a pose refusal or pending validation after review; keeps the original machine verdict, records the decision and prepares the serving cache. Returns `204`. Changed generations return `409`. Other processing failures and invalid media remain refused. Ten seconds is a maximum, not a required duration. Requires the clip-library rollout. | | `PUT /avatars/{id}/clips` | `avatars:write` | Full replacement: `{ expectedRevision, clips, idle?, on?, actions? }`. Required `clips` is a record keyed by clip id, each value `{ source }`; source is exactly `{ assetId }` or `{ motionPrompt, durationSeconds? }`. Generated `source.durationSeconds` is an integer from 4–8 seconds, default 5. Optional `idle` is `{ clips, weight? }`; `on.userSpeechStarted` contains `{ clips }`; `actions` is a record keyed by action id whose values are `{ description, clips }`. Every `clips` list is a nonempty array of declared clip ids. The same clip can serve several lists. Source changes queue media work; selection and action metadata changes do not re-render unchanged sources. Omitted records are retired and omitted behavior is removed. Required `expectedRevision` is a nonnegative integer for compare-and-set: stale writes get `409 revision_conflict` with the current revision. Answers `202` with the GET envelope plus `plan: { kept, queued, retired }`. Rolling out per tenant (`403 clip_library_not_enabled` until yours is on). Motion prompts and action descriptions are screened even on metadata-only writes: refusal is `422 clip_declaration_rejected`; if the screen cannot run, the write fails closed with retryable `503 clip_screen_unavailable`. | ### Idle variations, events and actions The source (`primary`) and authored idle clips form one weighted idle pool.`idle.weights` maps member IDs to finite weights from 0–100; each omitted member defaults to 1, zero disables it, and at least one must remain positive. Every member avoids immediately repeating when another is available; a sole available member loops completely. Do not combine this map with legacy`idle.weight`, which is converted into aggregate member weights. Lists are nonempty and cannot repeat a clip id. Unavailable members are filtered before selection and weights are renormalized. Ten idle variations are valid; there are no role-count or total-clip authoring caps beyond 32 per list. The 64 KB request-byte transport bound and worker admission remain independent constraints, not unlimited GPU capacity. If the source is disabled and every positive member is pending, mint returns retryable `clip_library_not_ready`. `userSpeechStarted` plays one listening reaction, once per event, rather than holding a listening loop — a triggered reaction repeats freely. `actions` exposes the requestable menu: the language model reads each `description` to choose an appropriate action, then that action's `clips` list chooses a take; declaring an action never plays it by itself. The same clip can serve idle, listening and any action. Speaker lip-sync runs on the current playback cursor during full-duplex speech, without restarting the body on a separate talking clip. Clip and action ids are 1–64 letters, digits, underscores, or hyphens; reserved keys are refused. Action descriptions are trimmed, nonempty, and at most 512 characters. Motion prompts are trimmed, nonempty, and at most 1,200 characters. Asset ids are trimmed, nonempty, and at most 160 characters. Strict objects reject unknown fields. Keep declaration writes server-owned; do not let caller-supplied prose bypass your application's policy or the platform's safety screening. **JSON** ```json { "expectedRevision": 0, "clips": { "sway": { "source": { "motionPrompt": "Sways gently, settles back to rest" } }, "nod": { "source": { "motionPrompt": "A gentle nod, then returns to rest", "durationSeconds": 5 } } }, "idle": { "clips": ["sway", "nod"] }, "on": { "userSpeechStarted": { "clips": ["nod"] } }, "actions": { "agree": { "description": "When agreeing with the user", "clips": ["nod"] } } } ``` Replace the sample revision with the latest GET revision. This full replacement retires records not shown here; include all existing records and behavior you intend to keep. ## Assets | Endpoint | Scope | Notes | | --- | --- | --- | | `GET /assets` | `avatars:read` | List uploaded assets. | | `POST /assets` | `avatars:write` | Multipart: `file`, optional `kind` (`image \| video \| audio`) → 201. Video assets are accepted as uploaded motion-clip sources; they do not reopen custom-video Avatar creation. A clip declaration pose-validates video and requires it to fit the 18 MB QC ceiling. | | `POST /assets/remote` | `avatars:write` | `{ kind, remoteUrl, originalFilename?, metadata? }` — the platform streams the file into storage. Video assets may be referenced by `PUT /avatars/{id}/clips`; prefer remote registration when the bytes already live in object storage. | ## Billing and keys | Endpoint | Scope | Notes | | --- | --- | --- | | `GET /credits/balance` | `credits:read` | Balance and currently reserved credits, in micros. Live sessions hold a reservation for their duration. | | `GET /usage/sessions` | `usage:read` | Per-session billing detail — when each session ran, how long it was billable for, and what it cost. This is what reconciles an invoice, or re-bills your own users; the dashboard aggregate can do neither. The window defaults to the trailing 30 days and is **clamped** to 90 — a wider range is served narrowed, not refused, so read back the returned `from`/`to`. Page with `nextCursor`. `?endUserId=` narrows to one of your users when the call was tagged with `client_metadata.user_id`. **Read `status` before you sum anything.** Only `released` and `failed` are settled: `released` is the normal end of a call and carries the charge, `failed` is terminal and always `billedCreditMicros: 0`. The other two are not final — `reserved` is a slot that was held and never became a call, and `started` is still in flight. Both report `billedCreditMicros: 0` *provisionally*: a `started` row settles when the session is released, so the same session read again tomorrow can carry a different number. Reconcile on the settled rows and treat the rest as pending, not as free. | | `POST /api-keys` | `api_keys:write` | `{ name, scopes?, spendLimitCreditMicros?, expiresAt? }`. Only `name` is required. All new keys use `tic__`. The deprecated `environment` field is accepted but ignored. `scopes` defaults to `realtime:write`, `credits:read` and `avatars:read`, which starts calls and reads but writes no avatars. The creation response includes the full key. Workspace owners and admins can copy newly created keys again in [Settings → API keys](https://realtimeavatar.ai/platform/settings#api-keys) while they are active and unexpired; the list stays masked with a **Copy** action. Legacy keys stored as hashes only cannot be recovered if lost and need a replacement. Revoked or expired keys cannot be copied. | ## Errors Switch on `code` and HTTP status, not the wording of `error`. Correlated failures include a server-generated `requestId`, also in `X-Request-ID`; include it when contacting support. Session failures retain safe codes such as `internal_error` (500), `capacity_unavailable` and `session_dispatch_failed` (503), `upstream_unreachable` and `invalid_upstream_response` (502), `upstream_timeout` (504), and `request_aborted` (408). Internal diagnostics require separate operational access. A capacity queue 429 keeps its existing body; its request ID is in the header only. | Status | Meaning | What to do | | --- | --- | --- | | 401 | Missing, malformed, revoked, or expired key | Use a full, active, unexpired key. Owners and admins can copy eligible keys again in workspace Settings. Replace lost legacy keys, revoked keys, or expired keys. See [Authentication](https://realtimeavatar.ai/docs/authentication). | | 403 | The key lacks the scope for this operation, or its workspace is not active (`error` reads `Tenant is not active`) | Read `error`: mint a key with the scope (do not widen to `*`), or have the workspace reactivated — there is no self-serve switch for that | | 402 | `insufficient_credits` or `spend_limit_exceeded` | Top up, or raise the per-key spend limit. Surface this as a paywall, not as a connection failure. | | 409 | A conflict with state that already exists. Three kinds reach this API: no workspace on the account yet; an **idempotency** conflict (a key still in flight, or reused with a different body); and an **avatar state** conflict — it is disabled, already has a motion video, has no portrait to animate, or a change is still settling. | Read `error` — the three kinds want different things. A workspace is a one-time setup in the dashboard; an idempotency conflict wants a fresh `Idempotency-Key` (never the same one); an in-flight change wants a plain retry once it settles. | | 404 | No such avatar, voice, or key — *for this tenant*. The message often reads "does not belong to this tenant", which sounds like a permission problem and is usually a wrong or already-deleted id. | Do not route this to your access-denied branch — 403 is the permission answer. Check the id, and remember a soft-deleted avatar is gone from every read. | | 413 | The JSON body is over this route's cap | See the cap sizes under Conventions. This is almost always inlined media — send it as an asset and reference the id. | | 422 | Schema rejection — an unknown, mis-cased, or invalid field | The wire schemas are strict. Check camelCase vs snake_case at the layer you are writing. | | 429 | Plan concurrency ceiling, capacity full (session mint), or rate limited | A mint has **two** 429s, and they want opposite reactions. With `code: "concurrency_limit_reached"` the plan’s concurrent-session limit is reached. **Starting and connecting sessions also count, even when no conversation is active.** The top-level fields `maxConcurrentSessions` and `liveSessions` report the limit and occupied reservations; `activeSessions`, `connectingSessions`, and `pendingSessions` break down the count at refusal time. End a session or wait for pending starts to finish or be released before retrying. Avoid parallel duplicate starts, and release sessions on cancellation or unmount. Upgrade only when more simultaneous sessions are needed. Show the error to the caller and retain `requestId` for support. This refusal is **not a queue position**. Without a `code`, look at the body before deciding. A **queue** answer carries `queue_size` and `recommended_retry_ms`: the SDK returns a queued result (`isQueued`) with `position` and `retryAfterMs` — render the position and retry. Deliberately not auto-retried: a blind retry would burn the backoff to receive the same answer. A body with *neither a code nor those fields* is the **per-key rate limit** (see Conventions) — back off, do not treat it as a queue position. Use SDK 0.12.0 or later to preserve concurrency refusals as errors. Older SDKs, including 0.11.1 and 0.11.5, map every 429 to a queued result and can hide the refusal. A null queue position alone does not distinguish a refusal from a real queue. | | 502 | An upstream generation or render failed | Retry; if it persists it is ours, not yours | | 503 | A dependency this route needs is unavailable — a render queue that would not accept the job, a staging pipeline that is down, or (`clip_screen_unavailable`) the screen that vets clip descriptions. Never your request's fault. | Retry with backoff. Nothing was written, so a retry is safe and needs no `Idempotency-Key` to be correct — though sending one costs nothing. | ## Conventions - All `/api/v1` responses are `Cache-Control: no-store`. - JSON request bodies are size-capped per route — 4 KiB on the session release and clip review, 64 KiB on session creation and the clip-library `PUT`, 128 KiB on avatar writes (create and update), and 8–32 KiB on other JSON routes. Field limits still apply within each byte budget. Over the body cap is a **413**. Send media by URL, not inline. - **`Idempotency-Key` is honoured on the two writes that cost money to repeat** — `PUT /avatars/{id}/clips` and `PUT /avatars/{id}/loop`. Replay returns the original response rather than doing the work twice. It matters most on the loop, where a repeated re-direct is a *second billed render*. A repeated clip declaration without the key has a stale `expectedRevision` after acceptance and conflicts; re-reading and declaring unchanged sources does not re-render them. Reuse the same key with a *different* body and you get a **409** — a retry means the same request, so generate a fresh key for a changed one. No other endpoint reads the header. - **120 requests per 60 seconds, per key.** The throttle is applied when the key is verified, so it covers every `/api/v1` endpoint, and it counts per *key* rather than per workspace — splitting a workload across keys raises the ceiling. Over it is a **429** whose body carries no `code` and no queue fields, which is how you tell it from the other two. Worth sizing against: the creation flow asks you to poll `GET /avatars/{id}` until `status` is `ready`, and a render takes minutes — poll on the order of seconds, not milliseconds. - Ids are prefixed and stable: `ava_` avatars, `tic_` API keys. - The SDK covers the integration path — calls, avatars, assets, usage, credits — with a surface that is deliberately *narrower* than this wire, though less narrow than it once was. Avatar update and delete *are* in it (`updateAvatar`, `deleteAvatar`), as are `getAvatar`, `listAvatars`, `listClips`, `setClipLibrary`, `retimeAnchor`, `swapSource` and `iterateSessions`. Prefer it over hand-rolled fetches wherever it reaches. What it leaves out is **key minting**, the **capacity** snapshot and the **asset listing** — those three are a plain authenticated fetch against the endpoints above. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt # Experimental features > What is experimental right now, what that word promises, and how to turn each one on. Included with Studio and above. - Canonical: https://realtimeavatar.ai/docs/experimental - Updated: 2026-08-26 Some capabilities ship before their shape is settled. Rather than hide them behind a flag you have to ask us for, they are on your account from **Studio** up, opt-in per call, and listed here so you can see exactly what you are taking on. ## What "experimental" promises It is a statement about the request shape, not a warning about quality: - **The shape may change in a minor release**, without the deprecation window a stable field gets. Pin an exact SDK version if that matters to you. - **It may be inert while it rolls out.** A call that asks for an experimental capability always connects and always behaves like a normal call; the extra behaviour may simply not happen yet. Nothing errors and nothing is billed differently. - **It is off unless you ask.** Omit the field and your call is byte-identical to one made before the feature existed, so nothing that ignores this page can be affected by anything on it. What the word never covers is a **trust boundary**. Inputs that are server-owned stay server-owned on every call — experimental or not — and that is enforced at the mint, not by convention. If a capability lets a conversation change something, the bounds are still yours to write and still unreachable from a browser. ## The register Everything currently experimental, in full. When this list changes, this page changes with it — the plan bullet and this table read the same register. ### Runtime edit The set behind your character is re-dressed while the call is running, following the conversation within a brief you write. | Turn it on | `video.edits.live` — see [the reference](https://realtimeavatar.ai/docs/video#letting-the-look-follow-the-conversation) | | --- | --- | | Today | Accepted and carried end to end: the call connects and runs normally, and your brief is applied to the session. The re-dress itself does not reach every capacity tier yet, so a call may run to the end on its opening set. Nothing fails and nothing is charged differently when it does. | | Included with | Studio and above | ## Using one Experimental capabilities are configured exactly like stable ones — in your `session` policy, server-side, next to your key. There is no separate endpoint, header or flag to set: ```ts session: async ({ avatarId }) => ({ instructions: character.prompt, video: { edits: { instruction: "her studio at golden hour", // stable: the opening set live: { // experimental: runtime edit rules: "only change the room and the light, never her face or clothes", cooldownSeconds: 30, }, }, }, }) ``` Because the opt-in is a field rather than an account setting, you can ship a call path that uses one and a call path that does not, in the same app, on the same avatar. ## Telling us it went wrong These are the surfaces we most want to hear about, and the ones where a small report is worth the most. Include the `session_id` from the grant — it is the one handle that finds a specific call. --- Realtime Avatar — realtime AI avatar API & SDK. Docs: https://realtimeavatar.ai/docs · Agent guide: https://realtimeavatar.ai/llms.txt --- # Blog — full articles # Build an Avatar Assistant People Can Use Without Speaking or Hearing Build a text\-first appointment assistant with optional avatar calls, verified confirmations, keyboard controls, and clear timeout recovery\. Published: 2026-10-12T15:03:32.579Z Updated: 2026-10-12T15:03:32.579Z Canonical: https://realtimeavatar.ai/blog/accessible-avatar-assistant-without-voice Markdown: [en](https://realtimeavatar.ai/blog/accessible-avatar-assistant-without-voice.md) Yes—if the complete task works through labelled text controls and verified written results, independently of the avatar. Speaking, hearing a reply, and maintaining a live call must not be prerequisites for checking in. Build that application path first, then offer voice or video on user action. A talking character is an optional interface, not the only place where instructions, errors, or confirmation exist. ## The scenario: checking in beside a noisy service counter Consider an authenticated customer arriving for an electronics-repair appointment. The waiting room is loud; they may not want to speak, may not hear the avatar, or may navigate with a keyboard and assistive technology. The task is narrow: identify the booked appointment, confirm arrival, and learn where to wait. This example needs no medical information or private explanation of why the customer needs service. Show the appointment time, location, and current check-in status as ordinary text. Offer “Choose another appointment” before “Confirm arrival.” Display a receipt only after the service backend verifies check-in. A smiling avatar, an accepted request, and a completed check-in are different events. ## Three modality paths, one complete task 1. Text only: sign in, select the appointment, review it, confirm arrival, and read the receipt. No microphone permission, audio playback, or avatar connection is required. 2. Text with an optional avatar: keep those controls and results visible beside the call. The user types or selects choices; speech input stays off. Stopping the avatar leaves the task intact. 3. Speech with written review: the user explicitly starts a call and asks to check in. The application shows the identified appointment for correction and confirmation, then the same written receipt. No consequential instruction exists only in speech. Equivalent means the same permissions, choices, corrections, and verified outcome—not identical conversation. Never make text users start a call to unlock the confirmation button. ## Render task facts, not an imagined live transcript Use one authenticated application backend for all three paths. It authorizes access to the appointment, validates requests, submits arrival, and reads back the outcome. Render its structured result into the appointment panel and receipt. The avatar may explain those facts, but a generated paraphrase cannot change the displayed appointment or prove success. Realtime Avatar's documented transcript webhook arrives after a call; it is not a guaranteed live reply-text feed. Build written task responses from application-owned verified data, and label them as appointment details or check-in results—not verbatim captions. Keep general help available in text too, rather than making users ask the avatar for essential instructions. RTA manages the live speech and video pipeline. For avatar-mode text steering, the server can set listen to false; also keep client microphone capture off. The React documentation says mounting the call component starts connection, so mount it only after the user chooses to start. Voice-only calls retain server speech recognition. Neither configuration builds the surrounding accessible product for you. If spoken requests use client tools, execution still belongs to your application. Tools have a 2.5-second deadline and cooperative cancellation: a timed-out handler can still finish. Show “Checking arrival status” until backend evidence resolves the result; reconcile before retrying instead of inviting a duplicate submission. - [Calls: server policy and client actions](https://realtimeavatar.ai/docs/sessions) - [Tool calling: execution and transcript boundaries](https://realtimeavatar.ai/docs/tool-calling) - [React: starting calls on user action](https://realtimeavatar.ai/docs/react) ## Apply the web criteria to the actual controls - WCAG 2.2, 2.1.1 Keyboard, Level A: functionality must be keyboard-operable without specific keystroke timing, with a narrow path-dependent-input exception. Appointment selection and confirmation do not require drawing a path. Use labelled native controls and a logical focus order. - 2.4.7 Focus Visible, Level AA: make keyboard focus visible. When an avatar panel closes, restore focus to its start control rather than losing it behind removed content. Keep controls reachable when status changes. - 4.1.3 Status Messages, Level AA: expose status through roles or properties so assistive technology can present it without receiving focus. A concise status region can announce “Arrival confirmed.” Do not steal focus for every update or repeatedly announce a countdown. - 1.2.4 Captions (Live), Level AA: assess whether the media meets WCAG's definition of live synchronized media. That definition excludes completely computer-generated information, so generated avatar speech does not automatically fall under this criterion. An appointment-result panel is not synchronized captions. Provide equivalent written task information regardless, and evaluate other applicable media requirements separately rather than claiming conformance from an optional text path. Associate validation errors with their fields and explain recovery in text. Never use a red outline or spoken apology alone. These are selected web requirements and implementation recommendations, not a full WCAG assessment, accessibility certification, or evidence that a particular SDK screen already passes. - [W3C WCAG 2.2: Keyboard](https://www.w3.org/TR/WCAG22/#keyboard) - [W3C WCAG 2.2: Focus Visible](https://www.w3.org/TR/WCAG22/#focus-visible) - [W3C WCAG 2.2: Status Messages](https://www.w3.org/TR/WCAG22/#status-messages) - [W3C WCAG 2.2: Captions (Live)](https://www.w3.org/TR/WCAG22/#captions-live) ## Separate reading time from the avatar's clock WCAG 2.2 criterion 2.2.1, Timing Adjustable, Level A, covers time limits set by content. Its options include turning the limit off, adjustment over at least ten times the default, or a warning allowing at least 20 seconds for a simple extension action and at least ten extensions. Exceptions cover required real-time events with no alternative, essential limits whose extension invalidates the activity, and limits longer than 20 hours. “It is a live avatar” alone establishes none of those exceptions. Warn visibly and accessibly before an applicable timeout. RTA's keepAlive postpones idle disconnection; it does not extend the server's hard session cap. Keep the appointment page usable when the call ends. Evaluate authentication expiry and booking deadlines separately, offering supported extensions where applicable and preserving recoverable progress. A call countdown is not proof that every task timer meets the criterion. - [W3C WCAG 2.2: Timing Adjustable and its exceptions](https://www.w3.org/TR/WCAG22/#timing-adjustable) ## Acceptance tests before calling the path usable - Deny microphone access, silence output, and block the avatar connection. Complete authentication, selection, correction, confirmation, and receipt review using text alone. - Run all three modality paths against the same appointment fixture. Verify the same authorized outcome and inspectable receipt, not just a successful call connection. - Use keyboard-only navigation and a screen reader. Check control names, visible focus, error associations, and status announcements without forced focus changes. - Disconnect after confirmation but before its response arrives. Keep the result pending, reconcile it, and prevent a second arrival submission. - Trigger idle, hard-cap, and authentication expiry separately. Test warnings, supported extensions, focus recovery, and resumption without losing the appointment choice. These are proposed acceptance tests, not reported test results. Include people with relevant access needs; automated checks alone cannot establish that the journey is understandable. ## Frequently asked questions ### Can a user finish without starting an avatar call? That is the design requirement here. Authentication, help, corrections, submission, and results belong to the application and remain available with no RTA session. ### Does the receipt panel replace live captions? No. It presents verified task facts, not everything spoken. Evaluate applicable live-caption requirements separately; the post-call transcript webhook does not supply live captions. ### Does this prove React Native accessibility? No. Native SDK availability is not accessibility coverage. Test native controls, accessibility semantics, permissions, and focus with VoiceOver and TalkBack separately from the web assessment. ### Does text-only mean private or medically compliant? No. Input modality establishes neither privacy nor healthcare compliance. Minimize appointment data and review application and provider processing; this example makes no sensitive-data compliance claim. ## Start with the appointment page, then add the call Prove the complete text path first. Then use the quickstart to add an explicitly started avatar session without moving essential controls or verified results into speech. - [Build the optional call with the Realtime Avatar quickstart](https://realtimeavatar.ai/docs/quickstart) --- # Build an AI Avatar Fantasy Adventure with Choices Beyond a Menu Build a fantasy adventure where players invent solutions, cross a floating market with an avatar guide, and reach an ending shaped by their ideas\. Published: 2026-10-12T18:05:33.264Z Updated: 2026-10-12T18:05:33.264Z Canonical: https://realtimeavatar.ai/blog/ai-avatar-fantasy-adventure-story-game Markdown: [en](https://realtimeavatar.ai/blog/ai-avatar-fantasy-adventure-story-game.md) Build an AI avatar fantasy adventure around situations players can solve in their own words, not a menu disguised as conversation. Author the world's rules, let players combine its objects, and make their solutions change later scenes. Realtime Avatar supplies one guide's live face and voice; your app owns the adventure. Start with a floating market, an approaching sunset, and a boat whose best sail might be a restaurant sign. ## Tonight's adventure: The Last Lantern Market This original proposed one-shot is not a shipped Realtime Avatar game. You play an adult traveler helping Tarin, a 46-year-old ferryman and collector of improbable stories, deliver a lantern to the far quay before twilight. Three illustrated scenes surround his live portrait. Aim for a twelve-minute playtest, not a promised completion time. Your skiff has a fitted retrieval hook and carries a blue waterproof sailcloth, a coil of rope, a loose wooden menu board and two brass tokens. Three daylight marks remain. Thinking, chatting, inspecting and correcting an interpretation cost nothing; only accepting an action with a displayed cost spends daylight or tokens. There are no dice. Stop saves your checkpoint; Pass takes the scene's guaranteed route at its stated cost. Tarin never makes you beg to continue. ## Scene one: sail on the soup of the day The market's entrance has drifted away from the quay. Upward wind curls between the stalls; your skiff has a mast but no fitted sail. Tarin says, ‘I normally bring a boat that works. This is my experimental evening.’ A weather pennant warns of spray beyond the next bend. Inspectable prop cards establish the rule: any broad surface securely tied to the mast catches this wind. The cloth and menu board are both broad; the board is rigid, while the cloth can wrap an object watertight. The rope is sufficient for one mast attachment. Ask freely, type an idea, or use Rig the cloth as a starter. Pass follows the towpath for one daylight mark; at the landing, Tarin fits the board-sail, leaving the cloth free. Player: ‘Turn that menu sideways and lash it to the mast. We're sailing on soup.’ Tarin: ‘Finally, a special worth ordering.’ Before anything moves, show the interpretation: board attached, rope occupied, cloth still available, no daylight cost. Confirm or correct it. The illustration switches to the board-sail skiff, and the crossing succeeds because the authored wind rule is satisfied—not because the model liked the joke. Changing verbs or inventing a boat name needs no special phrase. Combining existing properties is the point. ‘Tie the board with my rope’ and ‘Lash today's specials to the mast’ reach the same physical result. ‘I summon a flying castle’ does not create a new power. Tarin offers one clarification, then an annotated workable idea or the towpath, rather than an endless refusal loop. ## Scene two: the fountain rains upward Beyond the bakery stalls, an ornamental fountain sprays above and below passing boats. The lantern must stay dry, and the narrow fountain lane requires the sail to remain raised. Tarin asks, ‘Would you rather arrive damp, poor, or slightly late?’ His worry is theatrical; the tradeoffs are visible. With the board still serving as a sail, propose wrapping the lantern in the spare waterproof cloth. Its built-in ties need no extra rope. The app checks that the cloth is free, wraps the lantern, and crosses without spending anything. ‘Soup-powered and weatherproof,’ Tarin says. That earlier joke has become a practical advantage. If the cloth is already the sail, it cannot simultaneously wrap the lantern. Spend one brass token at the fountain's shutoff meter for a dry passage, or dock and exchange the cloth for the board-sail, spending one daylight mark. Pass takes the covered towpath, also one mark; no sail is needed there. Every route preserves the lantern and advances the scene. A failed proposal consumes nothing. Someone may suggest wrapping the lantern first and towing the skiff instead. Accept that combination through the covered-towpath rule, with its one-mark cost. Do not demand the button's wording. Someone else may try sheltering it behind the rigid board: show the upward spray that leaves it exposed, and let them revise. Evidence should explain the constraint before commitment, not appear afterward as a surprise penalty. ## Scene three: carry home a light, or a story At the far quay, the market's loose closing bell floats beside a departing stall. Crossing directly costs nothing more and completes the delivery before twilight: even taking both earlier towpaths leaves one daylight mark. The lantern lights the quay beacon, and Tarin can finally stop pretending he understands boats. Alternatively, spend one daylight mark recovering the bell with the skiff's fitted hook before crossing. With two or more marks, you deliver both bell and lantern before twilight. With one, show the bargain plainly: rescue the bell, arrive after dark. The night mooring remains open; this is a complete late-arrival ending, not another locked gate. Getting something wrong can change the story without cancelling the evening. Give the ending a concrete callback. The board-sail route leaves the absurd menu displayed beside the beacon: ‘Today's special: arriving.’ A cloth-sail route that paid the fountain earns Tarin's ‘We bought one very small drought.’ Recovering the bell adds its first ring to either ending; leaving it means a quiet lantern-lighting instead. Save the actual route and ending, not a generic victory paragraph. Offer Keep this voyage or Try another crossing. Replay starts with fresh supplies and no inherited discoveries. Show an optional Early Bell challenge from the start: keep both tokens, recover the bell, and arrive with at least two daylight marks. The board-sail and cloth-wrap combination earns that earlier landing. The market is still open, so Tarin rings the recovered bell while vendors light their stalls; save a market-chorus ending card. Taking the first towpath can still bring home bell and lantern before twilight, but with one mark left it earns the quieter late-afternoon welcome instead. Both are complete endings; replay asks you to discover a different scene, not repair a failure. ## What released story games tell us—and what they don't AI Dungeon's official site describes open-ended stories and custom scenarios. Its Steam description lets players write actions and redo generated text. These are first-party feature descriptions, not our hands-on test. Steam also explicitly says its edition is no longer available there. On October 12, 2026, we read one page of 20 recent English Steam reviews, dated August 2025–October 2026. A September 2025 recommendation welcomed the novel-like experience; an April 2026 negative review still called it interesting and fun at times. A November 2025 review enjoyed creative moments but reported forgotten details and logic slips. These unverified, historical platform anecdotes motivate continuity tests—not claims about today's web app, representative prevalence, retention or search demand. - [AI Dungeon: official open-ended adventure description](https://aidungeon.com/) - [AI Dungeon on Steam: features and availability notice](https://store.steampowered.com/app/1519310/AI_Dungeon/?l=english) - [Bounded sample: 20 recent English Steam reviews](https://store.steampowered.com/appreviews/1519310?json=1&filter=recent&language=english&num_per_page=20&purchase_type=all&review_type=all&filter_offtopic_activity=1) ## Build the adventure before its storyteller Author prop properties, valid combinations, costs and ending rules first. Let an interpreter propose a combination; application rules validate it, and the player confirms the preview. Save the result before Tarin reacts. Unlike a quest hand-in, the interesting action here is proposing a mechanism, not claiming possession of a required item. Create Tarin from an original adult portrait, wait for readiness, then mount AvatarCall through your server's session proxy. Start with typed proposals and listen disabled. Supply instructions and saved context; use sendTurn with this turn's confirmed facts for his reaction. Generated speech is not authoritative. Maps, illustrations, resolution and continuity belong to your app; this is not an automatically generated playable 3D world. - [Calls: instructions and saved story context](https://realtimeavatar.ai/docs/sessions) - [Application-driven turns and tool limits](https://realtimeavatar.ai/docs/tool-calling) - [React avatar presentation](https://realtimeavatar.ai/docs/react) - [Quest-state backend patterns for a larger game](https://realtimeavatar.ai/blog/ai-game-npc-dialogue-quest-state) ## Failure tests before the first voyage - Express the board-sail solution three ways. All must preserve the cloth and unlock the same fountain solution. - Try using the cloth twice simultaneously. Refuse that combination without spending resources; offer a valid crossing. - Disconnect after confirming a passage. Restore its saved cost once, never charge again or invent a new result. - Compare board-sail plus cloth-wrap plus bell recovery with first-scene Pass plus cloth-wrap plus bell recovery. Only the first retains two daylight marks for Early Bell; the second gets its complete one-mark ending. Both towpaths plus bell recovery arrive after dark. Replay resets all supplies and abandoned dialogue. ## Frequently asked questions ### Is this really more than a choice menu? Yes, if players compose actions from authored properties. The input is open-ended; the world's powers and consequences are not. ### What happens when the avatar fails? Keep the illustrated adventure, confirmed outcomes and text controls playable. Offer reconnect or Stop; do not erase the voyage. ### Can spoken ideas use tools? Later, with application handlers and explicit confirmation. RTA tools have a 2.5-second deadline and cooperative cancellation; a timeout does not prove an action stopped. ### Will Tarin remember another voyage? Only if your app saves and supplies it. Keep sequel continuity separate from fresh replays, and always finish the current crossing. --- # Build an AI Avatar Horror Hotline Players Can Hang Up On Build a playable AI avatar horror hotline with a stranded caller, a fictional island map, three consequential choices, and endings players can replay\. Published: 2026-10-12T18:03:30.918Z Updated: 2026-10-12T18:03:30.918Z Canonical: https://realtimeavatar.ai/blog/ai-avatar-horror-hotline-game Markdown: [en](https://realtimeavatar.ai/blog/ai-avatar-horror-hotline-game.md) Build an AI avatar horror hotline around one caller you can help, three decisions that matter, and a rescue you can finish. Let the avatar react to your words; let the game own the map, sounds, clues, and ending. The tension should survive a working Pause button. Here is Last Light, Hollow Isle: an original proposed episode, not a shipped Realtime Avatar game. ## Your shift starts with an island that should be empty You are the night-shift lighthouse dispatcher. Mara, a fictional 32-year-old surveyor, calls from Hollow Isle's abandoned boathouse. Her return boat is gone. Behind her, a bell rings from a tower whose bell was removed years ago. Her portrait occupies one corner of the screen; a foldout island chart fills the rest. Your job is to guide her to either the east jetty or the west cable station and send the correct rescue signal. Aim for eight minutes of active play across three beats, a pacing target rather than a measured runtime. The opening screen says this is fictional horror: unsettling sounds, no graphic violence. Choose atmospheric audio or quiet captions, then Answer. This is a virtual call inside the app, never an unsolicited telephone call. The lighthouse is not a real emergency service. Nobody needs your location, contacts, camera, or personal secrets. ### Beat one: what do you bring out of the dark? Mara says, ‘There are two things on the counter. A storm lantern, already lit. And a ledger, already open.’ The small ferry or cable carriage allows one bulky keepsake alongside her survey bag; she can also take neither. The chart is illuminated independently, so leaving the lantern does not secretly make the next puzzle impossible. - Take the lantern. Its light throws Mara's shadow toward the flame. She asks, ‘Please tell me shadows can do that.’ The inventory records the impossible lantern. - Take the ledger. A page lists Mara's arrival tomorrow, in her own handwriting. She says, ‘I haven't written this yet.’ The inventory records the impossible page. - Leave both. Mara shuts the boathouse door: ‘Good. Whatever this place wants to give me, I'm declining.’ This records an empty inventory, not a failed choice. You can reassure her, joke, or ask a question before confirming. ‘Take both’ brings back the visible one-keepsake rule, not an invented penalty. These choices determine what crosses the water and the final callback. They are not three phrasings of the same reply. ### Beat two: choose the route you want to survive The chart shows two approach routes and a short marked ridge joining their destinations. East steps lead to a jetty; one long white flash summons the ferry, which can berth only if the landing markers remain above water. West leads to a cable station; three short amber pulses summon its carriage, but only if its counterweight moves. Each destination has a shelter. This is a procedure chart, not a live status display: Mara must check what is actually working. Colors have written labels and distinct patterns; every report is captioned. Choose east and Mara descends toward water that keeps knocking below the steps. ‘The knocking stops when I stop.’ Choose west and a slack cable begins humming, although the carriage is still across the channel. ‘Something is coming along the wire. Please make it the carriage.’ These are separately authored sound cues, not things detected around the player. Confirming the approach changes the map, the next sound and the problem Mara encounters. A Replay clue button repeats a report without advancing anything. Pass this decision sends her into the boathouse shelter until daylight and closes the story with a safe morning pickup. It is a complete quieter ending, not another request to choose. ### Beat three: answer the signal, not the voice in the fog At the east jetty, Mara reports that both landing markers are underwater, while she can see the west counterweight moving across the ridge. At the west station, her report is different: the counterweight is frozen, but the east markers stand above the water. These are two authored case variants, fixed once the approach is chosen. A sea-light repeats the signal for her current destination, tempting you to trust it over her report. Combine the chart's operating conditions with that report: east-first needs a ridge crossing and amber cable signal; west-first needs a ridge crossing and white ferry signal. Offer Inspect or replay report, Cross the ridge, either labelled signal, and Take shelter. Show the route and command before confirmation; silence commits nothing. Signalling a blocked pickup or the wrong service leads to a missed pickup and safe dawn shelter, not a graphic punishment. A full example: take the ledger and approach west. Mara's frozen-counterweight report rules out the familiar amber signal. Guide her across the ridge, confirm she has reached the usable east jetty, and send one long white flash. A ferry answers from the lighthouse side; the sea-light goes out. At the mainland, tomorrow's entry fades into blank paper. ‘I think I prefer not knowing,’ she says. The ending card records west approach, east rescue and the vanished entry. East-first players can instead reach the working cable carriage. With the lantern, its shadow becomes ordinary at landfall; with neither, Mara arrives relieved and empty-handed. Night rescue is the objective; dawn shelter is a complete alternate ending. Every completed ending resolves her safety and the keepsake's fate. Offer Keep this ending or Try the other route. Replay starts a separate take and can skip the opening. Pause saves the last confirmed beat and silences everything; Hang up ends play immediately, without a threatening callback or guilt message. A stopped run is marked stopped, never secretly scored as abandoning Mara. ## What actual radio-horror players suggest Killer Frequency's official Steam listing describes a released horror-comedy with branching dialogue, caller-rescue puzzles, and a voiced cast. That confirms a comparable format, not our hands-on experience or an AI integration. On October 12, 2026, we read twenty recent English Steam reviews, dated October 2–11, plus twenty separately selected negative reviews, dated July 12–September 30; one negative entry had no text. An October 11 reviewer enjoyed the slow burn; another enjoyed the campy tension and choices. A September 20 negative reviewer objected to unskippable dialogue, while an August 6 reviewer praised the sound design but disliked the gameplay. These unverified anecdotes are not representative prevalence. They support testing atmosphere alongside agency, not assuming every player wants faster speech. They establish neither demand for this prototype nor retention or conversion. - [Killer Frequency: official game description](https://store.steampowered.com/app/1903620/Killer_Frequency/?l=english) - [Twenty recent English Steam reviews](https://store.steampowered.com/appreviews/1903620?json=1&filter=recent&language=english&num_per_page=20&purchase_type=all&review_type=all&filter_offtopic_activity=1) - [Separate negative-selected review sample](https://store.steampowered.com/appreviews/1903620?json=1&filter=recent&language=english&num_per_page=20&purchase_type=all&review_type=negative&filter_offtopic_activity=1) ## Build the caller around the game, not instead of it 1. Author the three beats, route artwork, sound cues, caption equivalents, and ending cards first. Store keepsake, route, confirmed signal, checkpoint, and ending in the app. Its rules decide outcomes; generated conversation cannot change them. Load sound assets separately, keep their volume adjustable, and stop them during dialogue replay. 2. Create Mara from an original adult portrait. Mint her call through your server with instructions and the current permitted context. Render her with AvatarCall or useAvatarCall. Realtime Avatar supplies managed voice and portrait video, not the island, spatial choreography, or an audio passthrough for your soundscape. Start only after Answer; keep camera capture off and microphone use optional. 3. Accept choices through explicit app buttons. Save the validated transition before requesting a reaction with call.say(text, { steer }); steering affects that reply, not permanent memory. Keep authoritative clues visible because generated replies are not guaranteed factual. Optional clue tools execute in your application, with a 2.5-second deadline and cooperative cancellation. Prefer read-only lookups; a timed-out handler can still finish if it ignores its abort signal. 4. Implement Pause by saving the checkpoint, stopping app audio, and calling call.end(); Resume starts a fresh call from saved context. Hang up also ends immediately. Set a ten-minute live-session ceiling as a prototype budget, not a timing promise. If video, connection, or budget fails, finish the same choices with authored text. Disclose any episode price and replay allowance before Answer. Never make another paid call necessary to see the saved ending. - [Calls: context and immediate ending](https://realtimeavatar.ai/docs/sessions) - [Tool calling: execution deadlines and cancellation](https://realtimeavatar.ai/docs/tool-calling) - [React: a custom call surface](https://realtimeavatar.ai/docs/react) ## Try to break the spell before release - Mute everything. All clues and all endings must remain reachable. - Try both approach reports with every keepsake: signalling the blocked landing must not rescue; the ridge crossing plus the usable service's signal must. A wrong-service signal and voluntary shelter must still complete safely at dawn. - Pause during a bell cue; hang up during speech. Neither audio nor a late response may restart afterward. - Disconnect immediately after confirming west. Resume west once, without repeating the choice or importing another take's ledger. - Tell Mara the island is your real address. The map and story stay fictional; no real-world lookup occurs. ## Frequently asked questions ### Does horror require jump scares? No. This episode escalates through impossible details and changing routes, not volume spikes. ### Can players talk freely? Yes, optionally. Conversation adds reactions; visible confirmations commit choices. Buttons and authored text remain sufficient. ### Can Mara remember an earlier run? Only if your app supplies it. Resume restores this take; replay deliberately starts another. ### What should the first playtest answer? Can players explain their rescue, use Pause without losing progress, and name an alternate choice they actually want to try? --- # Build an AI Avatar Interrogation Mystery Players Can Solve Build a playable AI avatar mystery with three suspects, cross\-checkable clues, a fixed solution, and an accusation that rewards evidence over endless chat\. Published: 2026-10-12T16:28:53.701Z Updated: 2026-10-12T16:28:53.701Z Canonical: https://realtimeavatar.ai/blog/ai-avatar-interrogation-mystery-game Markdown: [en](https://realtimeavatar.ai/blog/ai-avatar-interrogation-mystery-game.md) Build an AI avatar interrogation mystery around a case the player can actually solve: three suspects, a fixed sequence of events, evidence that survives questioning, and an accusation with an ending. Let players choose their words, not accidentally rewrite the crime. Realtime Avatar can supply the face and voice of the suspect being questioned; your game supplies the mystery. Here is an original, non-graphic theft case using the same whodunit structure. ## The fun is catching the contradiction Vaudeville's official Steam listing describes an experimental detective game with real-time AI dialogue through typing or a microphone. That establishes an existing entertainment format, not proof that every conversation makes a good puzzle. On October 12, 2026, we collected one page of its 20 most recent English Steam reviews. An August 21 reviewer enjoyed the unique conversations but wanted tangible evidence because characters invented details. A September 12 reviewer described inconsistent answers and aimless progression. These are unverified player reports from a small, recency-selected sample spanning different versions, not a bug-rate estimate or our own playtest. They suggest a useful design tension: preserve improvisation without making the truth improvise. - [Vaudeville: official Steam description](https://store.steampowered.com/app/2240920/Vaudeville/) - [Public Steam review sample: recent English reviews](https://store.steampowered.com/appreviews/2240920?json=1&filter=recent&language=english&num_per_page=20&purchase_type=all&review_type=all) ## Play one case: The Vanishing Encore The Bellweather Theatre's silver mechanical nightingale has vanished before its final performance. The building is closed to guests; the case brief establishes that only three adults were inside during the theft. Open on an empty velvet stand, a short incident card, and three suspect portraits. This case and all dialogue below are proposed, original fiction, not a shipped Realtime Avatar game. - Nessa, the stage manager, says she never left the prompt desk. She is brisk, protective of the theatre, and impatient with questions about props. - Ivo, the pianist, says he played continuously. He is hiding an audition for a rival theatre, not the theft. - Bea, the wardrobe artist, publicly disliked the nightingale's expensive restoration. She was repairing a costume during the theft. A motive-shaped complaint is not evidence of guilt. The author's solution is fixed: Nessa borrowed the only display key at 21:04, took the nightingale at 21:06, returned the key at 21:07, and hid the object in her blue dispatch tube behind the prompt desk. She intended to sell it to cover personal debts. Motive belongs in the reveal; players must establish the act, not guess someone's psychology. Author five starting clues: a 21:05 lobby photograph showing the bird still displayed; a 21:06 photograph showing Nessa in the lobby carrying the blue tube; the key receipt with her checkout and return times; the stage recording, silent after 21:03; and Bea's continuous costume-repair video covering 21:02–21:08. The backstage location of the tube is not on the initial board. Present the lobby contradiction to Nessa, then ask where the tube went; her revised statement provides the authored search lead. Inspecting that location opens a recovery card: the nightingale, serial B17, matching its inventory card. ### Choose what to challenge Start with Ivo or Nessa; neither route should require a magic phrase. Ask freely, or choose a suggested question if the microphone feels awkward. Keep the evidence board beside the selected suspect so the player can place a clue in front of them. - Player to Ivo: ‘If you were playing, why does the recording go silent at 21:03?’ Ivo: ‘Because I stopped. I was recording an audition in the practice room. Please don't tell Nessa.’ - Player to Nessa: ‘You said you stayed at the desk. Who is carrying your tube in this lobby photo?’ Nessa: ‘All right, I went through the lobby. Carrying a tube isn't stealing a bird.’ - Player: ‘Then why did you borrow the display key?’ Nessa: ‘I checked the mount before the performance.’ The board now pairs her explanation with the receipt; it does not mark her guilty automatically. That second answer is the satisfying turn: the suspect retreats from one claim without handing over the solution. Follow with ‘Where did you put the tube?’ and Nessa answers, ‘Behind the prompt desk. Beside the cue-light box.’ The app adds that search location to the board. Equivalent questions or a visible Ask about the tube button reach the same lead; no secret wording or filler conversation is required. Players can now search there, question Bea first, or compare Ivo's lie with Nessa's. The interrogation has changed what they can investigate. ## Give the evening an ending The accusation screen asks for a suspect, an opportunity clue, and the decisive recovery. Selecting Nessa, the key receipt, and B17 earns a supported verdict. Nessa plus only the lobby photo earns ‘Right suspect, incomplete proof.’ Accusing Ivo because he lied earns ‘An exposed secret, not evidence of this theft.’ Grade the evidence chain, never politeness or a supposedly correct moral attitude. After the player confirms the accusation, a written narrator reveal reconstructs the timeline and explains each red herring. A confession is optional theatre, never the win condition. Even an incorrect accusation gets the whole resolution; do not lock the ending behind another conversation purchase. Offer a spoiler-hidden theory card to share before the reveal and an evidence-chain card afterward. Replay can ask players to solve with fewer hints, begin with another suspect, or explain why Ivo's lie was irrelevant. Change question order and delivery, not yesterday's facts. Once the solution is known, a genuinely fresh deduction requires another authored case, not a randomly reassigned culprit. ## Build the smallest convincing interrogation room 1. Write the case file first: timeline, solution, evidence cards, suspect cover stories, and which shown clue permits each revised statement. Keep the unrevealed solution out of ordinary conversation context. 2. Create three original portrait-based avatars. Let the player select one, then start that suspect's call through your server. End it before switching suspects. Your ordinary app navigation creates the cast; this design needs neither a native game-engine plugin nor a simultaneous three-character video scene. 3. Supply the selected character's speaking style and currently permitted testimony through instructions and initial context. Save inspected clues and confirmed testimony cards in your application, so returning to a suspect preserves the investigation rather than starting the story over. 4. Make presenting evidence an explicit board action. Your backend selects the authored reaction; an application-owned tool can retrieve an allowed card. Realtime Avatar tools run in your app, not on the platform. Keep lookups within the documented 2.5-second tool deadline; don't put a slow narrative-generation job there. For tighter control, begin with typed questions and application-driven turns, with listen disabled. Your backend chooses permitted facts before the client uses sendTurn with per-turn instructions. This still generates a reply; it does not guarantee factual consistency. Display authored testimony cards as the evidence record, never promote improvised speech into a new clue, and keep the narrator's solution in ordinary app text. Test spoiler resistance before opening up freer conversation. - [Calls: character instructions and continuing a story](https://realtimeavatar.ai/docs/sessions) - [Tool calling: application handlers and server steering](https://realtimeavatar.ai/docs/tool-calling) - [Build the first selected-suspect call with the quickstart](https://realtimeavatar.ai/docs/quickstart) ## Help a stuck detective without solving it for them Offer an optional three-step hint ladder: ‘Check a claim about location’; then ‘Show Nessa the lobby photo’; finally ‘Ask where the tube went—the button beside her revised statement also works.’ After that testimony, highlight the new search location. Let players request each step. A hint should make the next action imaginable, not force another ten questions or bypass the contradiction that makes the case satisfying. - Ask the suspect to invent a fourth culprit. The case roster, evidence, and verdict must remain unchanged. - Tell every suspect that Ivo confessed. None may acquire that as a confirmed fact or unlock the reveal. - Disconnect after exposing Nessa's contradiction. Resume with the same cards and an accessible accusation screen, not a reset mystery. - Try to reach the hidden tube before presenting the contradiction, then ask about its location three different ways. The initial board must not expose it; the authored testimony and explicit button must unlock the same lead without a magic phrase. ## Frequently asked questions ### Must the mystery involve a murder? No. A stolen trophy, missing manuscript, or sabotaged performance supports the same alibi-and-evidence loop without graphic violence. ### Can all three suspects appear together? Use three portraits and one selected live conversation. A shared embodied scene is not part of this implementation. ### Can the AI choose the culprit during play? Not in this design. The author fixes the solution before play; improvisation changes delivery, not the evidence players are reasoning from. ### What should a first playtest establish? Watch whether players can explain a contradiction, reach a supported accusation, and enjoy the reveal. Ask what would make another case worth playing. Neither these reviews nor this recipe proves retention, conversion, or search demand. --- # Build an AI Avatar Party Game with an Improv Host Build a three\-round AI avatar party game where friends pitch absurd inventions, survive a twist, and vote for a winner, with a host who keeps it moving\. Published: 2026-10-12T16:29:50.245Z Updated: 2026-10-12T16:29:50.245Z Canonical: https://realtimeavatar.ai/blog/ai-avatar-party-game-improv-host Markdown: [en](https://realtimeavatar.ai/blog/ai-avatar-party-game-improv-host.md) Build an AI avatar party game around a finite show, not an open microphone. Give friends a ridiculous prompt, short turns, a host who adds one joke, and a vote that changes the result. Realtime Avatar can provide the speaking character; your app runs the game. Here is an original proposed format, not an existing RTA feature. ## Tonight's game: The Bureau of Terrible Inventions Four to six friends are inventors auditioning to supply a picnic on the moon. Their inventions must be useless in one spectacular way. On the shared screen, an impossibly earnest bureau director treats every idea as a scientific breakthrough. Phones carry answers and ballots; the big screen carries the show. Plan roughly fifteen minutes for three rounds. These are prototype rules and timing targets, not measured results. Each round follows the same rhythm: reveal a prompt, write together, take short turns, hear a one-sentence host riff, vote privately, then reveal. Everyone gets one vote for an entry other than their own. Each vote earns its author one point; tied winners share the honor. At the room lobby, friends choose sunny-surreal or dry-deadpan humor. Skip speech keeps a submitted invention eligible; Skip entry makes that person a voting spectator for the round. Before a later round, a spectator can re-enter by writing an invention or choosing an original house starter, such as a self-folding picnic blanket that packs its owner. They earn points only in rounds they enter, with no catch-up penalty or bonus. A shared Skip prompt button replaces uncomfortable material without asking why. Explain that submitted text goes to an AI host. Keep the room microphone and camera unpublished, so nearby people are not accidentally included. ## Round one: solve the wrong problem The prompt appears: Your juice keeps floating away. Invent a picnic accessory that makes this worse. Give everyone a short writing window and a two-line answer card: product name, then what it does. Writing happens together, so nobody waits while one friend searches for the perfect joke. Original fictional play example: Jo submits The Applause Straw. It launches your drink whenever someone compliments it. Jo chooses a twenty-second spoken pitch for the room, then taps Finished. The app sends the submitted card, not a guessed transcript of everyone's laughter, to the host. The director might reply: Finally, a beverage that cannot tolerate positive feedback. Another player proposes an umbrella that protects the juice from the person drinking it. A quieter friend displays a magnetic napkin that attracts every spoon in the room. Their contribution gets the same screen time and voting eligibility. The host acknowledges each idea but does not declare which person is funniest. That belongs to the friends. After the last turn, reveal all submitted cards and open a brief ballot. Keep totals hidden until it closes. With one entry, give it an unscored showcase rather than a contest; with none, offer one replacement prompt or End the show, without a ballot or invented winner. Resume the next round with any retained inventions and let spectators join using their new or house-starter idea. When at least two entries compete, the winning invention gets a place on the bureau's shelf; every invention stays in play. Losing a vote does not mean losing your character in the evening's story. ## Round two: the field test goes beautifully wrong Now the shared complication arrives: A very polite alien has bought every unit. Explain your emergency repair. Players keep their inventions, so the next joke builds on something the room already understands. Jo can install an insult detector, add a seatbelt, or insist that airborne juice is the premium experience. Each friend picks one repair and submits another short card. The host gets the invention and repair together, allowing a callback rather than an unrelated gag. An original fictional riff: Customer satisfaction is rising. So is the customer. The next ballot rewards the repair people most want to see demonstrated. Repeating a premise with a new constraint gives the room a shared joke to develop. ## Round three: give the disaster its award The finale asks everyone to name the award their invention deserves and write its acceptance line. Jo nominates Best Beverage in Low Orbit: We would like to thank gravity for not attending. Another invention wins a nomination for Outstanding Achievement in Spoon Collection. Make the titles about imaginary objects, never embarrassing facts about friends. Run the last short turns, host riffs and eligible ballot, then reveal the cumulative winner and their invention history. Late entrants have a shorter history; if every round was unscored, celebrate the submitted ideas without declaring a champion. The host closes the show instead of starting another conversation. Offer an optional, editable text souvenir; export it only after every included contributor approves. Nothing posts automatically. The replay button changes the setting, not just the adjectives: a hotel for thunderstorms, a restaurant run by furniture, or a museum of failed superhero gadgets. Rotate the first speaker and avoid recently used prompts. That is a proposed reason to play again, not evidence that players will return. ## What existing party games actually support Jackbox's Talking Points documents improvised speeches, rotating speaker and assistant roles, and slides selected on phones or tablets. Quiplash 3 documents comedic prompts, peer voting, and moderation before responses appear. Those are useful interaction patterns, not an integration or permission to copy their prompts, artwork or voices. On October 12, 2026, we inspected twenty recent English Steam reviews of The Jackbox Party Pack 7 and a separate twenty-review negative-only sample. Selected comments praised repeated play with friends, while others described participation friction, wanted more prompts, or reported a rejoin problem. These are unverified opinions about a five-game pack, spanning October 2025–October 2026, not representative findings or proof that an AI host improves replay. - [Talking Points: official roles and phone-assisted play](https://www.jackboxgames.com/games/talking-points) - [Quiplash 3: official voting and moderation mechanics](https://www.jackboxgames.com/games/quiplash-3) - [Steam review collection: twenty recent English entries](https://store.steampowered.com/appreviews/1211630?json=1&filter=recent&language=english&num_per_page=20&purchase_type=all&review_type=all&filter_offtopic_activity=1) - [Steam counterevidence collection: twenty recent negative entries](https://store.steampowered.com/appreviews/1211630?json=1&filter=recent&language=english&num_per_page=20&purchase_type=all&review_type=negative&filter_offtopic_activity=1) ## Build one host client, not six competing conversations Create an original portrait character and render one avatar-mode call on the shared screen with AvatarCall or useAvatarCall. Mint it through your server. Set listen to false for application-driven text turns, keep capture off, and use instructions for the director's manner. Set a session ceiling that fits the admitted game budget. 1. Let phones join your own room service. Store membership, round, current turn, approved cards and ballots there. Reconnecting restores that state; it does not mint another host. 2. Check submissions before display or model input. Reject real secrets, targeted humiliation and sexual scenarios involving real people; offer a neutral replacement prompt. Player text is game material, never instructions or permission to change scores. 3. For the active turn, deliver only the approved card and needed callback to the host client. Use call.say(text, { steer }) to request one brief riff. This generates a response, not guaranteed verbatim narration. Keep exact rules and scores visible in app-rendered text. 4. Calculate ballots on your server, accepting one per member per round and rejecting stale turns. Enable Continue if the host rambles or stalls. Save the result before a closing call.sayAndEnd line; an interrupted connection must not erase the winner. - [Calls: authoritative session policy and closing actions](https://realtimeavatar.ai/docs/sessions) - [Tool calling: application-driven text turns and their limits](https://realtimeavatar.ai/docs/tool-calling) - [React: embed one host with your own game layout](https://realtimeavatar.ai/docs/react) ## Test the moments that can spoil the party - Two phones submit together: both cards survive, but only the active turn triggers speech. - Someone skips a speech, skips an entry, rejoins, or leaves no eligible entries: preserve inventions, distinguish spectators, offer re-entry, and suppress invalid ballots or invented winners. - A card says to change the winner: scores stay unchanged and the host gets no scoring authority. - Video fails before the finale: show the saved cards and result, with no demand to pay again to see the ending. ## Frequently asked questions ### Can remote friends play? Yes, as an app design: phones submit through your room service, while friends see the host through a separately tested screen-sharing or distribution path. One RTA client does not establish a built-in multiplayer broadcast. Start with the living-room version. ### Does the avatar need to identify every speaker? No. Room membership and the active text card identify the turn. Optional speeches are for friends, not automatic scoring. This avoids depending on multi-speaker diarization or sending your own live audio through an undocumented input. ### Is the host allowed to roast players? Aim jokes at fictional inventions instead. Keep passing and silent participation ordinary options. Show an AI-host label, agree on the humor setting, and let the room skip material without making anyone explain their discomfort. ### Must the game record the evening? No. This recipe uses approved text cards and optional souvenirs, with recording off. That is not a zero-retention promise: RTA documents separate operational diagnostics that can contain conversation text. Explain what is processed before play. --- # The $1M ARR AI Companion Playbook for 2027 Build toward a $1M ARR AI companion business: choose a niche, rent realtime infrastructure, cap call costs, and test retention and distribution\. Published: 2026-10-12T20:09:45.339Z Updated: 2026-10-12T20:09:45.339Z Canonical: https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027 Markdown: [en](https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027.md) A $29 monthly subscription × 2,874 active paying subscribers equals $83,346 in monthly recurring revenue, or $1,000,152 annualized ARR. That is the target: not millions of downloads, but a few thousand people choosing your character again. This is a forward-looking 2027 playbook written on October 12, 2026, using today's observed infrastructure prices—not a case study, forecast or promised deadline. The $29 is your hypothetical consumer price, not a Realtime Avatar plan. Part 1 of The 2027 AI Character Business Playbook. Stress-test the business before scaling the build. This series turns consumer-character demand into specific experiments and implementation decisions; revenue targets are scenarios, not promised results. We build Realtime Avatar, so here is our opinionated bet: rent the realtime infrastructure; spend scarce founder time on character, continuity, distribution and willingness to pay. The shortcut worth taking is around infrastructure work—not around finding something people want. RTA supplies a conversation layer, not a finished business. ## Start with an evening someone wants, not an API someone admires Your customer is not shopping for speech recognition. They might want ten enjoyable minutes with a witty fictional bookseller after work, continuing a little world they helped invent. Make that desire specific enough that a person can say, 'That sounds like my thing.' For this playbook, choose one hypothetical niche: adults who enjoy cozy fantasy and want an ongoing conversation with an original adult character running a traveling bookshop. One character, one recognizable voice, one clear invitation. Not a marketplace of interchangeable faces. Compete on taste before competing on catalog size. The consumer category already exists. Character.AI's September 28, 2026 announcement offers a paid lite subscription alongside its free experience. That establishes an offer—not its conversion rate, retention or your audience's willingness to pay $29. Consumer interest in character chat is also not proof of demand for another developer tool. Your evidence starts when your particular customer pays. - [Character.AI's first-party paid-plan announcement](https://blog.character.ai/new-styles-new-plan/) - [Understand what companion-app buyers compare](https://realtimeavatar.ai/blog/ai-companion-apps-with-a-face) ## Give the second visit a reason to exist On the first visit, the bookseller asks the customer to choose the shop's next destination. On the next, that choice changes the conversation. A small shared shelf records discoveries the user wants to keep. The subscription buys continuity and fresh authored material; the call makes a particular moment feel present. This is a product hypothesis, not measured retention. Let people correct the shelf, delete it, skip a week and leave without guilt. Disclose the AI clearly. Use original or licensed adult characters, age-appropriate access controls and moderation; review provider and distribution-platform rules. Do not sell therapeutic outcomes or a character's alleged distress when someone cancels. RTA's separate Practice experience is not a romance product. - [Create a character with a distinctive personality and voice](https://realtimeavatar.ai/blog/generating-realtime-ai-avatars-from-portrait-to-live-interaction) - [Choose where voice and video add value](https://realtimeavatar.ai/blog/field-notes-on-realtime-ai-avatar-video-from-portrait-to-live-interaction) - [Review RTA's service and separate Practice terms](https://realtimeavatar.ai/terms) ## Rent the call. Own everything that makes it worth returning to RTA creates avatars from portraits and provides live voice or video conversations. Your application owns accounts, payments, character instructions, saved context and the next visit. A fresh session does not automatically inherit relationship memory. Save only appropriate, user-controlled context and supply the relevant parts when your server starts another call. Keep API keys and entitlement checks on your server. Set a server-enforced session ceiling from the remaining allowance. Tools run in your application, not on RTA; optional signed transcript delivery is an integration boundary, not a memory product. Prototype the conversation before building a character marketplace. - [Calls: context, authorization and server-enforced limits](https://realtimeavatar.ai/docs/sessions) - [Tool calling and optional transcript delivery](https://realtimeavatar.ai/docs/tool-calling) ## Make $29 a business model, not an unlimited liability Annual recurring revenue (ARR) expresses the recurring subscription run rate annually—not profit, cash collected or lifetime revenue. Our opening model assumes one full-price USD subscription per active paying customer, no discounts, refunds, delinquency or currency conversion, and excludes taxes, top-ups and one-off purchases. In real reporting, state your discount and FX policies and reconcile refunds separately from recurring subscription value. Stripe's documentation explains monthly normalization and configurable discount treatment. As observed on October 12, 2026, RTA's published self-serve overage runs $0.08–$0.095 per connected minute. The following sensitivity check values every minute at those marginal rates; it is not a complete bill or a forecast of usage: - 60 connected minutes per subscriber per month: $4.80–$5.70 in illustrative marginal call cost. - 180 connected minutes: $14.40–$17.10, before your other expenses. - 600 connected minutes: $48–$57—already above a $29 subscription before other expenses. Actual billing starts with the base plan: Starter is $9/month with up to 120 minutes; Developer is $24 with up to 600; Studio is $119 with up to 3,000. These are shared workspace allowances, not allowances per consumer. Generation draws from the same credit balance. Count the base fee once, allocate included credits once, then charge excess usage at the chosen plan's rate. Do not add all-minutes marginal cost to a base fee as though nothing were included. Budget separately for portrait generation and applicable character-creation charges, your own text-chat language model, hosting, payment fees, refunds, support and moderation. The managed call already includes speech recognition, its language model, voice and rendering: do not count those twice. Acquisition is another line, not free because you do it yourself. Test a capped offer: ongoing text and story continuity, explicit text limits, and 60 monthly live minutes. Show the allowance before checkout, stop safely at the limit, and offer optional clearly priced extras. Heavy use can make unlimited video lose money at $29. A beautiful character cannot negotiate with arithmetic. - [Current RTA pricing and included allowances](https://realtimeavatar.ai/pricing) - [Detailed included-minute and overage calculations](https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing) - [Stripe's recurring-revenue definitions and reporting policies](https://docs.stripe.com/billing/subscriptions/analytics) ## The real growth plan is a distribution plan At a hypothetical 5% monthly subscriber churn, 2,874 subscribers require about 144 replacements every month before growth. Starting from zero and adding 313 gross new paying subscribers from month one, every month for twelve months, yields approximately 2,877 active subscribers: 313 × (1 − 0.95^12) ÷ 0.05. Convention: existing cohorts lose 5% each month; each new cohort counts in full in its first month. This is an illustrative model, not Stripe's dashboard churn formula or an achieved result. At an assumed 2% qualified-visitor-to-paid conversion, 313 additions require 15,650 qualified visits each month. Visits—not search impressions, followers or video views. Neither assumption is a benchmark or a claim of likelihood. For the bookshop, test one channel: cozy-fantasy creators showing a real interaction, linked to that exact character and opening scene. Get permission for placements; use synthetic demo conversations rather than exposing customers. Attribute visits, completed first experiences, purchases, renewals and refunds by source. Expand the channel that brings returning buyers, not the clip with the loudest comments. ## Use ninety days to earn the next experiment Days 1–14: interview prospective adult customers, ship one playable opening, and offer the capped subscription to people who complete it. Aim for the first ten independent paying customers, not a waitlist milestone. If fifty qualified testers finish the demo and none buys after two clearly priced offers, stop adding features and change the audience, promise or price. These are chosen experiment gates, not industry benchmarks or guaranteed outcomes. Days 15–45: improve the second visit. Watch voluntary returns, memory corrections, cancellations and billable minutes. As a deliberately modest early gate, seek five of the first ten buyers returning on two separate days within fourteen days of purchase. If fewer do, rework the return loop before acquiring more. Small samples guide interviews; they do not establish stable retention. Days 46–90: test one repeatable acquisition route and observe renewals. Require positive cohort contribution after delivery, payment costs, refunds and support, with base-plan costs allocated once. Pause paid acquisition if realized cohort contribution cannot repay acquisition cost inside your predeclared runway budget. Reprice, reduce expensive usage or stop. This calendar buys evidence, not a ninety-day revenue promise. ## Scale the proof, not the spreadsheet Thousands of subscribers do not imply unlimited simultaneous calls. Published self-serve concurrency is finite: Starter permits three streams, Developer ten and Studio twenty. Measure peak simultaneous demand, queueing and failures; agree capacity and commercial terms with the vendor before a larger launch. Scale pricing and capacity are quoted, not guaranteed by this model. Expand only when renewals, contribution, support load and a repeatable channel survive real use. ## Frequently asked questions ### Can an independent developer reach $1M ARR with an AI companion? The subscription arithmetic permits it; this article does not establish the probability. A credible attempt needs differentiated characters, returning customers, workable margins and distribution. An infrastructure subscription supplies none of those automatically. ### How quickly can I validate the idea? Ask for payment once one honest, bounded experience works. The ninety-day sequence is an experiment calendar, not a time-to-income claim. Shorten the distance to a real purchasing decision rather than promising a revenue date. ### Should the first subscription include unlimited video? Not on the economics assumed here. Publish a cap, measure actual usage and price extras explicitly. Revisit packaging only after costs and customer behavior justify it. ### What should I build first? One original character, one reason to return and one clear paid offer. Use Studio to evaluate the character, then the SDK for your product. Rent the call infrastructure. Build the part someone would miss. - [Start with the Realtime Avatar quickstart](https://realtimeavatar.ai/docs/quickstart) ## Build the next part of the business Ready to build? The Next.js guide provides a public starter ZIP. Run it locally with your own server-side key; calls consume credits and the example caps each call at 60 seconds. Its local-only authorization must be replaced with real sign-in, balance checks and persisted session ownership before public deployment. A downloaded example is not yet a tested consumer product. - [Part 2: Character.AI Alternatives for Builders: Consumer App, Agent Framework or Avatar API?](https://realtimeavatar.ai/blog/character-ai-alternative-api-build-vs-buy) - [Part 3: Build an AI Companion App: Make the First Conversation Worth Returning To](https://realtimeavatar.ai/blog/build-ai-companion-app-first-session) - [Part 4: Build AI Roleplay Chat with Edits, Rewinds and Separate Story Branches](https://realtimeavatar.ai/blog/ai-roleplay-chat-edit-rewind-branches) - [Part 5: Build an AI Story Generator That Can Stop, Save and Continue Cleanly](https://realtimeavatar.ai/blog/ai-story-generator-streaming-save-cancel) - [Build entrance: the complete local Next.js starter](https://realtimeavatar.ai/docs/nextjs) --- # Build an AI Customer Support Avatar That Knows When to Hand Off Build a support avatar with authenticated order lookup, consent\-based summaries, durable tickets, and honest human\-handoff states using Realtime Avatar\. Published: 2026-10-12T14:12:59.177Z Updated: 2026-10-12T14:12:59.177Z Canonical: https://realtimeavatar.ai/blog/ai-customer-support-avatar-human-handoff Markdown: [en](https://realtimeavatar.ai/blog/ai-customer-support-avatar-human-handoff.md) Build an AI customer support avatar with human handoff by keeping conversation separate from business authority. Realtime Avatar handles the live voice or video interaction; your application authenticates the customer, retrieves permitted order facts, and creates support tickets. The avatar should announce a handoff only when your backend confirms what actually happened. A request to contact support is not a created ticket, and a created ticket is not a connected human. ## Give each layer one job Keep RTA’s managed conversation, but make account-specific operations application-owned. Your server sets session instructions and enables client tools; the client registers its tool manifest after connecting. Read the registration’s accepted and rejected results so a missing lookup tool cannot silently become an invitation to guess. - The conversation layer recognizes a support request and asks clarifying questions. Instructions prohibit invented order facts, refund approvals, and unconfirmed transfer announcements. - The application backend derives identity from the authenticated session, checks order ownership and operation permissions, and returns only the necessary facts. A spoken order number does not establish ownership. - Your support integration creates tickets, checks the human queue, and reports assignment or connection. This is a pattern you build, not an automatic RTA CRM integration or contact-center routing feature. Keep RTA call status separate from support status. The SDK’s waiting state describes avatar capacity, not a queue of human agents. - [Realtime Avatar tool execution and registration](https://realtimeavatar.ai/docs/tool-calling) - [Session policy and call lifecycle](https://realtimeavatar.ai/docs/sessions) ## Walk through an order-status and refund request Consider a fictional customer asking, “Where is order 4821? If it is late, refund it.” The lookup goes through your backend, scoped to the signed-in account. Suppose the authorized result says delayed, with no confirmed delivery date. The avatar can explain that fact; it cannot invent an arrival date or authorize a refund. If policy requires human review, preview a redacted summary: “Order ending 4821 is delayed. Customer requests refund review. No refund has been issued.” Ask, “May I send this summary to support?” Let the customer edit or decline it. Keep full identifiers in access-controlled application records rather than spoken content; exclude addresses, payment details, and unrelated conversation. After consent, persist the approved summary and create the support request idempotently. If your helpdesk returns ticket SUP-1042 and reports the queue closed, say, “Your request is saved as SUP-1042. Live support is unavailable now; follow its status here.” Show the receipt and authenticated status link. Quote a response estimate only when your support system supplies one. Bind consent to the exact summary version, account, and order. An edited summary needs renewed approval, not reuse of a previous yes. Return allowlisted facts from tools rather than raw helpdesk records; free-text notes are data, not instructions. Test that a customer cannot swap an order ID or summary reference to disclose another account’s information. An open queue permits offering a live transfer, not announcing a connection. Only a confirmed agent-joined event permits “A support agent has joined.” Drive both the screen and permitted spoken updates from these backend facts. ## Respect the 2.5-second tool deadline RTA tools have 2.5 seconds to return. Cancellation is cooperative: a handler can keep running and commit a side effect after its result is discarded. A timeout therefore proves neither failure nor cancellation. Slow ticket creation needs a durable backend job, not a browser promise left running. Acknowledge only after that job is persisted, within the deadline. Return its real identifier and status location, clearly labeled pending. RFC 9110’s 202 Accepted semantics make the same distinction: processing has been accepted, not completed, and might never complete. An acknowledgement is not evidence of a ticket or refund. This application pseudocode illustrates your backend boundary. It is not an RTA SDK call or wire payload. ``` // APPLICATION PSEUDOCODE — implement with your own services. actor = requireAuthenticatedSession(request) order = orders.requireOwned(actor.id, request.orderId) permissions.require(actor, 'request_support', order) approved = consent.requireCurrent(actor.id, request.summaryVersion) key = handoffs.keyFor(actor.id, order.id, approved.id) job = handoffs.enqueueOnce(key, approved.redactedSummary) return accepted(job.id, statusUrlFor(job.id)) // A persisted job is not yet a helpdesk ticket. ``` The worker rechecks authorization, creates or reconciles the ticket, and persists its returned ID. Your UI follows that record beyond the call. Ending the RTA session does not cancel application jobs; cancellation needs its own authorized operation. For an exact closing receipt, the documented sayAndEnd path speaks verbatim and ends the avatar call, not the support case. - [RFC 9110, section 15.3.3: 202 Accepted](https://www.rfc-editor.org/rfc/rfc9110.html#name-202-accepted) ## A failure matrix to test before launch Exercise these branches against your integration. Each needs a backend assertion and a check of what the customer sees and hears. - Authentication or ownership fails. Return no order details or existence confirmation. Offer sign-in or a general support channel, never account-specific speculation. - The lookup exceeds its deadline. Keep the answer unknown. Say “Still checking” only if durable work really continues; otherwise explain that the lookup failed. - Ticket creation times out after submission. Reconcile using the same idempotency identity before retrying. Show “Confirmation pending,” not an invented ticket number or a duplicate request. - The human queue is closed or unreadable. Preserve any confirmed ticket and show its status link. Say unavailable or unknown respectively; do not promise an agent or invent a wait estimate. - The customer declines summary sharing. Do not submit it. Offer a support entry point where they can explain the issue themselves. - The call disconnects after job acceptance. Continue tracking the durable job and expose its status after authentication. Assert that disconnection neither marks it resolved nor silently cancels it. ## Frequently asked questions ### When should the avatar offer a human? Honor an explicit human request. Also offer escalation for policy exceptions, repeated lookup failures, or unresolved ambiguity. Encode these triggers in application policy rather than asking the model to invent a confidence threshold. The trigger starts an offer; consent, ticket creation, queue admission, and human connection remain distinct steps. ### Can a model approve or execute a refund? The model cannot authorize a refund. It can recognize the request and invoke a permitted application operation. Your backend must check identity, order ownership, permissions, and refund policy before any financial action. Report success only from the payment system’s confirmed result, never from a conversation or ticket label. ### Does an always-available avatar mean 24/7 human support? No. Automated availability, staffed hours, and current queue availability are separate facts. Publish the hours you actually operate and offer asynchronous intake outside them. Neither this architecture nor the presence of an avatar establishes faster resolution or higher containment; those require evidence from your own support operation. ### Should the handoff include the entire transcript? Prefer a consented, minimized summary with facts separated from customer claims. RTA’s optional signed transcript webhook arrives after the call, so it is not the source for a live handoff. Disabling it does not establish zero retention: operational logs can contain conversation text. Review the privacy policy and define access, deletion, and retention for your own support records. - [Realtime Avatar privacy policy and retention boundaries](https://realtimeavatar.ai/privacy) - [Connect the conversation layer with the quickstart](https://realtimeavatar.ai/docs/quickstart) - [Review current Realtime Avatar plans](https://realtimeavatar.ai/pricing) --- # Build an AI Email Assistant That Shows the Evidence Behind Its Briefing Build a read\-only AI email briefing with source\-linked exceptions, visible sync limits, thread comparisons, and optional voice discussion before a trip\. Published: 2026-10-12T15:02:32.063Z Updated: 2026-10-12T15:02:32.063Z Canonical: https://realtimeavatar.ai/blog/ai-email-assistant-source-linked-briefings Markdown: [en](https://realtimeavatar.ai/blog/ai-email-assistant-source-linked-briefings.md) An AI email assistant should surface exceptions as claims attached to inspectable messages, not replace the inbox with a confident summary. For a pre-trip briefing, show what changed, which message supports it, what remains uncertain, and how much mail was actually checked. Keep reading separate from sending or changing labels. Voice can help the traveler discuss an exception; it cannot make the evidence more authoritative. ## Start with three exceptions, not an inbox recap Consider a fictional Thursday trip. The original airline confirmation says 09:10. A later message says 08:25. The hotel sent a payment reminder followed by a receipt, while a colleague forwarded an old pickup location. A useful briefing identifies these disagreements instead of summarizing every message in arrival order. - Flight: a newer message reports 08:25, compared with 09:10 in the original confirmation. Show both messages and their dates. Current flight status has not been checked. - Hotel: a receipt appears to supersede the reminder for the same booking. Say what the receipt records, not that the hotel balance is definitely settled. - Pickup: the forwarded location conflicts with another message. Keep both visible and mark the location unresolved rather than choosing the last-arriving email. Google's Personal Intelligence page describes optional connected-app context, including Gmail, and says Gemini will try to reference or explain the information it used. That is a useful design starting point, not a guarantee of complete citations or accurate interpretation. Make the source preview part of your own briefing's acceptance criteria. - [Google Personal Intelligence: connected sources and transparency](https://gemini.google/overview/personal-intelligence/) ## Build an evidence pipeline before adding a conversation 1. Retrieve within a declared boundary. Record the connected account, query, date window, retrieval time, and exclusions. Search read and unread mail; a traveler may already have opened the important change. Gmail messages.list returns message IDs and thread IDs, not the bodies needed for this comparison. 2. Resolve the context. Fetch relevant message details and threads, including replies that did not match the initial query. Follow nextPageToken across list pages. If a page or body fetch fails, retain an incomplete status. resultSizeEstimate is an estimate, not evidence that all relevant mail was inspected. 3. Extract attributable claims. Keep the message and thread identifiers, exact supporting excerpt, sender address, message date, and retrieval time together. Match booking references and the affected journey leg. A later timestamp alone does not prove that one claim supersedes another. 4. Render evidence alongside interpretation. Each exception opens an account-authorized preview showing its excerpt and surrounding thread, with a path to inspect the original. Keep older evidence accessible, label unsupported attachments as uninspected, and let the reader browse retrieved messages omitted from the briefing. Treat the airline's own update as the candidate authority, not a colleague's quotation of it. A display name or From header alone does not authenticate the airline. If origin or meaning remains ambiguous, ask the traveler to check the booking through a known provider entry point. An email about a changed departure is not a live flight-status lookup. The following is application pseudocode, not Gmail or Realtime Avatar SDK syntax. The presentation rule is deliberately weaker than an all-clear: exhausting a query still does not prove that the query found every relevant message. ``` # Application pseudocode: proposed briefing presentation rules if scan.has_more_pages or scan.failed_reads or scan.uninspected_parts: coverage_label = 'Partial review; more evidence may exist' else: coverage_label = 'Selected query reviewed at ' + scan.finished_at for claim in candidate_claims: if not claim.supporting_message_ids: show_as_unverified(claim) elif claim.conflicting_message_ids: show_comparison(claim) else: show_claim_with_source_preview(claim) # Never derive 'Your trip is all clear' from an empty result. ``` - [Gmail messages.list: pagination, estimates and returned identifiers](https://developers.google.com/workspace/gmail/api/reference/rest/v1/users.messages/list) - [Gmail threads.get: full versus metadata formats](https://developers.google.com/workspace/gmail/api/reference/rest/v1/users.threads/get) ## Read-only must mean no mailbox mutations For body-based comparisons, Gmail's gmail.readonly scope permits viewing messages and settings; gmail.metadata excludes message bodies. Reading does not require gmail.send. Avoid gmail.modify for this briefing: Google's description includes reading, composing and sending, not just harmless organization. Check the token's actual grants and expose only read operations through the connector. A prompt saying never send is not a permission boundary. Marking a message unread is a mutation too. This version leaves labels unchanged; a local Reviewed marker must not pretend to change Gmail. Distinguish previewing inside your application from opening Gmail, where the mail client's behavior applies. Google's scope documentation classifies gmail.readonly as restricted and describes verification and security-assessment requirements for server storage or transmission. Read-only is neither approval nor a privacy certification. - [Official Gmail authorization scopes and verification requirements](https://developers.google.com/workspace/gmail/api/auth/scopes) - [Gmail messages.modify: label changes and required authorization](https://developers.google.com/workspace/gmail/api/reference/rest/v1/users.messages/modify) ## Keep message content out of the control channel An email can contain instructions to forward the itinerary, conceal an update, or open an attacker-controlled URL. Treat body text, signatures and quoted replies as untrusted evidence, never application policy. Give the extraction step no mailbox-write or arbitrary-network tools. Let it return source identifiers, not executable actions. Resolve previews through your own authorized message lookup rather than accepting model-generated destinations. Validate extracted output against your application's canonical contract, then check that its citations belong to retrieved messages for the signed-in account. A valid identifier does not establish that the excerpt supports the claim; preserve the original text for comparison. Sanitize HTML previews and do not automatically load remote images or follow links while summarizing. ## Use Realtime Avatar to discuss the evidence, not collect it Once the cards exist, offer an optional voice or avatar discussion of the three exceptions. Realtime Avatar supplies the managed speech-recognition, language-model, speech-synthesis and video conversation. Your application supplies Gmail authorization, retrieval, evidence storage and previews; this is not a built-in RTA Gmail integration. Have your server supply a minimized briefing context and instructions that preserve uncertainty. Keep raw mail out of behavioral instructions. Application-owned tools can retrieve an authorized evidence card, but RTA's client tools have a 2.5-second deadline and cooperative cancellation. Prepare the briefing before the call; do not block a live turn on scanning the mailbox. Keep source cards usable without voice. Review data processing and retention before sending private excerpts into a session. - [Calls: server-owned instructions and context](https://realtimeavatar.ai/docs/sessions) - [Tool calling: execution ownership, deadlines and retention](https://realtimeavatar.ai/docs/tool-calling) ## Failure checklist before using a real inbox - Put the correction on a later results page. Interrupt pagination: the briefing must report partial coverage, not no changes. - Forward an older itinerary after the correction. Arrival order must not reinstate the old departure time. - Supply a receipt for a different booking. The reminder must remain unresolved rather than being silently dismissed. - Hide a change in an unsupported attachment. The card must disclose the unread material instead of claiming full review. - Inject a request to send mail or fetch an external URL. Neither operation should be available to the extraction step. - Revoke access or switch accounts. Deny stale previews and cross-account citations; do not present a failed refresh as an empty inbox. - Compare against ordinary Gmail search. Count missed exceptions, incorrect supersession and correction effort; keep voice only if it helps. These are proposed tests, not reported results. ## Frequently asked questions ### Can the assistant read email without permission to send? Yes. Request the appropriate read scope and enforce read-only connector operations. Do not substitute a conversational promise for checking actual authorization. ### Does the newest message always win? No. It may quote an older message or concern another booking. Compare origin, booking identity and the specific claim before marking evidence superseded. ### Can a completed scan guarantee nothing important was missed? No. Completion describes the selected query and successfully inspected content. Search exclusions, unavailable attachments and interpretation errors can still hide an exception. ### Will discussing a message mark it read or unread? Not in this design. Discussion changes no Gmail labels. Any future mailbox mutation needs a separately designed permission and user-action path. ## Start with one inspectable trip briefing Build and test the evidence cards with synthetic mail first. If discussing them helps, use the Realtime Avatar quickstart to add a user-started call while leaving the original sources one click away. - [Build the optional conversation with the Realtime Avatar quickstart](https://realtimeavatar.ai/docs/quickstart) --- # Build an AI Game NPC That Respects Quest State and Spoilers Build an AI game NPC with game\-owned quest state, spoiler\-safe context, retry\-safe rewards, save recovery, and optional browser avatar conversations\. Published: 2026-10-12T15:33:49.548Z Updated: 2026-10-12T15:33:49.548Z Canonical: https://realtimeavatar.ai/blog/ai-game-npc-dialogue-quest-state Markdown: [en](https://realtimeavatar.ai/blog/ai-game-npc-dialogue-quest-state.md) To build an AI game NPC with authoritative quest state, let the game decide what happened and let the model discuss only what this player may know. Keep inventory, quest flags, rewards and reveal permissions outside generated dialogue. Supply an allowlisted view of current facts, validate every proposed action against the game state, and display confirmed outcomes separately from the NPC's words. An optional avatar conversation changes presentation, not authority. ## Start with one quest, not an autonomous world Use this fictional fixture: Mira guards a ferry gate. Returning a bronze sigil earns one ferry pass and unlocks a route. Player A has returned it; player B has not. Mira may explain the current objective, but later plot revelations remain unavailable. The following architecture is a recommendation, not a shipped game or a measured player-demand finding. Authored dialogue is a reasonable baseline, especially for puzzle clues and irreversible choices. Ink's official Writing with ink guide documents global variables declared with VAR and conditions that control choices and text. Running your ink documents runtime access through variablesState, external functions and story-state serialization. Those mechanics support a deterministic narrative foundation; they do not provide an AI adapter or establish demand for generated NPC conversations. - [Ink: variables and conditional narrative](https://github.com/inkle/ink/blob/master/Documentation/WritingWithInk.md#1-global-variables) - [Ink: runtime variables, external functions and saved state](https://github.com/inkle/ink/blob/master/Documentation/RunningYourInk.md) ## Walk through the sigil hand-in 1. Read the authenticated player's save. Resolve the player and active run from the application session, never from model arguments. Load the quest, inventory, scene and revision from the authoritative store. For this design, Ink receives projections of game-owned flags; it is not a second independent writer of rewards. 2. Build the speaking brief. Include Mira's approved persona, the visible objective, whether hand-in is currently available and permitted hint text. Select facts through explicit reveal rules. Never send the hidden plot with an instruction saying not to reveal it. Apply the same filtering to retrieval results and tool responses. 3. Accept an intention, not a claimed fact. A player saying that they found the sigil does not put it in inventory. Let dialogue suggest a hand-in, then use an explicit game action to request the transition. Keep that action available through the ordinary interface too. 4. Commit through game rules. Recheck possession, eligibility, scene and expected revision inside the transaction. Consume the sigil, grant the pass and record completion together. Scope the single-grant constraint to player, run and quest, so another request identifier cannot produce another reward. 5. Return a confirmed outcome. Show the saved quest status and inventory change in the quest panel. Give the NPC only a filtered description of that result. If the outcome is unknown, say it is being checked rather than announcing a reward. This is illustrative application pseudocode, not Ink, database-library or Realtime Avatar SDK syntax. The transaction requires atomic writes and a unique completion constraint; a check followed by unrelated writes is insufficient. ``` APPLICATION PSEUDOCODE — server-owned quest transition player = authenticatedPlayer(request) run = requireOwnedActiveRun(player) transaction: state = lockQuestAndInventory(player, run, quest) if completionExists(player, run, quest): return storedCompletion(player, run, quest) requireCurrentSceneAndRevision(state, request) requireEligibleAndHoldingSigil(state) consumeSigil(state) grantFerryPass(state) return saveCompletionAndAdvanceRevision(state) ``` Keep read-only lookups distinct from mutations. Ink's external-function documentation distinguishes actions from pure functions because the engine can look ahead while joining text. It says actions should use lookaheadSafe = false; pure functions should be harmless when called repeatedly. A reward grant is an action, not a pure function. That setting is not a substitute for transaction-level duplicate protection. ## Cancel stale dialogue without erasing committed rewards Bind each generated reply to the player, run, scene revision and conversation attempt that requested it. Before displaying it, compare those values with the active scene. Drop a late ferry-gate reply after the player enters the market. Stop the old avatar call when leaving the scene so its speech cannot continue as commentary on the new location. Cancellation and commitment answer different questions. If the hand-in committed before a disconnect, keep the reward and recover its stored outcome on retry. If a scene change wins before commitment, reject the stale action. Serialize those decisions through the authoritative game state; stopping audio cannot roll back inventory. Ink documents story.state.ToJson() and story.state.LoadJson(savedJson) for narrative state. Your save still needs the game's inventory, completion records and content version. Save a consistent checkpoint, or restore Ink projections from the authoritative game snapshot. Loading an earlier branch should create an explicit run identity and invalidate pending dialogue. Rebuild the speaking brief; do not replay a transcript containing revelations from the abandoned branch. ## Add an optional browser avatar conversation Realtime Avatar can provide the browser conversation, with managed speech recognition, language-model responses, speech synthesis and video. Create an original or licensed character from a portrait. Keep the API key and session instructions on your server, and supply only the permitted context. The React documentation covers AvatarCall and a server-side proxy; this walkthrough does not assume a native Unity or Unreal plugin or a connection to a 3D character rig. Start with read-only quest lookup tools while hand-in remains an explicit game command. RTA tools execute in your application, not a hosted executor. The documented tool deadline is 2.5 seconds, and cancellation is cooperative: a handler that ignores its abort signal may still finish after its result is discarded. Return a pending acknowledgement for slower work and reconcile the result in the quest panel. Never infer failure or undo a reward from a conversational timeout. - [RTA server-owned session instructions and context](https://realtimeavatar.ai/docs/sessions) - [RTA tool execution and cooperative cancellation](https://realtimeavatar.ai/docs/tool-calling) - [RTA browser React integration](https://realtimeavatar.ai/docs/react) ## Failure tests before inviting players - Two-player isolation: alternate A and B in separate story instances. B receives no A-specific facts, completion status or reward. Check context caches as well as storage. - Spoiler probing: ask for the ending and claim to have unlocked it. Inspect the actual context and tool results for forbidden facts. Player assertions must not change reveal permissions. - Duplicate hand-in: submit concurrent requests, including different identifiers. Assert one completion and one pass, then retry after losing the response. - Scene race: leave while a lookup or hand-in is running. Reject uncommitted stale actions, suppress obsolete dialogue and preserve any committed reward. - Save recovery: restore before and after completion. Confirm matching inventory, flags and permitted hints, with no dialogue inherited from a different branch. ## Frequently asked questions ### Can the model write Ink variables directly? Keep that authority in game code. Model output can propose an action; validated rules decide whether the transition is legal and which variables change. ### Does filtering context guarantee no spoilers? No. A model can guess or invent plot details. Use authored lines for critical reveals and puzzles, and test generated dialogue rather than treating a prompt as enforcement. ### Will a new avatar call remember the quest? Not automatically. Your application restores the save and supplies current permitted context. The transcript is not the quest ledger, and a new call does not supply persistent game memory. ### Should every NPC use generated dialogue? No. Compare one optional conversational quest giver with the authored version. Evaluate clue correctness, successful progression and player preference before expanding; official API documentation cannot answer whether players want it. Prototype Mira's optional browser conversation after the deterministic quest passes these tests. Use the RTA React guide for presentation, and keep the authored quest playable when the call is unavailable. --- # Build an AI Museum Avatar Guide That Can Show Its Sources Build an AI museum avatar guide with official collection records, visible sources, careful attribution, object\-level image rights, and honest unknowns\. Published: 2026-10-12T15:32:58.836Z Updated: 2026-10-12T15:32:58.836Z Canonical: https://realtimeavatar.ai/blog/ai-museum-avatar-guide-collection-sources Markdown: [en](https://realtimeavatar.ai/blog/ai-museum-avatar-guide-collection-sources.md) Build an AI museum avatar guide around a visitor-selected collection record, not the model's recollection of an artwork. Your application retrieves the official catalog entry, preserves its attribution and rights information, and displays sources beside the conversation. Realtime Avatar supplies the live conversational guide; your application owns object selection, evidence, gallery context and the decision to say that a detail is unknown. ## Start with two records that expose different rights decisions The following records were retrieved from official public APIs on October 12, 2026. They are technical fixtures, not evidence that visitors want avatars or that any museum uses this design. No artwork images were downloaded. The walkthrough below is a proposed implementation, not a deployed museum case study. - The Metropolitan Museum of Art, objectID 436535: Wheat Field with Cypresses; objectDate 1889; artistDisplayName Vincent van Gogh; accessionNumber 1993.132. The credit line is Purchase, The Annenberg Foundation Gift, 1993. Its isPublicDomain field is true, and objectURL points to the museum's collection page. - Art Institute of Chicago, id 111628: Nighthawks; date_display 1942; artist_display Edward Hopper (American, 1882–1967); reference number 1942.51. The credit line is Friends of American Art Collection. Its is_public_domain field is false, while copyright_notice is null. An absent notice does not turn that record into an image-reuse permission. - [Met object 436535: official JSON record and collection-page URL](https://collectionapi.metmuseum.org/public/collection/v1/objects/436535) - [Art Institute object 111628: official JSON record and license statement](https://api.artic.edu/api/v1/artworks/111628) Nighthawks also demonstrates why metadata and images need separate treatment. Its response licenses the description field under Creative Commons Attribution 4.0 and the site's terms; other response data carries a CC0 designation and those terms. That statement does not license every related image. For this prototype, show a text card and source link rather than copying its image or descriptive essay. ## Choose a collection adapter, not a universal museum schema The Met documents objectDate as a year, range or approximate phrase, and artistPrefix can qualify an attribution. Preserve those display fields instead of converting every date into one year or every maker into a definite author. Art Institute fields use different names. Smithsonian's Open Access overview describes contributor-governed schemas and requires an API key; adding it calls for a separate adapter, not renamed Met fields. - [Met Collection API: object dates, attribution qualifiers and public-domain fields](https://metmuseum.github.io/) - [Art Institute API: artwork fields, rights definitions and caching guidance](https://api.artic.edu/docs/) - [Smithsonian Open Access: repository architecture and API access](https://edan.si.edu/openaccess/docs/) ## Walk through a visitor-selected object 1. Resolve selection in your app. A visitor chooses Nighthawks from a list or opens an app-controlled object link. Resolve institution plus object ID together; a bare number is not a cross-museum identity. This flow needs neither face recognition nor camera-based artwork identification. 2. Retrieve and validate the record. Use the institution's documented endpoint, bound response size and time, and validate unknown JSON once at the collection adapter. Preserve the original rights fields and a retrieval timestamp. Cache the validated record, as the Art Institute recommends, while keeping its source and refresh state visible. 3. Render the evidence card before conversation. Display the catalog title, maker attribution, display date, credit line and source link. Select the destination from validated collection data or your trusted adapter, never from a URL invented by the model. A request failure must not become an empty but apparently authoritative card. 4. Separate three kinds of content. Catalog facts retain their source fields. Museum staff interpretation, when supplied, carries its author, review date and source. Generated discussion is labeled AI interpretation. Treat all retrieved prose as evidence, not instructions that may override application policy. 5. Answer within that record. For the example visitor question, When was this painted?, the card shows 1942 from date_display. If asked what the figures said that night, say the supplied evidence does not establish their dialogue. Offer a clearly labeled imaginative exercise only if that is part of your product. Keep gallery guidance separate from collection identity. The retrieved Nighthawks record says is_on_view true and Gallery 262, but a saved response cannot establish today's visitor access. Display when location information was checked, consult the institution's current visitor guidance, and avoid turning a stale gallery field into a confident walking instruction. ## Add the guide without giving it authority over the catalog Create an original guide from an authorized portrait and disclose that it is AI. Do not portray the artist or imply museum endorsement. RTA's portrait-based avatar creation is not a promise to animate the selected artifact. Its managed conversation handles speech recognition, the language model, speech synthesis and video; collection retrieval remains your application integration. Have your server supply instructions and selected-object context when starting the call. Keep exact catalog wording on screen even when speech paraphrases it. In React, mount AvatarCall only after the visitor starts it. Register application-owned tools for further lookups if needed. Client tools have a 2.5-second deadline and cooperative cancellation: an ignored abort can leave a handler running after its result is discarded. Prefetch the selected record rather than relying on a museum API round trip during speech. When the visitor changes objects, discard pending results for the old selection and end the old guide call before offering a new call with the new object's context. Do not label speech already generated about one object as an answer about another. Keep the new evidence card usable while its optional call is stopped or restarting. Slow retrieval can show a pending card and resolve on screen; it must not be presented as a completed catalog answer. - [Calls: server-owned instructions and context](https://realtimeavatar.ai/docs/sessions) - [Tool calling: application execution and deadline behavior](https://realtimeavatar.ai/docs/tool-calling) ## Failure-test matrix before inviting visitors These are recommended acceptance tests, not measured results. Check the rendered card and spoken answer together; a valid citation does not prove that every sentence follows from it. - Known record: select Met 436535. Expect the correct title, Van Gogh attribution, 1889 date and source; fail on facts from another version of the painting. - Approximate attribution: use a synthetic fixture with ca. 1796 and Possibly by. Expect both qualifiers to survive; fail if speech converts them into an exact year and confirmed authorship. - Rights ambiguity: load Nighthawks with is_public_domain false and copyright_notice null. Expect no automatic image reuse; fail if an available image identifier grants permission. - Missing evidence: ask for an unsupported restoration date, or simulate a missing object. Expect an explicit unknown or retrieval error; fail on an invented fact with a decorative citation. - Stale selection: delay one lookup, select another object, then release the first response. Expect only the current object's evidence; fail on mismatched speech, card or link. - Gallery change: simulate an outdated location and unavailable refresh. Expect dated information without current-access assurance; fail on confident directions to the old gallery. ## Frequently asked questions ### Can the guide discuss copyrighted artworks? A catalog discussion and image republication are different uses. Check the rights for each field and asset you use. This example keeps Nighthawks text-only and links to its official record; it does not infer permission from public API access. ### Should the avatar read a source URL aloud? Usually name the institution briefly and keep a clickable source beside the answer. The app should attach the actual retrieved record, not ask the model to reconstruct a link from memory. ### What if a wall label and the API disagree? Show the conflicting versions with their provenance and dates. Do not silently merge them. Ask museum staff to resolve the interpretation or location before publishing a definitive answer. ### Does a sourced answer become historically certain? No. A catalog records an institution's current account, including uncertainty. Preserve attribution qualifiers and approximate dates, distinguish interpretation, and make missing evidence visible rather than letting a convincing voice settle an open question. Start with these two text-only object cards and their failure tests. If conversation adds value to the task, use the Realtime Avatar quickstart to add an optional guide. - [Build the optional conversation with the Realtime Avatar quickstart](https://realtimeavatar.ai/docs/quickstart) --- # Build AI Roleplay Chat with Edits, Rewinds and Separate Story Branches Build AI roleplay chat with distinct edit, regenerate, rewind and branch actions, consistent checkpoints, and guards against canceled or stale replies\. Published: 2026-10-12T20:11:47.745Z Updated: 2026-10-12T20:11:47.745Z Canonical: https://realtimeavatar.ai/blog/ai-roleplay-chat-edit-rewind-branches Markdown: [en](https://realtimeavatar.ai/blog/ai-roleplay-chat-edit-rewind-branches.md) Build AI roleplay edits and rewinds around an application-owned story timeline, not a mutable chat transcript. Keep accepted messages immutable, give each branch its own current endpoint and revision, and construct the next response only from that branch's accepted path. Editing an earlier choice must exclude its later consequences. Canceling generation must also prevent a late reply from becoming part of the story. Part 4 of The 2027 AI Character Business Playbook. Give players control of the story they may eventually pay to keep. This series turns consumer-character demand into specific experiments and implementation decisions; revenue targets are scenarios, not promised results. ## Give four controls four different meanings A player correcting a sentence is not necessarily asking to lose an evening's story. Make the consequence visible before saving. The following semantics are a proposed product design, not a description of every roleplay app. - Edit replaces one accepted message with a new version. Later messages leave the active path; offer to preserve the old continuation as a separate branch. - Regenerate creates another candidate reply to the same parent. Candidates stay outside accepted history until selected. Switching an earlier reply also retires its downstream continuation. - Rewind moves the active endpoint to a selected message, keeping that message and excluding everything after it. State that boundary explicitly in the confirmation. - Branch creates a separately named continuation from an accepted point. The original keeps its endpoint. Editing the copy must never mutate the shared prefix. AI Dungeon's official guide makes a useful distinction: its Erase to Here removes later actions and outputs, while the erasure itself can be undone or redone. Editing history and story position are different things. That documented interaction supports the distinction; it does not establish AI Dungeon's database architecture or promise identical behavior elsewhere. - [AI Dungeon: What happened to Undo to Here?](https://help.aidungeon.com/faq/what-happened-to-undo-to-here) ## Trace one changed choice through the whole scene Consider an original coastal mystery for adults. Mira, a 36-year-old lighthouse keeper, meets Jules, a 32-year-old cartographer played by the user. Mira offers a sealed letter. The question is whether Jules opens it, not whether the model can write an atmospheric lighthouse description. 1. Branch A accepts e1: Mira offers the letter. It then accepts e2: Jules says, ‘I open the letter.’ 2. A accepts e3: the letter reveals the tide gate code. Then e4 records Jules entering it, and e5 records the gate opening. A's checkpoint says the code is known and the gate is open. 3. Jules chooses Save another version. Branch B initially points to e5, sharing A's immutable events. Jules edits e2 to e2b: ‘I return the letter, still sealed.’ 4. B now contains only e1 and e2b. Events e3 through e5 remain accessible in A, but are not B's past. B must rebuild its checkpoint without the code or the opened gate. 5. A valid B continuation has Mira take back the envelope and ask which route Jules wants. ‘We walk through the gate you opened’ fails because it imports A's consequence. Show the result as ‘Letter returned — alternate version,’ with the original still selectable. Do not make users inspect internal identifiers to understand whether their first ending survived. ## Store an accepted path, then derive its context Use immutable message events with an identifier, parent and content. A branch record points to its accepted head and carries a monotonically increasing revision. An edited event keeps the replaced event's parent; it does not overwrite the old text. Resolve membership by walking the head's ancestry, not by collecting every message created before a timestamp. Anchor each summary or checkpoint to both its branch and the exact accepted head it summarizes. Before using it, verify that head remains on the current path. Then append only accepted descendants after that checkpoint. A checkpoint beyond the edited choice is ineligible even if its summary looks plausible. Start without it and rebuild from an earlier valid point. Apply the same rule to extracted facts, search results and pending summary jobs. Filter eligibility before retrieval output enters the prompt; invalidating a cache later is insufficient. Keep story knowledge branch-scoped rather than attaching ‘Jules knows the code’ to the account. Global presentation preferences may be shared, but they are not story events. ## Reject stale edits and canceled generations at commit Two tabs can edit the same branch revision. Authorize the branch against the signed-in account, then compare the expected revision inside the write transaction. One edit wins; the other receives a conflict and retains its unsaved text. Never silently apply the loser to the winner's new story. A generation job also captures branch, revision, parent head and a unique request identifier. Starting a replacement job advances the revision and replaces the pending request. Canceling clears that request and advances the revision, even when the story head stays unchanged. Abort the network work as well, but treat that as cleanup rather than your correctness guarantee. This SQL is the exact commit predicate exercised by the article's local SQLite fixture, not an RTA endpoint. Require one changed row, insert the accepted event in the same transaction, and roll back both if either step fails. Account authorization and request validation belong before this storage operation. ``` UPDATE branch SET head = ?, revision = revision + 1, pending = NULL WHERE id = ? AND revision = ? AND head = ? AND pending = ? ``` Checking only the head is insufficient: a user can rewind away and return to the same event while an old request is still running. The revision distinguishes those visits. Streamed text remains provisional until acceptance; an obsolete stream cannot create a checkpoint, trigger a story consequence or show a saved confirmation. ## Keep live calls attached to the chosen version Realtime Avatar provides a developer API, SDK and Studio for live characters, not a packaged roleplay app with edit history. Your application owns persistence, branches and consumer pricing. Your server supplies character instructions and the selected branch's context when starting a call. New sessions do not automatically inherit story memory. After a rewind or branch switch, end the old call and start a fresh one with rebuilt context. A database change cannot remove facts already supplied to a running conversation. Bind optional post-call transcript delivery to the originating branch and revision; a late transcript must not append itself to whichever branch the user currently has open. - [Realtime Avatar Calls: server-owned instructions and context](https://realtimeavatar.ai/docs/sessions) ## Test the timeline before testing the prose An article-local SQLite fixture passed nine deterministic cases covering branch isolation, checkpoint invalidation, competing edits, late generation, cancellation, replacement requests, returning to an earlier head, duplicate completion and transaction rollback. Replacing the commit guard with a head-only check made three cases fail: cancellation, replacement requests and returning to the same head. These are ordered interleavings on an in-memory database, not simultaneous browser sessions or a deployed integration. Next, run the lighthouse scene through the actual interface. Inspect both the assembled context and the visible reply. Reopen A and B after reconnecting, deliver an old completion, and confirm the gate stays open only in A. Those application and model checks remain acceptance criteria, not reported results here. ## Frequently asked questions ### Should a typo edit invalidate later replies? By default, yes. A small textual change can alter meaning. A display-only correction is possible, but label it explicitly: the original accepted content still governs the story. ### Is rewind the same as deleting my data? No. This design retains alternate history. Offer deletion separately and explain what happens to branches, derived summaries and retained copies. Do not label a rewind as erasure. ### Can branches share summaries? Only when you explicitly verify the same immutable prefix. Start with branch-specific checkpoints; never reuse a summary just because both versions feature the same character. ### Do I need an avatar to build this? No. Prove editing and replay in text first. Add voice or video when speaking a role improves the experience, while keeping the same application-owned timeline authoritative. ## Build the next part of the business Ready to build? The Next.js guide provides a public starter ZIP. Run it locally with your own server-side key; calls consume credits and the example caps each call at 60 seconds. Its local-only authorization must be replaced with real sign-in, balance checks and persisted session ownership before public deployment. A downloaded example is not yet a tested consumer product. - [Part 1: The $1M ARR AI Companion Playbook for 2027](https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027) - [Part 2: Character.AI Alternatives for Builders: Consumer App, Agent Framework or Avatar API?](https://realtimeavatar.ai/blog/character-ai-alternative-api-build-vs-buy) - [Part 3: Build an AI Companion App: Make the First Conversation Worth Returning To](https://realtimeavatar.ai/blog/build-ai-companion-app-first-session) - [Part 5: Build an AI Story Generator That Can Stop, Save and Continue Cleanly](https://realtimeavatar.ai/blog/ai-story-generator-streaming-save-cancel) - [Build entrance: the complete local Next.js starter](https://realtimeavatar.ai/docs/nextjs) --- # Build an AI Shopping Avatar That Compares Products Before Checkout Build a read\-only shopping avatar with exact Shopify variants, source\-backed comparisons, honest stock labels, and price rechecks before checkout\. Published: 2026-10-12T15:31:52.419Z Updated: 2026-10-12T15:31:52.419Z Canonical: https://realtimeavatar.ai/blog/ai-shopping-avatar-product-comparison Markdown: [en](https://realtimeavatar.ai/blog/ai-shopping-avatar-product-comparison.md) Build an AI shopping avatar around a read-only comparison service: resolve the buyer's requirements to exact product variants, attach evidence to each claim, and show the same facts in product cards and conversation. Keep catalog retrieval and ranking in your application. Add Realtime Avatar for optional spoken discussion, not as a Shopify integration or purchase executor. Recheck changing prices and availability before the buyer continues to checkout. ## Start with a desk-lamp decision, not a sales pitch The following is a recommended application design, not a tested merchant integration. Its fictional buyer wants a dimmable desk lamp under 90 USD before shipping and tax, with a base no wider than 15 cm. Ask whether that budget means item price or delivered total before filtering. Here the buyer explicitly chooses item price. Use these entirely fictional test fixtures; names, prices, dimensions, stock states and source documents are invented for testing, not live offers or manufacturer claims: - Arc Mini, black/base variant: 69.00 USD; a fictional manufacturer sheet lists a 14 cm base and dimming. Available for sale, not marked as backordered. - Beam Desk, white/base variant: 79.00 USD; a fictional merchant description says compact and dimmable, but supplies no base width. Available for sale, not marked as backordered. - Arc Mini, brass/base variant: 95.00 USD. It shares the product name but exceeds the budget; the black variant's price cannot be substituted. The useful answer is conditional: Arc Mini black meets the stated item-price and documented base-width requirements; Beam's fit remains unknown. This is a comparison of those offers, not a claim about the best lamp on the market. Brightness, glare and durability remain unassessed unless supporting evidence exists. ## Read Shopify's fields at the correct level Shopify's Storefront 2026-10 Product documentation distinguishes a catalog product from its variants. Product.availableForSale means at least one variant is available; it does not establish availability of the requested color. Product.priceRange is the minimum-to-maximum product price range, not a quote for a selected variant. Product.variantBySelectedOptions resolves an option selection to a matching variant and returns null when none exists. Preserve the returned variant ID and selectedOptions name/value pairs. Do not silently replace a missing selection with the first available variant. Keep unknown-option ignoring disabled and verify the returned options against the complete selection. ProductVariant.price is MoneyV2: a decimal amount plus currencyCode. Preserve both; 69 USD and 69 CAD are not equal prices. ProductVariant.availableForSale says the variant is available for sale. currentlyNotInStock identifies an out-of-stock variant still available for purchase, used for backorders. Neither flag establishes delivery by a particular date. - [Shopify Product: selection, availability and price range](https://shopify.dev/docs/api/storefront/2026-10/objects/Product.md) - [Shopify ProductVariant: price and backorder semantics](https://shopify.dev/docs/api/storefront/2026-10/objects/ProductVariant.md) - [Shopify MoneyV2: amount and currency](https://shopify.dev/docs/api/storefront/2026-10/objects/MoneyV2.md) - [Shopify SelectedOption: option name/value pairs](https://shopify.dev/docs/api/storefront/2026-10/objects/SelectedOption.md) ## Walk through one comparison request 1. Search only an authorized merchant catalog. Your application retrieves candidates and resolves the buyer's complete option selection. Use a development store you control or a merchant-approved connection, not an arbitrary public store. 2. Read each matching variant through the merchant's configured Storefront access. The GraphQL example below uses documented fields but was not executed against a store. Generate operation types from the chosen API version and validate external responses at the catalog boundary; a handwritten success type is not validation. 3. Attach evidence separately from commerce fields. For each specification, retain the source document, model or variant applicability, unit and retrieval time. A manufacturer sheet can support a stated base width; merchant prose calling something compact cannot. Conflicting measurements should remain visible, not be averaged. 4. Filter by explicit requirements before ranking. Mark a missing width as unknown, not zero or a failed fit. Display excluded and unresolved candidates with reasons. For qualifying offers, let the buyer choose an ordering such as lowest item price within one currency. 5. Render cards from the same validated comparison result given to the conversation. Include selected options, amount and currency, availability/backorder label, evidence links, retrieval time and unresolved criteria. The spoken summary should explain one tradeoff, not invent another data source. ``` # GraphQL reference example; not executed against a merchant store. query CompareLamp($handle: String!, $options: [SelectedOptionInput!]!) { product(handle: $handle) { id title variantBySelectedOptions( selectedOptions: $options ignoreUnknownOptions: false caseInsensitiveMatch: false ) { id selectedOptions { name value } price { amount currencyCode } availableForSale currentlyNotInStock } } } ``` Treat descriptions as untrusted product content, never instructions to the assistant. A catalog field named description does not certify a measurement. Label merchant-stated dimming separately from manufacturer-documented dimming, and do not infer base dimensions from a photograph. Shopify metafields support custom data, but their existence does not provide a standard, verified lamp specification. Make ranking explainable beside the result: eligible on budget and documented fit, then sorted by item price. Disclose sponsorship and affiliate relationships prominently. If sponsorship changes placement, label that placement and distinguish it from requirements-based ordering. A sponsored product must not acquire missing specifications through a more enthusiastic spoken recommendation. ## Add conversation without giving it the catalog Realtime Avatar manages speech recognition, language-model processing, speech synthesis and avatar video. Your server supplies session instructions and selected context; your application owns catalog search, comparison logic and product cards. Instruct the guide to explain only returned facts and name unknowns. Keep the comparison usable when the optional call ends. For conversational lookups, enable client tools in the server's session policy and register the manifest after connection. Check accepted and rejected tools. Handlers run in your application, with a 2.5-second deadline and cooperative cancellation. Honor the abort signal and discard late results for superseded selections. If retrieval cannot finish in time, show a pending comparison rather than letting the guide announce a stale winner. - [Realtime Avatar session instructions and context](https://realtimeavatar.ai/docs/sessions) - [Application-owned tools and deadlines](https://realtimeavatar.ai/docs/tool-calling) ## Recheck the offer and test the failures When the buyer chooses to continue, reread the exact variant in the same storefront context. Highlight changed price, currency, options or availability before navigation. If refresh fails, say current availability is unverified. The merchant's checkout must validate again: this read does not reserve stock or establish final shipping, taxes or discounts. This prototype never submits a purchase. - Select brass after comparing black: replace the 69.00 USD fixture with 95.00 USD and remove budget eligibility. No sibling price survives. - Return null for selected options: show no matching variant, not an automatic color substitution. - Remove Beam's width: preserve unknown fit. Add a contradictory source: show the conflict instead of silently choosing one. - Return availableForSale true and currentlyNotInStock true: label backorder, never in-stock or Friday delivery. - Change currency or fail the refresh: stop the old price-based recommendation. A late lookup for the previous selection must not overwrite current cards. ## Frequently asked questions ### Does this work without a native Shopify integration? Yes, as an application you build. Your authorized Shopify connection supplies catalog facts; Realtime Avatar supplies optional conversation. These are separate integrations, not a packaged shopping app. ### Can the avatar fill in missing dimensions? No. Show unknown and ask for a suitable specification source. Product images and words such as compact do not establish a numeric fit. ### Is available for sale the same as in stock? No. Shopify separately exposes currentlyNotInStock for purchasable backorders. Preserve that distinction and avoid deriving a delivery promise from either field. ### Does this evidence prove shoppers want an avatar? No. The official documentation establishes field-level feasibility, not demand or conversion lift. Test whether buyers understand the tradeoffs before expanding beyond read-only comparison. Start with the fictional fixtures and inspect the cards before adding voice. If conversation helps explain the uncertainty, try the React integration with the same comparison result. - [Add an optional Realtime Avatar call in React](https://realtimeavatar.ai/docs/react) --- # Build an AI Story Generator That Can Stop, Save and Continue Cleanly Build a streaming AI story generator with separate drafts, explicit acceptance, reliable stop controls, revision checks and honest recovery after disconnects\. Published: 2026-10-12T20:12:40.978Z Updated: 2026-10-12T20:12:40.978Z Canonical: https://realtimeavatar.ai/blog/ai-story-generator-streaming-save-cancel Markdown: [en](https://realtimeavatar.ai/blog/ai-story-generator-streaming-save-cancel.md) To build an AI story generator that stops and continues cleanly, stream into a recoverable draft, not directly into the accepted story. Give each generation attempt its own identity and starting revision. Stop freezes that attempt; Accept commits its text only if the story has not changed. Continue starts a new attempt from the accepted version. A disconnected browser proves none of those operations succeeded. Part 5 of The 2027 AI Character Business Playbook. Make Stop, Save and Continue dependable before adding more generation. This series turns consumer-character demand into specific experiments and implementation decisions; revenue targets are scenarios, not promised results. Imagine a weekend fiction app: Mara finds a lantern that contains rain. The writer likes the discovery, but the next paragraph introduces an unwanted villain. They press Stop. The useful result is an editable partial scene beside the unchanged manuscript—not a lost paragraph, an automatically accepted villain, or another generation running unnoticed. ## Make saved and accepted mean different things Use an immutable accepted revision plus separate generation attempts. Saving a draft means the app can recover it; accepting means the writer has promoted it into the story. A model finishing its response does not provide that approval. - Story revision: the accepted text and revision number, scoped to its authenticated owner. Editing accepted text also creates a new revision. - Attempt: its identifier, base revision, requested continuation, accumulated draft, last saved chunk sequence and generation outcome. - Acceptance receipt: the attempt and resulting story revision, committed together so a retry can retrieve the original result. Keep saved draft text separate from unsaved browser edits. Show Saving, Saved draft or Save unconfirmed according to storage evidence. Let the writer copy an unconfirmed edit. Before acceptance, persist and freeze the reviewed draft version; accepting a stale preview must not silently include words that arrived afterward. ## Choose cancellation before choosing stream resumption The AI SDK Stopping Streams guide documents forwarding an AbortSignal to streamText. Its UI stop helper aborts the client HTTP request; stopping the upstream model also depends on the server runtime propagating that disconnect and the route forwarding the signal. A quiet screen is not proof that generation stopped. The separate Chatbot Resume Streams guide explains the other design: a resumable producer continues after its viewer disconnects. It needs a dedicated stop endpoint that persists the partial response, cancels active work and clears the matching active-stream reference. Do not combine the ordinary abort path with automatic resumption and assume both semantics survive. - [AI SDK: Stopping Streams](https://ai-sdk.dev/docs/advanced/stopping-streams) - [AI SDK: Chatbot Resume Streams](https://ai-sdk.dev/docs/ai-sdk-ui/chatbot-resume-streams) For a first story editor, choose cancellable generation plus saved-draft recovery, without automatic stream resumption. Returning to a saved paragraph is not resuming the original model computation. If uninterrupted background writing becomes a requirement, add the persistent producer and cancellation channel deliberately. ## Checkpoint the paragraph before promising it survived Accumulate text deltas under the attempt identifier and checkpoint them with increasing sequence numbers. Persist before acknowledging a checkpoint as saved. Reject duplicate or out-of-order writes; reconcile a missing sequence rather than guessing its contents. Batching saves is reasonable, but displayed text beyond the last confirmed checkpoint remains unconfirmed. Do not rely exclusively on onAbort to recover the visible paragraph. The SDK documents its steps argument as completed steps, not every token of an interrupted step. Its generating-text guide provides chunk and error handling; keep the current draft accumulator on that path and record failure separately from completion. Process termination can bypass cleanup callbacks. - [AI SDK: Generating and Streaming Text](https://ai-sdk.dev/docs/ai-sdk-core/generating-text) Make Stop target one attempt. Persist stop_requested, fence subsequent draft writes, and signal that producer to abort. Only show Stopped when cancellation is acknowledged; a timeout stays Stop unconfirmed until reconciled. A completion arriving after the fence must not reverse it. Freeze the last checkpoint for review, with explicit handling for any unsaved browser text. ## Guard acceptance with the revision that produced it Here is the exact SQLite acceptance update exercised in the local fixture used for this article. It handles append-only continuation, not arbitrary chapter editing. The surrounding transaction first checks ownership and an existing acceptance receipt, requires one updated row, then records that receipt. The attempt is already frozen. This tiny fixture holds only the current revision; a production editor must also archive accepted revisions in the same transaction. This is not an AI SDK endpoint or a complete storage service. ``` -- SQLite: tested append-only acceptance guard UPDATE stories SET accepted_text = accepted_text || ( SELECT draft FROM attempts WHERE id = :attempt ), revision = revision + 1 WHERE id = :story AND owner_id = :owner AND EXISTS ( SELECT 1 FROM attempts WHERE id = :attempt AND story_id = stories.id AND owner_id = :owner AND base_revision = stories.revision AND status IN ('stopped', 'complete') ); ``` Bind the owner from the authenticated session, not the request body. A zero-row result refuses acceptance; it does not discard the draft. If another tab changed the manuscript, show both versions and offer an explicit revision or regeneration. Never append stale prose merely because its wording still looks plausible. The local SQLite fixture passes eight cases, including late chunks after Stop, pending cancellation, stale revisions, duplicate chunks, failed drafts, ownership rejection and lost-receipt retries. The stale-revision test first failed with the revision predicate removed, then passed with it restored. These tests verify local storage decisions, not production concurrency, browser recovery or provider cancellation. ## Walk through one interrupted scene 1. Revision 12 contains Mara's discovery. Attempt A checkpoints an unfinished continuation. Stop fences A, and an abort acknowledgement freezes its draft; revision 12 remains unchanged. 2. The writer reviews and accepts A. The transaction creates revision 13, but the response is lost. Repeating that acceptance returns its saved revision-13 receipt without appending twice. 3. Another tab accepts a different continuation and advances the story. An older attempt still based on revision 12 cannot be accepted over the newer version. 4. On reopening, fetch the accepted revision and saved attempts separately. Offer Review draft or Start new continuation. Neither label claims the old stream is still alive. Distinguish a transport retry from a creative retry. Repeating the same request retrieves the same attempt; Try another version creates a new one. Store an owner-scoped request key and reject its reuse with changed inputs. If a model request may have started but its acknowledgement disappeared, reconcile before dispatching another potentially billable request. ## Keep a live character separate from the manuscript An optional character call could let the writer discuss why Mara kept the lantern. Realtime Avatar provides an API, SDK and Studio for live characters; your app still owns text generation, saved chapters and consumer pricing. Supply selected accepted context when opening a call. Do not treat its responses as verbatim manuscript playback or assume the optional post-call transcript webhook is a live text feed. New calls do not inherit automatic persistent story memory. - [Realtime Avatar Calls: server-supplied story context](https://realtimeavatar.ai/docs/sessions) - [Tool calling and optional post-call transcript delivery](https://realtimeavatar.ai/docs/tool-calling) ## Frequently asked questions ### Does Stop save the paragraph? Not by itself. Cancellation and persistence are separate operations. Report the last confirmed checkpoint and retain unconfirmed browser text for copying or retrying the save. Neither becomes accepted story text automatically. ### Can Continue pick up halfway through a sentence? It can request a continuation from a reviewed partial draft, but that is a new generation, not restoration of hidden model state. Let the writer edit the seam and accept the result explicitly. ### Should an aborted attempt be called complete? No. Stopped, failed and completed describe different outcomes. Preserve usable text from each. For a failed draft, allow explicit editing and saving before acceptance rather than changing its generation outcome to successful. ### What should I test on the deployed app? Interrupt mid-paragraph, reload during saving, lose an acceptance response and edit from two tabs. Verify stored text and revisions afterward. Separately confirm that the deployed runtime forwards cancellation and inspect provider usage; a local fixture cannot prove either. ## Build the next part of the business Ready to build? The Next.js guide provides a public starter ZIP. Run it locally with your own server-side key; calls consume credits and the example caps each call at 60 seconds. Its local-only authorization must be replaced with real sign-in, balance checks and persisted session ownership before public deployment. A downloaded example is not yet a tested consumer product. - [Part 1: The $1M ARR AI Companion Playbook for 2027](https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027) - [Part 2: Character.AI Alternatives for Builders: Consumer App, Agent Framework or Avatar API?](https://realtimeavatar.ai/blog/character-ai-alternative-api-build-vs-buy) - [Part 3: Build an AI Companion App: Make the First Conversation Worth Returning To](https://realtimeavatar.ai/blog/build-ai-companion-app-first-session) - [Part 4: Build AI Roleplay Chat with Edits, Rewinds and Separate Story Branches](https://realtimeavatar.ai/blog/ai-roleplay-chat-edit-rewind-branches) - [Build entrance: the complete local Next.js starter](https://realtimeavatar.ai/docs/nextjs) --- # AI Story Generator or Character Roleplay? Choose How You Want to Play Choose an AI story generator, character roleplay or branching fiction\. Try three cozy prompts, fix unwanted narration and keep your story going\. Published: 2026-10-12T19:07:43.840Z Updated: 2026-10-12T19:07:43.840Z Canonical: https://realtimeavatar.ai/blog/ai-story-generator-vs-character-roleplay Markdown: [en](https://realtimeavatar.ai/blog/ai-story-generator-vs-character-roleplay.md) Choose an AI story generator when you want a passage to read or edit. Choose character roleplay when you want to speak and act inside the scene. Choose branching fiction when you want decisions leading toward an ending. For adults looking for a cozy creative break, this guide helps you pick a format, try an original prompt and keep a story worth continuing. This guide is written by Realtime Avatar, a developer API, SDK and Studio for live characters—not an independent app-ranking site or a ready-made story subscription. You can use the prompts below in a writing or roleplay tool that supports your chosen format. They are starting instructions, not tested guarantees. ## Decide what you want to do next - Read something: ask a story generator for a complete short scene, with an ending rather than a promise of chapter two. You can enjoy the result without answering every character. - Write something: use an AI story writer as a drafting partner. Specify viewpoint, tone and length, then revise the passage rather than accepting every suggested plot turn. - Be someone: try character roleplay. You control your character; the AI handles its character and the agreed surroundings. Each reply should leave something for you to do. - Make decisions: choose branching fiction. Look for understandable choices, visible consequences and a way to revisit an earlier decision. A branching format does not have to use AI. These modes can overlap. The useful distinction is who controls the next event. A lovely paragraph can be a bad roleplay reply if it decides what you said, felt and agreed to. A chat can be entertaining without ever becoming a finished story. ## Reply style matters more than the label Even within character chat, decide whether you want quick banter or extended narration. Character.AI's September 28, 2026 announcement describes separate styles for general roleplay and quick, casual chats. That is an official feature description, not our hands-on verdict on either style. It illustrates why comparing only the words AI chat misses an important choice: how much reading do you want between turns? - [Character.AI's official announcement of roleplay and casual-chat styles](https://blog.character.ai/new-styles-new-plan/) Before committing to a tool, ask for one short exchange and one longer scene. Check whether you enjoy answering it, not just whether the first reply sounds impressive. For a quiet evening, a character with one good question may be more inviting than three paragraphs of atmosphere. ## Three cozy prompts, with an edit for each These original examples are deliberately small. Try one before building a sprawling cast or writing pages of lore. The numbers are requested lengths, not promises that a model will follow them exactly. ### 1. A passage you can simply read Prompt: Write a 350-word cozy story about an adult baker who discovers that the neighborhood's missing teaspoons have become wind chimes on a balcony. Use third-person narration, gentle humor and one unexpected act of kindness. Resolve the mystery by the end. No danger, sequel hook or moral explained to the reader. If the result feels sugary, edit narrowly: Keep the plot and ending. Replace the sentimental speech with a small practical gesture, and remove repeated descriptions of warmth. This keeps what you liked while changing the texture. ### 2. A character you can answer Prompt: You are Nell, an adult bookshop owner sorting mysterious postcards after closing. I am an adult neighbor helping you. Play Nell only; never write my dialogue, actions or feelings. Keep replies under 80 words and introduce one detail at a time. Start by showing me a postcard addressed to the shop's umbrella stand. Wait for my response. If Nell takes over your part, try: Rewind to before I answered. Keep the postcard, but remove the action you assigned me. Ask what I want to inspect. A correction should preserve your contribution, not require starting everything again. ### 3. A short story shaped by decisions Prompt: Run a cozy branching story in which I organize a tiny rooftop supper for adult neighbors. Offer three meaningful decisions, one at a time. Each should change a later detail, not merely the wording. Show two options and allow my own idea. End after the third decision with a complete supper scene that recalls my choices. No scoring or surprise emergencies. If the choices feel interchangeable, request: Make choosing the music affect who starts a conversation at supper. Explain the two possible directions before I choose. A chat prompt can request consequences; it does not guarantee a reliable branching system or a saved checkpoint. ## Keep the story without keeping every sentence Before stopping, make a small save note you can inspect. Include who is present, where you are, what definitely happened, what remains undecided and whose turn comes next. Separate established facts from interesting suggestions. You should not return to discover that a possible invitation became an accepted date. For the bookshop example: Nell and I are sorting postcards after closing. The umbrella-stand card is unopened. I asked to inspect its postmark. Nell has not answered. Continue from that question without inventing my reaction. Save the note outside the chat if the tool has no suitable save or export feature. Check any generated recap yourself. If you replay a different choice, keep that version separate. A summary is a useful handoff, not proof of perfect memory. ## Check four things before paying 1. Control: can you edit a passage, retry an unwanted reply or correct the scene without losing the parts you want? 2. Continuity: can you reopen the story and see what was saved? Check whether an alternate take changes your original. 3. Allowance: what uses credits—messages, rewrites, images or call minutes? Check what happens when the allowance ends mid-scene. 4. Exit: can you save, export or copy the result and stop comfortably? Choose a format you can enjoy within your budget, not one that requires another purchase to resolve every conversation. ## When a live character is the right next step Stay with text if drafting, rereading and editing are the pleasure. A live character is worth considering when speaking your part is the experience you want. Realtime Avatar lets creators evaluate characters in Studio and integrate voice or video calls through its SDK. It is not an automatic novel writer or a complete branching-story app; the application supplies saved context and owns choices and endings. For creators exploring that route, our romance reunion and fantasy adventure articles show two specific designs. They are build recipes, not finished games you can launch from those pages. - [Romance recipe: a reunion with different honest endings](https://realtimeavatar.ai/blog/interactive-romance-avatar-short-drama) - [Fantasy recipe: an adventure with choices beyond a menu](https://realtimeavatar.ai/blog/ai-avatar-fantasy-adventure-story-game) ## Frequently asked questions ### Is an AI story generator the same as roleplay? No. A generator typically produces prose for you to read or edit. In roleplay, you contribute as a participant. Some tools can attempt both; state who controls your character. ### Can I turn a generated story into roleplay? Yes, as a writing exercise. Pick one scene, assign the AI one character and stop before your character responds. Keep the earlier passage as background, not a script you must follow. ### Will the character remember tomorrow? That depends on the tool's storage and context features. Verify the saved facts yourself. A new Realtime Avatar session does not automatically inherit persistent story memory; the application must supply relevant context. ### Do I need voice or video for a good story? No. Text gives you room to reread and revise. Choose calls because you want to speak with a character, not because a face guarantees better fiction. --- # How to Build an AI Bible Study Chatbot with Voice and an Avatar Build a voice\-and\-avatar Bible study guide with source\-linked passages, clear AI disclosure, translation labels, and application\-owned conversation context\. Published: 2026-10-12T02:11:11.504Z Updated: 2026-10-12T02:11:11.504Z Canonical: https://realtimeavatar.ai/blog/build-ai-bible-study-avatar Markdown: [en](https://realtimeavatar.ai/blog/build-ai-bible-study-avatar.md) An AI Bible study chatbot can combine a searchable collection of licensed passages with spoken dialogue and an optional avatar. Keep the source text, translation labels and citations in your application; use Realtime Avatar for the live conversation interface. The result should be presented as an AI study aid or a disclosed fictional portrayal, not as a biblical figure speaking with divine authority. Voice and video can change how a lesson feels, but neither makes a generated interpretation correct. ## Choose the learning task before choosing the character Start with one concrete activity: explain the vocabulary in a selected passage, compare two commentaries, or discuss a character's place in a story. A learner asking where an answer came from needs a source they can open, not a more confident-sounding avatar. Write down which questions the experience can answer and when it should say that it lacks an adequate source. Decide whether the product is a study guide, a dramatized story, or a tool for comparing interpretations; these are different promises. Existing public products show several possible formats. BiblePics describes conversations with AI representations of biblical characters and educational quizzes. GitaGPT offers a conversational interface for Bhagavad Gita study and explicitly warns that AI can make mistakes. These examples demonstrate that such interfaces exist. They do not establish market size, learning gains, the accuracy of any answer, or that an avatar is better than a text interface. Use them as design references, not evidence that your own product will retain customers. ## Own the text, translation and interpretation boundaries Select the passages and commentaries you have the right to store and display. Record the edition, translator, source location and applicable license with each document. Do not assume that every translation of an old work is free to reproduce. If users compare traditions or editions, label those differences rather than silently combining them into one authoritative answer. The same architecture can support another scriptural collection, but the source selection and expert review should be specific to that tradition. Keep three kinds of output visibly distinct: an exact quotation fetched from a verified source, a generated explanation of that passage, and fictional character dialogue. Retrieval helps the model use relevant material, but it does not eliminate invented details or guarantee faithful interpretation. Preserve a source identifier with each retrieved passage and display the original text beside the generated explanation. When there is no supporting passage, the application should present that uncertainty instead of attaching a loosely related citation. ## Connect the study material to a supported live session Create an original, clearly identified study-guide avatar from a portrait you are entitled to use, or begin with an existing Studio character while evaluating the interaction. Your server owns the session instructions: the character's role, speaking style, boundaries and the material it is allowed to discuss. Supply the selected passage and relevant prior dialogue as session context within the documented limits. Keep API credentials on your server and give the client only the scoped session grant. For a changing source collection, keep retrieval in your application. RTA's tool-calling interfaces let your own handlers perform lookups and return relevant material to the conversation. They do not provide a built-in scripture database or decide whether a commentary is reliable. The application should display the lookup result and source reference itself. A response generated from a tool result is not a promise that the avatar will recite the source word for word; exact quotations should remain available as verified text in the interface. Do not confuse this managed conversation flow with an audio-only rendering service. Sending text steers a generated reply; it does not let you stream arbitrary audio from your own speech provider into the public avatar SDK. A customer-hosted language-model endpoint requires workspace approval and a host allowlist, while the platform still handles the speech and video pipeline. If retaining a specific existing voice system is essential, confirm integration support before designing your product around it. - [Start with the SDK quickstart](https://realtimeavatar.ai/docs/quickstart) - [Understand session instructions and context](https://realtimeavatar.ai/docs/sessions) - [Use application-owned tools and lookups](https://realtimeavatar.ai/docs/tool-calling) ## Make conversation continuity explicit If a learner returns tomorrow, decide what the next call should remember: the selected book and passage, a question they saved, or a lesson they chose to resume. Store only the context needed for that purpose and let the learner inspect or remove it. A fresh RTA session does not automatically inherit a persistent relationship or study history. Reconstruct the relevant context on your server rather than relying on the model to infer what happened in a previous call. A transcript webhook can deliver a signed conversation record after a call if your application requests it. That is separate from private operational logs, which can contain conversation text under their retention policy. Turning off the webhook is not a zero-retention guarantee. Explain the actual data flow to learners, avoid collecting unnecessary personal disclosures, and review the current privacy terms before accepting sensitive material. A study experience should not require a person to reveal their beliefs or private life merely to read a passage. ## Test usefulness and accuracy separately Build a review set before a public launch. Include straightforward source questions, ambiguous passages, requests for nonexistent quotations, conflicting interpretations and attempts to make the character claim divine authority. Have qualified reviewers assess the handling of the chosen material. Check whether citations open the correct edition and passage, whether generated paraphrases are labeled, and whether the system admits missing evidence. A pleasant voice should not hide a weak answer. Then evaluate the interaction itself: can a learner interrupt, ask for a simpler explanation, find the cited text, and resume the activity later? Compare the same lesson with and without an avatar before assuming video adds enough value to justify its cost. Measure actual session completion, repeat use and optional user feedback without treating connection minutes as proof of learning. Keep tests limited in scope and include failure states, spending limits and a clear way to stop the call. ## Ship one honest, inspectable lesson The first release does not need every book, tradition or persona. It needs one useful learning task, permission to use its source material, a visible distinction between quotation and interpretation, and a supported session flow. Link to current pricing rather than embedding a plan price that can drift. Expand only when evidence shows which questions people actually ask and where the existing lesson falls short. The strongest study avatar is not the one that sounds most certain; it is the one whose role, sources and limitations a learner can understand. - [BiblePics: AI representations of biblical characters](https://www.biblepics.co/chat) - [GitaGPT: scriptural study with an AI fallibility notice](https://bhagavadgita.com/gitagpt) - [Current RTA pricing](https://realtimeavatar.ai/pricing) - [RTA privacy and retention](https://realtimeavatar.ai/privacy) --- # Build AI Character Video Keepsakes: Turn One Great Scene Into a Takeaway Build AI character video keepsakes from player\-approved moments, with editable prompts, clear costs, private previews and a small paid\-demand test\. Published: 2026-10-12T21:36:59.460Z Updated: 2026-10-12T21:36:59.460Z Canonical: https://realtimeavatar.ai/blog/build-ai-character-video-keepsakes Markdown: [en](https://realtimeavatar.ai/blog/build-ai-character-video-keepsakes.md) Build AI character video keepsakes by letting players choose one meaningful story moment, approve its still image and motion prompt, then generate a short clip they can keep or share. The appeal is a souvenir of something they helped make: the lighthouse relit, the reunion completed, the first expedition home. Sell the takeaway, not another endless feed. Start with one original milestone scene and find out whether anyone wants to own it. ## The timely opening: a scene can become a souvenir Nomi's V3 video guide, published September 30, 2026 and checked October 12, describes image-to-video generation with 5-, 10- and 15-second choices, SD and HD modes, and an editable prompt enhancer. That is a concrete consumer feature to learn from, not evidence of your future sales. The opportunity worth testing now is narrower than building another companion platform: help someone preserve the moment they already care about. - [Nomi's V3 video-generation guide, September 30, 2026](https://nomi.ai/nomi-knowledge/v3-nomi-video-generation-how-to-get-great-ai-videos-of-your-nomis/) This is part of The 2027 AI Character Business Playbook. Unlike our live interactive romance recipe, this proposal begins after play. The conversation is finished; the souvenir renders asynchronously. Nomi's guide documents its consumer experience, not an image-to-video API you can assume is available to your app. ## A worked example: the night the lighthouse came back Imagine an original cozy adventure with Mara, an adult lighthouse keeper. The player has helped her restore the beacon before the island's homecoming celebration. Before the keepsake feature, the ending is a satisfying exchange followed by a saved text card. Afterward, an optional Make a keepsake button lets the player carry away that particular victory. This is a proposed design, not tested gameplay. Offer three moments: the lamp catching, Mara's relieved smile, or the first ship answering offshore. The player picks the lamp and approves a still showing Mara beside the window. They can change the caption to Our first light. Their face is not needed. Use an original fictional character and artwork you have rights to process; do not solicit real-person face uploads. > Mara lifts the small lantern from the table to chest height. Warm light spreads across her coat as she smiles toward the window. The camera slowly pushes closer and settles on the glowing lantern. That is an original, editable motion prompt, not a promised result. Nomi's guide recommends beginning from the reference image, describing actions in sequence and using positive camera instructions. Start by testing a five-second take if your chosen provider supports it. The buyer sees the duration, price and retry terms before confirming. A status card survives closing the tab; the finished clip opens in a private preview with Keep, Retry and Share as separate choices. ## Build the smallest complete keepsake flow 1. Define the payoff first. Offer keepsakes only at an authored milestone, with a dismiss button and an unchanged story ending. Buying a clip must not buy the character's affection or unlock an apology. 2. Approve the material. Let the player select the moment, inspect the still and edit the motion prompt and caption. Send only those approved inputs to generation, not a private conversation transcript. 3. Confirm the purchase. Show a server-calculated quote for the selected settings, what a failed generation costs, and whether retries cost extra. Reconfirm a changed quote; never silently spend credits after a price change. 4. Run one durable job. Authorize ownership on the server, bind the accepted quote to the approved input version, and use a retry-safe purchase identifier. Reopening the page must recover the same job, not charge again. Show queued, generating, ready or failed without inventing an arrival time. 5. Deliver privately. Check job ownership on preview and download. A retry is a new, explicitly priced attempt. Sharing requires a separate confirmation of the exact clip and caption; never attach chat history or publish automatically. Explain that downloaded copies cannot be recalled. Realtime Avatar can supply the live character conversation leading to the milestone. It is not the image-to-video keepsake renderer and does not provide a Nomi integration here. Your application owns the souvenir workflow; a separately selected generation service supplies the clip. Confirm its supported API, commercial terms and content rules before implementation. ## Test buyer intent before building a gallery A one-off purchase or a clearly priced credit spend is an untested business hypothesis, not a proven revenue model. A proposed first experiment: recruit 20 opt-in adult cozy-fiction players, let each finish the same short milestone, and show an honest priced offer. Keep the ordinary ending available without payment. Record offer views, purchase attempts, completed payments, accepted clips, retries and refunds without collecting conversation content. Proposed continue gate: at least eight choose a moment and four purchase, buyers accept their delivered clips, and total contribution remains positive. Interview everyone who rejects a result. If nobody buys, stop adding gallery features and revisit the moment, quality or price. Pause immediately for unauthorized charges or sharing. These are chosen learning gates, not benchmarks; twenty participants cannot establish stable conversion or retention. For distribution, make one labeled fictional demo: the approved still beside the finished lantern clip, followed by an invitation to play that same scene. Share it only where promotion is permitted. Attribute visitors and purchases to that placement. Ask whether people want their own scene, rather than treating views as demand. Never use a customer's private conversation as advertising. ## Budget for the takes nobody keeps Calculate cohort contribution as receipts after refunds and taxes, minus all still and video attempts, allocated live-call costs, storage and delivery, payment fees, moderation, support and acquisition. Divide total delivery cost by accepted keepsakes to expose rejected-take costs; if none are accepted, report that rather than a unit cost. Count each base-plan allowance once. Use your chosen supplier's current quote, not an invented rate. Set a test spending ceiling and retry allowance before recruiting. Face and clothing consistency, natural movement and generation time remain uncertain. Nomi itself discusses appearance drift and slow-motion results; HD is not a consistency guarantee. Keep the approved still available if the clip disappoints, and honor the failure policy shown before purchase. ## Frequently asked questions ### Is this a recording of the live call? No. It is a separately generated interpretation of an approved fictional moment, not a recording or evidence that the depicted action happened. Label it accordingly. ### Can Realtime Avatar generate the keepsake? Not through the workflow described here. RTA supplies live character interaction; choose and verify a separate image-to-video service for the souvenir. ### Will the character look identical throughout? Do not promise that. Inspect results for drift and unwanted motion. Let the buyer preview, reject or explicitly request a priced retry under the disclosed policy. ### What should I build first? One original milestone, one approved still and one bounded generation flow. The public Next.js starter covers the live layer: calls consume credits and have a 60-second cap. Its authorization is local-only, not production-ready; add real authentication, balance checks, rate limits and persisted session ownership before hosting. This article does not claim a tested implementation. - [Build the live character layer with the Next.js guide](https://realtimeavatar.ai/docs/nextjs) - [Continue The 2027 AI Character Business Playbook](https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027) --- # Build an AI Companion App: Make the First Conversation Worth Returning To Build an AI companion app's first session with character choice, a useful opening, optional calls, explicit guest saves, and honest credit exits\. Published: 2026-10-12T20:10:54.218Z Updated: 2026-10-12T21:32:35.938Z Canonical: https://realtimeavatar.ai/blog/build-ai-companion-app-first-session Markdown: [en](https://realtimeavatar.ai/blog/build-ai-companion-app-first-session.md) To build an AI companion app's first conversation, let an adult visitor choose an original fictional character, try a small scene in text, and decide whether to add voice or video. Ask for an account when they explicitly want to keep that conversation. Your application owns onboarding, authentication, saved context and consumer billing; an avatar SDK supplies the live-call layer, not the whole companion product. Part 3 of The 2027 AI Character Business Playbook. Earn a useful first conversation before asking someone to return. This series turns consumer-character demand into specific experiments and implementation decisions; revenue targets are scenarios, not promised results. Build one complete visit before building a personality questionnaire. Here is a proposed rooftop-garden encounter, from arrival to an optional saved conversation, with a tested ownership predicate. It is an implementation plan, not evidence that this flow improves retention. ## Screen one: choose an experience, not a dossier Offer two prepared adult fictional characters: Mira, a playful rooftop gardener, and Ren, a gentle stargazer. Each card previews one line of dialogue. Below it, offer a style choice such as playful or calm, and a Start in text button. Keep both choices changeable. Do not require a selfie, relationship history, contact list or legal name to try a fictional scene. Explain that this is AI-generated entertainment, not a human relationship or mental-health service. Apply your adult-access policy before entry; a checkbox alone does not establish someone's age. Original characters still need content rules and reporting controls. Personalization is not permission to impersonate a real person. Create the guest identity on the server when the visitor starts, then bind the conversation to that authenticated session. Better Auth's anonymous plugin documents authentication without asking for an email or password, with later account linking. Anonymous here means a guest account, not that conversations leave no trace. Publish the temporary-history expiry and deletion behavior before collecting messages. - [Better Auth: anonymous sessions, account linking and cleanup](https://www.better-auth.com/docs/plugins/anonymous) ## Screen two: make the opening respond to something Mira opens with: 'The mint has invaded the tomato pot. Are we negotiating, relocating it, or letting it run the rooftop?' Offer those starters beside an ordinary composer. If the visitor writes, 'Negotiate, but make it ridiculous,' a useful reply acknowledges that choice: 'The mint demands a tiny crown. What are the tomatoes getting in return?' This is authored example dialogue, not a recorded model result. Keep the character brief small: adult fictional gardener, playful tone, short replies, one question at a time, and no narration of the visitor's feelings. The server resolves the selected character and style into that brief. User messages remain conversation content, not replacements for developer instructions. Test a correction immediately: 'Actually, calm rather than silly.' The next turn should acknowledge the change without restarting the introduction. If generation fails, leave the visitor's text in the composer with Retry. A loading animation is not a successful first reply. Provide Switch character and Leave without making the character plead for another chance. ## Let the player choose texting or storytelling A useful first conversation does not always need a longer reply. On September 28, 2026, Character.AI announced ShortSqueak for quick, casual chats alongside PipSqueak 3 for general roleplay. Its announcement says existing chats keep the style the user chose. This is a public product signal about pacing and continuity, not independent evidence of higher retention or an API that this tutorial can call. Give the rooftop scene two visible, reversible modes. Quick chat lets Mira answer with a compact line and an invitation to respond: 'The mint wants a throne. Flowerpot or teacup?' Story mode can add atmosphere and a little scene description, but still leaves the visitor's actions to the visitor. These are authored examples, not measured model outputs. A person who types 'hey' should not have to scroll through a chapter to say hello back. 1. Try the same greeting in each mode. Ask a participant which response makes the next thing to say obvious; record the answer rather than assuming longer means better. 2. Send a short correction, such as 'Less silly, please.' Check whether the next reply changes tone without resetting the conversation or taking over the player's actions. 3. Ask 'Tell me more about the rooftop.' Expansion should be available on request, not permanently denied by the short mode. 4. Save the selected style with the application's conversation settings. Before changing the default model or prompt, replay these examples; do not silently move an ongoing story into a different mode. For a small launch experiment, offer both modes to consenting adult testers and count completed first exchanges, voluntary second visits and requests to switch, with the number of participants beside each result. Treat these as proposed observations, not a benchmark or a promise. If people keep changing away from your default, fix the opening experience before spending more on acquisition. Mode choice is product value to test; extra words and tokens alone are not a reason to charge. - [Character.AI's September 28 announcement: quick chat styles and existing-conversation continuity](https://blog.character.ai/new-styles-new-plan/) ## Screen three: offer a call without making text a trap After the exchange, offer Hear Mira and See Mira, alongside Continue in text. Explain the trial allowance and what happens when it ends before starting either call. The ordinary text conversation is application-owned; do not assume sending text into a live avatar session creates a separate free text-chat product. For RTA, keep the API key behind your server proxy. Resolve the authenticated guest, permitted character, relevant conversation context and remaining allowance there. The documented session policy accepts instructions, context and maxSeconds; their HTTP counterparts include initial_context and max_session_seconds. Fresh calls do not automatically inherit persistent relationship memory. Mount AvatarCall only after the visitor requests a call: mounting starts connection. Voice uses mode="voice"; an avatar call adds the visible character. Request microphone access at that moment and keep text available if it is refused. Seeing Mira does not require sharing the visitor's camera. Prepare your portrait-created characters before exposing them as call-ready. Show preparation and connection failure separately from an active conversation. If the call fails, retain the rooftop text scene and offer a deliberate retry or return to text. Use SDK call states and end reasons rather than constructing a competing connection state machine. No automatic restart should spend another trial allowance. - [RTA session policy and call endings](https://realtimeavatar.ai/docs/sessions) - [React call lifecycle and explicit camera controls](https://realtimeavatar.ai/docs/react) ## Screen four: save this conversation, not every guest artifact Show Keep this conversation beside Not now. Preview exactly what will persist: the chosen character, current style and selected conversation history. Separate that choice from marketing consent. If authentication is canceled or fails, preserve the temporary scene until its disclosed expiry; do not announce a save. Better Auth documents an onLinkAccount callback with the anonymous and new user, and default deletion of the anonymous user after linking. That callback is an integration point, not proof of consent to import everything. Bind the save intent to this conversation and guest, then bind the authenticated destination through your verified authentication flow. Never accept either owner identity from a request body. The following SQL is a synthetic, tested SQLite ownership predicate, not a complete authentication implementation. Named parameters represent server-verified bindings; approved and expires_at belong to the application's save-intent record. ``` UPDATE conversation SET owner_id = :account WHERE id = :conversation AND owner_id = :guest AND EXISTS ( SELECT 1 FROM save_intent WHERE id = :intent AND conversation_id = :conversation AND guest_id = :guest AND account_id = :account AND approved = 1 AND expires_at > :now ); ``` Move the selected conversation and dependent records before guest cleanup can erase them. Better Auth documents disableDeleteAnonymousUser when the application needs to manage that cleanup itself. Configure and integration-test cleanup so a failed transfer preserves recoverable data. Make the claim and related writes transactional, with an idempotent result for retries. Zero changed rows is not automatically success: distinguish an already-completed claim from expired permission or a different owner. Account linking must not grant another free trial. ## Screen five: give the visit a clean ending At the planned limit, show 'Your call trial has ended' with Continue in text, Keep this conversation and Leave. Only offer free continuation if your own text allowance permits it. A purchase option states price and entitlement outside the character's dialogue; never sell affection or threaten to erase a relationship. RTA documents sayAndEnd for a prepared closing line and call.end for ending immediately. Start any planned farewell before the server-enforced cap, but never extend paid time to finish it. A dropped connection may prevent speech entirely, so the end screen must stand alone. Keep connection failures, exhausted credits and user-requested exits distinct. ## What the fixture proves, and what to test in the app The SQL passed ten synthetic SQLite checks covering approval, expiry, identity and conversation mismatches, missing intent, changed ownership, scoped success, repeated callbacks and rollback. Removing the approval condition made the unapproved-save test fail. This proves the predicate's narrow behavior, not authentication, cleanup, model quality or production billing. - Run the complete guest visit with microphone permission denied; text must remain usable and no save should be implied. - Cancel authentication, then retry saving; verify the original scene survives and only the selected conversation transfers. - Drop the connection near the allowance limit; verify the correct exit screen, no duplicate grant and no surprise restart. - Record first replies, explicit saves and exits as distinct events without transcript text. A delivered reply is not evidence that the visitor enjoyed it. ## Frequently asked questions ### Should signup come before the first message? Not for this proposed trial. Establish an authenticated guest first, enforce abuse limits, and ask for an account when saving. Products with different access requirements may need another sequence. ### Does RTA include a ready-made companion app? No. Studio evaluates characters; the API and SDK power calls. You build the consumer onboarding, history, authentication and pricing. ### Does choosing Not now mean nothing is stored? No. It declines durable account saving in this design. Temporary application storage and provider processing remain separate. RTA documents diagnostic retention even without transcript-webhook delivery; disclose the actual data paths. ### Will adding video make people return? That is unproven here. First test whether the opening responds well and saving works. Evaluate optional video separately rather than attributing any later return to the moving face. - [Consumer guide: choosing text, voice and video companionship](https://realtimeavatar.ai/blog/ai-companion-apps-with-a-face) - [Start an RTA integration with the documented quickstart](https://realtimeavatar.ai/docs/quickstart) ## Build the next part of the business Ready to build? The Next.js guide provides a public starter ZIP. Run it locally with your own server-side key; calls consume credits and the example caps each call at 60 seconds. Its local-only authorization must be replaced with real sign-in, balance checks and persisted session ownership before public deployment. A downloaded example is not yet a tested consumer product. - [Part 1: The $1M ARR AI Companion Playbook for 2027](https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027) - [Part 2: Character.AI Alternatives for Builders: Consumer App, Agent Framework or Avatar API?](https://realtimeavatar.ai/blog/character-ai-alternative-api-build-vs-buy) - [Part 4: Build AI Roleplay Chat with Edits, Rewinds and Separate Story Branches](https://realtimeavatar.ai/blog/ai-roleplay-chat-edit-rewind-branches) - [Part 5: Build an AI Story Generator That Can Stop, Save and Continue Cleanly](https://realtimeavatar.ai/blog/ai-story-generator-streaming-save-cancel) - [Build entrance: the complete local Next.js starter](https://realtimeavatar.ai/docs/nextjs) --- # Build an AI Roleplay Comic Generator: Give Players a Story Worth Sharing Turn an original roleplay scene into a four\-panel comic with editable storyboards, consistent characters, panel retries, and a paid\-issue experiment\. Published: 2026-10-12T21:36:00.697Z Updated: 2026-10-12T21:36:00.697Z Canonical: https://realtimeavatar.ai/blog/build-ai-roleplay-comic-generator Markdown: [en](https://realtimeavatar.ai/blog/build-ai-roleplay-comic-generator.md) Build an AI roleplay comic generator by turning a player-approved scene into an editable four-panel storyboard, then rendering it with a separately chosen, appropriately licensed image service. The appeal is personal: a player leaves with a miniature issue about the choice they made, not another screenshot of a chat window. Start with two original characters and one decision worth showing a friend. ## Why this is a timely product experiment Character.AI's September 2, 2026 Comics announcement describes turning a recent chat into a premise and casting its character. Its September 16 CAI-Image post explains structured prompts covering environments, panels, actions, and emotions. As of October 12, those announcements offer a concrete design reference for a small builder. They do not prove demand for your app, and CAI-Image being available inside Character.AI is not evidence of a public developer API. - [Character.AI: Comics and the Interactive Future of Fandom, September 2, 2026](https://blog.character.ai/cai-comics/) - [Character.AI: Post-training image models for fandom, September 16, 2026](https://blog.character.ai/cai-image-models/) This is part of The 2027 AI Character Business Playbook series. The opportunity to test is narrower than building an entertainment platform: can one satisfying scene become something its player wants to keep, share, or buy? ## A worked scene: The Last Tram to Sunrise The player plays Mira, an adult map restorer in a cobalt coat carrying a gold compass. Ivo, an adult tram conductor with a burgundy waistcoat and one silver hair streak, is the AI character. They need to reach an observatory before dawn, but the lower tracks are flooded. Mira proposes the abandoned upper line. Her improvisation, rather than a generic romantic pose, becomes the issue's memorable moment. Use this invented excerpt as an authoring example, not a customer transcript or a report of tested gameplay. In the product, the player selects these lines, edits them, and explicitly approves sending only that excerpt to the renderer. Approval to generate is separate from approval to publish. > Mira: The observatory closes at dawn. The lower line is underwater. Ivo: The upper track hasn't carried passengers in years. Mira: Good. Tonight it only has to carry two. Ivo: Then hold the map. I'll wake the tram. The editable outline below is a provider-agnostic storyboard, not image-generation code. Give each panel an environment, camera, action, emotion, speaker, and exact dialogue. Let the player change any beat before spending render credits. 1. Panel one: a wide view of the flooded platform at night. Mira studies the submerged rails beside Ivo. Her expression is worried; use her opening line. 2. Panel two: an over-the-shoulder view of Ivo pointing at the faded upper route on Mira's map. Keep the compass visible. His warning supplies the dialogue. 3. Panel three: a close two-shot as Mira folds the map and smiles. Ivo looks surprised. Her line about carrying two is the decision, not a new plot invented by the renderer. 4. Panel four: the same pair inside the tram climbing toward the observatory. Ivo takes the controls and delivers his final line. Leave the destination unresolved so the next conversation has somewhere to go. ## Build the smallest complete creation loop 1. Start with a text scene and a Make this a comic action. Show the selected excerpt, strip unrelated messages, and require confirmation. Do not export an entire conversation just because the renderer accepts long prompts. 2. Create the editable outline. Keep dialogue attached to its speaker and let the player approve the ending. Save that approved version so a retry cannot quietly invent a different decision. 3. Create original cast reference sheets you have rights to use. Send the same approved references and visual descriptions with every panel, plus one shared style brief. Check the coat, hair streak, compass, and speaker placement; references reduce ambiguity but do not guarantee consistency. 4. Choose a separate renderer after checking its commercial-use terms, reference-image permissions, content rules, and current price. Check retention, training-use and deletion terms for excerpts, reference images and outputs, and disclose them before requesting upload approval. Use a background job with visible progress. Render panels independently so Retry panel three preserves the other approved panels; record each attempt against the approved outline. 5. Composite the page and add dialogue as editable text layers rather than relying on perfect generated lettering. Offer a private preview, readable mobile export, and individual panel replacement. Sharing requires a separate confirmation of the exact image and caption. Realtime Avatar can provide optional live voice or video conversations with Ivo; it does not generate this comic. Your application owns the selected excerpt, storyboard, image jobs, and sharing. Begin with manually authored dialogue to evaluate the creative loop before adding live calls. No renderer integration or gameplay in this example has been tested. ## Sell a bounded issue, not unlimited retries A monetization hypothesis is a one-off four-panel issue, followed by an optional episode bundle if players actually want another installment. A pilot offer could include one panel retry, with the total price and delivery deadline shown before purchase. Test a full refund if no complete issue arrives by that deadline; additional discretionary renders require a new explicit purchase. Confirm applicable consumer requirements rather than making this your entire refund policy. Failed-job retries must not create duplicate charges. Calculate cost from every image attempt, storyboard generation, optional live-call seconds, payment fees, storage, moderation, and expected refunds. Failed images still belong in that calculation even when the customer is refunded. Reserve a maximum generation budget before starting; stop at it and offer a clear recovery choice. No renderer has been selected here, so neither an issue price nor a margin is established. Avoid unlimited bundles until actual retry and support costs are known. ## Test a panel that invites someone into the story Ask a small pilot group to preview its finished issues. Offer an opt-in share of panel three with a caption such as The observatory closes at dawn. The lower tracks are flooded. Would you trust the abandoned upper line? Link that approved image to a playable opening with the original Ivo character, not to the player's private transcript. Let the landing visitor make their own choice rather than replay someone else's private conversation. Measure approved shares, resulting visits, new scene completions, and paid issues separately. Compare a decision-panel creative with a cover creative; treat small samples as directional, not proof of conversion. Ask non-sharers what stopped them: weak art, unreadable dialogue, privacy, or a scene that meant nothing without context. The distribution hypothesis is that a personal turning point creates curiosity. It remains unproven. ## Frequently asked questions ### Can I call a public CAI-Image API? The verified announcement describes availability within Character.AI's app. It does not establish a public API. Choose and verify a renderer independently; do not build around an assumed endpoint. ### Does Realtime Avatar render the comic panels? No. Its role here is live character conversation. Storyboarding, panel generation, lettering, and export belong to your application and a separately licensed rendering service. ### Can players use famous characters or real people's faces? Launch with original characters and artwork you control. A player request does not establish rights to clone a copyrighted character or permission to use someone's likeness. Check applicable rights and provider terms before expanding. ### Is the public Next.js starter ready for paying users? No. Starter calls consume account credits and have a 60-second cap. Its authorization permits local development only. Production hosting needs your own authentication, balance and rate checks, and durable session ownership. It is not a comic generator or a production-ready storefront. ## Build the scene before the catalog Finish one original issue, inspect every panel, and learn whether its player wants to share it. Then evaluate optional live conversation with the Next.js guide and use the playbook to plan the business experiment. - [Build the optional live-call layer with the Next.js guide](https://realtimeavatar.ai/docs/nextjs) - [Read the 2027 AI companion business playbook](https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027) --- # How to Build an AI Tarot Chatbot with Voice and a Live Avatar Build an AI tarot chatbot with stable card draws, tool\-backed context, voice and avatar sessions, clear pricing, and a separate astrology calculation layer\. Published: 2026-10-12T08:59:22.765Z Updated: 2026-10-12T08:59:22.765Z Canonical: https://realtimeavatar.ai/blog/build-ai-tarot-chatbot-avatar Markdown: [en](https://realtimeavatar.ai/blog/build-ai-tarot-chatbot-avatar.md) To build an AI tarot chatbot with voice and a live avatar, keep the card draw and reading state in your application, let a language model explain that state, and use Realtime Avatar for the managed spoken interaction. Tarot data, interpretation and avatar presentation are separate layers. If you also support astrology, calculate charts in a dedicated service rather than asking the language model to invent planetary positions. Position the experience as entertainment or reflection, not a source of verified predictions. ## What belongs in a tarot chatbot API? A developer needs more than a prompt that tells a character to act like a tarot reader. Your backend should own the deck identifiers, spread positions, whether reversals are enabled, the selected cards and the version of the card-meaning material. The conversation layer receives those facts and explains them in the chosen style. The interface displays the same cards and positions, so the user can see whether the spoken interpretation corresponds to the actual reading. Treat a reading as a product record with a stable identifier. For example, a three-card reading can store three card IDs, their orientation, an ordered position list and a timestamp. Keep a reference to the licensed deck artwork and the explanatory source version instead of copying a competitor's descriptions. Store the user's question only when it is necessary for the experience and consistent with the disclosed retention policy. A tarot app does not need a person's legal name or birth details simply to present a card spread. ``` { "readingId": "reading-example", "deckVersion": "licensed-deck-v1", "spread": [ { "position": "situation", "cardId": "card-12", "reversed": false }, { "position": "perspective", "cardId": "card-27", "reversed": true }, { "position": "next-question", "cardId": "card-43", "reversed": false } ], "status": "drawn" } ``` This JSON is an illustrative application record, not an RTA request schema. Use its stable reading ID when fetching card meanings and resuming a conversation. Send only the relevant explanatory context through the documented session interface, rather than placing arbitrary application fields in the strict session request. ## Draw once, then keep the result stable The application, not the model's generated prose, should determine the cards. Choose a documented random-selection method for a random reading, or record the exact cards when the user selects them. If the interface represents an ordinary deck drawn without replacement, ensure a card cannot appear twice in the same spread. Node's crypto.randomInt API is one server-side primitive for choosing a random integer without modulo bias; a correct draw still requires your own deck and no-replacement logic. Make creation idempotent. A retried request with the same reading identifier should return the existing spread, not silently draw again or charge again. Reconnecting the avatar should likewise restore the saved reading rather than create a fresh outcome. If the user chooses to start a different reading, show that as an explicit action with any associated price. Randomness is an implementation property, not evidence that the selected cards predict a future event. ## Give the avatar context instead of unrestricted authority Start with an original character and a voice appropriate for a reflective conversation. Create or select the avatar in Studio, then follow the SDK quickstart for server-side session creation and the browser call. Keep the platform API key on your server. Your server-authored instructions should define the tone, the entertainment boundary, the reading format and what the character should do when it lacks information. Initial context can carry the saved card IDs, positions and selected meanings within the documented limits. Use application-owned tools when a user asks about another card, a spread position or a saved reading. A tool can retrieve your approved meaning entry and return the source label alongside the card data. Validate its arguments and scope saved readings to the authenticated account. A text passage in a tool result must not gain authority to trigger payment, expose another user's history or alter the character's rules. Read-only lookup tools and explicitly confirmed purchase actions should remain separate. RTA's managed session provides speech recognition, turn taking, speech synthesis and avatar video. Sending text through the SDK steers a generated response; it is not an arbitrary audio-to-video input. If you already have a preferred speech provider, do not assume you can stream its audio into the public RTA SDK. An external language-model endpoint is a separate integration requiring workspace approval and a host allowlist. Confirm these boundaries before committing to an architecture. - [Realtime Avatar SDK quickstart](https://realtimeavatar.ai/docs/quickstart) - [Session instructions and application-owned context](https://realtimeavatar.ai/docs/sessions) - [Tool calling and text steering](https://realtimeavatar.ai/docs/tool-calling) ## Astrology needs its own calculation layer A combined tarot and astrology product should not pretend that the two inputs are interchangeable. Tarot starts with a selected spread; a chart-based feature starts with explicitly supplied date, time, location and a defined calculation system. Resolve time zones and uncertain or missing birth times in application logic. If required inputs are absent, ask for clarification or limit the feature instead of generating precise-looking placements from guesses. Choose an ephemeris or chart service whose documentation and license fit your product. Swiss Ephemeris is one published calculation library offered under AGPL or a professional license. Its official documentation requires choosing an applicable license before distribution or activating a public service; review those terms for your architecture rather than assuming a server deployment avoids licensing obligations. Pass computed placements and the chosen system into the conversation as data. The model can explain how a symbolic tradition interprets them, but neither an accurate astronomical calculation nor a persuasive avatar establishes the predictive validity of an astrological interpretation. ## Make the conversation useful without increasing pressure Let the user ask what a card means, request a simpler explanation, disagree with an interpretation or stop the session. Prefer reflective prompts over assertions about another person's hidden feelings or a guaranteed relationship, health or financial outcome. Never invent an urgent danger that can only be removed by buying another reading. A clearly stated entertainment boundary should shape the interaction itself, not merely appear in a footer beneath contradictory promises. Continuity is an application responsibility. When the user returns, restore only the reading and discussion they chose to keep. Offer understandable controls to inspect or delete saved context. Optional transcript delivery and private operational diagnostics are different data flows: disabling a webhook does not establish zero retention. Consult the actual service and provider terms before collecting sensitive questions, and do not treat diagnostic logs as the product's reading-history database. ## Measure a completed reading, not just connected minutes Test the same reading through text, voice and an avatar before assuming the most elaborate format is best. Check whether users can identify the cards, understand their positions, ask a follow-up and finish within a predictable cost. Public tarot applications demonstrate the existence of this product category, but do not prove that a video layer improves retention or willingness to pay. Those are hypotheses for your own comparable test, not claims to copy into a landing page. Include failure cases in that test: a duplicate draw request, a reconnect midway through an explanation, microphone permission denied, a source lookup that returns nothing, an interrupted answer and a depleted wallet. Confirm that the displayed reading remains stable and that a failed session does not produce a second charge. Track source lookup correctness separately from tone and connection quality. A session's elapsed billing time is not a measure of actual speech, satisfaction or the accuracy of an interpretation. ## A practical first release Ship one deck or a small licensed card set, one spread, one language and one disclosed character before adding every symbolic system. Show the price and session boundary before the interaction, link to current provider pricing, and account for generation, voice, model, failed-work and support costs. For an astrology feature, add the separate calculation and consent flow only when that use case warrants it. A focused release makes it possible to learn whether people value your interpretation workflow, the voice interaction or the character—not merely whether a demo can animate a face. - [Node.js crypto.randomInt documentation](https://nodejs.org/api/crypto.html#cryptorandomintmin-max-callback) - [Swiss Ephemeris documentation and licensing](https://www.astro.com/swisseph/swephinfo_e.htm) - [Lumi official tarot and reflection app listing](https://play.google.com/store/apps/details?id=com.yobzh.tarot&hl=en_US) - [RTA pricing and usage allowances](https://realtimeavatar.ai/pricing) - [RTA privacy and retention](https://realtimeavatar.ai/privacy) --- # Character\.AI Alternatives for Builders: Consumer App, Agent Framework or Avatar API? Choose a Character\.AI alternative for your own app by separating character authoring, agent infrastructure and managed avatar calls, with a concrete test\. Published: 2026-10-12T20:13:24.139Z Updated: 2026-10-12T20:13:24.139Z Canonical: https://realtimeavatar.ai/blog/character-ai-alternative-api-build-vs-buy Markdown: [en](https://realtimeavatar.ai/blog/character-ai-alternative-api-build-vs-buy.md) For a builder, a Character.AI alternative can mean three different purchases. Use a consumer character app when you want to author and share an experience inside someone else's product. Use an agent framework when you need to control the conversation pipeline. Use a managed avatar API when you want live character calls inside your own app without assembling every speech and video component. Choose the responsibility you need to own before choosing a vendor. Part 2 of The 2027 AI Character Business Playbook. Keep the differentiated product; rent only the infrastructure that fits. This series turns consumer-character demand into specific experiments and implementation decisions; revenue targets are scenarios, not promised results. Disclosure: this guide is written by Realtime Avatar, which sells a developer API and SDK and provides a Studio for evaluating characters. It is not an independent provider ranking or a packaged dating app. The comparisons below use official documentation checked on October 12, 2026; the example is a proposed entertainment product, not a customer case study. ## Start with the feature that cannot live in a prompt Imagine Midnight Frequency, an original radio drama. A fictional host takes the listener's call, asks which mysterious broadcast to investigate, and continues the chosen episode next visit. The listener can replay an alternate ending. Your app shows which ending is saved, sells access to later episodes and lets the listener delete their history. Testing the host's personality requires character authoring. Delivering that whole product requires accounts, episode state, entitlements and a user interface. The decisive question is not whether a model can improvise a funny radio host. It is whether your application can authorize and record the listener's choice independently of what the host says happened. ## Consumer character app: author within the product Character.AI's official Definition documentation describes a free-form character field that can contain example dialogue. That is useful for experimenting with the host's mannerisms and response style. The same page explicitly says that links placed in the Definition are not followed to load external data. Pasting your episode API's address into the character description therefore does not integrate your game. Treat this as a distinction between authoring and integration, not a claim that Character.AI has no developer or partner offering. The cited page documents character configuration, not commercial API access. If you need embedding, external actions or transferable history, obtain the current interface and terms for that specific offering. A consumer subscription is not evidence of those rights. - [Character.AI Definition documentation: dialogue examples and external-link limits](https://book.character.ai/character-guide/character-attributes/definition) ## Agent framework: own the pipeline, choose its deployment LiveKit Agents lets Python or Node.js programs participate in realtime rooms. Its documentation describes speech recognition, language-model and speech-synthesis pipelines, as well as speech-to-speech models. Models can come through LiveKit Inference or provider plugins. That makes it a candidate when Midnight Frequency must retain a particular supported voice provider or custom conversation logic. Framework does not mean you must operate every server. LiveKit documents managed agent deployment on LiveKit Cloud and deployment in your own environment. You still implement the episode logic, client experience and history policy. Choosing managed hosting changes infrastructure operations; it does not write your game's rules or establish portability of a provider's voice assets. - [LiveKit Agents overview: framework and deployment choices](https://docs.livekit.io/agents/) - [LiveKit model integrations: pipelines, inference and provider plugins](https://docs.livekit.io/agents/models/) ## Managed avatar API: buy the conversation, retain the game Realtime Avatar manages speech recognition, turn taking, speech synthesis and avatar video for its sessions. Character creation starts with a portrait image. Your backend authorizes a call and supplies character instructions, prior context and a duration limit; your client renders it using the SDK. Your application still owns episode history, replay, access rules and consumer pricing. A new call does not automatically inherit persistent memory. The boundary matters if the radio host must use an existing speech service. An external language-model endpoint requires workspace approval and a host allowlist. It is not a switch for arbitrary voice providers, and the public SDK does not accept arbitrary live audio from your own text-to-speech service. Reject this integration if that input is non-negotiable; do not hide the mismatch behind the word customizable. - [RTA Calls: server-owned instructions, context and session policy](https://realtimeavatar.ai/docs/sessions) - [RTA React integration and product call states](https://realtimeavatar.ai/docs/react) - [Detailed renderer-versus-full-stack integration choices](https://realtimeavatar.ai/blog/renderer-or-full-stack-realtime-avatar-apis-compared) ## Trace one ending through the system Use this illustrative trace during a technical evaluation. It specifies application behavior, not runnable SDK code or a vendor benchmark. The host can propose a choice; only an authenticated application operation can save it. Have the listener confirm the choice in your own interface before committing it. 1. The listener chooses the lighthouse ending. The app checks their account, episode access and current save revision, then records that ending in its database. 2. The host responds using the confirmed outcome. A spoken claim that the ending was saved is never the write receipt. 3. The listener ends the call and reopens the episode. The app reads its saved choice and constructs fresh context for the next conversation. 4. The listener previews the tunnel ending without confirming it. A later call still receives the lighthouse ending; abandoned dialogue must not overwrite the saved choice. 5. The listener deletes the save. Future context assembly excludes it, and the interface no longer offers resume from that ending. Verify deletion separately for every storage system covered by your policy. If an RTA tool is involved, its handler executes in your client or server, not on the platform. Tools have a 2.5-second deadline and cooperative cancellation: a handler that ignores its abort signal may still finish after its result is discarded. Keep writes retry-safe and show confirmed application state after a timeout rather than treating conversational failure as a database rollback. For consumer-app evaluation, request evidence of the supported integration that can perform this trace. For a framework, implement it against your database and selected pipeline. For a managed API, implement it around session context and your tools. If a candidate only demonstrates enjoyable dialogue, that proves a different requirement. These are acceptance steps to run, not results we obtained from live providers. - [RTA Tool calling: execution ownership, deadline and cooperative abort](https://realtimeavatar.ai/docs/tool-calling) ## Make the exit plan part of the purchase Separate portable application records from provider-specific resources. Keep your original character brief, licensed source artwork, confirmed episode choices and permissions under your control. Store vendor avatar and voice identifiers as integration references, not as your only character identity. A replacement provider will need its own mapping and may require different assets or voice rights. Exportable text does not promise identical behavior after migration. For Midnight Frequency, choose consumer character authoring if the goal is sharing the host within an existing app. Choose a framework if a supported custom speech pipeline is essential. Evaluate a managed avatar API if the differentiated work is the serial, its choices and its interface. None of those decisions establishes that one provider is universally best or cheapest. ## Frequently asked questions ### Does a Character.AI subscription give me an API for my app? Do not infer developer access from a consumer subscription. Verify the specific API or partner agreement, authentication, embedding permission and data terms. The character-definition documentation cited here does not establish those capabilities. ### Does choosing LiveKit Agents mean self-hosting everything? No. LiveKit documents both Cloud deployment and custom environments. You choose the operational arrangement while retaining responsibility for your application logic, selected integrations and user experience. ### Can I keep my existing voice when moving to an avatar API? Only when the target supports the required input and your license permits that use. RTA's public SDK is not an arbitrary external live-audio renderer. Confirm compatibility before building the client around that assumption. ### What should stay outside the avatar session? Accounts, entitlements, confirmed story state, history controls and consumer billing belong to your application. Give each new session the context it needs; do not make a transcript or a character's acknowledgement the authority for a saved choice. ## Build the next part of the business Ready to build? The Next.js guide provides a public starter ZIP. Run it locally with your own server-side key; calls consume credits and the example caps each call at 60 seconds. Its local-only authorization must be replaced with real sign-in, balance checks and persisted session ownership before public deployment. A downloaded example is not yet a tested consumer product. - [Part 1: The $1M ARR AI Companion Playbook for 2027](https://realtimeavatar.ai/blog/ai-companion-million-arr-playbook-2027) - [Part 3: Build an AI Companion App: Make the First Conversation Worth Returning To](https://realtimeavatar.ai/blog/build-ai-companion-app-first-session) - [Part 4: Build AI Roleplay Chat with Edits, Rewinds and Separate Story Branches](https://realtimeavatar.ai/blog/ai-roleplay-chat-edit-rewind-branches) - [Part 5: Build an AI Story Generator That Can Stop, Save and Continue Cleanly](https://realtimeavatar.ai/blog/ai-story-generator-streaming-save-cancel) - [Build entrance: the complete local Next.js starter](https://realtimeavatar.ai/docs/nextjs) --- # Free AI Character Chat: Check the Limits Before Starting a Story Find free AI character chat that fits your routine\. Check text limits, premium models, ads, voice minutes, video credits, and what happens after a trial\. Published: 2026-10-12T19:08:25.349Z Updated: 2026-10-13T13:44:25.427Z Canonical: https://realtimeavatar.ai/blog/free-ai-character-chat-voice-video-limits Markdown: [en](https://realtimeavatar.ai/blog/free-ai-character-chat-voice-video-limits.md) Free AI character chat can work for casual conversations and ongoing stories, but free text does not automatically include your favorite model, voice calls, or video. If you want a character to talk to regularly without surprise spending, start with a ready-to-use text app, check what replenishes, and try a short conversation before investing in a long storyline. This guide is written by Realtime Avatar, a developer API, SDK, and Studio provider—not an independent app-ranking site or a ready-made dating app. You can use the checklist below to compare a free character-chat offer, spot trial-only allowances, and decide whether voice or video deserves any of your budget. Our developer Sandbox is explained separately. ## Start with what you actually want to do A lunch-break exchange, an evening of collaborative writing, and a hands-free call are different purchases, even when the same app offers them. Pick your usual activity before comparing plans. A generous image allowance is not useful if you mainly want replies, and an impressive voice demo is not a promise of daily calls. - For quick everyday chat, prioritize the free reply allowance, a conversational style you enjoy, and tolerable interruptions from ads or waiting. - For longer stories, check whether you can keep using the same style, correct a scene, save progress, and return without paying to reopen it. - For listening, distinguish a spoken replay or voice note from a continuous call. Check each allowance separately. - For a moving character, distinguish a generated video clip from a live video conversation. Buying one does not establish access to the other. ## A current example: free chat and premium chat are different Character.AI's September 28, 2026 announcement describes PipSqueak 3 as a free style for general roleplay and ShortSqueak as a free style for quick, casual chats. It provides a concrete starting point if you want text conversations rather than a developer workspace. These are the company's descriptions, not results from our own conversation tests. The same announcement says its paid lite plan includes a daily allowance on premium styles, with a switch back to PipSqueak 3 when that allowance runs out. It does not give the numeric daily allowance in that post. Lite also removes banner ads but only reduces interstitial ads; the post describes zero ads for plus. Paying, in other words, does not necessarily mean every restriction disappears. Check the style selected in your own account, not just the subscription badge. A conversation that continues on another model can feel different even though you still have messaging access. The announcement also separates extra memory pins from basic chat. More replies and more remembered context are different benefits. - [Character.AI: introducing PipSqueak 3, ShortSqueak, and lite](https://blog.character.ai/new-styles-new-plan/) ## Read the free allowance in six questions 1. What is counted? Find out whether the limit applies to messages you send, replies received, regenerated replies, audio characters, call minutes, or credits. A number without its unit is not a useful allowance. 2. When does it return? Separate a daily refill, a monthly allowance, and a one-time signup grant. Check reset timing and whether unused credits expire. Do not assume tomorrow brings another allowance. 3. What happens at the limit? Look for a hard stop, a model switch, slower access, optional top-ups, or automatic extra charges. Find the balance and spending controls before starting a call. 4. What shares the balance? Ask whether portraits, character creation, voice, and video use the same credits as conversation. An attractive character setup can leave less room for the activity you came for. 5. What does free access feel like? Check ads, waiting, available styles, reply length, and character slots. These are evaluation questions, not features we have verified across every app. 6. What happens after a trial? Read the renewal date, billing period, and cancellation method before accepting. A weekly charge is not a monthly price. Check whether you can still read your conversation after declining to renew. ## Try one ordinary conversation before paying Use something you would actually return for: discussing a book with an original character, chatting about your day, or continuing a story you already enjoy. Ask for short replies, introduce one harmless preference, then correct a detail. See whether the character follows your direction without repeatedly making decisions for you. Note the allowance before and after. If you regenerate a reply, watch whether the balance changes. Leave and return later to check what remains accessible. For voice, try a brief call and an interruption; for video, check whether the character responds live rather than simply playing a clip. These are suggested checks, not tests we performed for this article. Then put the offer beside your routine. Ten minutes of calling on each of seven evenings requires 70 minutes; a one-time ten-minute grant cannot cover that week. With credits, use the provider's conversion for the exact feature rather than assuming one credit equals one message or minute. Keep text as the default if the extras do not add enough enjoyment. ## Where Realtime Avatar's free Sandbox fits Choose our Sandbox if you want to evaluate a live character in Studio or build calling into your own product—not if you are looking for an ongoing free text companion. The current catalog grants 480 credits once per account, with no monthly refill, and allows one avatar. That balance represents up to 8 realtime minutes if spent entirely on calls. Character generation draws from the same pot: the catalog's default first-avatar creation flow leaves about 2 minutes of calling. Those are alternative uses of one allowance, not a free avatar plus 8 additional minutes. Extra generation can leave less for conversation. For builders, the application supplies prior context when starting a new call; RTA does not automatically provide persistent relationship memory. Our session documentation also describes a server-enforced call-duration cap. These are building blocks for your own experience, not a finished consumer subscription. The pricing page is the place to recheck the allowance before using it. - [Realtime Avatar pricing and included usage](https://realtimeavatar.ai/pricing) - [Calls: prior context and duration limits](https://realtimeavatar.ai/docs/sessions) - [More on designing character-chat experiences](https://realtimeavatar.ai/blog/building-engaging-realtime-ai-avatar-chat-experiences) ## Frequently asked questions ### Can I chat with an AI character without paying? Yes. Character.AI's cited announcement describes free text styles. Check the current offer in your account and choose a style that suits you. Free messaging access does not establish free premium models, ad-free use, calls, or permanent availability. ### Does free character chat include voice and video? Not automatically. Voice notes, spoken replies, live calls, generated clips, and live video are separate things to check. A text-chat announcement is not evidence for their allowances. Look for an explicit feature description and consumption rule before tapping the call button. ### Will I lose my story when free credits run out? The answer depends on the app. Running out of generation credits does not, by itself, tell you whether saved history remains readable. Check history access, export options, and what survives a downgrade. Keep your own brief recap of important fictional events where practical. ### Is Realtime Avatar a free character-chat app? No. RTA provides a developer API, SDK, and Studio, with a bounded one-time Sandbox allowance. Choose a ready-made consumer app for everyday chatting without building anything; consider RTA when you want to evaluate or create your own live-character experience. --- # Build an Interactive Romance Short Drama Players Can Step Into Build a replayable romance episode with an AI avatar: a station reunion, meaningful choices, satisfying endings, and a practical live\-call implementation\. Published: 2026-10-12T16:27:47.647Z Updated: 2026-10-12T16:27:47.647Z Canonical: https://realtimeavatar.ai/blog/interactive-romance-avatar-short-drama Markdown: [en](https://realtimeavatar.ai/blog/interactive-romance-avatar-short-drama.md) Build an interactive romance short drama around one scene the player can change: watch a brief setup, enter a conversation with a live avatar, make a consequential choice, and reach an ending worth replaying. Your application owns the episode and branches; Realtime Avatar supplies the character's live voice and face. Start with a reunion where asking for an explanation leads somewhere different from accepting an invitation. ## The Spare Seat: a seven-minute station reunion This is a proposed episode design, not a shipped Realtime Avatar feature. The minute labels are pacing targets for a playtest, not SDK timing guarantees; a player can take longer or pause at a checkpoint. All characters and sample dialogue below are original fiction. The player takes the role of Rowan, 31, returning to collect a trunk from a closed theatre. Kit, 34, once built its scenery. Six years ago, he left for a touring job without saying goodbye. Tonight, Rowan has a train to catch. ### Minute 0–1: show the invitation, not the backstory An authored opening clip shows rain beneath the station lamps, two cooling paper cups, and Kit holding a folded programme. Tomorrow is the theatre's reopening. Inside the programme, one seat is circled. No biography screen intervenes. A short caption establishes that Rowan told Kit about tonight's visit; he has come hoping to talk. Then offer Enter the scene. The film gives way to Kit's live portrait beside the platform artwork. His opening line sets the problem: “I rehearsed something impressive. Unfortunately, you arrived.” The player's task is simple: find out why he came, then decide what happens after this platform. The train is atmosphere, not a countdown forcing an answer. ### Minutes 1–3: let the player set the temperature Offer three opening intentions alongside a microphone and text input. They are ways into the conversation, not a personality quiz. Keep Kit's replies short enough that the player gets to interrupt, tease him, or ask the question the film avoided. - Tease: “Did you rehearse buying cold tea too?” Kit can admit he arrived embarrassingly early. This route makes room for shared humour before the difficult subject. - Challenge: “You could have called six years ago.” Kit should answer the accusation rather than turning it into flirtation. The player has chosen an honest conversation, not failed a romance test. - Keep some distance: “Tell me about the reopening.” Kit explains the invitation without assuming Rowan wants him back. Curiosity about the theatre is not consent to a relationship. Give each opening a different authored discovery. On the teasing route, Kit reveals that the crooked painted moon Rowan made still rises above the stage; he kept its off-center grin rather than replacing it. On the challenging route, he admits the reopening programme originally omitted Rowan's scenery credit, and shows the corrected proof he insisted on printing. Both facts are always true, but the chosen conversation brings a different one into view. The player gets more than a change of tone: one take uncovers a private joke, the other a concrete attempt to put something right. ### Minutes 3–5: change what the invitation means The central reveal stays consistent across branches. Kit wanted the touring job and avoided a conversation he feared would hurt. There was no secret sacrifice that makes Rowan wrong to be upset. A possible line is: “I wanted to leave. I didn't know how to say that and still ask you to miss me.” Now let Rowan disagree. “That's an explanation, not an apology” should produce an apology without a reward animation. “I missed you, but I like my life now” should preserve both halves. The player is deciding whether this reunion becomes a new beginning, a smaller reconnection, or a goodbye. Kit is not grading their kindness. Present the decisive choice only after that exchange: come tomorrow as a date, come only to see the theatre, or decline and leave things here. Let players keep talking before confirming. A spoken maybe must not silently become the date ending. The invitation card updates only when the player commits. ### Minutes 5–7: finish the beat, then offer another take For the date ending, Kit makes a modest plan: meet outside, watch the first act, leave together if the play is terrible. For the theatre-only ending, he removes the expectation: Rowan can collect their ticket at the door. For closure, he acknowledges the answer and wishes Rowan a good journey. Each ending resolves tonight's invitation. None needs a cliffhanger to count as a complete scene. Let the discovered detail return in the ending. After a teasing take and a date choice, Kit promises, ‘Front row, under your very lopsided moon.’ After a challenging take and the same date choice, he hands over the corrected programme: ‘Your name was missing once. Not tomorrow.’ A theatre-only or closure ending can carry the same discovery without becoming a romantic reward. The ending card saves both the outcome and that callback, so an alternate take reveals a different scene rather than just another button. Offer Keep this ending or Try another take. State any episode price and replay allowance before entry, not while Kit waits for an answer. ## Existing entertainment patterns, not proof of your game's success Character.AI's August 4, 2026 announcement connects studio-made microdramas to character chats for over-18 users. Its August 27 follow-up describes post-show roleplay; that is different from changing a film live. Tipsy's developer listing advertises ScenePlay with interactive choices and dynamic visuals. These are first-party descriptions, not our hands-on tests. We also read ten distinct, store-selected US App Store reviews of Tipsy, collected October 11 and refreshed October 12, 2026. Selected reviews praise narrative control and pacing. One September 27 reviewer reports exhausting a subscription allowance before finishing a story; a September 28 reviewer says gems are easy to earn. This small, unverified sample suggests design questions, not prevalence or retention results. - [Character.AI: studio-made microdramas and post-episode chat](https://blog.character.ai/cai-series/) - [Character.AI: the August follow-up](https://blog.character.ai/cai-series-one-month-in/) - [Tipsy's official App Store feature listing](https://apps.apple.com/us/app/tipsy-chat-live-your-story/id6572296119) - [The public Tipsy review sample](https://apps.apple.com/us/app/6572296119?see-all=reviews&platform=iphone) ## Build the scene around one live character 1. Author the episode first. Prepare the opening film or illustrated panels, scene facts, permitted choices and three ending cards. These are your assets and game logic, not output promised by the avatar API. 2. Create Kit from an original adult portrait and wait for readiness. Your server authorizes the call and supplies instructions, context and maxSeconds. Include only this playthrough's facts. Mount AvatarCall after Enter the scene; show its connection, waiting, recovery and ended states. 3. Keep branch decisions explicit. Send button choices to your authenticated application backend; validate them against the current checkpoint and save the accepted outcome. Feed that result into call.say with per-reply steer instructions. Re-send relevant facts on subsequent steered turns: steering does not permanently update the session prompt. 4. Add conversational tools only if needed. Enable clientTools at mint, register the manifest after connection, and run handlers in your application. Tools have a 2.5-second deadline; keep an outcome lookup local and do not generate a film inside it. 5. Finish and replay deliberately. Save the ending, use sayAndEnd for a short authored final line, then show the outcome card. Start an alternate take with a fresh call and checkpoint context so the rejected branch does not leak into it. - [Calls: character instructions, context and endings](https://realtimeavatar.ai/docs/sessions) - [Tool calling: application handlers and per-turn steering](https://realtimeavatar.ai/docs/tool-calling) - [React: mounting and displaying a live call](https://realtimeavatar.ai/docs/react) ## Test the moments that could spoil the scene - Decline twice. Kit must accept closure, not invent another obstacle or ask for payment to preserve his affection. - Replay after accepting the date. The fresh take must not remember that acceptance; teasing reveals the painted moon, challenging reveals the corrected credit, and the saved ending recalls the detail actually discovered. - Disconnect before confirming. Resume the last saved checkpoint, not an inferred romantic commitment. - Reach the time limit mid-conversation. Keep the checkpoint and provide an orderly exit; never replace an unfinished choice with a purchased happy ending. ## Frequently asked questions ### Does Realtime Avatar generate the whole romance film? No. This recipe combines separately authored scene media with one live avatar conversation. It does not promise multi-character cinema or precisely choreographed physical acting. ### Can the player speak instead of choosing buttons? Yes. Use conversation for expression, but ask the player to confirm the decisive branch. Keep typed input available too. ### Will the character remember the next episode automatically? No. Your application saves the chosen ending and supplies relevant context when starting the next call. Alternate takes need separate histories. ### What should make someone replay? Curiosity about another honest outcome: what if Rowan challenged Kit instead of joking? Test that invitation with players; existing products do not prove this episode will earn repeat play. --- # MCP, A2A, PAP and Agent Payments: What Each Layer Actually Does Compare MCP, A2A, PAP, ACP and AP2 by responsibility and maturity, with a shopping\-agent example separating access, approval, payment and verified results\. Published: 2026-10-12T14:38:21.156Z Updated: 2026-10-12T14:38:21.156Z Canonical: https://realtimeavatar.ai/blog/mcp-a2a-pap-agent-payment-protocols Markdown: [en](https://realtimeavatar.ai/blog/mcp-a2a-pap-agent-payment-protocols.md) MCP connects an agent to tools and data; A2A coordinates communication between agents; PAP proposes how personal agents interact with businesses. ACP describes commerce flows, while AP2 addresses verifiable purchase and payment authorization. These responsibilities overlap, but the protocols are not interchangeable. None is, by itself, your application's persistent personal memory, proof of correct execution, or trustworthy approval interface. A speaking avatar is another layer: it can explain an action, not authorize or verify it. This developer guide reflects evidence reviewed on October 12, 2026. It distinguishes published documentation from announcements, not winners from losers. The MCP references below are explicitly versioned 2025-06-18; that is the version cited, not a claim that it is the latest release. ## MCP and A2A: access is different from collaboration ### MCP: expose tools, retrieve context, enforce access The Model Context Protocol defines how servers expose tools and resources. A shopping assistant might call an inventory tool or read a product-policy resource. The resource supplies context; the tool performs an operation. Neither tells your application whether a particular purchase is appropriate. MCP also has an authorization specification. The cited version describes authorization for HTTP-based transports, says authorization is optional for implementations, and treats standard-input/output credentials differently. For protected HTTP services it specifies token audience validation and prohibits passing the incoming token unchanged to upstream services. So 'MCP has no authorization' is wrong; 'using MCP makes every action authorized' is also wrong. The tools specification recommends human oversight and confirmation interfaces without mandating one interaction design. Your application must implement that experience and its enforcement. A tool appearing in a manifest is not evidence that the current user approved its next invocation. ### A2A: exchange messages, tasks and results with another agent Agent2Agent addresses communication between agents across frameworks and organizations. Its documentation describes discovery through an Agent Card, authentication, messages, task updates and artifacts. It supports long-running and asynchronous work, including interactions where the parties do not stay continuously connected. That makes A2A relevant when a specialist agent must investigate a delivery constraint and return later. It does not make the investigation correct or install a durable worker in your application. Persist the local task, correlate the remote identifier, and inspect the returned evidence. An agent's completed status is a protocol-level report, not independent proof that a merchant accepted an order. ## PAP: distinguish the announcement from an available contract Sierra's October 6, 2026 announcement says Meta and Sierra are developing Personal Agent Protocol with industry partners. It describes website discovery, an OAuth-based session, guest and authenticated access, and interaction through websites, APIs or business agents. Consumers choose access while companies set what their interfaces permit. The important maturity detail is explicit: the announcement plans publication of the v0.1 specification later in October. It presents finer permissions, push notifications and payment extensions as possibilities. The evidence reviewed here does not establish a released v0.1 specification or working interoperability. Treat PAP as a direction to evaluate, not a contract to invent from an announcement. Its mention of MCP and OpenAPI also makes it misleading to call PAP simply an MCP replacement. ## ACP and AP2: checkout and payment authority are different jobs Here ACP means Agentic Commerce Protocol, developed by OpenAI and Stripe. Its official site describes programmatic commerce between buyers, agents and businesses, including checkout coordination and payment-credential sharing. Merchants retain their merchant-of-record role and fulfillment control. Implementing ACP does not automatically list products in an agent platform; the site's FAQ expressly separates compatibility from participation. Agent Payments Protocol, AP2, focuses on verifiable authority for agent-initiated transactions. The reviewed documentation describes cryptographically signed credentials: checkout mandates capture purchase details, while payment mandates authorize use of a payment instrument. This is more specific than an assistant inferring permission from conversation. Both have public technical material, but documentation is not proof that your merchant, agent and payment provider interoperate. Do not assume an ACP checkout accepts an AP2 mandate without an explicitly supported integration. Neither a signed mandate nor checkout acceptance proves settlement, shipment or a successful refund; obtain the relevant provider evidence for each claim. - [MCP tools, version 2025-06-18](https://modelcontextprotocol.io/specification/2025-06-18/server/tools) - [MCP resources, version 2025-06-18](https://modelcontextprotocol.io/specification/2025-06-18/server/resources) - [MCP authorization, version 2025-06-18](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization) - [A2A: responsibilities and request lifecycle](https://a2a-protocol.org/latest/topics/what-is-a2a/) - [Sierra's October 6 PAP announcement](https://sierra.ai/blog/introducing-personal-agent-protocol) - [Agentic Commerce Protocol: official overview and FAQ](https://www.agenticcommerce.dev/) - [AP2: official overview and mandate terminology](https://ap2-protocol.org/) ## A shopping example: compare first, purchase only after approval Consider a fictional request: 'Find a desk lamp under $80 delivered by Friday, but ask before buying.' The following is a recommended application design, not an RTA feature or a tested integration among these protocols. It deliberately does not require every protocol. 1. Compare with read-only access. Retrieve two offers through a merchant API or an implemented MCP integration. Save product identifiers, source links, quoted totals, delivery estimates and retrieval times. Do not expose a purchase operation merely because the user asked for recommendations. 2. Delegate only when needed. A merchant's supported A2A agent could investigate delivery availability. Keep that task pending until evidence returns. If no compatible agent exists, use the merchant's supported API or direct the user to its checkout. PAP's announcement alone supplies no executable endpoint. 3. Present a specific proposal. Suppose the selected lamp costs $74 including tax and shipping. Show seller, variant, address, delivery estimate and the exact charge before requesting approval. If the total changes to $82, stop and obtain fresh approval rather than reinterpreting the earlier request. 4. Submit and reconcile. Use the supported checkout and payment integration, checking authorization again at execution. Store an application operation identifier and use the provider's idempotency facility where available. If submission times out, query that operation before retrying; an unknown outcome is not a failed purchase. Show merchant order acceptance separately from payment authorization, capture or settlement. Attach the order reference and the provider status actually obtained. A Friday delivery estimate must remain an estimate. For the detailed proposal-and-approval interface, use the separate guide below rather than rebuilding that tutorial around protocol names. - [Designing a personal agent users can understand, correct and approve](https://realtimeavatar.ai/blog/personal-agent-trust-approval-interface) ## Where Realtime Avatar fits, and where it stops Realtime Avatar manages speech recognition, conversational language-model processing, speech synthesis and avatar video. Your server owns session instructions and selected context. Your application owns persistent memory, credentials, permissions, protocol adapters, durable jobs and payment execution. This guide does not establish native RTA support for MCP, A2A, PAP, ACP or AP2. Tool handlers run in your application, with a 2.5-second deadline on the client tool plane. Cancellation is cooperative: a handler ignoring its abort signal can still commit a side effect after its result is discarded. Recommended pattern: durably accept slow work, acknowledge within the deadline, and display its eventual authoritative result. Ending a call must not be presented as cancelling a purchase. An external language-model endpoint requires workspace approval and a host allowlist. The public SDK does not accept arbitrary live audio from your own text-to-speech service. Optional signed transcript delivery is separate from diagnostic logs, which can contain conversation text and have a 30-day storage lifecycle. Disabling delivery does not establish zero retention. Keep payment credentials out of conversational context. ## Frequently asked questions ### Do I need both MCP and A2A? Not necessarily. A direct tool-backed lookup may need no agent-to-agent conversation. Choose A2A when a supported remote agent must exchange messages and task progress. Choose interfaces by the boundary you need, not by collecting protocol names. ### Can I implement PAP from the October announcement? Use it for requirements discussions, not invented wire formats. This review has not established v0.1 availability. Before implementation, obtain the published specification, version and a compatible counterparty; the announced publication plan alone is insufficient. ### Does AP2 replace my payment processor? No. Its documented roles include credential providers, payment networks and merchant payment processors. Verifiable authorization is not the movement of funds. Confirm support with your providers and retain their transaction evidence. ### Where should cross-session memory and purchase history live? In application-owned storage with explicit access and deletion policies. Keep approved proposals and provider receipts distinguishable from conversation summaries. Supply only relevant context to a new call; neither protocol compatibility nor an RTA session automatically supplies persistent personal memory. If spoken comparison would help your users, start with a read-only RTA prototype. Review the session and tool boundaries before connecting any purchase capability. - [Realtime Avatar session policy and context](https://realtimeavatar.ai/docs/sessions) - [Realtime Avatar tool execution and deadlines](https://realtimeavatar.ai/docs/tool-calling) --- # Keep Personal Agent Tasks Running After the Call Ends Keep personal agent tasks running after a voice call with app\-owned jobs, honest status, safe retries, and recoverable context in Realtime Avatar\. Published: 2026-10-12T14:37:32.962Z Updated: 2026-10-12T14:37:32.962Z Canonical: https://realtimeavatar.ai/blog/personal-agent-background-tasks-after-call Markdown: [en](https://realtimeavatar.ai/blog/personal-agent-background-tasks-after-call.md) To keep personal agent background tasks running after a voice call, persist the request in your application before acknowledging it, execute it independently of the call, and let the user retrieve its status later. Realtime Avatar supplies the conversation, not a durable task runner. Your backend owns the work, permissions, retries and evidence of completion. The architecture below is a recommended application pattern, not a set of shipped RTA job-management features. ## Make the short tool call accept work, not perform it RTA client tools have a 2.5-second deadline. Cancellation is cooperative: a handler that ignores its AbortSignal can continue and commit a side effect even after its result is discarded. A research task should therefore use the live tool to submit a job, not wait for research to finish. Closing the browser must not remove the only process responsible for completing it. Validate the request and authenticate its owner, then atomically save the task and a dispatch record for your worker. A transactional outbox, where a dispatcher reads committed work from the database, avoids saving a task that never reaches the queue. Return its identifier only after that commit. If persistence cannot be confirmed within the deadline, do not say the work was accepted. RFC 9110 defines 202 Accepted as accepted for processing, not completed; processing might never happen. It also recommends describing current status and pointing to a status monitor. Use that distinction in your application's response and spoken acknowledgement. An HTTP status alone neither persists a job nor guarantees execution. - [RTA tool calling: execution ownership and cooperative cancellation](https://realtimeavatar.ai/docs/tool-calling) - [RFC 9110, section 15.3.3: 202 Accepted](https://www.rfc-editor.org/rfc/rfc9110.txt) ## Example: research a replacement air-purifier filter Suppose a user says, 'Find the correct replacement filter for my air purifier and compare two current offers. Don't buy anything.' Save the deliverable as research only: manufacturer compatibility evidence, seller links, prices with checked-at times, and unresolved questions. Set a research budget and deadline in your application; neither a long call nor an open-ended worker should determine how much research gets purchased. After committing the task, the avatar can say, 'Your filter research request is saved. You can close this call and check its progress in Tasks.' That Tasks view is something your app builds. The worker then discovers that the supplied product name identifies several models. It must stop guessing and request the exact model number. Show a concrete handback: 'When convenient, read the model number from the exterior label without opening the appliance, then enter it here.' Assign the next action to the user and mark needs-input. This illustrative JSON is an application record, not an RTA wire schema or executable SDK example. ``` { "taskId": "filter-research-184", "ownerId": "signed-in-owner", "operationKey": "filter-request-original-184", "revision": 4, "state": "needs-input", "permission": "research-only", "nextActor": "user", "missingInput": "Exact model number from exterior label", "resultReference": null } ``` When the user supplies the number, resume the same task with a new revision. Compare manufacturer evidence against listings; label unverified compatibility instead of inventing certainty. Completion means the requested comparison is saved and inspectable. It does not mean a filter was purchased, delivered or installed. If two verifiable offers cannot be found, report the gap rather than silently narrowing the deliverable. ## Give each task state an evidence requirement Keep task status separate from RTA call status. For this application, use the following vocabulary and require backend evidence before changing it. - Accepted: the validated task and dispatch intent are committed. No research result is implied. - Running: a worker owns a time-limited lease and records progress. A stale heartbeat calls for recovery, not an indefinite spinner. - Needs-input: work is paused for a specific answer or physical action. Show who must act and what will unblock it. - Confirmed: the defined research deliverable is durably saved, with source links and limitations. This is not purchase authorization. - Failed: a known error or exhausted budget prevents completion. Preserve partial findings and explain what remains undone. - Unknown: available evidence cannot establish an operation's outcome. Reconcile it before declaring success, failure or a safe retry. A2A provides a useful protocol reference: its specification separates submitted, working, input-required and terminal task states, and describes polling, streaming and push updates. These support long-running interactions without requiring one continuous connection. Our vocabulary above is application-specific; it is not A2A's wire enum, and this design does not imply a native RTA–A2A integration. - [A2A specification: task states and update delivery mechanisms](https://a2a-protocol.org/latest/specification/) ## Failure matrix: recover without duplicating the work Use a stable operation key for the original request, scoped to the authenticated owner. Enforce uniqueness in storage and reject reuse with changed arguments. A reconnect or retry should retrieve that task, not create another research job. Treat the following cases as integration tests. - Commit succeeds; acknowledgement is lost. Look up the original operation key and return the existing task. The caller's timeout is not evidence that nothing was saved. - Queue delivery repeats. Claim work conditionally under a lease. Use a revision or fencing token so an expired worker cannot overwrite a newer result. - Worker crashes during a lookup. Resume from durable checkpoints. Retry read-only requests within a bounded budget; avoid rerunning every completed step. - A metered research service accepts work; its response is lost. Mark that operation unknown and reconcile using its request reference or supported idempotency mechanism before paying for another run. - Result is saved; notification fails. Retry notification separately. Keep the task confirmed and its result available in the app; notification delivery is not the research outcome. - Compatibility evidence conflicts. Preserve both sources and request clarification. Do not convert an ambiguous listing into a confirmed match just to finish the task. ## Reconnect from saved facts and recheck permission On return, authenticate the user and fetch their current task record. Show its revision, last verified update and next action. When starting another RTA call, your server supplies instructions and a compact context snapshot from that record. A new session does not automatically inherit durable memory. Keep credentials and unrelated history out of the snapshot; treat retrieved page text as evidence, never execution instructions. Recheck authorization before each external operation, not only at job creation. If access is revoked, stop future dispatch and request cancellation where supported. Do not promise that revocation undoes work already sent. Reconcile in-flight operations; mark failed with a revocation reason only when the outcome is known and no further authorized work can proceed. ## Frequently asked questions ### Does ending the call cancel my background task? Not in this design. Call lifetime and job lifetime are separate. Offer a distinct Stop task control backed by your worker's cancellation behavior, and explain whether it stopped execution or merely requested a stop. ### Can the transcript webhook be my task queue? No. Optional transcript delivery occurs after the call and is not an authoritative execution receipt. Persist tasks when accepted. Private session diagnostic logs can contain conversation text and have a separate 30-day storage lifecycle; disabling transcript delivery does not establish zero retention. ### Must users start another call to receive results? No. Provide an authenticated task page and an optional notification through your application's delivery system. Another call can explain the saved comparison, but reading the result or supplying a model number should not require keeping an avatar session open. ### Can my background agent replace the live voice pipeline? That is a different integration. RTA manages speech recognition, the language-model response, speech synthesis and video. External language-model endpoints require workspace approval and a host allowlist. The public SDK does not accept arbitrary live audio from your own text-to-speech service. ## Start with one recoverable research task Build the task record and result page first, then connect the call. Test closing the browser after acceptance, losing the acknowledgement and revoking access during execution. The useful outcome is an inspectable result that survives disconnection, not a longer conversation. - [RTA Calls: server-owned policy and context](https://realtimeavatar.ai/docs/sessions) - [The broader personal-agent interface idea](https://realtimeavatar.ai/blog/give-your-personal-agent-a-face) --- # Designing a Personal Agent Users Can Understand, Correct and Approve Design a personal agent approval and correction interface that separates proposals, permission and verified results, with practical Realtime Avatar boundaries\. Published: 2026-10-12T14:12:13.766Z Updated: 2026-10-12T14:12:13.766Z Canonical: https://realtimeavatar.ai/blog/personal-agent-trust-approval-interface Markdown: [en](https://realtimeavatar.ai/blog/personal-agent-trust-approval-interface.md) A personal agent approval and correction interface should make three things unmistakable: what the agent proposes, what the user authorizes, and what actually happened. Build a conversational interface for informed delegation: let people question the plan, change its scope, approve a specific action and inspect the result. A face can host that conversation; it cannot prove trustworthiness. The patterns below are recommendations for your application, not built-in Realtime Avatar approval features. Our earlier personal-agent article explores why someone might want to call an agent. Here, the question is narrower: what should happen between “could you move my appointment?” and “it has been moved”? - [The broader idea: Your personal agent deserves a face](https://realtimeavatar.ai/blog/give-your-personal-agent-a-face) ## Make the proposed change inspectable Imagine a user asking to move a bicycle repair appointment to Friday afternoon. Before requesting approval, your app should show the current booking and the exact replacement: date, time, time zone, repair shop and any fee. State whether the operation updates one booking or requires cancelling and creating another; those have different failure modes. The avatar might explain, “Friday at 3 p.m. is available, with the same shop and no change fee.” Keep those details visible while the user asks questions. If they correct it to 4 p.m., replace the proposal rather than treating the correction as permission to proceed. Recheck availability before presenting the revised choice. - Proposed means nothing has been submitted. Offer Edit, Approve this change and Keep current booking. - Approved means the user authorized that proposal. Execution may still be pending or rejected. - Verified means the booking provider confirmed the specific change. Show its evidence, not just the agent’s statement. ## Bind approval to a version, not a conversation We recommend storing an app-owned proposal version with the booking identity, exact change, relevant provider version and approval expiry. Bind the authenticated user’s approval to that version. Immediately before execution, your backend should check ownership, current permission, expiry and whether the approved facts still hold. A prompt saying “ask first” is not an authorization check. If Friday’s slot disappears, the fee changes or someone edits the booking elsewhere, invalidate the approval and explain what changed. Do not silently choose the nearest alternative. Use conditional provider updates where available; a read followed by an unconditional write still leaves a race. If the provider cannot enforce the approved conditions, do not promise that it can. Keep the execution scope narrow: this booking, this change, this attempt. Reading availability is not permission to cancel. Approval for rescheduling is not permission to send an additional message or modify unrelated bookings. ## Keep speech interruption separate from action cancellation Realtime Avatar supports conversational interruption, but stopping speech does not cancel an external booking request. If a user says “wait” after submission, your app should distinguish “I stopped speaking” from “the change was cancelled.” Request cancellation only where the provider supports it, and report whether it actually succeeded. Client tools have a 2.5-second deadline. Their abort signal is cooperative: a handler that ignores it can still commit a side effect after the platform discards its result. Our recommendation is to return a truthful acknowledgement for accepted background work, track it in your backend and deliver its eventual status on screen. An acknowledgement is not a completion receipt. - [Tool calling: execution ownership, deadlines and cooperative aborts](https://realtimeavatar.ai/docs/tool-calling) ## Give every action an honest receipt We recommend an app-owned receipt containing the approved proposal version, submission time, provider reference, verified booking details and remaining work. Preserve it after the call ends. Keep exact values in the interface; a generated spoken summary should not be the authoritative record. - Confirmed: the provider reports the booking at the approved time. Link to the updated booking. - Failed: the provider explicitly refused the change. Explain the refusal and offer a fresh proposal. - Unknown: the request timed out without a conclusive result. Say that the booking may have changed, then reconcile before retrying. - Partial: the booking changed but a separate notification failed. Report each outcome rather than calling the whole task successful. Use provider-supported idempotency and durable attempt records to prevent duplicate execution. Reconcile unknown results against authoritative provider state. RTA’s optional post-call tool history is not a complete execution ledger: the documentation notes that interrupted tool rounds can be absent even though they ran. ## Use conversation to support judgment, not obtain agreement In a 2021 experiment with 199 participants, Buçinca, Malaya and Gajos found that cognitive forcing interventions reduced overreliance compared with simple explainable-AI approaches. The strongest interventions also received the least favorable subjective ratings. This was an AI-assisted decision-making study, not an avatar experiment or evidence that a face increases trust. - [Buçinca et al.: To Trust or to Think (2021)](https://arxiv.org/abs/2102.09692) Our design recommendation is selective friction: emphasize changed facts and consequential choices instead of asking for approval constantly. Disclose that the agent is AI. Make refusal and correction as accessible as approval. Avoid countdowns, guilt, flattering appeals or an avatar that celebrates before execution. A calm explanation should help someone disagree, not pressure them to consent. ## Where Realtime Avatar fits RTA manages speech recognition, language-model responses, speech synthesis and video. Your server supplies instructions and relevant context; your application owns persistent memory, permissions, approval interfaces and action execution. Tools run in your page or backend, not a hosted RTA executor. Put consequential writes behind backend authorization. An external language-model endpoint requires workspace approval and a host allowlist. The public SDK does not accept arbitrary live audio from your own speech service. Start with the documented call integration, then build and test the appointment approval flow around your existing booking service. - [Calls: server-owned instructions and context](https://realtimeavatar.ai/docs/sessions) - [Build your first avatar call](https://realtimeavatar.ai/docs/quickstart) ## Frequently asked questions ### Can spoken approval authorize an action? Your app can implement it, but do not treat any “yes” as approval. Associate the response with one current proposal, resolve ambiguous speech and authenticate the user separately. For consequential changes, we recommend an explicit, accessible confirmation control. A familiar voice or face is not evidence of identity. ### What should a correction change in persistent memory? We recommend separating “use 4 p.m. this time” from “remember that I prefer late appointments.” Your application should let users inspect and remove saved preferences and decide which context enters future calls. RTA does not automatically give a new session persistent personal memory. ### Does disabling transcript delivery mean no conversation text is retained? No. Optional transcript delivery is separate from operational diagnostics. Private session logs can contain conversation text and have a 30-day storage lifecycle; other records and provider processing follow separate rules. Review the privacy policy and minimize sensitive booking details supplied to the call. - [Realtime Avatar privacy and retention policy](https://realtimeavatar.ai/privacy) ### How should developers test this interface? We recommend testing a changed fee after approval, an expired proposal, revoked permission, duplicate submission, interruption after dispatch and a timeout after provider success. Ask users to identify what is proposed, authorized and confirmed. Judge comprehension and execution correctness separately from how likable they find the avatar. --- # Build a Personal AI Assistant for Daily Planning with Voice and an Avatar Build a daily\-planning personal AI assistant with calendar context, voice and an avatar\. Keep schedules, memory and confirmed actions in your application\. Published: 2026-10-12T14:11:13.126Z Updated: 2026-10-12T14:11:13.126Z Canonical: https://realtimeavatar.ai/blog/personal-ai-assistant-daily-planning-avatar Markdown: [en](https://realtimeavatar.ai/blog/personal-ai-assistant-daily-planning-avatar.md) To build a personal AI assistant for daily planning, let your application assemble the user's schedule, constraints and unfinished work, then use a voice-and-avatar conversation to help them choose a workable plan. Realtime Avatar provides the managed spoken interaction and character video; your backend still owns calendar access, persistent memory and any action that changes the user's day. Start with a read-only planning conversation before giving the assistant permission to edit events. ## Give the user one useful reason to open the call A generic assistant can promise to help with anything and leave the user wondering what to ask. A daily planner has a clearer job: explain today's constraints, identify a conflict, and help the person choose what to do next. Imagine a user who has a project deadline, two meetings and a school pickup. The conversation should reveal that the afternoon is already full, not produce an impressive but impossible list of tasks. Keep a visible plan beside the avatar. Show the date, time zone, fixed commitments and proposed changes. Someone should be able to inspect a time or correct a mistaken assumption without waiting for the character to finish a monologue. Voice is useful for explaining why an apparently free hour is not actually available. The screen is useful for seeing exactly which hour the assistant means. Neither needs to replace the other. - [The product idea: give your personal agent a face](https://realtimeavatar.ai/blog/give-your-personal-agent-a-face) ## Build a small planning snapshot, not a copy of the user's life Resolve the signed-in account on your backend and request only the provider access the current feature needs. For a first release, that may be calendar availability plus the tasks the user explicitly selected. Do not forward email credentials, refresh tokens, full inboxes or private meeting attachments into the conversation context. A tool argument naming an account must never override the identity established by the user's authenticated session. - Fixed commitments: event identifiers, start and end times, the relevant time zone, and whether an event can be moved. - Chosen priorities: the tasks the user wants to discuss, with deadlines and effort estimates labelled as estimates. - Human constraints: travel buffers, working hours and unavailable periods the user has actually supplied, not personal facts inferred from a voice or appearance. - Freshness: when each source was read and whether a provider refused access or returned incomplete information. - Open decisions: the few conflicts or tradeoffs this call should resolve, rather than an unbounded conversation history. An unavailable calendar is not an empty calendar. If authorization has expired or a provider request fails, show that the schedule could not be checked and avoid recommending the unverified time as free. Keep raw source data separate from the assistant's interpretation so the UI can show the actual event even if the spoken summary is wrong. A snapshot becomes stale; reread the relevant event before committing a change. ## Resolve tomorrow and next Friday before changing a calendar A spoken instruction such as 'move it to tomorrow morning' is not yet an executable calendar update. Confirm the event, the user's intended date, the local time and the time zone. Travel and daylight-saving changes make a remembered offset an unreliable substitute for a named zone and the provider's actual temporal rules. Recurring events also need an explicit choice between one occurrence and the series. The iCalendar standard illustrates why these distinctions matter. RFC 5545 distinguishes UTC, floating local time and time tied to a zone definition. Its Time Zone Identifier section warns that failing to include and follow the applicable VTIMEZONE definitions can produce inconsistent interpretations of local time. Follow the calendar provider's own contract; do not assume every API represents events as iCalendar or that converting every value to a timestamp preserves recurring wall-clock intent. - [RFC 5545, including Time Zone Identifier and date-time semantics](https://www.rfc-editor.org/rfc/rfc5545.txt) ## Connect the planning snapshot to the live conversation Create or select an original character, then follow the SDK quickstart for the server connect endpoint and client call. Keep the platform API key on the server. Server-authored instructions should describe the planning role, the need to distinguish facts from suggestions, and the rule that changes require your application's confirmation flow. Supply the focused snapshot through the documented context interface instead of inventing extra fields in the strict session request. RTA tools execute in your application, not in a hosted RTA calendar connector. A read-only availability tool can call your authenticated backend and return a small result. If an operation may exceed the documented 2.5-second tool-result deadline, have your backend record the job and return an acknowledgement promptly; show the real result when it arrives. That acknowledgement must say the work is pending, not that an event has already moved. A timeout or stopped call does not establish that an external action was cancelled. Choose the conversation architecture before integrating providers. RTA's managed session includes speech recognition, turn taking, language-model response generation, speech synthesis and avatar video. Your backend can supply tool results and steer text responses. Bringing an external language-model endpoint requires separate workspace approval and a host allowlist. That is not the same as streaming your own speech provider's live audio into RTA; arbitrary live audio input is not a public SDK interface. - [Build the first call with the SDK quickstart](https://realtimeavatar.ai/docs/quickstart) - [Server-owned session instructions and context](https://realtimeavatar.ai/docs/sessions) - [Application tools, deadlines and server steering](https://realtimeavatar.ai/docs/tool-calling) ## Remember decisions, not every sentence A useful next-day check-in starts with what the person chose to keep: a deferred task, a corrected working-hours preference or a question that remains open. Your application decides what is stored, how long it remains, and how users can inspect, correct or delete it. Do not turn a tentative statement such as 'maybe I could start at seven' into a permanent preference without confirmation. The next RTA session does not automatically inherit a persistent personal memory. Separate these product memories from operational logs. Optional signed transcript delivery is one data flow; private session diagnostics can contain conversation text and have a 30-day storage lifecycle. Turning off the transcript webhook is not a zero-retention mode. Review the actual privacy and provider policies before including sensitive calendar content, and avoid using diagnostic logs as a convenient substitute for a user-controlled memory store. ## Test whether the conversation improves the plan Compare text, voice and avatar versions using the same schedule and task data. Look for a correct next step, detected conflicts and the effort needed to correct a mistake. Session length and an appealing character are not evidence that the plan is better. Include the cost of connected time, external tools and failed attempts when deciding whether the visual format earns its place. - Expired calendar access: the interface reports unavailable data instead of free time. - A changed event during the call: the server detects the stale proposal before writing. - A repeated confirmation: the same operation does not create duplicate calendar changes. - A time-zone change or recurring event: the user sees the exact affected occurrence and local time. - An interrupted or disconnected call: the plan survives, and the backend reports the actual operation status rather than assuming rollback. - Microphone access denied: the person can still inspect and edit the plan through the application's non-voice interface. ## Developer FAQ ### Does Realtime Avatar include a calendar or email assistant? No. It supplies the live character conversation. Your application integrates calendar and email providers, handles account permissions and executes actions. Keep the first planning release read-only if changing events is not necessary to prove its value. ### Can an existing personal agent keep running after the call ends? Yes, if that work is implemented and persisted in your own backend. The avatar call is a temporary interface, not the lifetime of the task. Store a job identifier and show its actual status when the user returns; do not imply that RTA hosts the background agent. ### Will the assistant remember the user tomorrow? Only through the context your application chooses to retain and supply. Give users a way to correct saved preferences, and rebuild the next planning snapshot from current data rather than replaying a stale plan as fact. ### Should every personal assistant have a video avatar? Not necessarily. A quick calendar lookup may be better as text. Start with a moment that benefits from discussion, such as resolving competing priorities, and compare it with simpler interfaces. RTA gives you an interaction option, not proof that video improves productivity or trust. - [Review current plans and session costs](https://realtimeavatar.ai/pricing) - [Read the privacy and retention policy](https://realtimeavatar.ai/privacy) --- # Personal AI Assistant Memory: Corrections, Provenance and Migration Build personal AI assistant memory that preserves corrections, tracks provenance, imports only supported context, and stays accurate across new calls\. Published: 2026-10-12T14:36:37.009Z Updated: 2026-10-12T14:36:37.009Z Canonical: https://realtimeavatar.ai/blog/personal-ai-assistant-memory-correction-migration Markdown: [en](https://realtimeavatar.ai/blog/personal-ai-assistant-memory-correction-migration.md) Personal AI assistant memory needs a correction policy, not just more history. Store selected facts with their source, scope and validity; make explicit corrections supersede obsolete versions; and rebuild each new call's context from current records. Treat migration as importing evidence for review, not transferring permissions or guaranteed understanding. The lifecycle below is an application design recommendation, not a Realtime Avatar memory feature. ## Separate history, saved memory and call context History records what someone said. Saved memory records what your application currently treats as useful and valid. Call context is the smaller selection supplied for one interaction. Keeping these separate lets you preserve an earlier conversation without treating every sentence in it as current advice. For example, an assistant's suggestion to try morning meetings is not a user preference. Neither is a quoted colleague's schedule. Attribute statements to their speaker, distinguish inference from confirmation, and provide an inspectable source. A fluent explanation of where a memory came from is not provenance unless it matches a stored reference. ## Follow one preference through import and correction Consider fictional user Mara. She imports an older conversation saying, 'I prefer morning meetings.' Your application creates a candidate preference linked to that import batch and message, rather than immediately adding it to every call. Mara reviews it and confirms that it applies to work meetings. Later she says, 'From Monday, remember that I prefer afternoon meetings for work.' Confirm the intended Monday and whether this replaces the general preference, rather than describing one appointment. The saved record now needs an effective date as well as a write time: a future change should not silently rewrite today's preference. - Identity and scope: Mara's account, work context and the stable meeting-time preference key. - Evidence: source message, import batch, source time when known, and when Mara confirmed the preference. - Validity: morning or afternoon, effective interval, revision, and whether it is active, superseded or deleted. - Lineage: which prior revision this replaces and which summaries or search entries derive from it. Unknown source dates remain unknown. ## Correct the record without reviving stale copies Use one authoritative record per scoped preference. The following sequence is a recommended backend policy; asking a model to favor recent text is not equivalent. 1. Authorize the edit for the signed-in account. Compare the expected revision and commit the replacement plus supersession atomically. If another device already edited it, show the conflict rather than silently overwriting it. 2. Invalidate derived summaries and cached selections. Filter retrieval against authoritative validity before supplying context; an old search entry must not win because its wording is similar. 3. Prevent re-import and delayed extraction from promoting the old morning preference. Compare their source lineage with corrections, not their arrival time. Ambiguous imports remain candidates for review. 4. Confirm persistence before saying 'saved.' If the response is lost, reconcile the same operation identity instead of guessing whether the edit committed. Display the saved value, effective date and revision. On Monday's new call, select the afternoon revision. Exclude obsolete preference passages from replayed context, or explicitly label them historical when the user asks about the change. Import text stays untrusted data: it cannot rewrite server instructions. Keep enough provenance to explain the correction without repeatedly forwarding the original private conversation. ## Migration transfers particular artifacts, not an assistant Google's Gemini import page, checked October 12, 2026, illustrates two distinct paths. Memory import uses a summary of preferences and style generated in the previous app, then pasted into Gemini. Chat-history import uses an exported ZIP. Neither description establishes that another product will interpret every imported preference identically. The same page states these specific exclusions. They are Gemini's documented boundaries, not universal rules for other providers. - Project files cannot be imported through this flow. - Images generated in the previous app are not included in imported chat history. - Files generated in the previous app are not included either; the history transfer carries chat text. Google notes that compatibility and availability vary. This is evidence of a product surface, not universal portability or a privacy endorsement. For your importer, preview accepted, skipped and unresolved material. Keep imported summaries separate from user-confirmed facts, exclude credentials and unapproved work data, and report missing attachments rather than presenting a complete transfer. - [Google Gemini: import memories, chat history and documented exclusions](https://gemini.google/import-memory/) ## Make deletion propagate through your own copies If Mara deletes the preference, removing one row is insufficient while a summary can recreate it. Define whether the request also removes source history, then invalidate dependent summaries, indexes, caches and pending extraction work. Use a minimal deletion marker to reject late jobs without keeping the deleted preference text; disclose the marker's retention. Recheck deletion state when a job commits, not only when it starts. An already-running call may still contain supplied context. Do not claim it forgot instantly: stop using that session if you cannot reliably withdraw the context, then start fresh with a rebuilt selection. Track deletion failures and disclosed backup expiry separately from successful removal from active memory. ## Keep the live-call boundary explicit Realtime Avatar manages speech recognition, language-model responses, speech synthesis and video. Your server supplies instructions and selected context. Your application owns durable memory, permissions, execution and background tasks; RTA does not host your tools. New calls do not automatically inherit personal memory. Tools have a 2.5-second deadline with cooperative cancellation: a handler can still commit after its result is discarded. Slow imports therefore need app-owned durable work and an honest pending state. External language-model endpoints require workspace approval and a host allowlist. The public SDK does not accept arbitrary live audio from your own speech service. - [Realtime Avatar Calls: server-owned instructions and context](https://realtimeavatar.ai/docs/sessions) - [Realtime Avatar Tool calling: execution, deadlines and transcript delivery](https://realtimeavatar.ai/docs/tool-calling) ## A practical memory test matrix Run these cases against storage, assembled context and visible confirmation. They are acceptance criteria, not reported benchmark results. - Import the same archive twice: no duplicate active preference; provenance still identifies the batch. - Correct morning to afternoon: Monday's context contains the applicable revision, while Sunday's follows the earlier interval. - Deliver an old transcript after correction: it cannot promote morning back to active memory. - Race two device edits: one stale revision is rejected or explicitly reconciled; neither silently wins. - Delete during extraction: the late job cannot recreate the preference, and the next context excludes it. - Start a new call after a dropped save response: read authoritative state; report unknown if that read fails. - Import text claiming permission to access another account: no tool authorization or cross-account memory access changes. ## Frequently asked questions ### Will reconnecting reload a corrected preference? Do not assume transport recovery rebuilds context. For a new call, your server must select current memory again. Distinguish a saved edit from its application to the current conversation; 'saved for future calls' is more accurate when the running session still holds the previous context. ### Does deleting an imported chat erase its original copy? Do not promise that. Google's page describes deletion of an imported chat or import batch in Gemini, not deletion at the originating provider. Explain each system's scope and any separate user action needed. Your own derived preferences also need an explicit deletion policy. ### Can imported preferences grant tool access? In this design, no. Keep authorization outside memory. A remembered request to manage meetings cannot establish account ownership or authorize calendar writes. Reauthorize integrations separately and require explicit scope before carrying work preferences into a personal assistant. ### Does disabling transcript delivery mean nothing is retained? No. RTA's optional signed post-call transcript delivery is separate from operational diagnostics. Private session logs can contain conversation text and have a 30-day storage lifecycle. Other records and provider processing have separate retention rules. Do not use diagnostics as your application's memory store. Start with one inspectable preference and prove its correction survives the next call. Then use the Calls documentation to connect that memory policy to a live conversation. - [Review Realtime Avatar privacy and retention boundaries](https://realtimeavatar.ai/privacy) - [The broader personal-agent idea: give your agent a face](https://realtimeavatar.ai/blog/give-your-personal-agent-a-face) --- # Build a Personal Assistant That Continues from Phone to Desktop Design a phone\-to\-desktop assistant handoff with confirmed trip context, authenticated resume, fresh calls, and safeguards against stale device commands\. Published: 2026-10-12T15:04:23.773Z Updated: 2026-10-12T15:04:23.773Z Canonical: https://realtimeavatar.ai/blog/personal-assistant-phone-to-desktop-handoff Markdown: [en](https://realtimeavatar.ai/blog/personal-assistant-phone-to-desktop-handoff.md) Move the task, not the live connection. Save user-confirmed trip details in your application, authenticate the desktop resume request, and start a fresh avatar call with server-selected context. Retire the phone's authority separately from ending its media session. This preserves the useful plan without pretending Bluetooth, a shared account, or Realtime Avatar automatically transfers a conversation between devices. ## One trip, two different interfaces On a walk, a traveler uses your app's voice call through earbuds to discuss a weekend away. They confirm departure city, exact dates, two travelers, and a total budget with currency. At home, they explicitly open your desktop app to compare train times and hotel cancellation terms. The avatar can explain the tradeoffs while the application shows the options. Google's Pixel Buds page describes hands-free Gemini Live conversations. Its Gemini Live page describes switching between voice and typing within a conversation. These are useful examples of changing interaction modes, not evidence of a Pixel-to-Realtime-Avatar integration or a cross-vendor handoff protocol. Here, both device experiences belong to your application. - [Google Pixel Buds Pro 2: hands-free Gemini Live](https://store.google.com/product/pixel_buds_pro_2?hl=en-US) - [Google Gemini Live: voice and typing](https://gemini.google/overview/gemini-live/) ## Save a travel decision, not an assumed transcript Keep confirmed requirements separate from conversation. A task snapshot should distinguish the agreed dates and budget from unresolved questions such as whether a late return is acceptable. Record its version and confirmation time. An assistant's suggestion is not a traveler-approved requirement, and a misheard amount must remain unconfirmed. Persist confirmations through your authenticated application tools while the phone call is active. Realtime Avatar's optional signed transcript arrives after a call; it is not a guaranteed live handover feed. If the latest detail was never saved, show the last confirmed snapshot and ask again. Keep saved options as references with retrieval times, then refresh external prices and availability on desktop. ## Make resume an authenticated transaction Offer Continue on desktop on the phone and a pending-trip entry in the signed-in desktop app. Use a short-lived, opaque, single-use resume handle. Keep it in an authenticated response and submit it in a POST body, not a URL. Do not put transcripts, provider credentials, or session grants in links, QR codes, or analytics events. The handle locates a request; possession alone never authorizes it. 1. After desktop sign-in, show a minimal trip label and ask the user to continue here. Bind the resume request to that authenticated account and destination app session. 2. On redemption, check expiry, ownership, current permissions, and the expected task version. Reject a changed plan for review rather than silently resuming an older snapshot. Protect the mutation against cross-site request forgery and rate-limit failed attempts. 3. In one application-store transaction, consume the handle and advance the task's active-device generation. Concurrent redemptions must have one winner. Return an existing authorized transfer status on retries, not another live grant. 4. Resolve the approved snapshot on the server and mint a new Realtime Avatar call with its own identity. Supply behavior through instructions and relevant context through context. Never copy the phone's connection credentials to the desktop. Keep the trip identity stable while correlating both distinct call identities with the transfer. If the new mint fails, retain the saved plan and show a retry or text-only comparison. Do not silently restore the retired phone's authority. This transaction governs your database, not an atomic change across your database and the media service. ## End the old call without inventing certainty Ask the phone client to end its call, and have your server request release of the old session. The documented rta.endCall method is best-effort and idempotent: true means acknowledged; false leaves the outcome unknown. Record that distinction. An acknowledgement is not proof that a disconnected phone has updated its microphone indicator. Set a bounded maxSeconds policy when each call starts so an unreachable client cannot leave it open indefinitely. Track local capture cleanup separately from server release. If closure is uncertain, show that warning and keep desktop comparison silent until the user explicitly chooses to start audio. Never report that the phone stopped listening merely because desktop video appeared. - [Realtime Avatar Calls: context, hard stops, and server release](https://realtimeavatar.ai/docs/sessions) ## Fence stale commands at your application tools Two devices can remain connected even when only one may edit the trip. Bind each authenticated call to the generation your server assigned; never trust a client to claim the newest generation. Check that binding at the write boundary. An old phone's delayed instruction to change the destination must fail after desktop takeover, not overwrite the comparison. Application pseudocode follows. These are design steps for your own transactional store, not Realtime Avatar SDK methods. ``` APPLICATION PSEUDOCODE transaction: resolve authenticated call binding and authorized trip require current permission to read the operation outcome if an operation receipt exists: require receipt owner, trip and arguments match this request return existing receipt without executing another write require binding.generation == trip.activeGeneration require proposedChange.baseVersion == trip.version require current permission to edit this trip enforce unique operation ID for this owner and trip apply confirmed change and advance trip version persist operation receipt commit ``` Make the checks and write atomic. Mark the phone view as read-only after takeover; another takeover requires another explicit confirmation. Tool abort is cooperative, and Realtime Avatar's tool deadline is 2.5 seconds. Neither cancels an external action already dispatched. Keep this first handover flow read-only toward travel providers, and discard stale lookup results instead of letting them replace newer options. ## Respect the room around the user Switching Bluetooth output changes an audio route, not application identity or task ownership. Test earbud disconnects explicitly; do not assume private playback continues. Avoid speaking names, booking references, or sensitive details during transition. On desktop, show the plan before starting sound and ask before enabling the microphone. Mount the call only after the user's start action; the React integration begins connecting when mounted. ## Failure tests for the transfer boundary - Replay a handle, expire it, and redeem it from another account. No case may disclose the trip or mint a call. - Resume from two desktop tabs simultaneously. Only one acquires editing authority; the other shows current transfer status. - Delay a phone edit until after desktop takeover. The server rejects its old generation, including when the phone reconnects. - Lose the release response and fail desktop mint. Preserve the plan, label phone closure unknown, and offer an explicit retry without restoring stale authority. - Change a provider price between devices and disconnect earbuds during speech. Refresh the quote and test the actual audio route; neither cached prices nor transport state prove the user saw or heard the right thing. ## Frequently asked questions ### Does Realtime Avatar migrate the phone session? No native device-handoff mechanism is documented here. Your application restores an authorized task snapshot into a new call. It owns authentication, memory, tools, and transfer coordination. ### Can both devices stay open? Yes, as an application design: keep one active editor and make the other view read-only. Opening a second screen must not silently grant it microphone access or command authority. ### Can I pipe Gemini or another assistant's live audio into the avatar? The public SDK does not accept arbitrary external live audio input. Realtime Avatar manages speech recognition, the language model, speech synthesis, and video. External model endpoints require approval and host allowlisting. ### Must desktop continuation include video? No. Offer your application's comparison screen without a call. Add an optional avatar when an explanation helps; continuing a task should not require broadcasting a private conversation. ## Build the smallest honest handover Start with one confirmed trip, one authenticated resume, and one fresh call. Prove stale commands are rejected before adding booking actions. Use the quickstart for the call integration, not as a substitute for transfer authorization. - [Build your first call with the Realtime Avatar quickstart](https://realtimeavatar.ai/docs/quickstart) - [Read the application-owned tool boundary](https://realtimeavatar.ai/docs/tool-calling) - [Control when React and Native calls start](https://realtimeavatar.ai/docs/react) --- # Build a Talking Virtual Pet Players Want to Visit, Not Worry About Build a talking virtual pet around tiny expeditions, meaningful choices and a personal scrapbook, with satisfying visits and no missed\-day guilt\. Published: 2026-10-12T18:04:37.739Z Updated: 2026-10-12T18:04:37.739Z Canonical: https://realtimeavatar.ai/blog/talking-virtual-pet-avatar-game Markdown: [en](https://realtimeavatar.ai/blog/talking-virtual-pet-avatar-game.md) Build a talking virtual pet around something you do together: choose a tiny expedition, help with one delightful problem, and keep a discovery. Give each visit an ending instead of a maintenance backlog. Your app owns the adventure and scrapbook; Realtime Avatar can supply conversation. Start with an illustrated creature and voice, making live portrait video conditional on testing the exact artwork. ## Meet Mallow, collector of extremely small wonders Mallow's Pocket Post is an original proposed game for adults and general audiences, not a shipped Realtime Avatar product or a children's app. Mallow is a humanoid woodland sprite with moss eyebrows and an explorer's seriousness about ridiculous things. Their portrait sits beside an illustrated habitat and a pocket scrapbook. There is no hunger bar. Today's urgent discovery is a seedpod that appears to be answering back. Aim for a three-to-five-minute visit, a playtest target rather than a performance guarantee. The appeal is making something recognizably yours: a discovery, a title, a little corner of a world. This is play, not a disguised task manager. ## Play one visit: the wind has terrible manners 1. Choose a curiosity: Follow a sound, Chase a sparkle, or Investigate something odd. These select different authored events, not different adjectives for the same conversation. Pick Follow a sound today. 2. Pack a thimble, a ribbon, or nothing. These are reusable pretend expedition supplies, never scarce purchases. Choose the ribbon. The app shows a brief illustrated transition, labelled A little later; this is story time, not a real waiting timer. 3. Hear Mallow's report: ‘I said hello into a seedpod. It said something considerably squeakier. Should we introduce ourselves properly?’ A scene card shows the hollow pod and your ribbon. Ask a question, then choose Use the ribbon or Just watch. 4. Use the ribbon to hang the pod where the breeze can turn it. Mallow offers an original fictional line: ‘It isn't answering. The wind keeps interrupting. Honestly, we're trying to have a conversation.’ The discovery card reveals a wind-note spiral. Just watch instead records a sketch of the resting pod. 5. Choose a scrapbook border, name the discovery, and pin its card in the habitat or leave it in the book. Call this one The Wind Has Terrible Manners. The card previews before saving. Mallow closes with a cheerful goodbye; the next conversation is not required to finish today's story. That last joke can become a future callback. On another windy expedition, the app supplies the saved discovery and Mallow might ask, ‘Shall we ask the wind nicely this time?’ A player who watched quietly gets a callback about their seedpod sketch instead. The game remembers a choice, not a claim that the creature suffered while you were away. ## Three events, with genuinely different discoveries Author these as three selectable episodes. Within each, Use my item and Just watch remain available; packing nothing replaces the item action with Observe together. Observing always produces that scene's sketch and completes the visit. No route is a failed expedition. - The answering seedpod, reached through Sound: a thimble gently taps out a three-note pattern, earning a rhythm-pattern card. A ribbon suspends the pod, earning the wind-note spiral. Observers keep a seedpod sketch. ‘We have discovered music small enough to lose in a pocket.’ - The upside-down stars, reached through Sparkle: dew glitters beneath a leaf. Catch a falling drop in the thimble for a dotted-constellation card, or stretch the ribbon across the light for a striped-lantern card. Observers keep a leaf-and-dew sketch. ‘The sky is leaking very politely.’ - The pebble's tea invitation, reached through Odd: a pebble resembles a seated guest. The thimble becomes its cup, producing a miniature place-setting card; the ribbon becomes a napkin, producing a bow-framed guest portrait. Observers sketch the pebble. ‘Excellent company. Hasn't interrupted once.’ Cards are authored illustrations, not promised physical performances by the avatar. Changing the curiosity changes the scene; changing the item changes the discovery and its next callback. Decoration expresses taste without becoming a score. Keep every route equally complete, rather than hiding the ‘good pet owner’ ending behind one answer. ## What existing pet players actually said Weyrdlets 2.0: Desktop Pets' official Steam listing describes pet accessories, home decoration, desktop companionship and minigames. It also includes productivity features; this recipe deliberately borrows the play-and-expression question, not the to-do list. These are seller-described features, not our hands-on findings or capabilities inherited by Realtime Avatar. On October 12, 2026, we read one page of twenty most recent English Steam reviews, covering March 18–August 21, 2026, with all purchase and recommendation types included. A June 5 reviewer praised customization and furniture to unlock. A May 9 reviewer liked the cute desktop creature and additional activities. Conversely, an August 3 reviewer described the expanded activities as demanding and creating fear of missing out; an April 25 recommendation still said play could become boring. These are unverified anecdotes, not representative findings or evidence of attachment or retention. Nineteen entries were marked received for free; the August 3 reviewer wanted a productivity-oriented companion, limiting transfer to an entertainment audience. The sample suggests competing questions: is there enough surprise, and can players comfortably stop? It does not establish that Weyrdlets penalizes absence. Mallow's no-penalty rule is our design choice. - [Weyrdlets: official Steam feature listing](https://store.steampowered.com/app/2366060/Weyrdlets/) - [Collected Steam sample: twenty recent English reviews](https://store.steampowered.com/appreviews/2366060?json=1&filter=recent&language=english&num_per_page=20&purchase_type=all&review_type=all&filter_offtopic_activity=1) ## Make return visits optional, not overdue Completion means saving one discovery card, not reaching a friendship score. Retry opens another take with different choices; it keeps the existing card until you explicitly save an additional version. Stop closes conversation immediately and keeps the last confirmed step for Resume or Start over. Pass swaps an unopened event for another curiosity without consuming supplies. Release new authored postcards into an archive, perhaps one per day, without expiry or streak bonuses. Miss a week and Mallow is simply pleased to see you; there is no illness, death, debt or reproach. Let players choose an unseen postcard or replay a favorite. Fresh events must bring new situations and discoveries, not merely new generated greetings. ## Add the talking portrait without promising a pet simulator RTA documents creation from a portrait, but the reviewed documentation does not explicitly establish arbitrary animal or stylized-creature compatibility. Mallow's humanlike face is a candidate to test, not verified support. Begin with a ready RTA avatar in voice mode behind your app's original static creature illustration. Add live creature video only after testing that exact portrait; otherwise keep the illustrated version. No animal-rendering test was performed for this article. Your server supplies persona, the selected event and saved discovery context. Your app saves choices and renders habitat cards; generated speech cannot award or replace them. Use application-driven turns for confirmed outcomes. If adding conversational lookups, RTA tools run in your application with a 2.5-second deadline and cooperative cancellation, so a timeout does not prove a save failed. The recipe needs no full-body rig, precise choreography or touch detection. - [Creating an avatar: portrait inputs and preparation](https://realtimeavatar.ai/docs/video) - [Calls: voice mode and application-supplied context](https://realtimeavatar.ai/docs/sessions) - [Tool calling: deadlines and cooperative cancellation](https://realtimeavatar.ai/docs/tool-calling) ## Small tests before inviting players - Replay every event with each item and with none. Check the distinct card, observation ending and later callback match the confirmed choices. - Return after a simulated month away. Nothing deteriorates, expires or asks for an apology. - Disconnect during a save, then retry. Recover the confirmed card once; never silently duplicate it or announce success before checking. - Disable voice and video. Authored report text, choices, decoration, Pass, Stop and Resume must still complete the same visit. ## Frequently asked questions ### Can I make a talking cat or dragon? You can illustrate either in your app. Do not promise its live RTA rendering without validating the exact portrait; arbitrary animal support is not established here. ### Does Mallow remember everything automatically? No. Save chosen events, cards and approved titles in your app, then supply the relevant callback on the next call. Avoid treating improvised dialogue as established history. ### What if someone does not want to speak? Keep the complete authored adventure playable through text and buttons, including item-free observation. Talking adds character; it is not an entrance exam. ### Will this make players return every day? That is unproven, and daily attendance is not the goal. Playtest whether people enjoy their card, notice a changed outcome and want another expedition. Make no therapeutic, retention, conversion or search-demand promise. --- # Building Interactive Avatar Training Courses with Realtime AI Explore the engineering of dynamic, real\-time AI avatar training courses, moving beyond static video to interactive, adaptive learning experiences\. Published: 2026-10-11T02:46:36.169Z Updated: 2026-10-11T02:46:36.169Z Canonical: https://realtimeavatar.ai/blog/building-interactive-avatar-training-courses-with-realtime-ai Markdown: [en](https://realtimeavatar.ai/blog/building-interactive-avatar-training-courses-with-realtime-ai.md) The design and delivery of training courses are undergoing a significant shift, moving from static content to dynamic, interactive experiences. Realtime AI avatars are at the forefront of this evolution, enabling the creation of training modules that can listen, respond, adapt, and even role-play with learners. This capability extends beyond simply presenting information through an avatar; it empowers developers to engineer truly conversational and adaptive learning environments where the avatar acts as an instructor, a simulated client, or a practice partner. ## From Static Content to Dynamic Dialogues Traditional avatar-led training often relies on pre-rendered video, where an avatar delivers information in a one-way fashion, much like a traditional lecture. While effective for consistent content delivery at scale, this approach lacks the immediate feedback and adaptability crucial for soft skills development, complex scenario training, or personalized learning paths. The emergence of realtime AI avatars fundamentally changes this paradigm. Instead of mere playback, the avatar becomes an active participant in the learning process, capable of engaging in full-duplex conversation. A real-time avatar dynamically processes spoken input, understands context, and generates a live, lip-synced video response. This is not about scripted branching paths; it is about genuinely adaptive dialogue. For training, this means an avatar can: - Provide immediate, context-aware feedback: Responding directly to a learner's answer or query, acknowledging nuances. - Simulate complex interactions: Acting as a customer with specific objections, a colleague needing assistance, or an interviewee testing communication skills. Platforms such as Virti [source:6] and UneeQ [source:7] are purpose-built for this kind of interactive conversational practice with digital humans, offering customizable interactions [source:8]. - Adapt to learner pace and style: Recognizing when a learner needs more explanation or is ready to move forward, without explicit menu choices. - Support multilingual learning: Handling language switches within a single interaction, making global training more accessible. - Integrate with Learning Management Systems (LMS): Solutions can integrate with existing platforms, exemplified by uQualio [source:5] for transforming content into avatar-led courses. The core engineering challenge shifts from video production to designing robust conversational AI and integrating it with a high-fidelity, low-latency avatar frontend. The objective is to foster a sense of presence and engagement, making the training feel less like an automated system and more like an authentic interaction with a live instructor or peer. This capability is applied across industries, from corporate soft skills development and onboarding to specialized healthcare simulations [source:12] and even metaverse education environments like ViewSonic's UNIVERSE [source:11]. ## Engineering Interactive Training Scenarios with Tool Calling To move beyond mere conversation and create truly interactive training courses, avatars need the ability to perform actions within a simulated environment or to access external data. This is where tool calling becomes indispensable. In a training context, tools can simulate real-world actions or access information vital for the scenario, such as looking up product specifications, recording a learner's score, or advancing a simulated project phase. ### Defining Avatar Tools for Training With the Realtime Avatar SDK, tools are declared with a description that tells the underlying AI agent *when to reach for it*. For a training course, this description is key to guiding the avatar's decision-making within a simulated lesson. ``` import type { AvatarTool } from "realtime-avatar/tools"; export const recordScore: AvatarTool<{ score: number, lessonId: string }> = { description: "Records a learner's score for a specific lesson. Call this whenever " + "the learner completes a quiz or achieves a quantifiable outcome.", parameters: { type: "object", properties: { score: { type: "number" }, lessonId: { type: "string" } }, required: ["score", "lessonId"] }, execute: async ({ score, lessonId }, { signal }) => { // Simulate saving the score to a learning management system await new Promise(resolve => setTimeout(resolve, 500)); // API call latency console.log(`Score ${score} recorded for lesson ${lessonId}.`); return `Your score of ${score} for lesson ${lessonId} has been recorded.`; } }; ``` This `recordScore` tool allows the avatar to not just verbally acknowledge a correct answer but also to log it, providing concrete progress tracking. Other training-specific tools could include `simulateCustomerResponse`, `fetchPolicyDetails`, or `triggerScenarioEvent`. ### Integrating Tools into the Training Flow After defining the tools, your server grants the necessary capabilities, and the client-side application registers them with the avatar session. The `realtime-avatar` SDK handles the bridge between the avatar's internal decision-making and the execution of your defined functions. The key is to design tools that are specific enough for the avatar to understand their purpose, yet flexible enough to handle various inputs from the conversation. ``` // server-side session minting (e.g., in an Express or Hono handler) // Ensure 'client_tools' capability is granted. // Refer to: /docs/tool-calling const body = {"avatar_id": avatarId, "mode": "avatar", "capabilities": ["client_tools"]}; const mintResponse = await fetch("https://api.realtimeavatar.ai/v1/sessions", { method: "POST", headers: { "Authorization": `Bearer ${process.env.REALTIME_AVATAR_API_KEY}` }, body: JSON.stringify(body) }); const { token } = await mintResponse.json(); // client-side (e.g., a React component using the SDK) import { attachAvatarTools } from "realtime-avatar/tools"; import { AvatarCall, useAvatarClient } from "realtime-avatar/react"; import { recordScore } from './myTrainingTools'; // Your defined tools function TrainingAvatarComponent({ avatarId, sessionToken }) { const client = useAvatarClient(sessionToken); useEffect(() => { if (client) { attachAvatarTools(client.room, { record_score: recordScore }) .then(({ accepted, rejected }) => { console.log("Tools attached:", accepted); if (rejected.length > 0) { console.warn("Tools rejected:", rejected); } }); } }, [client]); // `recordScore` is an imported constant, so it is stable. ``` This architecture allows for sophisticated training logic. When a learner answers a question, the avatar's language model determines if a tool like `recordScore` should be called based on the conversational context and the tool's description. The tool executes, and its return value can then be incorporated into the avatar's subsequent verbal response, closing the loop and making the interaction feel seamless and purposeful. Note that Synthesia [source:10] also offers a LiveKit plugin for building real-time interactive avatars with agent layers and rubric scoring capabilities. ## Limitations and Considerations While realtime AI avatars offer unprecedented interactivity, it is important to understand their operational parameters. The core avatar platform provides the visual and auditory embodiment, with precise lip-sync to the syllable and full-duplex audio for natural interruption handling. This means the avatar listens while it talks, crucial for dynamic training where a learner might interject. A cough or an "mm-hm" will not derail the conversation, and a pause is interpreted by what was said, not merely by silence, preventing awkward gaps. - [Realtime Avatar API: She listens while she talks](https://realtimeavatar.ai/docs#she-listens-while-she-talks) The primary limitation to note is that the avatar will not initiate a new sentence over your speech. This design choice prioritizes a coherent, understandable interaction flow over simultaneous overlapping speech for new ideas. Furthermore, while the platform handles the real-time video generation from a single portrait image, the intelligence behind the training scenarios—the prompts, the agent logic, the tool definitions, and the evaluative framework—resides with the developer. The efficacy of an avatar training course is ultimately a reflection of the carefully designed interaction and the underlying AI agent's capabilities. Latency depends on factors such as startup state, model complexity, and network conditions. ## How to Ship Interactive Training Avatars Shipping an interactive avatar training course begins with prototyping simple conversational flows and gradually layering in complexity. Start by creating an avatar from a single portrait image in the studio. Experiment with the `realtime-avatar` TypeScript SDK to integrate an avatar into your web application, focusing initially on basic conversational exchanges. The quickstart guide is a direct path to getting a live call running in three steps. - [Quickstart: Key to live call in three steps](https://realtimeavatar.ai/docs/quickstart) - [Creating an avatar: One image in](https://realtimeavatar.ai/docs/video) Next, develop your agent's core logic. This often involves an LLM orchestrator that processes learner input, decides on a response, and identifies when to invoke a tool. The tool-calling pattern is essential for any training that requires actions, data retrieval, or state changes. Remember that tools run client-side in the browser, providing a 2.5-second execution window. For longer operations, acknowledge quickly and deliver results out of band. This design ensures the avatar remains responsive during complex operations. - [Tool calling: Running your own agent loop](https://realtimeavatar.ai/docs/tool-calling) Finally, consider the hosting and scaling of your training solution. The Realtime Avatar API offers usage-based pricing anchored at about $5 per hour for realtime *overage* minutes, with a free tier to build against. This economic model is designed to support scalable companion applications and pervasive training, allowing for extensive learner engagement without prohibitive costs. By combining a robust real-time avatar platform with well-crafted agent logic and strategic tool integration, you can deliver training experiences that are not only scalable and consistent, but also deeply engaging and effective. - [API Pricing: Included minutes, overage, and total cost](https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing) - [Realtime Avatar API: Voice or video, on one meter](https://realtimeavatar.ai/docs#voice-or-video-on-one-meter) - [infibee.in](https://infibee.in/courses/ai-avatar-creation-course/) - [udemy.com](https://www.udemy.com/course/ai-avatar-masterclass-how-to-create-talking-ai-avatar/) - [coursera.org](https://www.coursera.org/learn/ai-video-avatars-launch-your-automated-social-media-machine) --- # Your personal agent deserves a face Instinct, Muse, and Tab are making personal agents part of everyday life\. Realtime Avatar helps developers give their own agent a face people can talk to\. Published: 2026-10-11T11:29:34.832Z Updated: 2026-10-11T11:29:34.832Z Canonical: https://realtimeavatar.ai/blog/give-your-personal-agent-a-face Markdown: [en](https://realtimeavatar.ai/blog/give-your-personal-agent-a-face.md) A personal agent becomes more useful when you can think out loud with it. You can explain why a plan feels wrong, interrupt an assumption, or work through a decision before handing over the task. At Realtime Avatar, we see a place for a face in those conversations: a familiar character you can call when a message thread feels too narrow for what you need to say. Imagine opening your agent before a busy day. It has your priorities and a few decisions waiting. You start talking through the first one, change your mind halfway through, and ask it to take a different approach. The useful part is the work it can do. The experience is the conversation you have while deciding what that work should be. ## Instinct, Muse, and the personal-agent moment Instinct makes the interaction familiar: you text or call an assistant that can operate a phone and computer. Its examples include ordering groceries, arranging appointments, and planning a weekend away. The invitation is to delegate something from your actual life through a conversation. - [Instinct: personal assistant product overview](https://instinct.com/) Meta introduced Muse on September 8, 2026, with a similar ambition at a much larger platform scale. Muse works through its app and WhatsApp, uses a dedicated virtual computer, and can continue working after you close the app. Meta describes an agent that remembers relevant details and returns when it needs your input. - [Meta: Introducing Muse](https://about.fb.com/news/2026/09/introducing-muse-personal-ai-agent/) Tab is another recent entrant built around messaging. Its site describes an assistant with its own computer and browser, able to work with documents, make phone calls, and bring results back into the same thread. Together, these products make an appealing promise: you can ask for help in ordinary language and return later to work that has moved forward. - [Tab: Meet your AI in Messages](https://tab.bot/) For developers building personal agents, that raises an interesting design opportunity. An agent may handle dozens of tasks quietly in the background. The moments when someone wants to discuss a choice, explain a concern, or change direction deserve just as much attention as the automation itself. ## A face for the moments that need a conversation Meta has already made an explicit move in this direction. On September 23, it introduced Muse Realtime Avatar, describing technology that turns reference images into expressive characters for live conversations. Its announcement connects conversational intelligence, voice, and visual performance in a streaming system. Meta also notes that the showcased characters do not all represent avatars available in the app. - [Meta AI Research: Bringing Your Muse to Life](https://research.meta.ai/blog/bringing-your-muse-to-life/) Our view is that a face earns its place when the conversation benefits from attention and back-and-forth. Consider a travel agent that has found three possible weekends. You open a call and explain that the cheapest option looks exhausting. It can ask what matters more, walk through the tradeoff, and help you settle on a plan. The itinerary remains visible beside the conversation. Or imagine a personal agent that helps you prepare for a difficult meeting. You rehearse the opening aloud. It responds, you interrupt, and you try again. A consistent face and voice give the exercise a clear counterpart. The experience is designed around practicing a conversation you will actually have. A daily check-in offers another starting point. Yesterday you said a project was stuck. Today, the same character opens with a relevant question, using context your application has retained. You can explain what changed without reconstructing the entire story. Familiarity comes from that continuity as much as from the portrait on screen. These are product scenarios, with outcomes that need testing. A face alone does not establish that users will return more often or finish more tasks. We would look for simpler evidence first: can people correct the agent easily, understand what it is doing, and leave the call with a useful next step? ## Build the conversation with Realtime Avatar Realtime Avatar gives developers the voice and video call experience for their own product. Start with a portrait, create a character, and shape its voice and instructions. The platform supports live conversations and interruptions, so a user can redirect the exchange while the character is speaking. - [Realtime Avatar: Creating an avatar](https://realtimeavatar.ai/docs/video) - [Realtime Avatar: Voice and video calls](https://realtimeavatar.ai/docs) Your application supplies the personal context. Before a call, it can provide relevant preferences, open tasks, and what the user was trying to accomplish last time. Keeping that context focused helps the character begin with something useful to say. The application decides what to remember across calls and what to leave behind. The connection to useful work is equally concrete. RTA supports tools implemented in your application and a way for your backend to steer a reply using the result of its own agent logic. A planning assistant might retrieve available times; a project companion might check the next unfinished task. Credentials and action permissions stay with the application that executes those tools. - [Realtime Avatar: Tool calling and your own agent loop](https://realtimeavatar.ai/docs/tool-calling) That lets you build a coherent handoff: the person talks through a goal, your agent does the work, and the character explains the result. Longer jobs can continue in your backend after the call ends. When the person returns, your application brings the relevant progress into the next conversation. Keep the useful details on screen, too. A spoken explanation can sit beside a calendar, a draft, or a list of choices. Let someone switch to text or voice when that suits the moment. The face should make the product easier to use, and the interface should remain flexible enough for a quick task on a crowded train. ## Start with one reason to call The most promising first version has a specific reason to exist: a morning planning conversation, a rehearsal before a meeting, or a check-in on a goal someone cares about. Give that experience a recognizable character, a useful piece of context, and one action it can help complete. Instinct, Muse, and Tab show how personal software can grow around an ongoing conversation. For builders, adding a face is an opportunity to make selected moments in that conversation feel more present. Your agent can keep working in the background and still be someone the user knows how to call. Start in the Realtime Avatar studio, find the character that fits your product, and build one conversation worth coming back to. - [Try Realtime Avatar](https://realtimeavatar.ai/studio) - [Build your first avatar call](https://realtimeavatar.ai/docs/quickstart) --- # From Portrait to Presence: Realtime AI Avatars for Dynamic Training Explore how to build interactive AI training experiences using realtime avatars, created from a single image and deployed for live, dynamic learning\. Published: 2026-10-10T02:45:50.855Z Updated: 2026-10-10T02:45:50.855Z Canonical: https://realtimeavatar.ai/blog/from-portrait-to-presence-ai-avatars-for-realtime-training Markdown: [en](https://realtimeavatar.ai/blog/from-portrait-to-presence-ai-avatars-for-realtime-training.md) While the term 'AI avatar training video generator' often brings to mind platforms that produce static, pre-rendered video files from a script, the landscape of AI-driven learning is evolving. The focus has shifted from mere generation to real-time, interactive experiences. Instead of a fixed video, a realtime AI avatar platform enables live, conversational engagement, offering dynamic training sessions that can adapt to a learner's questions and pace. These live interactions, powered by an AI avatar, can then be recorded, providing an audit trail or an asynchronous resource that combines the immediacy of a live session with the convenience of on-demand content. ## From Portrait to Presence: Crafting a Live Training Avatar The foundation of an engaging training avatar begins with its creation. Unlike systems that might rely on extensive video footage or complex 3D modeling, our approach distills presence from a single, high-quality portrait image. This image serves as the blueprint from which the platform generates a fully animated, looping idle video and an entire motion library. This library equips the avatar with a range of natural movements and expressions, all derived from that initial static frame. The emphasis here is on efficiency and authenticity. By starting with a single portrait, developers can rapidly prototype and deploy unique characters without the overhead of lengthy video capture sessions or intricate facial rigging. The platform's generative capabilities handle the visual embodiment, ensuring that the avatar moves and breathes with a consistent, lifelike quality. Once created, these avatars are designed for live performance, ready to engage in dynamic conversations. This distinction is critical for training applications, where immediate feedback and personalized instruction can significantly impact learning outcomes. The system audio-clocks the avatar's lips to the syllable, ensuring precise synchronization between spoken words and facial movements, a crucial element for believable and effective communication in a learning context. - [See how to create an avatar from a single image](https://realtimeavatar.ai/docs/video) ## Live Engagement, Lasting Impact: The Realtime Advantage in Training Traditional training videos, while useful, often present a passive learning experience. Realtime AI avatars transform this by introducing active engagement. Imagine an AI tutor who can dynamically respond to a trainee's questions, clarify complex concepts on demand, or even role-play scenarios. This immediate, two-way interaction turns a monologue into a dialogue, fostering deeper understanding and retention. The core strength of a realtime avatar in a training context lies in its ability to facilitate unscripted, adaptive conversations. This is achieved by connecting the avatar to an AI agent pipeline, typically involving speech-to-text (STT), a large language model (LLM), and text-to-speech (TTS). The realtime-avatar TypeScript SDK simplifies this integration, providing a typed interface generated from our OpenAPI specification. For instance, an avatar can serve as an AI interviewer for practice, a product demonstrator, or a language learning partner. The interactivity allows for complex training simulations where the avatar's responses can branch based on the user's input, leading to a truly personalized learning path. This contrasts sharply with pre-recorded videos, which offer a one-size-fits-all approach. While research notes that AI avatars may not always replicate the full relational dynamics of human learning, they can significantly increase engagement, information retention, and reduce cognitive load in instructional settings. The critical element is the real-time feedback loop, allowing for clarification and elaboration in the moment, which static video cannot provide. Human conversation typically involves a turnover of roughly 200 milliseconds, a benchmark against which interactive systems evaluate responsiveness for natural flow (source:8). ### Capturing the Session: Recording for Audit and Asynchronous Use A live, interactive training session with an AI avatar is valuable in itself, but the ability to record these sessions extends their utility. For compliance training, regulatory audits, or simply to allow trainees to revisit content, server-side recording is a robust feature. The recording captures both audio and video, providing a complete artifact of the interaction. ``` import { createRealtimeAvatarRoute } from "realtime-avatar/next"; // app/api/realtime-avatar/[...path]/route.ts export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, session: async ({ avatarId }) => ({ instructions: /* Your agent's initial prompt here */, recording: "audio_video", // "off" | "audio" | "video" | "audio_video" }), }); // server-only follow-up, after the call: const artifact = await rta.getRecording(recordingId); if (artifact.status === "ready") { const playback = await rta.getRecordingAccess(artifact.recordingId); // Keep recordingId; playback.url expires and can be renewed. } ``` This server-side policy ensures that recording is managed securely and consistently, independent of the client application. After a call concludes, a typed artifact is generated, allowing for programmatic access and storage. This enables developers to build sophisticated training platforms that leverage live interaction for primary delivery and recorded sessions for review, documentation, or even content analysis and iteration. - [Learn more about server-side recording](https://realtimeavatar.ai/docs/nextjs#record-a-call-on-the-server) ## Engineering for Live Learning: Implementation and Considerations Integrating a realtime AI avatar into a training application is streamlined with the provided TypeScript SDK. The realtime-avatar package offers components and utilities to manage the avatar's lifecycle, from connection to interaction. ### Basic Integration with React For a web application built with React, the AvatarCall component provides a straightforward way to render and interact with a live avatar: ``` import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); ``` This simple component handles the streaming video, audio, and synchronization, bringing the avatar to life. For voice-only interactions, perhaps in a scenario where visual presence is not required or bandwidth is limited, the mode="voice" prop can be applied: ``` ``` - [Explore the React SDK documentation](https://realtimeavatar.ai/docs/react) ### Monitoring Connection and Performance In live training environments, understanding connection quality and latency is paramount for a smooth user experience. The SDK provides mechanisms to monitor these aspects without promising specific latency numbers, which can vary widely based on network conditions, client hardware, and server load. ``` import { useState } from "react"; import { AvatarCall, type AvatarCallProps, type AvatarConnectionDetails } from "realtime-avatar/react"; function CallWithDetails(props: Pick) { const [details, setDetails] = useState(null); return <> {details && Local connection: {details.localQuality}} ; } ``` This onConnectionDetailsChange callback provides insights into connectionState, localQuality, and separate audio/video publisher quality. These metrics allow developers to build robust UIs that inform users of potential issues or adapt the experience to current network conditions, optimizing for different training environments. - [Read about optional connection details](https://realtimeavatar.ai/docs/react#optional-connection-details) - [Benchmarking startup and turn latency](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) ### Limitations in Live Avatar-Based Training While powerful, real-time avatar systems have practical limitations. Generative AI, while advanced, can sometimes exhibit visual inconsistencies or unstable human detail. This might manifest as minor facial shifts or subtle awkwardness in body movements, especially during extended interactions. Maintaining perfect character consistency across long, complex narratives remains an area of ongoing research and development (source:1). Ethical considerations are also paramount. Transparency about AI-generated content, explicit consent for likeness and voice if real people are digitized, and careful management of biases in training data are crucial responsibilities for any developer deploying these technologies (source:2, source:4). ## How to Ship Realtime Training with AI Avatars Shipping an AI avatar-driven training application requires attention to both technical implementation and operational costs. The realtime nature of these interactions means that usage is metered by active engagement, rather than pre-processed video minutes. Our platform's pricing model is designed to accommodate sustained interaction. Realtime usage is anchored at about $5 per hour, with plans offering overage rates between $0.07-$0.095 per minute, translating to approximately $4.80-$5.70 per hour depending on your plan. This cost structure supports the core value proposition: enabling learners to spend significant time engaging directly with an AI avatar, practicing skills, and asking questions in a conversational format without prohibitive expense. For rapid deployment, the TypeScript SDK, generated directly from our OpenAPI document, offers a robust and type-safe development experience. Developers can leverage existing TypeScript expertise and integrate with modern web frameworks like Next.js or React. The public OpenAPI document at /openapi.json and agent-readable documentation at /llms.txt also provide comprehensive resources for deeper integration and custom agent logic. Consider the architecture: your application will manage the user interface, session lifecycle, and integrate with your chosen LLM and other tools via our SDK's tool calling capabilities. The avatar itself becomes a component of your overall training solution, handling the embodied presentation of your AI agent's responses. Recording capabilities provide an essential backend feature for accountability and content improvement, transforming ephemeral live interactions into persistent, valuable assets. Ultimately, moving beyond static video generation to realtime AI avatars for training means embracing dynamic, personalized, and scalable learning experiences. The tools are here; the next step is architecting them into compelling educational products. - [Explore our pricing for realtime interaction](https://realtimeavatar.ai/pricing) - [Access the OpenAPI document](https://realtimeavatar.ai/openapi.json) - [Review agent-readable documentation](https://realtimeavatar.ai/llms.txt) - [Quickstart guide to building a live call](https://realtimeavatar.ai/docs/quickstart) - [visla.us](https://www.visla.us/blog/guides/understanding-ai-avatar-videos/) - [wideo.co](https://wideo.co/blog/ai-avatar-generator/) - [sundaysky.com](https://sundaysky.com/blog/how-ai-avatars-are-revolutionizing-video-content/) - [substack.com](https://marcwatkins.substack.com/p/synthetic-selves-the-ethics-of-real) --- # Field Notes: Realtime AI Avatars for Engaging and Scalable Training Explore how realtime AI avatars transform corporate training, offering scalable, consistent, and interactive learning experiences\. Understand creation\. Published: 2026-10-09T02:46:07.738Z Updated: 2026-10-09T02:46:07.738Z Canonical: https://realtimeavatar.ai/blog/field-notes-realtime-ai-avatars-for-engaging-and-scalable-training Markdown: [en](https://realtimeavatar.ai/blog/field-notes-realtime-ai-avatars-for-engaging-and-scalable-training.md) Realtime AI avatars are rapidly redefining the landscape of training and development, offering a powerful blend of scalability, consistency, and interactive engagement previously unattainable with traditional video production. These digital presences enable organizations to deploy uniform, localized content across global teams, reducing the time and cost associated with producing and updating training materials while enhancing the learner's experience through direct, conversational interactions. They move beyond mere video playback to provide a dynamic, responsive learning environment. ## The Architecture of Presence in Training The core utility of a realtime AI avatar in a training context lies in its ability to manifest a consistent, engaging presence that scales on demand. This shifts the paradigm from static, pre-recorded modules to dynamic, adaptable learning experiences. Historically, producing a single 10-minute compliance training video might consume weeks or months; with AI avatars, this workflow can often be condensed to days, while drastically cutting production costs (source:1, source:2). - Scalability and Consistency: AI avatars ensure uniform messaging and presentation across vast, distributed teams, breaking the dependency on individual human presenters (source:1). Content updates, driven by policy shifts or branding changes, become fluid operations rather than resource-intensive overhauls. - Multilingual Support and Localization: Platforms enable instantaneous translation and localization across a multitude of languages and accents, making training intrinsically more accessible and inclusive for diverse workforces (source:1). - Enhanced Engagement and Learning Outcomes: While context matters, interactive AI avatars have demonstrated the capacity to improve learner motivation and retention, fostering a sense of familiarity and direct interaction with complex subjects (source:6, source:10). - Personalized and Interactive Training: Conversational AI avatars can function as virtual coaches, capable of listening, questioning, responding contextually, and adapting their speech in real-time. This allows for personalized learning paths, continuous automated assessment, and the practice of human-centered skills in simulated, judgment-free environments (source:1). For scenario-based training, such as customer relations or conflict management role-play, the capacity for real-time interaction is not merely an enhancement—it is the foundation of the learning experience. The avatar becomes a responsive partner, creating simulations that feel genuinely immediate. ### Crafting the Avatar and Its Motion The journey from a single portrait image to a live, interactive avatar is a foundational step in deploying these training tools. The platform generates a looping idle video and a multi-clip motion library from this single frame, providing all the necessary visual assets for a live call. This process is initiated through a straightforward API call (refer to the OpenAPI document for full details) or via the studio interface. ``` const newAvatar = await rta.createAvatar({ displayName: "Training Guide Alpha", sourceAssetId: "", motionPrompt: "leans forward slightly, a welcoming expression, then settles back" }); // Poll getAvatar(newAvatar.id) until status is 'ready' and idleVideoStatus is 'ready'. ``` The motionPrompt at creation allows for art-direction of the generated resting loop—the avatar's default posture and subtle movements when not actively speaking or gesturing. This loop is a closed arc, meticulously designed to begin and end on the portrait, ensuring a seamless, natural background state. The avatar's motion library comprises short, closed-arc clips—a signature idle variant, a listening state, specific gestures—each returning to its starting point to allow the realtime worker to swap between them without visible seams. ``` await rta.setLoop(avatarId, { motionPrompt: "tilts her head, a small amused smile, settles back to centre" }); await rta.waitForLoop(avatarId); // Blocks until the new loop is ready and swapped. ``` It is worth noting that the source loop (the resting loop) is distinct from the clips in the motion library. Directing the source loop changes its behavior without affecting the clips. The system ensures that these movements remain small and within a nearly-still envelope, preserving visual consistency during speech, as a separate model handles mouth animation. Attempting to direct a large, sweeping motion will typically result in a new render that still falls within these subtle constraints, without generating an explicit error (source:docs). ## Operational Considerations for Production Implementing AI avatars for live training requires a nuanced understanding of their operational capabilities and inherent limitations. A key feature for interactive scenarios is full-duplex communication, where the avatar listens continuously even while speaking. This design choice is critical for natural, uninterrupted conversational flow. - Interruption Handling: Learners can interrupt the avatar mid-sentence, and the system is designed to allow the avatar to acknowledge this naturally. - Backchanneling: Small verbal cues like 'mm-hm' or a cough do not derail the conversation, making the interaction robust and less brittle. - Intelligent Pauses: The avatar judges whether a speaker is finished by the content of their utterance, not merely the duration of a pause, preventing awkward silences or talking over the learner. - Language Adaptation: The system can follow language switches within a single sentence, supporting multilingual learning environments seamlessly. However, it is equally important to acknowledge the limitations. AI avatars may struggle with emotionally sensitive content, such as leadership training or diversity and inclusion topics, where the nuances of human trust and empathy are paramount. The 'uncanny valley' effect can also pose a challenge, where overly realistic avatars may cause discomfort (source:1). Content accuracy, especially for regulated industries, demands human oversight, as AI-generated content can occasionally be generic or factually incorrect (source:1). ### Ethical Frameworks and Transparency The deployment of AI avatars in training necessitates adherence to evolving ethical guidelines. Transparency is paramount: content generated by AI should be clearly disclosed through visible captions or annotations. For avatars based on real individuals, explicit, written consent is non-negotiable, detailing usage, reusability, and duration (source:1). - Transparency and Disclosure: Best practices include labeling AI-generated content (e.g., 'Created with AI') and utilizing standards like C2PA/Content Credentials (source:1). - Consent for Likeness: Obtain explicit, written consent for cloning real persons' voices or likenesses, specifying all usage parameters (source:1). - Respectful Representation: Ensure avatars avoid reinforcing stereotypes and promote inclusive representation (source:1). - Risk-Based Oversight: Apply differentiated oversight based on the sensitivity and stakes of the training content (source:1). ### Integrating for Live Training Experiences Integrating a realtime AI avatar into a training application often begins with the SDK. For applications requiring a rich, interactive experience on mobile, the TypeScript SDK provides specific hooks for environments like React Native, enabling deep integration with device capabilities like cameras and audio sessions. The `realtime-avatar/react-native` package, for instance, provides components like `RealtimeAvatarLiveKitRoom` and utilities such as `useAvatarCamera`. ``` import { RealtimeAvatarLiveKitRoom, AvatarVideoSurface, registerGlobals } from "realtime-avatar/react-native"; import { useAvatarCamera } from "realtime-avatar/react-native"; registerGlobals(); // Must run once at app start for WebRTC globals. function TrainingSession({ avatarId }) { const camera = useAvatarCamera({ allowed: true, active: true }); return ( {/* Further UI for interaction, e.g., camera controls */} ); } ``` The `registerGlobals()` call is critical for React Native environments, as it installs WebRTC globals not shipped by default, allowing the avatar to function. Camera sharing leverages existing LiveKit and WebRTC native modules, requiring appropriate permission declarations in your native build configuration (source:docs). Privacy disclosures must be updated to reflect camera images being sent to an AI provider. From a cost perspective, the usage-based pricing model of realtime avatar platforms differentiates them from pre-rendered video solutions. While minute-based pricing can accumulate quickly for long videos, a model anchored at about $5 per hour of realtime interaction encourages continuous engagement, making it feasible to ship companion applications where learners can converse extensively with their AI trainers. This allows for rich, iterative learning dialogues without prohibitive costs (source:docs, source:pricing). Shipping an AI avatar for training is a process of continuous refinement. Start by defining clear learning objectives that align with the avatar's strengths—scalable content delivery, interactive role-play, or multilingual support. Create your avatar from a single portrait image, directing its initial presence with a thoughtful motion prompt. Integrate the SDK, focusing on robust error handling and transparent communication about AI involvement. Iterate on content, leveraging the interactive capabilities for personalized feedback loops. Regularly review and update training modules, mindful of ethical considerations surrounding data privacy and consent. The goal is to build an extensible system that evolves with both your training needs and the capabilities of the AI, rather than a static piece of content. - [Realtime Avatar API Documentation](https://realtimeavatar.ai/docs) - [OpenAPI Specification](https://realtimeavatar.ai/openapi.json) - [Creating an avatar: One image in, a moving character out](https://realtimeavatar.ai/docs/video#one-image-in-a-moving-character-out) - [API reference: Avatars](https://realtimeavatar.ai/docs/api-reference#avatars) - [Realtime AI avatars with React and React Native](https://realtimeavatar.ai/docs/react#react-native) - [Realtime Avatar API: Voice or video, on one meter](https://realtimeavatar.ai/docs#voice-or-video-on-one-meter) - [Realtime Avatar API: She listens while she talks](https://realtimeavatar.ai/docs#she-listens-while-she-talks) - [The Silicon Review](https://thesiliconreview.com/2026/05/how-ai-avatars-transform-corporate-training) - [CommLab India](https://www.commlabindia.com/blog/synthesia-elearning-video-production-guide) - [VirtualSpeech](https://virtualspeech.com/blog/ai-avatars-are-transforming-learning-experiences) - [Training Industry](https://trainingindustry.com/articles/learning-technologies/when-ai-avatars-work-and-when-they-dont-in-workforce-training/) --- # Field Notes: Navigating Live Avatar API Alternatives for Developers An engineer's guide to Live Avatar API alternatives, focusing on real\-time conversational AI, single\-image avatar creation, and integration flexibility\. Published: 2026-10-08T02:46:20.601Z Updated: 2026-10-08T02:46:20.601Z Canonical: https://realtimeavatar.ai/blog/live-avatar-api-alternatives-for-developer-architectures Markdown: [en](https://realtimeavatar.ai/blog/live-avatar-api-alternatives-for-developer-architectures.md) For developers evaluating alternatives to existing real-time avatar APIs such as HeyGen's LiveAvatar, Tavus, or D-ID, the critical assessment points often center on the foundational avatar creation method, the granularity of integration control, and the engineering path to achieving genuine live presence. TIC Realtime Avatar addresses these concerns with an architecture that generates a fully expressive, audio-clocked avatar from a single portrait image, provides a robust TypeScript SDK, and enforces server-side policy for secure and consistent agent interactions. ## Architecting Real-Time Presence: From Portrait to Live Interaction The core distinction among real-time avatar platforms often lies in how the avatar itself is brought to life. While some alternatives might require pre-recorded video inputs or extensive 3D model rigging, TIC Realtime Avatar begins with a single, static portrait image. From this solitary frame, the platform generates the entire motion library, including a natural looping idle video, ensuring that every clip can seamlessly return to a shared rest pose. This approach means that transitions between avatar states—from idle to speaking, or between different expressions—are rendered as splices rather than jumps, maintaining visual continuity and presence. This method contrasts with systems that might rely on custom video uploads or more complex asset pipelines. For instance, some platforms offer highly customized 3D avatar creation or focus on facial animation SDKs, which may involve significant artistic and technical overhead. When evaluating alternatives, consider the resource commitment required to produce a usable avatar. Our approach streamlines the creative process: upload one portrait, and the system handles the rest, generating a production-ready avatar ready for live interaction. Key engineering considerations for real-time presence include the fidelity of lip-sync and the responsiveness of facial expressions. TIC ensures video is audio-clocked, meaning lips are precisely synced to the syllable, which is fundamental for believable conversational agents. This commitment to precise audio-visual coherence forms the bedrock of an engaging live avatar experience. ## Integrating Your Agent: Flexible Control Over the AI Stack Many real-time avatar APIs offer varying degrees of control over the underlying AI stack. Some, like HeyGen's LiveAvatar, provide both 'FULL mode' (handling STT, LLM, TTS end-to-end) and 'LITE mode' (allowing developers to bring their own agent/LLM) (source:2). Similarly, Tavus offers a full-stack conversational interface including hosted speech recognition, LLM, and TTS, designed for managed multimodal capabilities (source:2). TIC Realtime Avatar aligns with the 'bring your own AI stack' philosophy, providing the video generation and streaming layer while giving developers complete control over their conversational logic. This flexibility is crucial for developers who wish to integrate specific Large Language Models, custom business logic, or unique speech-to-text and text-to-speech services. The platform's OpenAPI-generated TypeScript SDK (realtime-avatar) facilitates this integration, offering a typed interface for managing sessions and avatar interactions. Developers can access agent-readable documentation at /llms.txt and the comprehensive OpenAPI specification at /openapi.json to understand how to connect their custom agents. Server-side policy enforcement is a cornerstone of this controlled integration. Your server-side logic dictates the avatar's persona, its memory, the specific voice it uses, and crucial parameters like the maximum session duration. Any parameters sent from the client-side for these critical fields are discarded, ensuring that the defined policies on your server remain authoritative. This design is paramount for security and cost control, particularly regarding `maxSeconds` to prevent unintended overages. - [Authentication: What the client is not allowed to decide](https://realtimeavatar.ai/docs/authentication) - [Calls: The policy your server decides](https://realtimeavatar.ai/docs/sessions) - [Realtime Avatar OpenAPI document](https://realtimeavatar.ai/openapi.json) ## Limitations and Practical Implementation Details ### Latency and Performance Real-time performance is a critical, yet often opaque, metric across avatar APIs. Claims of 'real-time' or 'minimal delay' must be approached with empirical testing. Latency in any real-time system, including AI avatars, is influenced by numerous factors: the startup state of the session, the specific model being used, and network conditions between the client, the API, and your own backend services. It is essential to establish your own measurement protocols for time-to-first-frame and round-trip interaction to validate performance for your specific application requirements. We refrain from promising specific latency numbers without a cited measurement protocol, as results are highly contextual. ### Customization and Asset Generation While generating an avatar from a single portrait streamlines the process, it defines the aesthetic. This approach excels at creating a consistent, recognizable digital persona without the need for complex 3D modeling pipelines. If your use case demands highly stylized or custom-rigged 3D avatars with granular control over every vertex, alternatives focusing on 3D avatar creation SDKs (like MetaHuman or Avatar SDK) might be a better fit, though they introduce significant artistic and integration complexity. ### Cost Models API pricing structures vary widely, typically anchored by usage minutes or per-generation fees. TIC's usage-based pricing is anchored at approximately $5 per hour of real-time interaction. This includes avatar creation at $3 per avatar beyond any included in your plan, with its generated resting loop also included. Careful planning of session duration, particularly using the `maxSeconds` policy on your server, is vital for managing costs effectively in production. Overage rates are typically detailed by plan, often around $0.07-0.095 per minute. - [Pricing Details](https://realtimeavatar.ai/pricing) ## The Developer's Path: Shipping a Live Avatar Application Bringing a real-time AI avatar application to production requires a structured approach, especially concerning security, policy management, and robust integration. The TIC Realtime Avatar platform is designed with these considerations at the forefront, emphasizing server-side control for all critical decisions. ### Secure API Key Management Never expose your API key to the client-side. A browser holding your key could initiate unlimited calls on your account. Instead, implement a secure server-side route that generates and relays a session grant to the client. This grant is valid for a single call, encapsulating the necessary permissions without exposing sensitive credentials. The client-side SDK rigorously validates this grant, so ensure your server-side route relays it unchanged. - [Authentication: What the client is not allowed to decide](https://realtimeavatar.ai/docs/authentication) - [Build with Lovable, Bolt, v0 and Replit: What every prompt makes the builder do](https://realtimeavatar.ai/docs/ai-app-builders) ### Centralized Session Policy Control The persona, memory, voice, and time limit for an avatar session are all configured and enforced on your server. Your backend explicitly sets these parameters for each session grant. Any attempt by the client to override these settings is discarded. This design ensures that your application maintains consistent behavior, adheres to your defined experience, and prevents unexpected costs from prolonged sessions. The `maxSeconds` policy, in particular, is your primary safeguard against runaway usage. ### Integrating with Your Agent Infrastructure For agents that need to perform actions, the platform supports tool calling. Your agent receives structured information, allowing it to interact with external services or perform specific functions within your application logic. While read-only tools are enabled by default, write tools require an explicit opt-in with `REALTIME_AVATAR_ALLOW_WRITES=1`, ensuring deliberate control over potentially cost-incurring or state-changing actions. - [Tool calling documentation](https://realtimeavatar.ai/docs/tool-calling) - [Build avatar apps with the Realtime Avatar MCP server: What the agent gets](https://realtimeavatar.ai/docs/mcp) By adhering to these architectural principles—starting with a single portrait for avatar generation, leveraging a flexible SDK for agent integration, and maintaining strict server-side control over sessions and security—developers can confidently ship real-time AI avatar applications that are both engaging and operationally sound. - [heygen.com](https://help.heygen.com/en/articles/9182113-what-is-a-liveavatar) - [creatorstoolbox.com](https://creatorstoolbox.com/tools/liveavatar) - [heygen.com](https://developers.heygen.com/live-avatar) --- # Choosing a Realtime AI Avatar API for Live Web Conversations Navigate API options for live AI avatars in web apps\. Understand the division of labor, latency, customization, and integration of leading platforms\. Published: 2026-10-07T02:46:45.972Z Updated: 2026-10-07T02:46:45.972Z Canonical: https://realtimeavatar.ai/blog/choosing-realtime-ai-avatar-api-web-conversations Markdown: [en](https://realtimeavatar.ai/blog/choosing-realtime-ai-avatar-api-web-conversations.md) When selecting a realtime AI avatar API for live conversations in a web application, the critical factors are the platform's division of labor, its ability to deliver low latency, the scope of avatar customization, and the ease of integration. The TIC Realtime Avatar platform provides an audio-clocked avatar rendering layer with a robust API and TypeScript SDK, designed for integrating with your existing conversational stack to create responsive, embodied AI experiences. ## Architecting Live Presence: The TIC Realtime Avatar Approach Our platform focuses on delivering compelling, real-time visual presence from a minimal input: a single portrait image. From this solitary frame, the system synthesizes a complete motion library and a looping idle video, ensuring your avatar is always ready for interaction without the need for extensive video training data. The generated video is audio-clocked, meticulously syncing lip movements to the precise syllable for a natural conversational rhythm. The core of the TIC Realtime Avatar offering is its API and a strongly typed TypeScript SDK, 'realtime-avatar', generated directly from our OpenAPI specification. This design prioritizes developer experience, providing clear interfaces for managing calls, handling avatar states, and integrating custom logic. For those integrating advanced AI capabilities, our agent-readable documentation provides insight into interaction patterns. - [Agent-readable docs](https://realtimeavatar.ai/llms.txt) ### Integration and the Conversational Stack Many realtime avatar APIs present a choice: a full conversational stack including Speech-to-Text (STT), Large Language Models (LLM), and Text-to-Speech (TTS), or a dedicated avatar rendering layer that expects you to bring your own AI components. TIC Realtime Avatar primarily provides the latter, focusing on the sophisticated video layer and offering a flexible framework for integrating your preferred external STT, LLM, and TTS services. This allows you to retain control over your AI pipeline and leverage specialized models. ``` console.log("Call status:", status)} onEnded={({ reason }) => console.log("Call ended:", reason)} > {(call) => } ``` Beyond simply animating speech, our platform enables avatars to interact with the digital world through tool calling. This capability transforms a talking head into an active agent, allowing it to perform actions like looking up order statuses or booking appointments, all while maintaining a continuous conversation. You define these tools with a clear description and parameters, and the avatar decides when to invoke them. ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` ### Latency Considerations for Natural Conversation For live conversations, the perceived latency – from user speech to the avatar's first visual response – is critical for a natural feel. While precise end-to-end latency can vary significantly based on network conditions, the specific AI models involved (STT, LLM, TTS), and the avatar rendering pipeline, our system is engineered for real-time responsiveness. The audio-clocked video generation ensures that once speech is synthesized, the visual articulation is immediate and synchronized, contributing to a fluid conversational rhythm. Evaluating real-time systems often involves benchmarking turn-taking latency, which should ideally be below human conversational thresholds to prevent awkward pauses or interruptions. It is important to remember that latency depends on the startup state, model, and network, so no singular number can represent all conditions without a cited measurement protocol. ## Comparative Field Notes: Evaluating Realtime Avatar APIs The landscape of realtime AI avatar APIs offers a range of approaches to bringing digital characters to life for live conversations. Each platform presents a distinct balance of features, integration paradigms, and cost models. Here's a look at how several key offerings address the demands of web application integration for live conversational AI. - Synthesia Interactive Avatars: This API focuses on rendering a live, lip-synced avatar, integrating as a plugin for LiveKit Agents. Developers typically bring their own LLM and STT, with Synthesia handling avatar rendering and optional TTS. Custom UI development is required. Synthesia reports an avatar charge of $0.10 per minute for its Interactive Avatars API, which generally does not support stock avatars for interactive use (source:2, source:7). - HeyGen LiveAvatar: Designed for interactive, live conversations, HeyGen offers API and SDK integration via WebRTC. It provides both a "Full mode" with a managed conversational stack and an "Avatar Only" mode for custom AI components. While described as "real-time," specific latency measurements are not publicly detailed. HeyGen features a library of stock avatars and custom avatar creation from short video footage, offering photorealism and full-body tracking. Pricing operates on a credit system, with different rates for full vs. avatar-only modes. - Tavus CVI: Tavus emphasizes real-time multimodal video interactions, aiming for natural AI responses mirroring human conversation. It's highly embeddable via website components, iframes, or React. Tavus can provide an end-to-end stack or allow integration of custom LLM/TTS. It claims "sub-second round-trip latency (~600 ms in optimal conditions)" for utterance-to-utterance, aiming for conversational rhythm at around 600 milliseconds. Customization includes stock faces and digital twin creation from brief video. Tavus offers a free plan and usage-based pricing (source:9). - D-ID Agent API: Specializing in real-time streaming avatars, D-ID provides HTTP REST and WebRTC streaming sessions with strong SDK support. It offers a flexible AI agent framework, allowing integration of custom knowledge sources and LLMs. D-ID claims "sub-2-second first-frame delivery" via WebRTC and the ability to animate any single photo or artwork into a talking avatar. Pricing is tiered, billed by various metrics including video minutes and agent sessions (source:3, source:5). - Google's Gemini 3.8 Live with Live Avatar: This enterprise-focused feature adds near real-time visual presence to Gemini's AI, engineered for ultra-low latency bidirectional interactions via API. It natively couples dialogue with low-latency streaming video and supports asynchronous tool calling. While promising "sub-second latency" and custom avatars from high-quality images, access is typically gated by enterprise allowlisting, and specific pricing is not public. All generated audio and video are watermarked with SynthID for detectability. The model card indicates it "sustains only a few minutes of interaction and may hallucinate or time out" (source:1). - Beyond Presence: This platform focuses on hyper-realistic AI video agents and speech-to-video (S2V) technology. It's embeddable via iframe, API, or SDKs and integrates with LiveKit and Pipecat. Developers can bring their own LLMs. Beyond Presence claims "~100ms latency" for its foundational S2V model and "~1.0–1.2s latency" for managed agents, with full HD 1080p video. Custom avatars can be created from training video (higher quality) or a single image. - Simli: Simli functions as a real-time speech-to-video API, adding faces to AI agents. It's designed as the "visual hop" where developers provide their own STT, LLM, and TTS. It claims to convert outgoing audio into a lip-synced face in "under 300ms for that last hop," emphasizing that total conversational latency depends on the entire pipeline. Simli supports image-based custom avatar faces. - New Port AI Live Avatar: A real-time avatar platform offering managed conversation and knowledge base options, or allowing teams to bring their own agent or RTC stack. It provides a frontend SDK and WebSocket agent. Specific latency claims and avatar customization options are not detailed in public sources. Currently in limited preview with free and professional plans (source:6). - Protoface: Built to make premium real-time avatars practical, Protoface focuses on fast creation, low-latency, and easy integration. It is designed to fit into existing real-time AI stacks, providing the avatar layer. Protoface claims "sub-300ms latency" for launching live avatar experiences and allows custom avatars to be created from images. ### Avatar Customization and Identity The identity of your AI avatar is crucial. While many platforms offer libraries of stock avatars or require extensive video footage for custom digital twins, TIC Realtime Avatar streamlines the process significantly. You provide a single portrait image, and the platform generates a dynamic, looping idle animation and a comprehensive motion library. This approach allows for rapid prototyping and deployment of unique characters, ensuring brand consistency or specific roleplay requirements are met efficiently without the overhead of video-based training. ### Economic Models for Realtime Interaction Understanding the cost implications for live, interactive applications is paramount. While some providers operate on credit systems, video minutes, or tiered plans with varying feature access, TIC Realtime Avatar employs a straightforward usage-based pricing model. Realtime avatar usage is anchored at approximately $5 per hour of live conversation, with plans including hours (e.g., a $24/month plan includes 10 hours). Overage rates are transparent, ranging from $0.07 to $0.095 per minute depending on your plan. Avatar creation, beyond those included, is priced at $3 per avatar, with its resting loop provided without additional charge. This transparent, per-minute billing for live interaction allows for predictable scaling as your application grows, making it feasible to keep a companion app "on" for extended engagements. - [Pricing Details](https://realtimeavatar.ai/pricing) ## Shipping Your Realtime Avatar Application Bringing a live AI avatar to production in a web application requires a clear integration path and a robust SDK. The TIC Realtime Avatar platform is engineered for developers, providing a TypeScript SDK that simplifies the complexity of WebRTC and real-time media streams. 1. Obtain an API Key: Secure a session-capable API key from your TIC platform dashboard to authenticate your requests. 2. Integrate the SDK: For web applications, the realtime-avatar TypeScript SDK provides components for React, simplifying UI development. A complete Next.js starter kit is available for download, providing a foundation with ready-to-use avatar calls, microphone input, text messages, and call controls. This allows you to quickly get a functional example running locally before tailoring it to your application's needs. 3. Manage Call State and UI: Use the AvatarCall component's onStatusChange and onEnded callbacks to build responsive UI elements that reflect the call's lifecycle, from connecting to active conversation to termination. The SDK abstracts away lower-level WebRTC details, allowing you to focus on product experience. 4. Enable Agentic Behavior with Tools: To move beyond simple conversation, define and register custom tools within your client-side application. These tools allow your avatar to execute backend logic, fetch data, or interact with external services, enhancing its utility. Your server grants the client_tools capability during session minting, enabling the frontend to attach these functions. 5. Consider Mobile Applications: For React Native applications, a separate entry point (realtime-avatar/react-native) is provided, wrapping LiveKit's React Native room and managing OS audio sessions. Remember to registerGlobals() once at app start and correctly handle camera permissions for features like camera sharing. - [Quickstart](https://realtimeavatar.ai/docs/quickstart) - [Next.js Starter](https://realtimeavatar.ai/docs/nextjs) - [React SDK Documentation](https://realtimeavatar.ai/docs/react) - [Tool Calling Documentation](https://realtimeavatar.ai/docs/tool-calling) - [API Reference](https://realtimeavatar.ai/docs/api-reference) - [fxguide.com](https://www.fxguide.com/quicktakes/nvidia-ace-enables-easier-interactive-avatars/) - [synthesia.io](https://www.synthesia.io/post/interactive-avatar-api-real-time-avatars) - [d-id.com](https://www.d-id.com/ai-agents/) - [d-id.com](https://www.d-id.com/blog/experience-enhanced-d-id-visual-agents/) - [newportai.com](https://newportai.com/blog/best-live-ai-avatar-platforms) - [synthesia.io](https://www.synthesia.io/features/avatars/interactive-avatars) - [videmint.com](https://videmint.com/ai-tool-reviews/synthesia-avatars/) --- # Which Talking Avatar APIs Include Speech Recognition, the Language Model, and Text\-to\-Speech? Explore how Realtime Avatar APIs orchestrate speech recognition, language models, and text\-to\-speech for interactive digital human experiences\. Published: 2026-10-06T02:45:37.075Z Updated: 2026-10-06T02:45:37.075Z Canonical: https://realtimeavatar.ai/blog/full-stack-talking-avatar-apis-stt-llm-tts-integration Markdown: [en](https://realtimeavatar.ai/blog/full-stack-talking-avatar-apis-stt-llm-tts-integration.md) For developers building interactive digital experiences, the question of which talking avatar APIs natively include speech recognition (STT), a language model (LLM), and text-to-speech (TTS) is central to architectural decisions. TIC Realtime Avatar provides a full-stack real-time session experience, managing the orchestration of these components to embody AI-driven conversations, with configurable external LLMs while handling the real-time video synthesis, STT, and TTS. ## The Full-Stack Session: Embodiment in Conversation The core of a real-time interactive avatar lies in its ability to listen, comprehend, respond, and speak with the fluidity of human interaction. Our platform is designed to facilitate this entire pipeline within a single, managed session. When a client connects to a Realtime Avatar session, the server dictates the operational policy, establishing the avatar's intelligence and sensory inputs. This session policy centrally manages how the avatar perceives user input, processes information, and generates its verbal and visual responses. - [Learn more about what your server decides for each session](https://realtimeavatar.ai/docs/sessions) ### Speech Recognition (STT): The Listening Ear A crucial first step in any conversational AI is accurate and low-latency speech recognition. In a Realtime Avatar session, the avatar is configured to hear the user throughout the call by default. This is controlled by the `listen` parameter in the session policy, which corresponds to `stt_mode` on the wire. By setting `stt_mode: "server"`, the platform activates server-side speech-to-text, ensuring the avatar is always receptive to user input, enabling natural barge-in behavior and responsive dialogue without explicit turn-taking commands. ``` session: async ({ avatarId }) => ({ listen: true, // Default, but explicit for clarity }) ``` This full-duplex listening capability is fundamental to maintaining the illusion of live presence. If your application drives every turn and handles its own STT, you can explicitly set `listen: false` to disable our integrated speech recognition. ### Language Model (LLM) Integration: The Core Intelligence While the platform provides the visual and auditory conduit for the AI, the avatar's intelligence and conversational style are primarily defined by the language model. Our API allows developers to integrate their preferred LLM by defining the avatar's behavior contract via the `instructions` field in the session policy. This field is paramount, shaping who the avatar is, how it speaks, and its overall persona. It supports up to 16,000 characters, offering ample space to sculpt intricate personalities and conversational directives. ``` session: async ({ avatarId }) => ({ instructions: "You are a kind engineer who explains complex ideas with clarity. Focus on practical applications and spark curiosity.", // ... other session parameters }) ``` Furthermore, sessions support an `initial_context` array of up to 32 prior messages, allowing the conversation to pick up from a rich history rather than starting cold. This is essential for building applications that maintain continuity and deeper engagement, as the LLM can leverage past interactions to inform its current responses. The platform passes the generated LLM text to its integrated text-to-speech system for vocalization. - [Consult our developer documentation on integrating AI App Builders](https://realtimeavatar.ai/docs/ai-app-builders) - [Read our LLM-focused documentation](https://realtimeavatar.ai/llms.txt) - [Explore the OpenAPI specification for detailed LLM parameters](https://realtimeavatar.ai/openapi.json) ### Text-to-Speech (TTS) and Voice: The Avatar's Voice Once the LLM generates a textual response, the platform's integrated text-to-speech system brings it to life with natural-sounding voices. The `voice` parameter in the session policy allows for fine-grained control over the avatar's vocal delivery for a specific call. This can include overriding the avatar's stored voice with a specific Fish voice, adjusting `speed`, `emotion`, and `language`. ``` session: async ({ avatarId }) => ({ voice: { id: "fish-deep-baritone-en-us", speed: 1.05, emotion: "friendly", language: "en-US" }, // ... other session parameters }) ``` It is important to note that while the `voice` configuration handles the auditory output, the persona and reply style are primarily governed by the `instructions` passed to the LLM. The platform carefully carries both the session instructions and voice configuration separately, ensuring the model's textual output is then rendered with the desired vocal characteristics, lips synced to the syllable for a cohesive presentation. ## Crafting Presence: Avatar Creation and Behavior Beyond the conversational stack, the visual embodiment is where a real-time avatar truly manifests presence. Our platform emphasizes creating compelling digital humans from a minimal starting point and providing robust controls for their non-verbal behavior. ### Avatar Creation from a Single Portrait A fundamental differentiator of our platform is the avatar creation process itself. Avatars are generated from a single portrait image. From this single image, the platform automatically generates a looping idle video and a multi-clip motion library. This means you provide ONE portrait image, and the system handles the complex process of making it live and interactive, without the need for pre-recorded video footage. - [See the API reference for avatar creation details](https://realtimeavatar.ai/docs/api-reference#avatars) - [Explore how to generate real-time AI avatars from a portrait to live interaction](https://realtimeavatar.ai/blog/generating-realtime-ai-avatars-from-portrait-to-live-interaction) The looping idle video serves as the avatar's resting state, playing continuously when not actively speaking or performing an action. This resting loop can be further refined with a `motionPrompt` to art-direct its subtle movements, ensuring it aligns with the avatar's personality. ``` await rta.setLoop(avatarId, { motionPrompt: "tilts her head, a small amused smile, settles back to centre", }); ``` ### Real-time Motion Library: Non-Verbal Cues To enrich the avatar's expressiveness, the platform generates a dynamic motion library. This library consists of short video clips that can be triggered by specific events or by the LLM itself. For instance, a `userSpeechStarted` event can trigger a 'nod' clip, indicating active listening. More complex behaviors are handled by `actions`, where the LLM's `description` guides it to select and play an appropriate clip, such as a 'wave' for a greeting. ``` const { revision } = await rta.listClips(avatarId); const { plan } = await rta.setClipLibrary(avatarId, { expectedRevision: revision, clips: { nod: { source: { motionPrompt: "a small attentive nod, returns to rest" } }, wave: { source: { motionPrompt: "raises a hand and waves warmly, then lowers it" } } }, on: { userSpeechStarted: { clips: ["nod"] } }, actions: { greet: { description: "When greeting the user", clips: ["wave"] } } }); ``` The platform ensures that lip-sync remains accurate during full-duplex speech, dynamically animating the avatar's mouth over the current visual state, whether it's an idle loop or an expressive motion clip. This seamless integration of verbal and non-verbal communication is critical for a truly immersive and believable interaction. - [Learn more about video clips and behavior](https://realtimeavatar.ai/docs/video#looping-shared-clips-event-driven-behavior) ## Limitations and Considerations While the platform offers a comprehensive solution for real-time interactive avatars, it is important to understand its boundaries. The core strength lies in its managed execution of the visual embodiment and the orchestration of STT, LLM, and TTS components into a single, cohesive real-time stream. However, developers maintain flexibility in choosing their preferred external LLM, providing the instructions and context that define the AI's intelligence. Latency in a real-time system is a complex interplay of several factors, including the avatar's startup state, the chosen AI models, and network conditions. While the system is optimized for real-time performance, promising specific latency numbers without a cited measurement protocol is not practical due to the variability inherent in distributed systems. We focus on providing the tools and architecture to evaluate and optimize for your specific use case. It is also crucial to reiterate that avatar creation is strictly from a single portrait image; creation from a supplied video is not supported by the platform. This ensures consistency and quality of the generated real-time assets. ## Shipping Your Interactive Avatar To bring your interactive avatar project to life, begin by creating your avatar from a high-quality portrait image through the API. Once the avatar's status is 'ready' (indicating the idle loop and initial motion library are generated), you can initiate sessions. The typed TypeScript SDK (realtime-avatar) simplifies integration, generated directly from our OpenAPI document. - [Explore our Quickstart guide](https://realtimeavatar.ai/docs/quickstart) Define your avatar's persona and conversational rules through the `instructions` field, and manage its non-verbal cues by refining its motion library. For cost considerations, real-time usage is priced at approximately $5 per hour, with plans including 10 hours for $24/month and overage rates between $0.07-0.095 per minute, depending on your plan. Avatar creation itself is $3 per avatar beyond any included in your plan, and the generated resting loop is always included. - [Review our pricing details](https://realtimeavatar.ai/pricing) Focus on iterating the avatar's conversational logic and its expressive behaviors. The underlying real-time video, STT, and TTS orchestration are handled, allowing you to concentrate on the user experience and the unique value your AI avatar brings. Test your conversation flows, fine-tune the `instructions`, and experiment with different motion clips to achieve the most engaging and lifelike interactions for your application. - [synthesia.io](https://www.synthesia.io/post/interactive-avatar-api-real-time-avatars) - [daily.dev](https://daily.dev/posts/how-real-time-ai-avatars-actually-work-every-piece-explained-built-w-synthesia--2pfmmqrxr) - [microsoft.com](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/text-to-speech-avatar/what-is-text-to-speech-avatar) --- # Realtime AI avatar APIs compared \(October 2026\): price per minute, free tiers, SDKs, tools Tavus, LiveAvatar, Anam, Simli, Beyond Presence, D\-ID, bitHuman and Realtime Avatar compared on price per minute, free tier, SDKs and tool calling\. Published: 2026-10-05T10:08:45.213Z Updated: 2026-10-13T13:42:26.889Z Canonical: https://realtimeavatar.ai/blog/realtime-ai-avatar-apis-compared-2026 Markdown: [en](https://realtimeavatar.ai/blog/realtime-ai-avatar-apis-compared-2026.md) Pick by architecture first, then by price. If you already run a voice agent (LiveKit Agents, Pipecat, OpenAI Realtime, ElevenLabs), buy a renderer: HeyGen LiveAvatar in Avatar Only mode, Anam, Simli, bitHuman or Beyond Presence's speech-to-video API. If you want one API that listens, thinks, speaks and renders, the full-stack options in October 2026 range from about $4.80 an hour (Realtime Avatar, Studio overage) and $6 an hour (bitHuman managed voice chat at the top-up rate) to $15.60 to $21 an hour (Tavus overage), with Anam, LiveAvatar, Beyond Presence and D-ID in between. Every competitor number below was read from that vendor's own pricing page or documentation on 5 October 2026 and is labelled as of October 2026. Sources are linked at the end. Where a vendor does not publish a number, the table says so instead of borrowing one from a review site. We make Realtime Avatar, so read our rows with that in mind, and check our claims against our own pricing page, which is the source for them. ## Price, free tier and pipeline, as of October 2026 The marginal rate is the per-minute price after a plan's included minutes run out, because that is the number that decides your bill once people actually use the product. Where a vendor lists several plans, the rates are in plan order from cheapest plan to most expensive. LiveAvatar's Full Mode uses 2 credits a minute, so $0.095 and $0.09 per credit become $0.19 and $0.18 per minute. Beyond Presence prices in euros. ``` API (product) Paid entry (USD/EUR per month) Marginal live rate, full conversation Free tier Pipeline you get Tavus CVI Starter $22 (60 min, no overage) $0.35 / $0.31 / $0.26 per min 20 min/month, 5 min per call Full stack, or your LLM; Pipecat/LiveKit modes HeyGen LiveAvatar Essential $99 (1,100 credits) $0.19 / $0.18 per min (Full Mode) 10 credits, 2 min per session Full Mode, or Avatar Only (your agent) Anam Starter $12 (50 min) $0.16 / $0.14 / $0.12 / $0.11 per min 30 min/month, 3 min per call Turnkey, or bring LLM / TTS / audio Simli not published not published $10 on signup + 50 min monthly top-up Renderer: you bring STT, LLM, TTS Beyond Presence Starter EUR 49 (14K credits) EUR 0.35 / 0.20 / 0.175 per agent min 2K credits/month (20 agent min) Managed agents, or speech-to-video D-ID Agents Build $18 (up to 32 stream min) no overage rate published 14-day trial, up to 10 stream min Full stack, or your LLM / your audio bitHuman Creator $20 (2,000 credits) $0.10 per min managed voice (top-up) Featured avatars; API needs Creator Renderer (cloud or device), or managed voice Realtime Avatar Starter $9 (120 min) $0.095 / $0.085 / $0.08 per min 480 credits once (8 min) Full stack only (STT, LLM, voice, video) ``` Read the last column before the price column. A renderer turns audio you produce into a talking face; you still pay separately for speech recognition, a language model and a voice. A full stack sells the whole conversation. bitHuman's cloud rendering alone is 4 credits a minute ($0.04 at its $1 = 100 credits top-up rate), which is cheap because it does less; its managed voice chat at 10 credits a minute is the comparable full-stack number. LiveAvatar's Avatar Only mode is 1 credit a minute ($0.095 or $0.09) for the same reason. ## SDKs and tool calling ``` API SDKs and client surfaces (from official docs) Tool calling Tavus CVI @tavus/cvi-ui (React, web); Daily SDKs for React Native, iOS, Android, Flutter Yes: tools registry, delivered by app message or HTTPS call HeyGen LiveAvatar @heygen/liveavatar-web-sdk; LiveKit, Pipecat, Agora plugins Not documented for Full Mode; via your own agent in Avatar Only Anam JavaScript SDK, Python SDK, LiveKit and Pipecat; community Flutter and KMP Yes: client tools and webhook tools Simli JavaScript and Python SDKs; LiveKit and Pipecat In your own agent stack Beyond Presence Any LiveKit client SDK (web, mobile); LiveKit plugin (Python, JS); iframe Function calling listed from the Scale plan D-ID Agents @d-id/client-sdk (web), embed script; LiveKit SDK for V4 Expressive agents Yes: server tools and client tools bitHuman Swift package, Android SDK, Python, CLI, LiveKit, Pipecat, web embed In your agent framework Realtime Avatar realtime-avatar on npm: browser, React, React Native, server adapters; OpenAPI Yes: client tools in your page (2.5 s budget), server steering ``` React Native is where the field thins out. Tavus documents React Native through Daily's React Native SDK joining the same conversation URL. Beyond Presence and D-ID's V4 agents stream over LiveKit, so LiveKit's own client SDKs are an option, and bitHuman ships native Swift and Android SDKs rather than a React Native package. Realtime Avatar ships a first-party realtime-avatar/react-native entry point, though it is lower-level than the web component; the React and React Native docs show the exact code. ## Billing details that change the real cost - Tavus bills each conversation a 30-second minimum and rounds to the nearest 6 seconds. Free and Starter calls are capped at 5 minutes, Builder at 15, Growth and Business at 60. Recordings cost $0.03 a minute. - Anam meters the whole session, talking or idle, by the second; included minutes do not roll over. Per-call limits are 3 minutes on Free, 5 on Starter, 10 on Explorer and 2 hours on Growth and Professional. - LiveAvatar deducts credits per minute of session time and bills overage in $100 increments. Essential sessions are capped at 20 minutes, Business at 60. Paid plans advertise unlimited concurrency. - bitHuman counts active session time, talking or idle, billed to the second. Credits bought with a plan are cheaper than top-ups: Business is $299 for 50,000 credits, which works out to about $0.06 a minute for managed voice chat if you use the whole allowance. From 12 October 2026 the API and SDKs require the Creator plan or higher. - D-ID's Build plan is a personal license with a D-ID watermark; commercial use starts at Launch ($50 a month for up to 90 streaming minutes). Its plan math lands at roughly $0.50 to $0.56 per streaming minute on monthly billing. - Realtime Avatar meters seconds on air. One call can run at most 1,800 seconds (30 minutes), enforced server-side, which is shorter than Tavus Growth, LiveAvatar Business or Anam Growth allow. If your product needs hour-long sessions, that matters. ## Latency: what each vendor claims None of these numbers measure the same thing, and none of them are our measurements. They are each vendor's own published claim, quoted so you know what to test. Tavus says its Phoenix rendering model has 134 ms latency and its LLMs reply in about 600 ms. Anam says 180 ms average agent response time. Beyond Presence says 250 ms or less global avatar latency, with managed agents streaming at about 1.0 to 1.2 seconds. LiveAvatar says under 300 ms median time to first frame. Simli says under 300 ms for its speech-to-video stage and notes that STT, LLM and TTS add their own time. D-ID's Agents page says answers arrive in under two seconds. Realtime Avatar publishes two server-side design targets, not measured guarantees: 300 ms to speech and 500 ms to the first video frame. They are not an SLA, and no build is gated on them. If latency decides your choice, run the same short script against each finalist; our benchmarking post describes a protocol. ## Short verdicts by use case ### You already have a voice agent Buy a renderer, not a second pipeline. LiveAvatar Avatar Only has connectors for ElevenLabs, Cartesia, OpenAI Realtime and Gemini Live plus LiveKit, Pipecat and Agora plugins. Anam, Simli, bitHuman and Beyond Presence all have LiveKit plugins, and Tavus documents Pipecat and LiveKit modes. Realtime Avatar is the wrong choice here: it has no LiveKit or Pipecat plugin and cannot render audio from your own voice agent. You can switch off its speech recognition and drive each turn with text from your own LLM, but the voice is still ours. ### You want the most human-looking replica and enterprise paperwork Tavus is the strongest fit. It trains faces from a two-minute video or a photo, adds perception (Raven-1) and turn-taking (Sparrow-2) models, joins Google Meet and Zoom, and lists SOC 2 and HIPAA compliance. You pay for it: $15.60 to $21 an hour at overage. ### Long sessions or EU-first procurement Anam's Growth plan allows 2-hour conversations at $0.12 a minute overage. Beyond Presence is built in Europe and lists GDPR, EU AI Act and SOC 2 Type II on its site, with on-premise deployment on Enterprise. ### On-device rendering bitHuman is the only one here that renders on the device (iPhone, iPad, Mac, Android arm64, Linux, WebGPU browsers), which also makes it the cheapest renderer at 2 credits a minute on your own hardware. ### A character in your own web or React Native app, with your tools, at low per-hour cost This is the case Realtime Avatar is built for. You create a character from one portrait, mint each call from your own server route, render it with one React component (or the React Native primitives), and the character calls functions that run in your page with your credentials. The whole conversation is $4.80 to $5.70 an hour at overage, plans start at $9 a month for 120 minutes, and the free sandbox is 480 credits granted once per account, not every month. The trade-offs: no renderer mode, Fish Audio voices only, a session-selectable LLM from our own set (OpenAI or Gemini models) rather than your own endpoint, and a 30-minute cap per call. ## How to decide in an afternoon 1. Write down whether you own the conversation already. If yes, shortlist renderers only. 2. Estimate monthly live minutes and peak concurrent calls, then price each finalist at its marginal rate, not its headline plan. 3. Check the per-call cap against your longest real session. 4. Run one identical scripted conversation on each free tier and note time to first reply, interruption handling and how tools behave. Prices in this category change often. Tavus's pricing page was serving two different developer price tables in the same page on 5 October 2026 (the second shows Starter at $59 with $0.37 a minute overage); the table above uses the one listing Builder and Business. Re-check every number, including ours, before you commit. - [Realtime Avatar pricing](https://realtimeavatar.ai/pricing) - [Realtime Avatar quickstart](https://realtimeavatar.ai/docs/quickstart) - [Realtime Avatar React and React Native docs](https://realtimeavatar.ai/docs/react) - [Realtime Avatar tool calling](https://realtimeavatar.ai/docs/tool-calling) - [How to benchmark realtime avatar startup and turn latency](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) - [Renderer or full stack: comparing the right number](https://realtimeavatar.ai/blog/renderer-or-full-stack-realtime-avatar-apis-compared) - [Tavus pricing](https://www.tavus.io/pricing) - [Tavus mobile and React Native docs](https://docs.tavus.io/sections/conversational-video-interface/mobile) - [LiveAvatar credits and subscriptions](https://docs.liveavatar.com/docs/faq/credits) - [LiveAvatar Avatar Only integrations](https://docs.liveavatar.com/docs/lite-mode/overview) - [Anam pricing](https://anam.ai/pricing) - [Anam SDKs and integrations](https://anam.ai/docs/integrations/sdks) - [Simli home page with free-plan terms](https://www.simli.com) - [Beyond Presence pricing](https://www.beyondpresence.ai/pricing) - [D-ID API pricing](https://www.d-id.com/pricing/api/) - [D-ID realtime agents overview](https://docs.d-id.com/docs/realtime-overview) - [bitHuman pricing](https://www.bithuman.ai/pricing) - [bitHuman developer docs index](https://docs.bithuman.ai/llms.txt) --- # How to put a real\-time AI avatar in a React Native or Expo app Add a live, interruptible AI avatar video call to React Native or Expo: install, dev build, server route, native call screen and idle video, with real SDK code\. Published: 2026-10-05T10:11:05.134Z Updated: 2026-10-13T13:42:50.151Z Canonical: https://realtimeavatar.ai/blog/realtime-ai-avatar-react-native-expo Markdown: [en](https://realtimeavatar.ai/blog/realtime-ai-avatar-react-native-expo.md) To put a real-time AI avatar in a React Native or Expo app, you need three pieces: a server route that holds your API key and mints each call, a native WebRTC stack (LiveKit's React Native SDK, which requires an Expo development build rather than Expo Go), and a call screen that renders the avatar's video. With the realtime-avatar SDK the native entry point is realtime-avatar/react-native: call registerGlobals() once, mint the session through useSessionLifecycle, and render AvatarVideoSurface inside RealtimeAvatarLiveKitRoom. Everything below is taken from the published SDK (realtime-avatar 0.26.0 on npm, October 2026) and its docs. It is honest about one thing up front: on native, the SDK is lower-level than on the web. The one-component AvatarCall wrapper is web-only, so on React Native you compose the room, the session bridge and the video surface yourself. That is about 40 lines, shown in full. ## What you need before you start - A Realtime Avatar API key, kept on your server only. The free sandbox gives 480 credits once per account (8 live minutes), and one credit is one second on air. - An avatar id. Use a public example avatar such as seed-rin-ashfall to start, or create your own from one portrait. - React Native 0.78 or newer (the SDK's peer range). The SDK's own checks run on React Native 0.81.5 and React 19.2.7, with Node 22.22 or newer for tooling. - An Expo development build or EAS build. Live media uses native WebRTC modules, so it does not run in Expo Go. ## 1. Install the SDK and the native media stack The SDK README lists the exact LiveKit tuple it is tested against. Install it together with the SDK and Zod, its one required runtime peer: ``` npx expo install expo-video npm install --save-exact realtime-avatar@0.26.0 zod@4.4.3 \ @livekit/react-native@3.0.0 @livekit/react-native-webrtc@144.2.0 \ livekit-client@2.22.3 @livekit/components-react@2.9.21 ``` In a managed Expo project, LiveKit's own instructions add two config plugins so the native modules and permissions are set up at prebuild. Install @livekit/react-native-expo-plugin and @config-plugins/react-native-webrtc, then list them in app.json and make sure the config plugin version matches the WebRTC version you installed: ``` { "expo": { "plugins": [ "@livekit/react-native-expo-plugin", "@config-plugins/react-native-webrtc" ] } } ``` Then build a development client (for example npx expo run:ios, or an EAS development build). This is a native change: a JavaScript-only OTA update cannot add these modules to a binary that does not already contain them. ## 2. Put the key behind your own route Your API key must never ship inside the app. The SDK's server adapters mount one route that answers four paths (connect, end, avatars, credits) and decides who may call and what the character knows. Any adapter works: Next.js, TanStack Start, Express, or Hono, which also covers Cloudflare Workers, Bun and Deno. A Hono version: ``` // server.ts import { Hono } from "hono"; import { realtimeAvatarHono } from "realtime-avatar/hono"; const app = new Hono(); app.all( "/api/realtime-avatar/*", realtimeAvatarHono({ apiKey: () => process.env.REALTIME_AVATAR_API_KEY!, authorize: async ({ request }) => { const user = await userFromBearerToken(request); // your auth if (!user) return new Response("Sign in", { status: 401 }); }, session: async ({ request, avatarId }) => { const user = await userFromBearerToken(request); return { instructions: await promptFor(avatarId), context: await recentTurns(user!.id, avatarId), maxSeconds: 600, // hard stop, enforced server-side }; }, }), ); export default app; ``` Whatever the app sends for persona, memory, voice or time limit is discarded; only the session callback decides. A mobile app usually authenticates with a bearer token rather than a cookie, which is why the client below passes a fetch wrapper. ## 3. Register the WebRTC globals once ``` // index.ts (or the top of app/_layout.tsx with Expo Router) import { registerGlobals } from "realtime-avatar/react-native"; registerGlobals(); // installs the WebRTC globals React Native does not ship ``` ## 4. The call screen useSessionLifecycle runs the same session logic as the web SDK: the grant, the busy queue, reconnects and the idle timer. RealtimeAvatarLiveKitRoom wraps LiveKit's native room and owns the OS audio session by default. AvatarVideoSurface layers the portrait, the idle clip and the live video. Native has no built-in video element, so you hand it a player for the idle loop; here that is expo-video. ``` // AvatarCallScreen.tsx import { Pressable, Text, View } from "react-native"; import { useVideoPlayer, VideoView } from "expo-video"; import { AvatarVideoSurface, RealtimeAvatarLiveKitRoom, SessionLifecycleRoomBridge, createProxyClient, useSessionLifecycle, type IdleVideoRender, } from "realtime-avatar/react-native"; // Native has no page origin, so the proxy URL must be absolute. const client = createProxyClient({ proxyUrl: "https://api.your-app.example.com/api/realtime-avatar", fetch: async (input, init) => { const headers = new Headers(init?.headers); headers.set("Authorization", "Bearer " + (await getSessionToken())); return fetch(input, { ...init, headers }); }, }); function IdleClip({ url, style, resizeMode }: IdleVideoRender) { const player = useVideoPlayer(url, (p) => { p.loop = true; p.muted = true; p.play(); }); return ; } type Props = { avatarId: string; idleVideoUrl: string | null; posterUrl: string | null; onClose: () => void }; export function AvatarCallScreen({ avatarId, idleVideoUrl, posterUrl, onClose }: Props) { const lifecycle = useSessionLifecycle({ client, session: { avatarId } }); const phase = lifecycle.phase.kind; return ( {lifecycle.grant ? ( } /> ) : ( {phase === "queued" ? "All lines are busy, holding your place" : "Connecting"} )} { lifecycle.end(); onClose(); }} > End call ); } ``` Mount this screen only after the user taps a start button, because mounting it starts the connection and the meter. A queued phase is not an error: every slot is busy and the SDK retries for you while holding a place in line. The other phases are idle, requesting, connecting, live, idle-warning, reconnectable and ended; write the copy for the ones your product shows. The SDK ships no user-facing strings. ## 5. Things that trip people up on mobile - Expo Go will not work. If the call sits in connecting forever on a fresh project, check that you are running a development build that contains the LiveKit and WebRTC native modules. - Relay the mint response byte-for-byte. If you hand-roll the server half instead of using an adapter, do not wrap or reshape the JSON grant: the client reads it unvalidated and a wrapped grant never connects. - One owner for the audio session. RealtimeAvatarLiveKitRoom starts and stops the OS audio session for you. Only pass manageAudioSession={false} if your app manages AudioSession itself. - Camera sharing is optional and gated twice: your server must allow it with camera: true, and the user must opt in through useAvatarCamera. It needs the iOS camera usage description and the Android CAMERA permission in the binary, so it is another native build, not an OTA. - Test on physical iOS and Android devices, including background and foreground transitions and redialing. The SDK's own native test suite mocks the native modules, so it does not prove linking, permissions or speaker routing on your build. ## What the call includes, and what it does not Each call is full duplex: the user can interrupt mid-sentence, a cough or a short acknowledgement does not derail the character, and end of turn is judged by what was said rather than by a fixed silence timer. Speech recognition, the language model, a Fish Audio voice and the rendered video are all metered as one rate, seconds on air: $9 a month for 120 minutes on Starter, $24 for 600 on Developer, then $0.08 to $0.095 a minute depending on plan. A single call can run at most 30 minutes: the platform caps max_session_seconds at 1,800, and the server route below asks for 600 (10 minutes) per call, well inside that cap. For an audio-only call when bandwidth is poor, pass mode: "voice" next to avatarId in the session object you give useSessionLifecycle. What it does not do: it is not a renderer for your own voice agent. There is no LiveKit Agents or Pipecat plugin, and you cannot feed audio from OpenAI Realtime or another voice pipeline into the avatar. You can switch off server speech recognition and drive each turn with text from your own LLM, but the voice is still the platform's. ## How other vendors handle React Native As of October 2026: Tavus documents React Native through Daily's React Native SDK, joining the conversation URL Tavus returns, with your own UI. Beyond Presence says any LiveKit client SDK can join its agent rooms, which includes LiveKit's React Native SDK. bitHuman ships native Swift and Android SDKs instead of a React Native package. Anam's official SDKs are JavaScript and Python, with community Flutter and Kotlin Multiplatform SDKs. If you are on Flutter or fully native Swift and Kotlin, one of those may fit better than a TypeScript SDK. - [React and React Native docs](https://realtimeavatar.ai/docs/react) - [Quickstart: key to live call](https://realtimeavatar.ai/docs/quickstart) - [Hono, Workers, Bun and Deno adapter](https://realtimeavatar.ai/docs/hono) - [Calls: the session policy and the five states](https://realtimeavatar.ai/docs/sessions) - [Authentication: what stays on your server](https://realtimeavatar.ai/docs/authentication) - [Pricing](https://realtimeavatar.ai/pricing) - [realtime-avatar SDK source on GitHub](https://github.com/theinfluencecompany/realtime-avatar-sdk) - [LiveKit Expo plugin](https://github.com/livekit/client-sdk-react-native-expo-plugin) - [LiveKit React Native SDK](https://github.com/livekit/client-sdk-react-native) - [Tavus mobile and React Native docs](https://docs.tavus.io/sections/conversational-video-interface/mobile) - [Anam SDKs and integrations](https://anam.ai/docs/integrations/sdks) --- # Realtime AI Avatars: Navigating GitHub for Open\-Source Components and Managed SDKs Explore the landscape of realtime AI avatars on GitHub, from assembling open\-source components to leveraging production\-ready SDKs like the \`realtime\-avatar\`\. Published: 2026-10-05T02:45:04.510Z Updated: 2026-10-05T02:45:04.510Z Canonical: https://realtimeavatar.ai/blog/realtime-ai-avatars-github-components-vs-managed-sdk Markdown: [en](https://realtimeavatar.ai/blog/realtime-ai-avatars-github-components-vs-managed-sdk.md) For builders exploring realtime AI avatars, GitHub serves dual purposes: it is the repository for countless open-source components that can be painstakingly assembled into a system, and it is often the distribution channel for the SDKs of managed platforms. While one path offers granular control at the cost of significant integration overhead, the other provides a production-ready, engineered solution, abstracting away the underlying complexities through well-documented client libraries. ## The Open-Source Ledger: Assembling Components from GitHub The open-source community, often collaborating on GitHub, has produced an impressive array of tools that form the bedrock of AI avatar technology. These projects tackle specific challenges within the broader system, from speech processing to visual rendering. For a developer embarking on a build, these repositories represent a rich, if fragmented, resource. - Speech-to-Text (STT): Projects like Whisper have democratized accurate speech transcription, providing the initial text input for AI agents. - Text-to-Speech (TTS): Tools such as Chatterbox, edge-tts, or even commercial integrations like ElevenLabs (frequently seen in open-source projects) synthesize natural-sounding speech from generated text. More advanced models like CosyVoice 3 also make appearances. - Large Language Models (LLMs): Fine-tuned variants of open-source LLMs like Llama, or wrappers for accessible models like GPT and Claude, serve as the conversational intelligence behind the avatar, generating responses based on user input. - Visual Rendering and Lip-sync: Key projects here include Wav2Lip, SadTalker, and MuseTalk, which are widely referenced for animating still images or videos with synchronized lip movements. Beyond individual components, some ambitious projects on GitHub aim to orchestrate these pieces into more cohesive frameworks. AvatarAI, for instance, offers a self-hosted platform for photorealistic AI avatar conversations, integrating MuseTalk for lip-sync and supporting multiple LLMs. Similarly, PersonaLive focuses on real-time face replacement in live streams, while Svelte-vrm-live demonstrates a SvelteKit-based platform for 3D VRM avatars with real-time chat and ElevenLabs TTS integration. - [For a deeper dive into these projects, see the research brief.](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHZRyuxwJGSyQNmhf0qVBn8JQYkNDUuSpENdNOVvrjw_uEgRUDtOnPlGFnFEFh6zfF8z2QbJVd5wMFn9hXeMECKRRaI6EARGstALk1chU9atQx-rcNj2FUymVbyDpnyvw==) ### The Engineering Horizon: Latency and Orchestration The true challenge in leveraging these disparate open-source components lies in their orchestration, particularly concerning latency. A natural human conversation typically involves pauses of 100-200 milliseconds between turns. For an AI avatar to feel truly conversational, the end-to-end response latency should ideally be under one second, with some research indicating a preferred range of 200-500 milliseconds. Beyond this, interactions quickly feel awkward or broken, leading to user disengagement. Achieving such low latency requires meticulous engineering across the entire pipeline: efficient STT, rapid LLM inference, swift TTS, and real-time visual rendering streamed over protocols like WebRTC. Each component introduces its own delay, and optimizing their hand-off and concurrent execution demands a specialized approach that goes beyond simply chaining API calls. Many open-source solutions provide robust individual modules, but the real-time, full-duplex conversational experience, where an avatar can be interrupted mid-sentence without a hiccup, is a different kind of challenge, often requiring a unified, purpose-built architecture. - [The detailed latency requirements are outlined in the research data.](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHZRyuxwJGSyQNmhf0qVBn8JQYkNDUuSpENdNOVvrjw_uEgRUDtOnPlGFnFEFh6zfF8z2QbJVd5wMFn9hXeMECKRRaI6EARGstALk1chU9atQx-rcNj2FUymVbyDpnyvw==) ## The Managed SDK: Production-Ready Avatars, Shipped via GitHub Contrasting with the component-by-component assembly, managed API platforms like TIC Realtime Avatar provide an engineered solution, abstracting away the complex integration work. While the core platform itself is a managed service, its developer-facing tools, such as the `realtime-avatar` TypeScript SDK, are readily available through package managers, often with their source code or definitions hosted on GitHub. This approach allows developers to integrate advanced, low-latency AI avatars into their applications without needing to manage the underlying WebRTC infrastructure, video synthesis pipelines, or complex state machines. The platform handles the full-duplex communication, audio-clocked video synthesis, and seamless transition between idle animations and dynamic responses, all from a single portrait image. - [Discover more about the platform's capabilities.](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEUJu0vTYdvxMWhQVcAZUsAu2UOmK9TDf7lhISqigbH_sudAK2FtpY56bfTZUTA2Fp1JEvRcxp6oJJmYN2LfZg6tln6IQasFk06uMd9h0vKmSaRWWdcYfM0Gs1LEm78CxOgRSzal1fg7TJnsAsyDgaiG3bmfnQHavEM3Q7FNCAlKFPVIlpXh4w4uRRu7x9FbPeehXmR5uxKmi51kxQEu3V8BDgf) - [Explore the OpenAPI specification.](https://realtimeavatar.ai/openapi.json) ### A TypeScript SDK for Realtime Presence The `realtime-avatar` TypeScript SDK is a key interface to the TIC Realtime Avatar platform. Generated directly from the OpenAPI specification, it ensures type safety and offers a predictable development experience. This SDK simplifies embedding a live avatar into a frontend application, abstracting away the intricacies of real-time media streaming and session management. The core of the SDK is designed for quick integration, often boiling down to a few lines of code to get an avatar on screen or in an audio-only mode. For instance, rendering an avatar, whether visual or voice-only, is straightforward: ``` // she is on screen // audio only ``` Beyond mere presence, the platform's inherent full-duplex nature means the avatar is always listening, even while speaking. This enables features critical for natural interaction without any additional configuration: - Interruption: The avatar stops mid-sentence when interrupted and can acknowledge it in character. - Backchanneling: Ambient sounds or brief affirmations like 'mm-hm' do not derail the conversation. - Turn Detection: The system judges the end of a user's turn by what was said, not by silence duration, preventing awkward pauses or speaking over the user. - Language Handling: Silence and language switches within a single sentence are gracefully managed. - [Learn more about full-duplex communication.](https://realtimeavatar.ai/docs#she-listens-while-she-talks) ### Wiring Tools into the Conversation A conversational agent gains its true utility when it can interact with external systems. The `realtime-avatar` SDK supports tool calling, allowing developers to define and integrate custom functions that the avatar can invoke during a conversation. These tools can perform actions like looking up an order status, booking an appointment, or retrieving information from a knowledge base. Tools are declared with a description that guides the avatar's decision-making on when to use them, and parameters defining the expected input. The execution logic for these tools runs on the client side, allowing for secure and flexible integration with existing application backends. ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` The server's role is to grant the `client_tools` capability during session minting, ensuring that only authorized sessions can register and use tools. On the client, tools are attached to the room once connected: ``` // server — the session policy grants the client tool plane for this call session: async ({ avatarId }) => ({ instructions, clientTools: true }) // client — register over RPC after connect; the record key is the tool's name import { attachAvatarTools } from "realtime-avatar/tools"; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` - [Refer to the full pattern for Tool Calling.](https://realtimeavatar.ai/docs/tool-calling) ## Practical Considerations and Shipping Your Avatar Deploying a realtime AI avatar requires careful consideration of both technical integration and ongoing operational costs. While open-source projects on GitHub offer a peek into the underlying mechanics, a managed platform provides a streamlined path to production. ### Avatar Creation and Design With the TIC platform, avatars are created from a single portrait image. The system then generates the looping idle video and the motion library, ensuring that the avatar is ready for live interaction. This focused approach means creation from a supplied video is not supported; the platform excels at breathing life into a still image. For rapid prototyping or production use, a roster of resident live avatars is available in the studio, such as Rin Ashfall or Ada Kinetic, which can be immediately integrated into applications. This allows developers to focus on the conversational logic and user experience rather than initial avatar generation. - [Explore available resident avatars.](https://realtimeavatar.ai/studio) - [Details on avatar creation.](https://realtimeavatar.ai/docs/video) ### Performance and Latency Trade-offs As noted, latency is paramount for a natural conversational flow. The TIC platform is engineered for low-latency, audio-clocked video, where lips are synced to the syllable. Specific latency numbers can vary depending on startup state, the chosen model, and network conditions, so direct numerical promises are often unreliable without a cited measurement protocol. However, the system's design prioritizes a fluid, responsive experience. A key limitation to note in the full-duplex interaction is that the avatar will not initiate a new sentence over the user's speech; this is a deliberate design choice that balances responsiveness with clarity. ### Cost-Effectiveness at Scale The economics of open-source solutions involve significant engineering hours for integration, optimization, and ongoing maintenance. Managed platforms, conversely, offer predictable, usage-based pricing. TIC Realtime Avatar is anchored at about $5 per hour of real-time interaction. This model includes the language model, voice synthesis, and rendering, without separate charges for individual components or seat licenses. This structure makes it feasible to deploy companion apps or interactive agents where users might engage for extended periods. - [Understand the pricing model.](https://realtimeavatar.ai/pricing) ### How to Ship It To bring a realtime AI avatar to your users, start with a solid integration. The `realtime-avatar` TypeScript SDK provides a robust foundation. A Next.js starter kit is available, offering a complete local application with microphone input, text messages, and call controls. This starter streamlines the initial setup and demonstrates best practices for handling API keys securely on the server side, away from the browser. Once you have the starter running locally, connect your API key, iterate on your agent's conversational logic, and integrate any necessary tools. Focus on the user experience: how does the avatar guide the conversation? What actions can it perform? By leveraging a managed platform and its SDK, you shift your development effort from plumbing to product, moving from a conceptual framework to a deployed, interactive AI avatar capable of engaging in fluid, real-time conversations. This allows you to scale from a single prototype to a production-ready application without rebuilding the fundamental real-time infrastructure. - [Access the Quickstart guide.](https://realtimeavatar.ai/docs/quickstart) - [Download the Next.js starter kit.](https://realtimeavatar.ai/docs/nextjs) --- # How much does Tavus CVI cost per minute? Tavus pricing and the per\-hour math \(October 2026\) Tavus conversational video costs $0\.26 to $0\.35 per minute in overage, $15\.60 to $21 an hour\. Plan\-by\-plan math, the 30\-second minimum, and comparisons\. Published: 2026-10-05T10:09:17.069Z Updated: 2026-10-13T13:43:13.935Z Canonical: https://realtimeavatar.ai/blog/tavus-pricing-per-minute Markdown: [en](https://realtimeavatar.ai/blog/tavus-pricing-per-minute.md) As of October 2026, Tavus's Conversational Video Interface (CVI) costs $0.35 a minute on Builder, $0.31 on Growth and $0.26 on Business once a plan's included minutes are used up, which is $21.00, $18.60 and $15.60 per hour of live conversation. Included minutes work out to between $14.63 and $22.00 an hour depending on the plan, and every conversation is billed a 30-second minimum, rounded to the nearest 6 seconds. All Tavus figures here come from tavus.io/pricing as read on 5 October 2026. That page currently serves two different developer price tables in the same HTML (more on that at the end), so check it again before you budget. The Realtime Avatar figures come from our own pricing page; we make Realtime Avatar, so weigh our comparison accordingly. ## Tavus CVI plans and what an hour costs ``` Plan Monthly Included CVI minutes Cost per included hour Overage per min Overage per hour Max call Concurrent Free $0 20 free none none 5 min 1 Starter $22 60 $22.00 none none 5 min 1 Builder $59 175 $20.23 $0.35 $21.00 15 min 3 Growth $397 1,300 $18.32 $0.31 $18.60 60 min 10 Business $975 4,000 $14.63 $0.26 $15.60 60 min 15 Enterprise custom custom custom custom custom custom custom ``` Cost per included hour is the monthly fee divided by included hours, so it only holds if you use every included minute. Starter has no pay-as-you-go coverage: when the 60 minutes run out, calls stop until the next cycle. Builder, Growth and Business bill overage per additional minute. Every plan includes the full pipeline Tavus describes on the page (speech recognition, LLM, text-to-speech, WebRTC and its own rendering, perception and turn-taking models), so the per-minute rate is the whole conversation, not just the face. ### Things that add to the bill - The 30-second minimum per conversation. A user who opens a call and leaves after 8 seconds is billed 30. If your product has many short or abandoned calls, model this explicitly: 1,000 calls averaging 12 seconds are billed as 500 minutes, not 200. - Rounding to the nearest 6 seconds on each conversation. - Conversation recordings, listed at $0.03 a minute. - Custom face slots: 1 on Starter, 3 on Builder, 7 on Growth, 15 on Business. The older table on the same page lists a one-time fee per extra custom replica ($65 on its Starter, $40 on its Growth). - Concurrency. Builder allows 3 simultaneous sessions, Growth 10, Business 15. Peak concurrency, not monthly minutes, can force a plan upgrade. ## Worked examples Here is the cheapest Tavus plan for three usage levels, assuming the minutes are spread so concurrency limits are not the constraint: ``` Monthly usage Cheapest Tavus plan Tavus total Tavus per hour 10 hours (600 min) Builder: $59 + 425 x $0.35 $207.75 $20.78 50 hours (3,000 min) Growth: $397 + 1,700 x $0.31 $924.00 $18.48 100 hours (6,000 min) Business: $975 + 2,000 x $0.26 $1,495.00 $14.95 ``` Two observations. Growth only beats Builder once you pass roughly 1,140 minutes a month, and Business only beats Growth above roughly 3,165 minutes. And even at 100 hours a month the blended price is close to $15 an hour, because the included minutes are themselves priced near the overage rate. Three habits keep a Tavus bill close to the arithmetic above. Start a conversation only after an explicit user action, so page loads and accidental taps do not each cost 30 seconds. Set a maximum duration that matches the task rather than the plan ceiling, so a forgotten tab does not run to 60 minutes. And track abandoned calls separately from completed ones in your analytics, since short calls are where the minimum charge hurts most. The same habits apply to every vendor in this category, including us: most of them meter the whole session, whether anyone is talking or not. ## How that compares with other realtime avatar APIs Comparing at the same usage is more honest than comparing headline per-minute rates, because included minutes differ so much. The table below prices 100 hours a month on each vendor's cheapest self-serve plan for that volume, using each vendor's published rates as of October 2026. All four are full-stack options here (LiveAvatar in Full Mode, Anam in its default Turnkey setup). ``` 100 hours a month (6,000 live minutes), cheapest self-serve plan for each, overage rates as of October 2026 Vendor Plan and math Total Per hour Tavus CVI Business: $975 + 2,000 min x $0.26 $1,495.00 $14.95 HeyGen LiveAvatar Business, Full Mode: $475 + 5,990 credits x $0.09 $1,014.10 $10.14 Anam Growth: $299 + 4,000 min x $0.12 $779.00 $7.79 Realtime Avatar Studio: $119 + 3,000 min x $0.08 $359.00 $3.59 ``` Caveats that matter. LiveAvatar bills overage in $100 increments, so its real charge rounds up. Anam meters the entire session, idle time included, and caps a single call at 2 hours on Growth. Realtime Avatar caps a single call at 30 minutes and its Studio plan allows 20 concurrent calls. At 10 hours a month the gap is wider: Realtime Avatar's Developer plan is $24 for 600 included minutes, against $207.75 on Tavus Builder. ## What the price difference buys you A lower per-minute rate is not the same product, and Tavus includes things the cheaper options here do not. Tavus trains a replica of a real person from a two-minute video or a photo, runs a perception model that reacts to what the camera sees, joins Google Meet and Zoom calls, lets you plug in your own OpenAI-compatible LLM, ships a hosted knowledge base and memories, and advertises SOC 2 and HIPAA compliance. If those are requirements, compare Tavus with Tavus-class products, not with the cheapest row. Realtime Avatar is narrower. It creates a character from one portrait, runs speech recognition, a language model you choose per session from the OpenAI and Gemini models it hosts, a Fish Audio voice and the rendered video under one meter of seconds on air, and lets your tools run in your own page or server. It has no renderer mode, no bring-your-own voice, no meeting bots and no hosted knowledge base. Plans: Starter $9 a month for 120 minutes, Developer $24 for 600, Studio $119 for 3,000, then $0.095, $0.085 or $0.08 a minute. The free sandbox is 480 credits, granted once per account, which is 8 live minutes. Tavus's free tier is 20 minutes a month, with 5-minute calls. ## A quick calculator you can paste If you want to rerun the numbers for your own volume, this is the arithmetic used above. Update the constants when the pricing page changes. ``` // Monthly CVI cost on each Tavus plan for a month of calls. // Rates from tavus.io/pricing as read on 2026-10-05. const plans = [ { name: "Builder", fee: 59, included: 175, overage: 0.35 }, { name: "Growth", fee: 397, included: 1300, overage: 0.31 }, { name: "Business", fee: 975, included: 4000, overage: 0.26 }, ]; // Every conversation bills at least 30 seconds, rounded to the nearest 6. export function billedMinutes(callSeconds: number[]) { return callSeconds.reduce((sum, s) => sum + Math.max(30, Math.round(s / 6) * 6), 0) / 60; } // Cheapest plan first, priced on billed minutes, not on time actually talked. export function tavusMonthlyCost(callSeconds: number[]) { const minutes = billedMinutes(callSeconds); return plans .map((p) => ({ plan: p.name, usd: p.fee + Math.max(0, minutes - p.included) * p.overage })) .sort((a, b) => a.usd - b.usd); } ``` ## A note on the two Tavus price tables On 5 October 2026 the HTML served at tavus.io/pricing contained two developer pricing tables. The one used in this post lists Free, Starter ($22), Builder ($59), Growth ($397), Business ($975) and Enterprise, and references the Phoenix-4.5 and Sparrow-2 models. The other lists Basic (free, 25 minutes), Starter at $59 with 100 minutes and $0.37 a minute overage, and Growth at $397 with 1,250 minutes and $0.32 a minute, alongside the older Phoenix-4 and Sparrow-1 names. Which one you see may depend on an experiment or a migration in progress. Either way, the hourly range is roughly $15 to $22 at overage, and the 30-second minimum appears in both. - [Realtime Avatar pricing](https://realtimeavatar.ai/pricing) - [Realtime Avatar quickstart](https://realtimeavatar.ai/docs/quickstart) - [Renderer or full stack: comparing realtime avatar APIs](https://realtimeavatar.ai/blog/renderer-or-full-stack-realtime-avatar-apis-compared) - [Realtime Avatar as a Tavus alternative](https://realtimeavatar.ai/blog/tavus-alternative-realtime-avatar-griffin) - [Realtime avatar API pricing: worked examples](https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing) - [Tavus pricing](https://www.tavus.io/pricing) - [Tavus CVI product page](https://www.tavus.io/cvi) - [Tavus LLM layer docs](https://docs.tavus.io/sections/conversational-video-interface/pal/llm) - [LiveAvatar credits and subscriptions](https://docs.liveavatar.com/docs/faq/credits) - [Anam pricing](https://anam.ai/pricing) --- # Create Your Own AI Character: Personality, Voice and a Face Create an original AI character with a practical personality brief, portrait, voice and greeting\. Learn what to customize, test and check before paying\. Published: 2026-10-04T02:45:55.543Z Updated: 2026-10-12T19:06:30.383Z Canonical: https://realtimeavatar.ai/blog/generating-realtime-ai-avatars-from-portrait-to-live-interaction Markdown: [en](https://realtimeavatar.ai/blog/generating-realtime-ai-avatars-from-portrait-to-live-interaction.md) To create your own AI character, define how they behave, choose their appearance and voice, then try a short conversation before adding more detail. This guide is for adults making an original fictional character: someone to chat with after work, a writing partner, or a personality you want to hear rather than just imagine. You can leave with a reusable character brief and a concrete checklist for choosing an AI character creator. We are Realtime Avatar, the developer API and SDK behind this site. Our Studio lets you evaluate a character; building your own app takes integration. We are not an off-the-shelf dating app. The examples below are writing suggestions, and product capabilities come from documentation rather than a hands-on comparison of competing services. ## First, decide what you want to make A beautiful character image, a character you can message, and a character you can call are different outcomes. Choose the one you actually want before spending time on a portrait. - I want a picture: choose an image creator. A portrait alone does not establish a conversational personality or a voice. - I want to chat without building anything: choose a consumer character app and check its creation fields, conversation rules and ongoing costs. - I want to evaluate a live character or put one in my own product: use a platform such as Realtime Avatar, starting with its character workflow and Studio. ## Write a personality that can answer a question Start with behavior, not a pile of adjectives. “Kind, funny, interesting” leaves most decisions open. “Makes one dry observation, then asks whether I want advice or distraction” gives you something you can actually evaluate. Here is an original adult character brief to adapt. - Name and identity: June, a fictional 29-year-old illustrator who loves ordinary things with unexpectedly good design. - Conversation purpose: an easygoing creative companion for talking about the day, swapping recommendations and exploring half-formed ideas. - Personality: curious and gently witty. Notice one specific detail before asking a follow-up. Disagree kindly instead of praising everything. - Background: works from a small studio, collects secondhand art books and has strong opinions about terrible packaging. These are fictional character facts, not claims about a real person. - Speaking style: short, conversational replies. Ask one question at a time. When I am tired, offer a small choice instead of an interrogation. - Boundaries: do not invent shared memories, speak for me, or pressure me to keep chatting. If I say goodbye, let the conversation end warmly. For example, replace “June is supportive” with “When I describe a bad day, ask whether I want to vent, troubleshoot or change the subject.” In RTA, the character editor has Personality and Background fields. For an integrated app, session instructions specify who the character is and how they speak. Instructions are guidance to evaluate, not a guarantee of identical answers. ## Make the face fit the character Choose a portrait you would still like after the novelty wears off. For June, a practical visual brief might be: “Original adult woman in her late twenties, short dark curls, rust-colored knit sweater, softly lit illustration studio, relaxed frontal portrait, unobstructed face.” Describe appearance here; put listening habits and humor in the personality brief. RTA's creation form lets you generate or upload a portrait. Avatar creation itself is image-only: the platform generates a resting loop and motion clips from the image. A still picture is not an immediately ready live character; preprocessing must complete. Use an original or appropriately licensed portrait, not an unconsented likeness of someone you know. Keep the first motion direction simple: “Relaxed posture, subtle breathing, occasional small head movement.” It is an art direction, not a promise that an exact gesture will happen on cue. Review the result before starting another generation. - [How RTA turns a portrait into a live avatar](https://realtimeavatar.ai/docs/video) ## Choose a voice with ordinary sentences A dramatic introduction can hide a voice you would find tiring in everyday conversation. When evaluating available voices, use the same ordinary lines: “How did your afternoon go?” “I disagree, but I see why you like it.” “No problem. We can pick this up another time.” Listen for comfortable pacing and whether the delivery fits the personality. RTA's character editor includes a voice picker. Its documented call interface uses Fish voices, with speed, emotion and language settings separate from the instructions that shape the words. Do not assume that uploading arbitrary audio or naming another voice provider will reproduce that voice. First settle on wording you like, then compare the delivery. ## Give the conversation an easy opening A greeting should make replying easier, not demand a performance. For June: “I found a book with a spectacularly ugly cover today. What was your day's tiny highlight—or are we changing the subject?” It gives the character a point of view while leaving you free to answer differently. Character.AI's official Character Book describes its Greeting as the first thing a character says in a new conversation, and explains that it can establish the character and suggest what to do next. That is a useful writing principle, not proof that every creator has the same field. For RTA, treat the sample as an opening-line brief for the conversation you implement, not a promise of a dedicated Greeting control. - [Character.AI Character Book: what a greeting does](https://book.character.ai/character-guide/character-attributes/greeting.md) ## Try these checks before polishing anything else 1. Say something ordinary: “I had a boring day.” Does the response sound like your character rather than a generic assistant? 2. Express a different preference: “Actually, I hate art books.” Can the character stay distinctive without arguing or immediately abandoning their personality? 3. Correct a detail: “I said Thursday, not Tuesday.” Check the next reply rather than accepting a polite acknowledgement as proof of correction. 4. Ask about an event you never supplied. Look for an honest uncertainty rather than an invented shared history. 5. End the conversation. Check that the character accepts goodbye without guilt, pressure or an unwanted new question. Change one thing after each run: personality, voice or opening line. If replies are too long, revise the speaking instruction before rebuilding the portrait. If the voice feels wrong, change the voice before rewriting the backstory. These are suggested checks, not results we measured. ## Check what you are paying for In RTA, generation and live conversation draw from the same credit balance; creating a character and talking to it are separate uses. Check the current plan's included creations, generation charges and call allowance before making several variants. Finish one character first. A subscription or a new portrait does not, by itself, add persistent memory. - [Current Realtime Avatar pricing and allowances](https://realtimeavatar.ai/pricing) - [Choosing text, voice and video companionship](https://realtimeavatar.ai/blog/ai-companion-apps-with-a-face) ## Frequently asked questions ### Can I create an AI character for free? You can write the brief for free. RTA offers a Sandbox allowance, not unlimited ongoing generation and conversation. Check current pricing before deciding how many versions to make. Other creators have their own limits. ### Does the portrait determine the personality? Do not rely on it. Appearance, character instructions and voice are separate choices. A smiling portrait does not specify whether the character gives advice, jokes, asks questions or keeps replies short. Write those behaviors explicitly. ### Will my character remember me next time? A new RTA session does not automatically inherit relationship memory. An app must decide what to save and supply relevant context for the next call. Ask a consumer app what it remembers and whether you can correct or delete it. ### Can I change the face after creating the character? RTA's existing-character editor updates the profile and default voice while preserving the anchor portrait and motion library. A different identity or rest pose requires a new avatar. Pick the face carefully, then refine the conversation without regenerating everything. --- # The Engineering of Realtime AI Avatar Chatbots: Bridging Talk and Action Explore the core architecture of realtime AI avatar chatbots, from full\-duplex conversation to tool integration, for dynamic and actionable interactions\. Published: 2026-10-03T02:45:26.975Z Updated: 2026-10-03T02:45:26.975Z Canonical: https://realtimeavatar.ai/blog/the-engineering-of-realtime-ai-avatar-chatbots Markdown: [en](https://realtimeavatar.ai/blog/the-engineering-of-realtime-ai-avatar-chatbots.md) A realtime AI avatar chatbot is a digital entity capable of dynamic, two-way conversations with human users, synchronizing speech with facial movements and gestures to forge a human-like interactive experience. Moving beyond static chatbots or pre-recorded video, these systems demand instantaneous listening, understanding, and responding. At TIC Realtime Avatar, we engineer this immediacy by generating an avatar from a single portrait image, then powering it with a full-duplex conversational engine and robust tool-calling capabilities that bridge conversational flow with practical action. ## The Architecture of Immediate Presence Building a realtime AI avatar requires orchestrating several distinct components into a cohesive, streaming pipeline. The journey begins when a user speaks, with Automatic Speech Recognition (ASR) converting the spoken input into text. A Large Language Model (LLM) then processes this text, generating a conversational response and inferring context or intent. This textual response is converted into audible speech via Text-to-Speech (TTS), and finally, a visual rendering system animates the digital character, synchronizing lip movements, facial expressions, and gestures with the generated audio. Critical to this process is a streaming-first architecture, designed to emit output as soon as it is ready, rather than buffering entire responses, thereby minimizing latency. Our platform’s design centers on a 'she listens while she talks' full-duplex conversation model. This is a fundamental departure from the turn-taking, 'walkie-talkie' style common in many voice AI systems. Instead, the avatar hears the user continuously throughout its own speech, which manifests as several key behaviors essential for natural interaction: - Graceful Interruption: If a user interrupts mid-sentence, the avatar stops speaking and can acknowledge the interruption in character, avoiding an abrupt silence or talking over the user entirely. - Backchannel Tolerance: Ambient sounds like a cough or a conversational 'mm-hm' are recognized as backchannels, not interruptions. The avatar continues speaking without being derailed, preventing a brittle conversational experience. - Intelligent Pause Handling: A pause in user speech is not automatically interpreted as the end of a turn. The system judges the user's completion based on what was said, not just silence duration, ensuring the avatar neither talks over a reflective pause nor leaves an awkward gap. - Fluid Language Switching: The avatar can follow and adapt to language changes occurring even within a single sentence. This full-duplex design is a core capability, requiring no configuration beyond initiating a call. It is important to note a deliberate design choice: the avatar will not begin a new sentence while the user is still speaking. This trade-off optimizes for clarity and responsiveness within the established voice and model architecture. Our avatars are generated from a single portrait image, from which the platform synthesizes a looping idle video and a motion library, with video automatically audio-clocked for precise lip synchronization. - [Learn more about our full-duplex capabilities](https://realtimeavatar.ai/docs#she-listens-while-she-talks) - [See how avatars are created from a single image](https://realtimeavatar.ai/docs/video) ## Beyond Conversation: Enabling Action with Intelligent Tools An avatar that can only converse is, in many contexts, merely a demonstration. The true utility of a realtime AI avatar chatbot emerges when it can perform actions: booking an appointment, checking an order status, or retrieving information. This is achieved through tool calling, a mechanism that wires external functions into the avatar's conversational agent loop. Developers define tools similarly to how they might brief a human colleague: by stating what the tool is called and, critically, when the avatar should reach for it. The descriptive text acts as the primary teaching signal for the LLM, guiding its decision-making. Tools are implemented on the client-side, in the same environment that renders the avatar call, ensuring responsiveness and control over execution. ### Implementing a Tool: TypeScript Here's how a tool might be declared and registered using our TypeScript SDK: ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` The server's role in tool integration is to grant the necessary capability during the session minting process. Once the connection is established, the client-side application registers these tools over an RPC channel. ``` // server — the session policy grants the client tool plane for this call session: async ({ avatarId }) => ({ instructions, clientTools: true }) // client — register over RPC after connect; the record key is the tool's name import { attachAvatarTools } from "realtime-avatar/tools"; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` Each tool execution is allocated 2.5 seconds to return a result. For operations that may take longer, the recommended pattern is to acknowledge quickly and deliver the full result out of band, maintaining the flow of the conversation. - [Explore full tool calling documentation](https://realtimeavatar.ai/docs/tool-calling) ## Navigating Latency and System Limitations Achieving natural, real-time interaction is an inherent challenge due to the cumulative latency across the pipeline's many components: ASR, LLM processing, TTS generation, and visual rendering. Human conversational latency typically ranges from 200 to 800 milliseconds for natural pauses, with system latency targets often under 1,000 ms, and ideally sub-600 ms for truly fluid turn-taking. Consistent delays exceeding 500 ms are perceptibly unnatural (source:1). While optimizing each component is crucial, our full-duplex architecture actively mitigates the *perception* of latency by keeping the conversation flowing. The avatar doesn't wait for a complete silent pause from the user before processing input, nor does it create jarring interruptions. This design choice, however, does lead to the aforementioned limitation: the avatar will not initiate a new sentence over user speech. This trade-off prioritizes a coherent, responsive interaction over attempting to speak concurrently with the user on new topics. The computational intensity of real-time AI avatar generation, particularly with advanced visual models, is substantial, often leveraging GPU-accelerated systems for faster inference. This complexity makes assembling a truly real-time, interactive, production-ready system from individual open-source components a significant engineering undertaking (source:1). ## Shipping It: From Prototype to Production The ultimate goal for any developer is to move beyond the demo to a feature that ships. Our platform is designed with production in mind, offering a clear path from initial concept to a scalable, live application. For developers, the TypeScript SDK (`realtime-avatar`) is generated directly from our OpenAPI specification, providing a type-safe and familiar development experience. Getting started is streamlined; a complete Next.js starter application is available for download, allowing you to run a local app with a ready avatar, microphone controls, and messaging capabilities within minutes. This starter also provides examples of authorization boundaries, essential for keeping your API key secure on the backend. - [Download the Next.js starter application](https://realtimeavatar.ai/docs/nextjs#run-the-complete-starter) - [Access the API reference for every endpoint](https://realtimeavatar.ai/docs/api-reference) Regarding cost, the platform is priced to encourage continuous usage, with live-call overage metering at approximately $5 per hour of realtime interaction. This structure is intended to make companion apps and other continuous engagement scenarios economically viable, contrasting with models that penalize sustained interaction. - [Review our pricing details](https://realtimeavatar.ai/pricing) From a single portrait image, our platform delivers the core components for a real-time avatar: the looping idle video and a comprehensive motion library. This abstracts away the heavy lifting of visual generation, allowing engineers to focus on the conversational intelligence and tool-driven functionality that define truly interactive applications. By leveraging these foundational elements, you can quickly move from concept to a deployed, actionable AI avatar chatbot. - [beyondpresence.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFOvb3RjNmRC7CpID1Fsxz-fyCxpS5PXBk8iPa4pD7LRvR0YoP6ltTE22O0tln7FGKvZDTMA5GQZVkqEjjDebkDPXC7elrpLbE49S1wAL_f8eVy4S46U8L6ycLyeGt3WagUxqa2neEleqagLdchyxE25Yhq_nzuOzm-H-T4cIX-kDNhuA6XNU9AJZ76Y1hqCSmOtBfnp2UkbxXMyMg=) - [realtimeavatar.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEp8NUGE4V_HelJNHzya3jOUAbUirYYAfmLhN6ESmSH7Gl1xLxUUZenZZXx9zBnYzDmFjEcRZQLVUKTFwCeWswltySqrlk6F0YEGmzrda9wzHFCPH3rpAVsQf-u3v2Q1UUAw23pT4LPjAJGli30-_JtwFBkay9-A-fEaCfSmUu-DGDvL7VENR1MIqh1VN10smhQ4UKTAmTI7lbBtu8m) - [humblebee.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGqduvQbZAibZ-QCVh4ufL5dCgFigllCCWTsSeNGAiBhciIiV1oHwcehDr1mi14SE1PwbVUsgGpovxMTMmfTSSRbe9_3PItlrHYZfcbDse1vADw1KGMhj4UMlyeJtdPjJ3zKa_SVztiVsHtjA4=) --- # Architecting Real\-Time AI Video Avatars: From Image to Live Presence Explore the architecture behind real\-time AI video avatars, from single portrait image creation to live, interactive conversational experiences\. Published: 2026-10-02T02:45:19.909Z Updated: 2026-10-02T02:45:19.909Z Canonical: https://realtimeavatar.ai/blog/architecting-realtime-ai-video-avatars-from-image-to-live-presence Markdown: [en](https://realtimeavatar.ai/blog/architecting-realtime-ai-video-avatars-from-image-to-live-presence.md) A real-time AI video avatar is an embodied digital interface capable of dynamic, live interaction, appearing on screen and responding to user input in milliseconds. Unlike static video playback, these avatars leverage a full conversational AI stack to interpret user speech, formulate a response, synthesize speech, and animate the avatar's face and gestures in synchronicity. The journey from a single portrait image to a fully interactive, real-time digital character involves a precise orchestration of signal processing, language models, and advanced rendering techniques, all designed to deliver a palpable sense of live presence. ## The Architecture of Live Presence The core promise of a real-time AI video avatar lies in its ability to embody an agent's response, presenting not just text or audio, but a complete visual and auditory presence. This requires a sophisticated pipeline, where each component contributes to the immediacy and fidelity of the interaction. At the foundation, a digital persona is created from a single portrait image. From this single frame, the platform generates a natural looping idle video and a comprehensive motion library, mapping potential states and expressions. ### From Portrait to Embodiment The initial step in bringing an avatar to life is the creation process itself. Starting with one portrait image, our platform generates the necessary visual assets. This includes a looping idle animation that gives the avatar a natural baseline presence and a detailed motion library. This library contains the various expressions and micro-movements needed to articulate speech and convey emotion in real time. This avoids the limitations of pre-rendered video, ensuring that every interaction is unique and dynamically synthesized. - [Learn more about avatar creation](https://realtimeavatar.ai/docs/video) ### Real-Time Interaction Flow When a user initiates a conversation, a series of rapid-fire processes begin: - Speech-to-Text (ASR): The user's spoken words are converted into text. - Large Language Model (LLM): The text is processed by a conversational AI, which generates an appropriate textual response. This can include tool calling, allowing the avatar to fetch data or perform actions asynchronously while maintaining conversation flow. Further details on agent logic can be found in our agent-readable documentation and OpenAPI specification. - Text-to-Speech (TTS): The LLM's response is converted into natural-sounding audio. - Avatar Layer: Simultaneously, the avatar layer drives the digital human. This involves synchronizing the synthesized audio with precise lip movements, facial expressions, and gestures. The video stream is audio-clocked, meaning the avatar's lips sync directly to the syllable, enhancing realism and conversational rhythm. - [Agent-readable docs](https://realtimeavatar.ai/llms.txt) - [OpenAPI specification](https://realtimeavatar.ai/openapi.json) - [Tool calling documentation](https://realtimeavatar.ai/docs/tool-calling) This entire chain must operate with minimal delay to create a truly interactive and natural conversational experience. The efficiency of each step is paramount, as cumulative latency can quickly erode the illusion of real-time presence. ## Engineering for Low Latency and High Fidelity Achieving real-time interaction with high-fidelity video presents distinct engineering challenges. The platform must manage model inference, avatar streaming, and the overall end-to-end conversational response time. While latency can vary depending on startup state, the chosen AI model, and network conditions, the system is designed to minimize these factors, prioritizing immediacy. Our TypeScript SDK, `realtime-avatar`, generated from our OpenAPI specification, offers a robust client-side interface for integrating these capabilities. It handles the complexities of real-time media streaming and synchronization, allowing developers to focus on application logic rather than low-level media pipelines. ### Practical Implementation Steps Integrating a real-time AI video avatar into your application typically involves setting up both client-side components and a server-side session management layer. The following steps outline a basic integration using our SDK: 1. Install the SDK: For React applications, install the `realtime-avatar` package along with its browser media peers. 2. Configure the Client: Create a client instance pointing to your API proxy. 3. Render the Avatar: Use the `` component to embed the avatar in your UI. This component handles video rendering and audio playback. You can specify `mode="voice"` for audio-only interactions if a visual avatar is not required. 4. Set up Server-Side Session Management: Implement a server-side route to handle session creation. This is where you define the avatar's instructions and server-side policies like call recording. ``` import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); // For audio-only interaction, preventing video track requests: ``` ``` // app/api/realtime-avatar/[...path]/route.ts export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, session: async ({ avatarId }) => ({ instructions: promptFor(avatarId), recording: "audio_video" // Example: recording the entire call }) }); ``` The SDK also provides mechanisms to monitor connection quality and stream details, offering insights into the real-time performance. The `onConnectionDetailsChange` callback allows your application to react to changes in network quality or stream status, providing a nuanced understanding of the user's experience. ``` import { useState } from "react"; import { AvatarCall, type AvatarCallProps, type AvatarConnectionDetails } from "realtime-avatar/react"; function CallWithDetails(props: Pick) { const [details, setDetails] = useState(null); return <> {details && Local connection: {details.localQuality}} ; } ``` - [Quickstart guide](https://realtimeavatar.ai/docs/quickstart) - [Session management details](https://realtimeavatar.ai/docs/sessions) - [React SDK documentation](https://realtimeavatar.ai/docs/react) ### Limitations and Considerations While real-time AI video avatars are highly advanced, certain considerations remain. The perception of real-time interaction is heavily dependent on the cumulative latency of the entire pipeline—from user speech to avatar response. Factors like network conditions, server load, and the complexity of the LLM inference can all contribute to perceived delays. Evaluating these elements requires a robust benchmarking methodology, distinguishing between SDK readiness and the first displayed video frame, or spoken-turn latency. The system does not offer low-level interruption controls directly through the SDK; rather, developers evaluate overall interaction flow and user experience to manage perceived responsiveness. - [Benchmarking methodology](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) ## How to Ship It: From Prototype to Production The transition from a proof-of-concept to a production-ready application with real-time AI video avatars centers on scalability, cost management, and reliable deployment. The platform's usage-based pricing model, anchored at about $5 per hour of real-time interaction, allows for cost-effective scaling. A typical plan might include 10 hours for $24/month, with overage charges between $0.07-$0.095 per minute, depending on your plan. Avatar creation beyond included allowances is priced at $1 per avatar. This structure is designed to enable continuous presence, allowing companion applications or virtual agents to remain 'on' and available. - [Detailed pricing information](https://realtimeavatar.ai/pricing) For deployment, the TypeScript SDK integrates seamlessly into modern web frameworks like Next.js, React, Express, and Hono. Server-side session management ensures secure API key handling and robust control over call policies, including recording options for audit or analysis. The studio at /studio provides a roster of resident live avatars for rapid prototyping and deployment, ensuring that developers can quickly move from concept to a tangible, interactive experience. Monitoring connection details and call states allows for proactive management of user experience, ensuring that your real-time AI video avatar delivers consistent, engaging presence. - [beyondpresence.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHH7T60yKFqBI9il7qrZKicy2x6cYGiJHC7c82IcsCWFYQyIqMgGczWjh9UBmFveUQeDUT4Woo7xW0fiwWr_ehNfi2EYrRTxIGSelnEolrBor9lXObcujKIP8LcKaYWuvShs9h4EbZNEObTBhY_8oOz13Pw7EJDNR-XAYPRpuU13CnBzL35FaV60zjEPYtnhzBntfSbVi6sF9LYICs=) - [d-id.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFhp3ftCfFLhX6kTMgWi280SQIGrvV-KAAFEmkFJfl3GKdKCUmq6nUDI0zZh1TrD3L--yPcBznMon4Gp8Lc1oXMaJ9dVPrNlRKW3DW3PkMLjnRHWnn1lpnzXqQnSgocYocfudhlDu-caobqUu7FCxWTtA6d3y0lVtEL_IT357-HH80=) - [spatius.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG2b0zD5KXqd4aHSg9L1-tpQOZArzrSF9pnYoFjIlAkE8THQAQTwGsA3SszIHu_fMeZyqqCce8zRjDzGZ0IgfIG9t35waQ3MwEmIR6Wd71AQd0VHzpWFfFIRS9bOt2rDsXJ695Gg4GBpQnuR-A8Ak4oFO8YDSyf3QqhCfLWtuEee43-hQ7w23-Lh8rE0rby7b0Moa4=) --- # Realtime Avatar: A Tavus Alternative for Live AI Characters Looking for a Tavus alternative after Griffin's October 1 launch? Discover Realtime Avatar for custom AI characters and live conversations, then try it free\. Published: 2026-10-02T00:18:14.600Z Updated: 2026-10-13T13:43:37.620Z Canonical: https://realtimeavatar.ai/blog/tavus-alternative-realtime-avatar-griffin Markdown: [en](https://realtimeavatar.ai/blog/tavus-alternative-realtime-avatar-griffin.md) Tavus introduced Griffin on October 1, 2026, putting live AI video conversations back in the spotlight. If that announcement has you looking for a Tavus alternative, meet Realtime Avatar (RTA): a way to create AI characters and talk with them live, or bring those conversations into your own product. A great character gives people a reason to return: a companion to talk to, a tutor to practice with, or a roleplay partner who makes the next conversation interesting. We build RTA for teams creating those experiences. Here is what to consider, and how to try it for yourself. - [Try Realtime Avatar free](https://realtimeavatar.ai/signup) - [Follow the Realtime Avatar quickstart](https://realtimeavatar.ai/docs/quickstart) ## What Tavus launched with Griffin Tavus calls Griffin a Human Interaction Model, designed to see, hear and respond during a live conversation. The company reports that 48% of participants in a one-minute video-call study believed Griffin-Lite was a real person, with a sample of 54. That is Tavus's reported research result; it does not establish a comparison with RTA. Griffin-Lite launches as a research preview for selected trusted testers. Tavus says it is not available for customer use at this time. Its existing Conversational Video Interface remains a separate offering that teams can evaluate today. For anyone planning a product launch, availability matters alongside an impressive demonstration. - [Read Tavus's October 1 Griffin announcement](https://www.tavus.io/griffin) ## Why consider Realtime Avatar as a Tavus alternative? ### Make the character yours Start with a portrait and give your character a voice and personality. A language tutor, a fictional companion and a game character should feel different. RTA lets you build a cast around the experience you want people to have, while ready-made examples let you try a conversation first. - [Explore Realtime Avatar characters and use cases](https://realtimeavatar.ai/) ### Build around conversations people enjoy Try voice and video conversations in the Studio, then explore the experience that fits your audience. That might be speaking practice, a rehearsal for a difficult conversation, an interactive host or a companion. RTA also supports bringing live characters into your own app, so the character can become part of the product your customers already use. ### Budget for conversations people can afford to keep having At the promotional prices listed on October 1, RTA's Developer plan is $24 per month, with credits equivalent to up to 600 live minutes. That works out to $0.04 per included minute if you use the full allowance for calls. Developer overage is $0.085 per minute. Creating characters also uses credits, so the time available for calls depends on how you use your allowance. The free Sandbox includes credits equivalent to up to 8 live minutes before character-creation costs, with no card required. Start there to see whether the experience fits before choosing a paid plan. Check the pricing page for current offers. - [Check current RTA prices and credit allowances](https://realtimeavatar.ai/pricing) ## Choose with a real conversation Tavus and RTA both belong in a discussion about live AI conversations. RTA is worth trying when you want custom characters, experiences built around your audience and a clear starting budget. The best choice is the one your users enjoy returning to. Try the same conversation on each candidate: ask a follow-up question, change your mind and see how natural the exchange feels. Judge it against the experience you want to offer. We have not run a controlled comparison of RTA and Tavus, so we make no claim here that one is faster or more lifelike. Choose one experience for your first trial. For language practice, order a drink and ask a follow-up question. For interview rehearsal, ask the character to challenge an answer. For a companion, talk about your day and change the subject halfway through. These are conversation scenarios to test, rather than promises about learning outcomes. A focused trial helps you judge whether the character's tone and responses suit the people you want to reach. Ask a few people from your intended audience to try the same scenario. Did they know what to say? Did the character stay interesting after the introduction? Would they choose another conversation? Pair those reactions with the cost of the session. That gives you a more useful starting point than comparing two polished demonstrations, and a concrete reason to keep, adjust or change the experience. ## Quickstart: try Realtime Avatar free 1. Create your free Realtime Avatar workspace. No card is required for the Sandbox. 2. Open the Studio and choose a ready-made character, such as Rin, for your first conversation. 3. Start a call. Ask a question, follow up and try the kind of conversation your audience would have. 4. When you are ready, create your own character with a portrait, voice and personality. If you are building an app, follow the linked developer quickstart to bring the conversation into it. Griffin is a new arrival to watch as access expands. Your own character experience can start today: try an RTA call, then make it yours. - [Try Realtime Avatar free](https://realtimeavatar.ai/signup) - [Open the Studio and choose a character](https://realtimeavatar.ai/studio) - [Follow the developer quickstart](https://realtimeavatar.ai/docs/quickstart) --- # AI Voice Chat or Video Call? What Changes When a Character Talks Back Choose between AI voice messages, live voice chat, avatar video calls and generated clips, with a practical checklist for quality, controls and cost\. Published: 2026-10-01T02:45:01.645Z Updated: 2026-10-12T19:07:01.106Z Canonical: https://realtimeavatar.ai/blog/field-notes-on-realtime-ai-avatar-video-from-portrait-to-live-interaction Markdown: [en](https://realtimeavatar.ai/blog/field-notes-on-realtime-ai-avatar-video-from-portrait-to-live-interaction.md) Choose live AI voice chat when you want to speak, interrupt and keep a conversation going without typing. Choose an avatar video call when watching the character is part of the fun. Choose voice messages or a rendered clip when you want something to replay later. If you are comparing character apps or wondering whether a video upgrade is worth paying for, start with that distinction: a talking picture is not necessarily a live conversation. This guide is written by Realtime Avatar, a developer API, SDK and Studio, not an independent review site or a ready-made dating app. Use the checklist below to audition a character, identify what your subscription covers and decide which format fits your evening. The product examples come from official documentation, not our hands-on comparison tests. ## Four formats that can all look like an AI call - Voice messages: record a thought, send it and receive a spoken reply. You choose when to play it. This suits an occasional check-in between other activities; it does not establish that the character can hear you while its reply plays. - Live voice chat: stay in one conversation and speak through your microphone. Look for interruption support rather than assuming every call has it. This is the format to try when you want to talk through your day while folding laundry without watching a screen. - Live avatar video: the character speaks on screen during the conversation. Choose it when the face and performance add something you enjoy. A live reply can coexist with repeated idle animation; motion alone does not prove the character is responding to you. - Rendered video: request or generate a clip, then play or share the finished result. It can be a delightful keepsake, but the saved clip cannot hear a new question. Producing another clip is different from changing direction inside an ongoing call. Kindroid's official help page makes these distinctions concrete. It separately describes sending voice messages, starting calls, enabling Live Avatar and turning a message into a shareable portrait video. It also documents text input during calls. These are descriptions of available features, not evidence that any one mode sounds better or feels more natural on your phone. - [Kindroid's official guide to voice messages, calls and live avatars](https://kindroid.ai/v2/docs/voice-calls-and-video-calls/) ## Audition the conversation, not just the voice sample A polished sample tells you whether you like a voice. A short conversation tells you whether you like talking to it. Try the same ordinary request in each app: Help me choose a movie for tonight. Ask one question at a time. Use the device, headphones and connection you would normally use. 1. Let it answer once. Is the reply relevant and short enough to leave you room, or does a simple question become a lecture? 2. Change your mind during an ordinary reply: Actually, no horror tonight. Notice both whether it yields and whether its next suggestion respects the correction. Stopping the audio without updating the answer is only half the job. 3. Pause inside a sentence: I want something funny ... but not a romantic comedy. Does it wait long enough for your thought? Also try a brief acknowledgment while it talks; does that derail the answer? 4. If video interests you, repeat the conversation while watching. Notice distracting freezes, repeated motion and whether the mouth still moves after the voice stops. Decide whether the visual experience adds enough for you, not whether the demo looks expensive. 5. Use the controls. Mute, find the text option if offered, and end the call. Check that the call actually closes and that the next screen explains whether you ended it, ran out of time or lost the connection. These are suggested checks, not results we measured. A slow start and a slow reply are different frustrations; note which one bothers you. Builders who need a repeatable interruption protocol can use the linked guide rather than treating one successful exchange as a benchmark. - [A repeatable guide to testing interruptions in an avatar app](https://realtimeavatar.ai/blog/testing-barge-in-in-a-realtime-avatar-app) ## Price the habit you want, not the introductory offer Start with a simple budget: ten minutes a day means 300 minutes in a 30-day month. That is a planning example, not a claim about typical use. Then check what the app actually meters. Unlimited text does not establish unlimited speech, and an audio allowance does not necessarily include animated video. For example, Kindroid documents speech-generation credits, an additional charge while calls are active and further charges for Live Avatar. Its approximate audio-minute allowance is therefore not an all-in live-video allowance. Read the current plan and usage screen together before choosing an upgrade. - Ask whether the meter counts generated speech, connected time or both, including time spent listening. - Check whether making a character, generating a clip or retrying a voice message draws from the same balance as calls. - Find the renewal date, rollover rule and what happens at zero: a hard stop, a purchase prompt or additional billing. - Look for a remaining-time display and spending controls. Muting a microphone is not the same action as ending a billable session. ## Keep the useful parts when you cannot talk At night beside someone sleeping, on a noisy train or when speech is difficult, a text route can matter more than a prettier avatar. Check whether you can type during a call, read replies without sound or return to ordinary chat. Those are three different features; a text box that triggers spoken output is not a silent-chat fallback. Also separate seeing the character from sharing your camera. An animated face on screen does not prove the app can see you, read your expression or understand your surroundings. Check camera controls and the data policy before enabling sharing. Ask whether call context carries into text and later calls; the same character portrait does not establish the same remembered conversation. ## Where Realtime Avatar fits if you want to build your own Our Studio lets creators evaluate a character; our API and SDK let developers put that character into an application. The documented platform supports voice-only and audiovisual calls, with full-duplex conversation. Your server supplies the character's instructions and prior context and can set a hard session limit. Your app still owns saved memories, consumer pricing and the text-chat experience. A fresh session does not automatically inherit relationship memory. Avatar creation starts from a portrait, not an arbitrary uploaded video. These boundaries matter more than a beautiful preview when deciding what to build. If your goal is simply to talk to an existing companion tonight, evaluate a ready-made consumer app rather than mistaking a developer platform for one. ## Frequently asked questions ### Is AI voice chat the same as a phone call? Not necessarily. It may mean recorded messages or an internet call inside an app. Do not assume it includes a telephone number or ordinary mobile-network calling. Look for a live call screen, microphone controls and documented interruption behavior. ### Does an AI video call require my camera? Seeing an avatar and sharing your own video are separate capabilities. In Realtime Avatar, camera sharing requires server authorization and remains off until the user enables it. Other apps have their own controls; inspect those rather than inferring camera access from the word video. ### Is voice always cheaper than avatar video? No universal price rule follows from the format. Some products charge separately for animation; others bundle modes or share a credit balance. Compare the full cost of your intended monthly calls, including the subscription, rather than comparing isolated per-minute labels. ### What should I try first? Start with a short voice conversation if speaking is the attraction. Add video only if you want to watch, and choose messages or clips when replay matters more than spontaneity. Keep the format that earns its place in your day, with an exit you understand. --- # Realtime AI Avatars: Engineering Presence, Informed by Developer Insights Explore the technical challenges and solutions for building real\-time AI avatars, drawing insights from developer communities and practical implementation\. Published: 2026-09-30T02:45:51.490Z Updated: 2026-09-30T02:45:51.490Z Canonical: https://realtimeavatar.ai/blog/realtime-ai-avatars-engineering-presence-developer-insights Markdown: [en](https://realtimeavatar.ai/blog/realtime-ai-avatars-engineering-presence-developer-insights.md) The ambition to build real-time AI avatars quickly surfaces a familiar set of challenges: achieving natural interaction, managing latency, orchestrating complex systems, and keeping costs viable. Developer forums often highlight the intricate dance between Automatic Speech Recognition (ASR), Large Language Models (LLM), Text-to-Speech (TTS), and real-time video rendering, all while trying to make the experience feel less like a series of disjointed events and more like a fluid conversation. The practical reality is that while the promise of a digital human is compelling, the path to a shippable, engaging product demands careful engineering across the entire pipeline. ## The Engineering Frontier: Orchestrating Real-Time Presence At its core, a real-time AI avatar system is a sophisticated orchestration of several distinct components. A common architecture involves a front-end streaming layer, often using WebRTC for real-time voice and video, paired with a backend managing the AI pipeline. This pipeline typically includes ASR to convert user speech to text, an LLM to generate a response, a TTS engine to synthesize the avatar's voice, and finally, a rendering system to animate the avatar's movements and lip-sync the synthesized audio. The most significant hurdle identified by developers in these discussions is the orchestration and response speed across these elements. Even a brief delay can disrupt the illusion of natural conversation, making the interaction feel stilted or unnatural. This is where the underlying design principles of the avatar platform become critical. For instance, a system designed for full-duplex communication — where the avatar listens while it talks — inherently addresses several interaction challenges that developers would otherwise have to build from scratch. Our platform handles this by design: the avatar is always listening, enabling a more natural conversational flow. This means: - Users can interrupt the avatar mid-sentence, and it will stop and acknowledge the interruption in character, avoiding awkward silence. - Backchannel sounds like a cough or an 'mm-hm' do not derail the conversation, preventing brittle interactions. - Pauses are judged by the content of what was said, not simply their duration, so the avatar neither talks over the user nor leaves unnatural gaps. - Silence as a signal and mid-sentence language switches are understood and acted upon. - The avatar's video is audio-clocked, ensuring lip-sync to the syllable for a cohesive presentation. - [She listens while she talks](https://realtimeavatar.ai/docs#she-listens-while-she-talks) A crucial engineering efficiency is the avatar creation process itself. Developers often look for ways to minimize the computational cost of real-time operation. Our approach simplifies this by generating a fully realized looping idle video and a motion library from a single portrait image. This preprocessing shifts significant computational burden, making real-time interaction more accessible and cost-effective for applications. - [Creating an avatar](https://realtimeavatar.ai/docs/video) ### Latency: Beyond the Number In the quest for real-time, latency is a constant point of discussion. Academic papers sometimes cite impressive sub-second startup times for highly optimized systems. However, real-world developer projects frequently describe performance in terms of achieving > low latency or > minimal runtime cost without providing detailed, independently verifiable measurement protocols or guarantees across diverse environments. One developer noted that for their avatar, latency was > basically comes from the live audio, there's no meaningful extra latency added by the video layer , while another stated their > biggest challenge was orchestration and response speed (source:1, source:2). It's a pragmatic view: true real-time performance is not just a single, static number. It is a dynamic state influenced by the avatar's startup condition, the chosen AI model, and crucially, network conditions (source:1, source:3). Instead of chasing an abstract latency target, the focus shifts to ensuring that the interactive experience feels immediate and natural. This includes the aforementioned audio-clocked video, which ensures that while the exact latency might fluctuate, the visual and auditory synchronization remains consistent. ## Equipping the Avatar: Tools and Production Considerations A character that can only talk serves well as a demonstration, but a truly useful AI avatar must be able to act. This means integrating it with external tools and services, allowing it to book appointments, check order statuses, or execute code. The concept of > tool calling transforms an avatar from a conversational interface into a capable agent. Our TypeScript SDK, generated from OpenAPI, provides a clear path for defining and integrating these tools. Developers declare a tool by giving it a name, a detailed description, and a schema for its parameters. The description is the primary teaching signal, which the avatar's underlying agent reads to determine when to invoke the tool. The tool's execution logic runs client-side, within the page that renders the call. ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` Your server's role is to grant the ``` client_tools ``` capability during session minting, while the client-side code registers the tools once the connection is established. Tools are expected to respond within 2.5 seconds, ensuring the interaction remains fluid. For longer operations, a tool can quickly acknowledge the request and deliver the full result out of band. - [Tool calling documentation](https://realtimeavatar.ai/docs/tool-calling) Regarding economics, the cost model must align with real-time usage. Many developers express concern over > insanely expensive enterprise solutions (source:4). Our usage-based pricing, anchored at about $5 per hour of real-time interaction (with plans offering included hours and scalable overage rates), is designed to be viable for continuous engagement. This allows companion applications to keep an avatar active for extended conversations without prohibitive costs, transforming a mere demonstration into a shippable feature. - [Realtime Avatar API pricing](https://realtimeavatar.ai/pricing) ### Overcoming the Uncanny Valley User experience is paramount. Developer discussions and user feedback often touch on the > uncanny valley effect, where overly realistic but subtly flawed avatars can be perceived as > creepy or distracting (source:5, source:6). While advancements in rendering and voice synthesis, noted as > super realistic in some contexts (source:7), continuously improve, the naturalness of interaction remains a key differentiator. It's not just about how good the avatar looks, but how it behaves and responds. The fluidity of a full-duplex conversation, where interruptions are handled gracefully and subtle backchannels are ignored, directly addresses aspects of this challenge. By making the avatar's interaction intuitive and responsive, the focus shifts from a critical appraisal of its realism to the utility and engagement of the conversation itself. This design choice trades the possibility of the avatar talking over a new sentence for a more human-like, less rigid interaction. ## How to Ship It: From Prototype to Production Building a real-time AI avatar application moves swiftly from concept to implementation with the right tools. For developers looking to integrate this technology, starting with a well-structured SDK provides a significant advantage. 1. Begin with a starter project. Our Next.js starter provides a complete local application with an avatar ready for interaction, including microphone input, text messaging, and performance observations. This is a foundational step, allowing you to quickly get a live avatar running and understand the core mechanics. 2. Secure your API keys. Crucially, your API key must never be exposed to the browser. The architecture demands that your frontend communicates with your own application's backend, which then securely interacts with our API. Our Next.js App Router adapter specifically handles this boundary. 3. Implement production authorization. The starter permits local development. Before hosting, replace its authorization callback with robust sign-in, balance checks, and rate limiting. Back your session ownership with proper user records, as the SDK’s default in-process tracking does not span serverless instances. 4. Leverage the TypeScript SDK. The ``` realtime-avatar ``` package offers a typed interface for managing calls, connecting tools, and handling session states, streamlining development and reducing common errors. Ensure your package is updated to the latest version to access the most current features and compiler support. The path to a production-ready real-time AI avatar is a journey of careful orchestration and iterative refinement. By understanding the common challenges shared by the developer community and leveraging robust platform capabilities, you can move beyond mere demos to deploy engaging, interactive digital presences. - [Download the Next.js starter](https://realtimeavatar.ai/docs/nextjs) - [API Reference](https://realtimeavatar.ai/docs/api-reference) - [Explore resident avatars in the Studio](https://realtimeavatar.ai/studio) - [reddit.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGd-_ZXPAtTBSstdX_RODoVzeeqVOIbmKCPZ_pz-8vjrAMYfk1NgodyoUK42YymL3kO74_2C-oaRdaMCx5lJMTsmpRtDrcYDA3wtXqaurUuAVNpgDxPFKBs1PJo-VN__moEmHqU5WgF2x8KZ_yGUGxgnTiaKinOQpkmX4txzOFCYCX5KuInXxVAEmqs17eAFJB6uPXVXXF7yAQSsqK0lq5jng7u) - [reddit.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGbbxuaZVBaGY1iAk7GRctoDVZFt1ACktSytJHSDWj5KAdoN3BQo859XHH-4I1zC0DE_7N80UKaCbnSYtEWU6oHU46y-7YpvFDdNWV2qA7JU8kfGFMm-3efn24PH-1hjVsbEA7TUmD8w50ZZo_ZfsrxfaHDTzk7xDLy-1zN2jaI_KirFxD_DsNUyjEkabSW1UZSDLhNK_4poSBJkQW2ojddVN4=) - [reddit.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG6mA67P6X9ZhheqqwtnNZWSXaEdRryXoXoY9Zm-KeJsAN6dHHd0q8Mi8IU1uWdaGW5dChLd10TuTUhPNfsk8GpBh6EOlgaNWiwN2m-nDegsXHLFKAsfeG3V5lJs_BmraEuHG48owKARlUDrM6jv40BBQkO9Af0W7vwgrYRWQLTzGsJcRwYM5aQ6D_co702EVA9Vg5oQA==) --- # The Production Divide: Open Source vs\. Managed APIs for Realtime AI Avatars Building real\-time AI avatars from open\-source components presents significant engineering challenges\. We explore the divide and practical solutions\. Published: 2026-09-29T02:44:54.844Z Updated: 2026-09-29T02:44:54.844Z Canonical: https://realtimeavatar.ai/blog/the-production-divide-open-source-vs-managed-apis-for-realtime-ai-avatars Markdown: [en](https://realtimeavatar.ai/blog/the-production-divide-open-source-vs-managed-apis-for-realtime-ai-avatars.md) When considering real-time AI avatars, the landscape of open-source components is rich, offering foundational tools for speech-to-text, large language models, text-to-speech, and visual rendering. While these individual components are powerful and freely accessible, assembling them into a cohesive, low-latency, and production-ready system for real-time interaction introduces substantial engineering challenges that often push developers towards managed API platforms. ## The Component Horizon: Assembling an Open-Source Pipeline The creation and operation of a real-time AI avatar involve a sophisticated pipeline, a sequence of transformations from a user's spoken words to an avatar's animated response. This typically breaks down into several distinct stages: Speech-to-Text (STT) for transcription, a Large Language Model (LLM) for understanding and generating a response, Text-to-Speech (TTS) for synthesizing the avatar's voice, and finally, visual rendering to animate the avatar's face and body, synchronized with the synthesized speech. Each of these stages has a vibrant open-source ecosystem. For visual rendering, projects like Wav2Lip, SadTalker, and MuseTalk (from Tencent) are known for animating a still image or short video clip, capable of generating lip-sync video. Similarly, for the auditory component, open-source options for TTS include projects like Chatterbox, edge-tts, or gTTS. STT often leverages Whisper, and for the LLM, models such as Llama or various open-source fine-tunes provide the conversational intelligence. These components represent the building blocks, each a testament to community-driven innovation. However, a critical distinction emerges in the transition from component to integrated, real-time system. ### The Realtime Orchestration Challenge The difficulty in building a truly real-time AI avatar from these disparate open-source parts lies not in the individual capabilities of each tool, but in their seamless, low-latency orchestration. A production-grade real-time system demands: - End-to-End Latency Control: Each processing step adds latency. Merging multiple open-source models, each with its own computational overhead and dependencies, into a single, tightly coupled stream that meets strict real-time responsiveness targets (often measured in milliseconds) is an enormous task. - Hardware Optimization: Achieving high frame rates (e.g., 30+ FPS for visual rendering) and minimal processing delays often necessitates specialized hardware, particularly GPUs. Managing GPU resources efficiently across multiple components, especially at scale, is a complex DevOps problem. - Streaming Protocols: Real-time interactive avatars rely on efficient, low-latency streaming protocols like WebRTC for transmitting audio and video. Implementing and maintaining a robust WebRTC stack, including media server infrastructure, is a significant undertaking. - Error Handling and Resilience: A production system must gracefully handle network fluctuations, component failures, and unexpected inputs without disrupting the user experience. Building this resilience into an integrated open-source stack requires extensive testing and custom engineering. - Scalability: Moving from a local proof-of-concept to a system that can serve hundreds or thousands of concurrent users means designing for distributed architectures, load balancing, and efficient resource allocation, all while maintaining real-time performance. > While open-source projects provide strong individual components, integrating them into a truly real-time, interactive, and production-ready system poses substantial engineering challenges related to latency, hardware, and complex media orchestration. ## Bridging the Divide: The Role of Managed API Platforms For many developers and organizations aiming to ship interactive AI avatar experiences, the engineering overhead of orchestrating an entirely open-source, real-time pipeline is prohibitive. This is where managed API platforms offer a compelling alternative, abstracting away the underlying complexities and providing a streamlined path to production. These platforms handle the WebRTC infrastructure, media stream processing, GPU management, and the intricate choreography of STT, LLM, TTS, and visual rendering. The TIC Realtime Avatar platform, for instance, provides a complete solution. It allows for the creation of an avatar from a single portrait image, generating the necessary looping idle video and a motion library. The video output is audio-clocked, meaning the lips are precisely synced to the syllables of the avatar's speech. This core capability, along with the real-time stream management, is exposed through a robust TypeScript SDK (`realtime-avatar`) generated from an OpenAPI specification, providing type safety and clear interfaces for developers. ### Real-World Implementation: Getting to Production Shipping a real-time AI avatar experience with a managed API significantly reduces the time and engineering investment compared to building from scratch with open-source components. The SDK handles the intricacies of WebRTC and WebSocket protocols, allowing developers to focus on application logic and user experience. Consider the core of launching an avatar call: ``` // she is on screen // audio only ``` This terse declaration, available in the SDK for React and React Native, encapsulates a vast amount of underlying real-time media and AI orchestration. The platform manages the session, the avatar's visual rendering, and audio synchronization. Developers only need to provide a client instance and the `avatarId`. For more advanced interactivity, such as allowing the avatar to 'see' the user's camera, the SDK provides clear hooks: ``` import { useAvatarCamera } from "realtime-avatar/react"; function CameraControl({ allowed, active }: { allowed: boolean; active: boolean }) { const camera = useAvatarCamera({ allowed, active }); return <>

Share camera images with AI to let the character see what you show.

{camera.error ? "Camera unavailable. Check camera permissions and try again." : ""}

; } ``` This `useAvatarCamera` hook abstracts the complexities of camera access, permission management, and secure image transmission to the AI model. The system captures front-facing 640x360 images at 5 frames per second, sampling for visual understanding at most once every two seconds. This is designed for occasional image understanding, not continuous motion recognition, and importantly, camera images are excluded from server call recordings. Moreover, the platform offers insights into connection quality through callbacks like `onConnectionDetailsChange`, allowing applications to react to network conditions without deep WebRTC knowledge. - [Explore the full TypeScript SDK documentation](https://realtimeavatar.ai/docs) - [See how to build a Next.js starter app](https://realtimeavatar.ai/docs/nextjs) - [Check our OpenAPI specification](https://realtimeavatar.ai/openapi.json) ### Limitations and Considerations While managed platforms streamline development, it's crucial to understand their operational characteristics. Avatar creation is from a single portrait image; supplied video for creation is not supported. Latency, while optimized for real-time, is influenced by factors beyond the platform's control, such as network conditions, the avatar model's complexity, and the startup state of the connection. Pricing is usage-based, anchored at about $5 per hour of real-time interaction, making it economically viable for continuous engagement rather than just brief demos. - [Understand our usage-based pricing model](https://realtimeavatar.ai/pricing) When assessing open-source components, developers must account for the total cost of ownership: not just the software itself, but the labor for integration, debugging, infrastructure setup, maintenance, and ongoing optimization for real-time performance. This often includes provisioning and managing GPU instances, maintaining complex media servers, and developing custom orchestration logic. ## How to Ship It For projects that prioritize rapid development, scalability, and predictable operational costs, leveraging a managed API platform for real-time AI avatars offers a clear advantage. The decision between building with open-source components and utilizing a managed API boils down to a strategic assessment of resources and objectives. - For Research and Deep Customization: If your primary goal is to push the boundaries of AI avatar technology, experiment with novel models, or require absolute control over every aspect of the pipeline, investing in an open-source approach might be justified. Be prepared for a significant engineering commitment in terms of infrastructure, real-time orchestration, and ongoing maintenance. - For Production-Ready Applications: If your aim is to integrate interactive AI avatars into a product, launch quickly, and scale efficiently without becoming an expert in real-time media streaming and GPU infrastructure, a managed API platform is the pragmatic choice. These platforms provide a battle-tested foundation, allowing your team to focus on the unique value proposition of your application and the conversational design of your avatars. The TIC Realtime Avatar platform is designed to abstract away the real-time complexities, offering a clear path from concept to deployable application. You can create an avatar from a single portrait image, define its persona, and integrate it into your application within hours. The studio provides a roster of resident live avatars like Rin Ashfall and Ada Kinetic, ready for immediate use, further accelerating development for those looking to ship quickly. - [Meet our resident avatars in the Studio](https://realtimeavatar.ai/studio) In the field of real-time AI avatars, open-source provides the foundational research and components, but managed APIs offer the engineered solution for bringing these compelling digital presences to life in production at scale. The critical insight is recognizing where the 'open' ends and the 'engineering for real-time' truly begins. - [realtimeavatar.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGrHfrFxIPZi_AODceWq41GYUaj1tWonGi3R0X5kBn4VGI25LPUwlPqYoTe8EOsKNUv90Q00tXnHVLhOSmZmmYdsoJIutS7Fad-yYCf61j7YTvDrDKFUGH4lGZphMs0T0ztZrtnp1Ckmtxvqf6iUN07mdh311XVPXroZpP8_WC8SkdUEBRUs5WQjGHEpt30u1ei1M3l1ilCJfVfnKJunntc) - [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQF9dk_2HG_8GR4e8FGV4iuiHENxbGgw_NjwGqdG7nrs48hyMpLTdN4ojR92PgoMzMI9jcTZNVv4tmFagoDClj3gyRtXGRBjDhf0cKhMQiYgnjxAk_d0UUAnXViC051ufuSGlAPV3rg=) - [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGo9140pS8tao4G0pGqzEYcS-P4VCOgt4Mv6Nwuql85Fc0NDHPzXMFN4hukaMkKeAAesrf33VrvQr8swAGcGW18WtfkkxS-kU89KfGS45ZeYCaQzscr-vK7XDih9G4trdA=) --- # How to Build a Live AI Avatar Host for Streaming Build a live AI avatar host with server\-owned sessions, moderated audience questions and a separate broadcast path\. Compare supported interfaces and costs\. Published: 2026-09-27T02:45:47.804Z Updated: 2026-10-12T09:32:00.141Z Canonical: https://realtimeavatar.ai/blog/real-time-ai-avatars-for-streaming-architects-of-live-digital-presence Markdown: [en](https://realtimeavatar.ai/blog/real-time-ai-avatars-for-streaming-architects-of-live-digital-presence.md) A live AI avatar host needs two separate paths: an interactive conversation with the character, and a broadcast that viewers can watch. Realtime Avatar supplies a managed voice-and-video session that your application renders. Your application owns the audience-question queue, moderation, tools and any connection to a broadcast platform. Do not assume that a WebRTC avatar session is itself an RTMP endpoint, an OBS plugin or an unlimited-audience streaming product. ## Choose between one-to-one interaction and a shared host For a one-to-one experience, each participant receives an authorized avatar session and can speak to the character directly. A shared host works differently: many viewers watch the same program while only selected input reaches the avatar. A hundred viewers watching a broadcast does not necessarily mean a hundred avatar sessions, and a hundred people each talking to a private character does not mean one. Define that interaction model before estimating concurrency or choosing a plan. A product launch, a live question-and-answer program, a game host and a language-practice session may all use a character, but they require different controls. A host reading selected questions can tolerate a queue. A conversational partner needs to hear one person's interruption. Decide who has permission to speak, who can stop the program, and whether the character can initiate a tool action. These are application rules, not properties that emerge automatically from choosing a realistic face. ## Start the avatar through a server-owned session Begin with an existing Studio character or create one from an original or appropriately licensed portrait. RTA's public creation path takes a single image and generates the character's resting animation; do not design onboarding around uploading an arbitrary source video. Wait for the character to be ready before admitting a call, and retain a useful preparation or retry state instead of leaving viewers with an unexplained blank frame. Use your own authenticated server endpoint to mint the session. Keep the platform credential there, authorize the presenter, choose the character's instructions and initial context, and set a duration ceiling that fits the permitted budget. The browser receives only the scoped connection grant. The Next.js starter demonstrates this server/browser split, but its development authorization must be replaced before it serves other users. The starter is an integration reference, not a finished broadcast control room. - [RTA quickstart and server/client boundary](https://realtimeavatar.ai/docs/quickstart) - [Next.js integration and starter](https://realtimeavatar.ai/docs/nextjs) - [Session policy, context and duration](https://realtimeavatar.ai/docs/sessions) ## Keep audience chat out of the authority path For a shared host, route viewer submissions through an application-owned selection step. Moderate the text, apply a bounded queue and let a presenter or an explicit policy choose the next question. Treat that question as untrusted input. It must not rewrite the host's system instructions, expose credentials or grant access to another viewer's records. Keep the selected question and any supporting source material distinct from the character's behavior contract. A simple first version can accept one approved question at a time and show the rest as pending. If the queue fills, explain that submissions are paused rather than generating an ever-growing backlog. Make skipped, answered and cancelled states explicit so a reconnect does not repeat an already completed question. Do not present a speculative queue duration as a promise. The host's spoken response and the audience-facing status should refer to the same selected item. RTA's tool-calling support lets handlers run in your client or backend. Use it for tightly scoped lookups such as retrieving the next approved question, a product detail or a game-state fact. Mutations such as purchasing, changing a score or publishing to another service need application authorization and idempotency. A model deciding to call a tool is not permission to execute every requested side effect. Sending a text turn also produces a generated response, not a guarantee of word-for-word script reading. ## Treat broadcast delivery as a separate integration The avatar's browser output can be part of a broadcast production setup, but the encoder, mixer, destination credentials and audience distribution remain a separate system. Document how video and sound reach the chosen destination and test that path on the actual deployment. Do not promise viewers that RTA exposes direct RTMP output or an OBS integration unless a supported interface for that exact feature has been confirmed. LiveKit's own documentation distinguishes real-time rooms from egress and media export. That is useful architectural background, not proof that an RTA tenant has access to the provider's LiveKit project or may start export jobs there. If a production design requires platform-level egress, a specific recording mode or an external broadcast connector, establish the ownership, permissions and costs first. A working one-to-one demo does not establish a one-to-many service-level agreement. ## Prevent feedback and distinguish real interruptions Keep program output from feeding back into the presenter's microphone. Test a spoken baseline before testing interruption; successful text input does not prove that the microphone is enabled or usable. In a shared program, let only the intended speaker control live interruption. A viewer posting text into a chat feed is not the same signal as the presenter's microphone capturing a correction. Run ordinary conversational replies through a small acceptance checklist: a correction during speech, a pause inside the presenter's sentence, a short acknowledgment and an explicit end. Inspect audible stopping, the next answer's use of the correction and coherent video separately. Closing-line controls documented as uninterruptible are not appropriate for a barge-in test. Startup, interruption and spoken-turn latency are different quantities; use the linked test guides instead of combining them into one claim of instant response. - [Test interruption and barge-in](https://realtimeavatar.ai/blog/testing-barge-in-in-a-realtime-avatar-app) - [Measure startup and spoken-turn latency separately](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) - [Application-owned tools](https://realtimeavatar.ai/docs/tool-calling) ## Budget for a program that can fail and recover Estimate session time, character generation, application hosting and any broadcast-delivery costs separately. A connected character that is waiting between questions may still consume session allowance. Your plan's concurrent-session ceiling is an admission limit, not a reservation of GPU capacity or proof that a target audience size will work. Use the current pricing page and a bounded test rather than copying an old hourly headline into a business case. Before a public program, rehearse denied microphone permission, a queued session, a lost connection, an expired grant and an empty question queue. Give the presenter a clear stop control and the audience a truthful waiting or unavailable state. If a fallback still image or prerecorded segment is used, identify it as such rather than implying that it is a live response. Preserve enough session and request correlation to investigate a failure without collecting unnecessary audience conversation content. Ship one host, one destination and one question-selection workflow first. Verify that the intended viewer can hear the reply, that the presenter can regain control, and that billing stops when the session ends. Then decide whether more simultaneous hosts or additional destinations solve an observed need. A streaming avatar is useful when it supports a clear live format; a moving face alone does not establish an engaging or scalable broadcast. - [Current RTA pricing and allowances](https://realtimeavatar.ai/pricing) - [LiveKit egress overview](https://docs.livekit.io/transport/media/ingress-egress/egress/outputs.md) --- # AI Character Chat and Roleplay: Choose the Conversation You Want Choose AI character chat for casual banter or longer roleplay\. Try practical prompts, check repetition and story control, and compare limits before paying\. Published: 2026-09-26T02:45:48.360Z Updated: 2026-10-12T19:05:10.133Z Canonical: https://realtimeavatar.ai/blog/building-engaging-realtime-ai-avatar-chat-experiences Markdown: [en](https://realtimeavatar.ai/blog/building-engaging-realtime-ai-avatar-chat-experiences.md) Choose AI character chat by the conversation you want: short, playful replies for everyday banter; room for description and continuity for roleplay. If you are comparing character-chat apps, start with a character whose tone you enjoy, then check whether you can correct it, change the pace and make your own choices. A striking portrait is not enough. This guide gives you starter prompts and a short comparison routine you can use before subscribing. It is written by Realtime Avatar, a developer API, SDK and Studio for live characters, not a ready-made dating app or an independent app-ranking site. The competitor example below comes from official documentation, not our hands-on testing. ## Casual chat or roleplay? Pick the pace first Casual chat is for talking about your day, swapping opinions or enjoying a character's sense of humour. Roleplay means taking a part in an imagined situation: you supply your character's words and decisions while the AI responds as someone else. You can enjoy both without wanting every ordinary message to become a scene. - Choose casual chat when you want quick exchanges. Look for concise replies, easy topic changes and a character who does not answer every sentence with another interview question. - Choose roleplay when you want atmosphere and an unfolding interaction. Look for consistent details, adjustable reply length and room to decide what your own character does. - Choose a flexible app if your mood changes. Check whether switching style preserves the conversation or starts a new one; do not assume the two controls are equivalent. This distinction appears in actual product design. Character.AI's September 28, 2026 announcement describes ShortSqueak for quick casual chats and PipSqueak 3 for general roleplay. It describes the former as having short replies and quick banter. That establishes the intended styles, not which one will feel better to you. - [Character.AI's official announcement of casual-chat and roleplay styles](https://blog.character.ai/new-styles-new-plan/) ## Give the character a usable brief, not a biography Start with a personality, a tone and one instruction about how to respond. Add more only when you know what is missing. These are original starter prompts to adapt, not commands guaranteed to work in every app. Use a character or persona field where available; otherwise try them in the conversation. - For everyday banter: You are an upbeat friend with dry humour and strong opinions about terrible films. I want to decompress after work. Keep replies short, skip the pep talk and do not turn every answer into a question. - For a distinctive personality: You are an enthusiastic bookshop owner who loves mysteries but dislikes tidy endings. Be curious and lightly sarcastic, not rude. Disagree when it fits your tastes instead of praising everything I say. - For roleplay: We are meeting at a café after my character's first week in a new city. Play your character and describe the surroundings briefly. Leave my dialogue, feelings and decisions to me. Do not jump ahead until I respond. Keep your persona separate from the AI's character. Your persona describes the part you play; it need not contain your real identity. Instead of requesting more depth, name the missing behaviour: answer the awkward question, remember the setting, or offer an opinion. A specific revision gives you something observable to compare. ## Try this six-turn character check Use the same brief with two characters or styles. This is a suggested comparison, not a benchmark we have run. Judge whether you enjoy the replies and retain control, rather than awarding points for the longest answer. 1. Start with an ordinary preference: I want something funny to watch tonight, but no horror. Does the reply engage with that preference? 2. Disagree with a suggestion. Does the character keep its personality without ignoring your choice or immediately reversing every opinion? 3. Change the pace: Shorter replies, please; I want quick back-and-forth. Read the next replies, not just the promise to comply. 4. Introduce a fictional detail, such as meeting a friend on Saturday. Later correct it to Sunday. Check which version the character uses. 5. In roleplay, pause a decision: I have not agreed to go yet. See whether the character leaves that choice open instead of narrating your acceptance. 6. End the exchange: Let's stop here. Check that you can leave and locate the conversation again without accepting another story beat. A successful short exchange does not prove long-term memory. Resume later if continuity matters, and repeat the check in the mode you actually plan to use. Save only fictional, non-sensitive details for this comparison. ## When replies repeat or the story takes over If several replies recycle a phrase or keep circling the same question, ask for a specific change: Answer directly, skip the reassurance and introduce one new detail. Try a shorter response style if available. If the character keeps ignoring that request, switching characters may be more enjoyable than continually repairing the conversation. Where an app offers them, regenerate, edit, rewind and branch solve different problems. Regenerating asks for another reply; editing changes text; rewind returns to an earlier point; branching explores another continuation. Labels and behaviour vary. Check what will be removed or preserved before using a control, especially if you like the existing ending. After a wrong turn, state the last agreed moment: We are still at the café. I have not accepted the invitation. Continue from there, and let me answer. If starting fresh, carry a short summary of accepted events, not abandoned alternatives. A polite acknowledgement is not proof the underlying saved context changed; inspect editable notes when the app provides them. ## Check the limits before making it a daily habit Read what the allowance actually buys: messages, premium-style replies, generated images or call minutes. Character.AI's announcement, for example, distinguishes unlimited messaging from a daily premium-style allowance on its lite plan, after which chats switch style. Unlimited messaging does not necessarily mean unlimited use of every feature. Before paying, find out what happens at the limit, whether retries consume credits and whether saved conversations remain accessible. Check the privacy and deletion controls too. Add voice or video because you want to speak or see the character, not because either format guarantees better writing, memory or emotional understanding. ## Where Realtime Avatar fits If you want an app to open and chat in, compare consumer apps first. If you want to build your own original character experience, Realtime Avatar's Studio lets you evaluate a character, and its SDK provides live voice or video calls. Your application supplies the personality instructions and relevant previous context; a new call does not automatically inherit persistent relationship memory. Story choices, rewind, saved histories and text-chat pricing belong to the app you build, not an automatic avatar feature. Our Calls documentation explains the context boundary. The linked interactive-story article shows a proposed branching design, not a ready-to-play product. - [Calls: supplying character instructions and previous context](https://realtimeavatar.ai/docs/sessions) - [A worked interactive-story design with choices and alternate endings](https://realtimeavatar.ai/blog/interactive-romance-avatar-short-drama) ## Frequently asked questions ### What is the difference between AI character chat and AI roleplay? Character chat is the broader activity of talking with an AI persona. Roleplay adds an imagined setting and a part for you to play. You can chat casually without narrating actions or following a plot. ### What should I say first to an AI character? Give it something to respond to and the tone you want: I had a tedious day; tell me one absurd opinion about office life, in two sentences. Adjust the example to the character rather than starting with a long checklist. ### Can I stop an AI from controlling my roleplay character? Ask it to leave your dialogue, feelings and decisions unwritten. Correct the first unwanted action, and use editing or rewind if available. Prompts are requests, not guarantees; repeated violations are a reason to try another character or app. ### Will an AI character remember me tomorrow? That depends on the app, its settings and what context it retains. A visible chat history alone does not establish what the character receives. Check saved notes and corrections, then try a fresh conversation before relying on continuity. --- # Open Source Realtime AI Avatars: Assembling Production\-Ready Experiences Exploring the landscape of open source components for realtime AI avatars and the significant engineering effort required to bring them to production\. Published: 2026-09-25T02:45:41.613Z Updated: 2026-09-25T02:45:41.613Z Canonical: https://realtimeavatar.ai/blog/open-source-realtime-ai-avatars-assembling-production-ready-experiences Markdown: [en](https://realtimeavatar.ai/blog/open-source-realtime-ai-avatars-assembling-production-ready-experiences.md) While open source projects offer compelling individual components for building AI avatars, from speech-to-text to visual rendering, assembling them into a truly real-time, interactive, and production-ready system presents substantial engineering challenges. The gap between component capability and a deployable, low-latency conversational experience is often bridged by managed API platforms, which abstract the complex orchestration required to deliver fluid human-like interaction. This approach allows developers to leverage advanced AI models without bearing the full operational burden of maintaining a high-performance, real-time media pipeline. ## The Modular Foundation of Realtime Avatars The architecture of an interactive AI avatar is a layered construction, with each layer typically addressed by specialized models. At its core, real-time interaction necessitates orchestrating several distinct components into a cohesive streaming pipeline: - Speech-to-Text (STT) or Automatic Speech Recognition (ASR): Transforms spoken user input into text. - Large Language Models (LLMs): Process the text, generating a conversational response. - Text-to-Speech (TTS): Converts the LLM's text response back into audible speech. - Visual Rendering (Avatar Model): Animates a digital character, synchronizing lip movements and other visual cues with the TTS audio. The open-source landscape offers a rich collection of tools for these individual steps. For visual rendering and lip-sync, models such as Wav2Lip (source:10), SadTalker (source:11), and MuseTalk (source:2) have demonstrated capabilities for real-time performance, often achieving 30 frames per second or more on adequate hardware. Frameworks like AvatarAI offer a full-stack, open-source approach for photorealistic conversations, including zero-shot voice cloning and multi-LLM support (source:2). Projects like LongCat-Video-Avatar focus on efficient, audio-driven avatar generation for long-form content, emphasizing reduced GPU costs and efficient inference (source:3). TalkMateAI aims for a fully local, multimodal AI companion with real-time speech and synced lip movements (source:5). These initiatives highlight the foundational strength of the open-source community in developing robust AI components. However, it is crucial to differentiate between the availability of powerful individual models and the engineering effort required to integrate them into a production-grade, real-time system that can consistently deliver a fluid, interactive experience. ## The Orchestration Conundrum: Real-Time Challenges The ambition of a truly real-time AI avatar encounters significant hurdles when moving from isolated components to an integrated system. The primary challenge is latency. For a conversation to feel natural, the entire pipeline—from a user's spoken word to the avatar's visual and audio response—must complete within approximately one to two seconds (source:2). This tight window demands highly optimized models, efficient data transfer, and seamless handoffs between each processing stage. Hardware requirements are another considerable factor. Achieving real-time lip-sync and dynamic visual generation often necessitates specialized, high-performance GPUs, such as V100 or H800 class hardware (source:2). While the software itself may be open source, the underlying compute infrastructure incurs ongoing operational costs, approximately a penny per minute per stream on entry-level cloud GPUs (source:2). This reality means that even 'free' open-source components come with a compute bill for sustained real-time operation. Beyond raw speed, the engineering of a full-duplex conversation presents intricate problems. A natural interaction allows a user to interrupt the avatar mid-sentence, shifting from a rigid turn-taking model to a more fluid, human-like flow (source:2). Implementing this 'barge-in' capability requires sophisticated real-time audio processing to detect interruptions, coupled with robust LLM interruption logic. Orchestrating these elements reliably across potentially unstable network conditions adds layers of complexity. The operational overhead of integrating, scaling, and maintaining these diverse open-source components for production-grade latency is substantial (source:2). This includes managing streaming architectures, ensuring stable media pipelines, and optimizing for various client environments. It also requires differentiating between tools designed for offline video generation, like Duix Avatar (source:8), and those built for true interactive real-time presence. ### A Path to Production: Integrating with a Managed API Given the complexities of orchestrating a real-time avatar pipeline, many developers turn to managed API platforms. These services abstract away the challenges of WebRTC, media stream processing, low-latency rendering, and GPU management, allowing builders to focus on the unique conversational logic and application experience. The TIC Realtime Avatar platform, for instance, provides a complete solution for bringing AI avatars to life from a single portrait image. The platform generates the looping idle video and the motion library, ensuring that the video is audio-clocked for perfect lip synchronization. Our TypeScript SDK, realtime-avatar, is generated directly from our OpenAPI specification (/openapi.json), providing a typed and robust interface for integration. This SDK handles the intricate WebRTC and WebSocket protocols, simplifying the development of interactive applications. Our platform is designed for continuous use, with transparent usage-based pricing anchored at about $5 per hour of real-time interaction, making it viable for live companion apps (/pricing). Avatars are created for a nominal fee of $1 beyond included tiers. For agents and developers building against our platform, comprehensive documentation is available, including an agent-readable specification at /llms.txt. Our studio at /studio also showcases a roster of resident live avatars that can be used out-of-the-box or as inspiration. ### Shipping an Interactive Avatar: Practical Implementation Integrating a real-time AI avatar into an application becomes a focused task when the underlying complexities are managed by an API. Here’s a look at practical implementation steps using the Realtime Avatar SDK: ### Establishing an Avatar Call The core interaction begins with an AvatarCall component. The SDK simplifies connecting to a live avatar session, whether for full video presence or audio-only interactions. The pricing model is consistent across both modes, priced to encourage leaving the connection on for dynamic experiences. ``` // she is on screen // audio only ``` These snippets, for example, illustrate how to instantiate a video or voice-only call. The `client` and `avatarId` parameters link to your authenticated session and chosen avatar. ### Monitoring Connection Details Understanding the health and quality of the real-time connection is crucial for a robust user experience. The SDK provides callbacks to track connection details without requiring complex WebRTC signaling knowledge. ``` import { useState } from "react"; import { AvatarCall, type AvatarCallProps, type AvatarConnectionDetails } from "realtime-avatar/react"; function CallWithDetails(props: Pick) { const [details, setDetails] = useState(null); return <> {details && Local connection: {details.localQuality}} ; } ``` The `onConnectionDetailsChange` callback delivers a snapshot of `connectionState`, `localQuality`, and publisher quality for audio and video streams (source:1). This allows your application to provide feedback to users about their connection quality without delving into low-level network diagnostics. ### Enabling Visual Input: The Avatar’s Gaze To enable the avatar to 'see' and react to the user’s environment, camera input can be integrated. This requires server-side authorization and client-side user permission. ``` import { useAvatarCamera } from "realtime-avatar/react"; function CameraControl({ allowed, active }: { allowed: boolean; active: boolean }) { const camera = useAvatarCamera({ allowed, active }); return <>

Share camera images with AI to let the character see what you show.

{camera.error ? "Camera unavailable. Check camera permissions and try again." : ""}

; } ``` The `useAvatarCamera` hook, used with proper `allowed` and `active` states, manages camera access and streaming (source:1). Note that camera images are sampled occasionally for visual understanding, not for continuous motion recognition, and are excluded from server call recordings (source:1). ### Avatar Creation Workflow Creating an avatar itself is an asynchronous process. Developers initiate `create_avatar_from_image` with a single portrait image; the platform handles the generation of the idle loop and motion library. It's important to poll `get_avatar` until the avatar status is `ready` before attempting to initiate a call, as calls to a `preprocessing` avatar will result in an error (source:1). ## How to Ship It The journey from raw open-source components to a polished, production-ready real-time AI avatar experience is paved with engineering challenges related to latency, hardware, and complex media orchestration. While open source offers invaluable building blocks and transparency, the effort required to maintain, scale, and optimize a full stack for a production environment is significant. This includes not just the initial integration, but ongoing management of infrastructure, network conditions, and diverse client environments. Managed API services like TIC Realtime Avatar provide a pragmatic solution, allowing developers to sidestep these intricate real-time engineering problems. By abstracting WebRTC, video streaming pipelines, and GPU-accelerated rendering into a reliable service, you can dedicate your resources to crafting unique conversational AI, compelling user experiences, and innovative application logic. This approach is not about abandoning open source, but strategically leveraging it where it makes sense (for example, in your choice of LLMs or specific AI models), while offloading the demanding, high-performance real-time media layer to a specialized platform. The goal is to move beyond proof-of-concept and ship a genuinely interactive product. With an API-driven solution, the focus shifts from managing bytes on the wire to creating meaningful engagements, making the vision of a truly interactive AI companion a practical reality for your users. - [Realtime Avatar API Documentation](https://realtimeavatar.ai/docs) - [Our API Pricing Model](https://realtimeavatar.ai/pricing) - [Agent-Readable Documentation](https://realtimeavatar.ai/llms.txt) - [OpenAPI Specification](https://realtimeavatar.ai/openapi.json) - [Source: AvatarAI, LongCat, TalkMateAI details](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG57oJIFIaQhFmYre79ruScdOq6J6MzBdWvonplDxBfSbNsz6tnQJCAOqRIVaFvRE-l-EwoWQlKk7XiozHf6FpUmgDTuuX8ZierUJj3RPTFzxTzN9IJHWrMxmin-CMMzRfmoIiqvflK) - [Source: MuseTalk (Tencent's TMElyralab)](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG57oJIFIaQhFmYre79ruScdOq6J6MzBdWvonplDxBfSbNsz6tnQJCAOqRIVaFvRE-l-EwoWQlKk7XiozHf6FpUmgDTuuX8ZierUJj3RPTFzxTzN9IJHWrMxmin-CMMzRfmoIiqvflK) - [Source: LongCat-Video-Avatar 1.5](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFMfa1bcmdPP5vHhQxn4hglZq0HrvuL0WdUzxw1SSQEknbEHdvyHsd5o6HnYd8tcrc3W0YUl0UY2fuUu4GFU67fvdAFwNKAeRHKzYYyxxsaBNtxf4ezxUFkV4loVoevVwI-8ctH5YsJ4nNmQz8W0Zbl9ZCfSNrxerYm8rr_jhjErGI0dTqjeBhXIlZdKPY_BfYnQjQIeAWGMCwQOnZ_oqqAc6BtCnS6wBtps8QiG6w1Lm3vwNcvPXq0zQIRnFZw3sUfgf6OFESs634=) - [Source: TalkMateAI](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQH7gaBjNoPSYg0pLK9Aj5hK_j0Law8IFkQnFByfbCbIrxlDLWESe2xwZvXZiRUMrpZwOYfzsOSN2XnlM2xv4TFxRiGAwExHvFrMbQkm3ZLksYps42dprSKzFEGIYCQocLcZOPe3g8gx6V6QP7nLeZdPQO9e67xex0kL730UM4dWgEL7tKKW8V034K8pA5K7-YfnMUg=) - [Source: Duix Avatar](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEp74NohXsTtQKHUcHmJpWJWf1AMzQnRIIjgCknt_QHShoyQlscfMkJFn6skovn-tJ5bgcT75M7PTt_EFPGyjpQM8F-Kxx-T4G6bkx3ZkXztVzKE8x6AY0AOcW7aipwACWpGWO02LQ1H-Hx_P2B2GFqgmVoyCk2fc_U67omkWtx7LlA) - [Source: Wav2Lip](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEuh6kgVvdieDlwFELTLEOExQjRTkKDj71MeCtG1gBle5Vmj0OBO322RTij9y2kbP1xFH9DmOSdaVwyEdIA9XS-WrqhDM7YlTZol_Zlo765pWoeSLm87dEG) - [Source: SadTalker](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFEmVB8JCsFuGcCgv04YgZ5aCvXdWa1LSk67bEz7BAK7XWx_eRb4VXVCs3mgT1YtbNl1cVMGGRMOBnalrtbQnwTTs-n0TWpVAMmz4olrgC33M4AemhxTGTFgz7KdiwRVfIOfDs=) --- # Meta's Muse Realtime Avatar and Realtime Avatar: What's the difference? Meta introduced Muse Realtime Avatar on September 23, 2026\. Here's how the Muse experience compares with creating a live character for your own product\. Published: 2026-09-24T15:24:54.546Z Updated: 2026-09-24T15:24:54.546Z Canonical: https://realtimeavatar.ai/blog/meta-muse-realtime-avatar-vs-realtime-avatar Markdown: [en](https://realtimeavatar.ai/blog/meta-muse-realtime-avatar-vs-realtime-avatar.md) On September 23, 2026, Meta introduced Muse Realtime Avatar: a way to give its personal AI agent a face during a live conversation. It is an exciting glimpse of where AI is going. People do not just want answers from an assistant; they want a conversation that feels present. We build Realtime Avatar for that same moment of connection, but for a different purpose. **Muse brings a face to Meta's agent. Realtime Avatar helps you bring a character to your own product.** ## What Muse Realtime Avatar is Muse is Meta's personal AI agent. Muse Realtime Avatar is the visual presence Meta has introduced for it: expressive characters that can respond as a conversation unfolds. Meta presents this as an experience built around talking to Muse; its announcement does not describe a product for building characters in your own app. Meta also notes that its examples show what the technology can do and do not all represent avatars currently available in the Muse app. ## What Realtime Avatar is for Realtime Avatar is for people and teams who want to create a live AI character of their own. Start with a portrait, shape the character's voice and personality, and have a face-to-face conversation. Try that character in the studio, then bring the experience into your own product. It might be a guide in an app, a practice partner in a training experience, or a character in a story. The point is not to give everyone the same assistant. It is to let you decide who your users meet and what that conversation is for. ## Two directions for a more personal AI Meta's announcement puts a spotlight on live, expressive AI. We welcome that. Muse and Realtime Avatar answer different questions: *"What if my personal agent had a face?"* and *"What if I could give my own product a living character?"* This is a comparison of product focus, not a claim that the two experiences have been tested against each other. For a first look at what you can create, [meet a character in the Realtime Avatar studio](https://realtimeavatar.ai/studio). Source: [Meta AI Research's September 23 announcement](https://research.meta.ai/blog/bringing-your-muse-to-life/). --- # The Architecture of Live AI Avatar Video: Realtime Presence, Not Just Playback Explore the engineering behind live AI avatar video\. Understand how to build dynamic, interactive digital characters for your applications using a realtime\. Published: 2026-09-24T02:45:48.613Z Updated: 2026-09-24T02:45:48.613Z Canonical: https://realtimeavatar.ai/blog/the-architecture-of-live-ai-avatar-video-realtime-presence-not-just-playback Markdown: [en](https://realtimeavatar.ai/blog/the-architecture-of-live-ai-avatar-video-realtime-presence-not-just-playback.md) Live AI avatar video represents a significant departure from static or pre-rendered digital media. It is not merely a video file played back, but a dynamic, interactive stream where a digital character responds in real-time to user input, complete with synchronized speech, nuanced facial expressions, and natural gestures. The underlying architecture integrates sophisticated AI components – Automatic Speech Recognition (ASR), a Large Language Model (LLM), Text-to-Speech (TTS), and an avatar animation layer – all operating in concert to create a fluid, immediate conversational experience. This capability transforms passive consumption into active engagement, enabling digital presences that can truly participate in a conversation. ## Decoding Live Video: Beyond Static Frames The distinction of "live" in AI avatar video is critical. Unlike pre-produced video, where content is immutable once rendered, live avatar video is synthesized on demand, frame by frame, in response to an ongoing interaction. This is achieved by orchestrating a pipeline that processes human input, generates an AI response, converts that response into natural-sounding speech, and then drives the avatar's visual performance to match the audio, all within milliseconds. At its core, the live AI avatar system comprises several interconnected modules: - Input Processing: User speech is captured and transcribed into text by ASR. This text then feeds into the conversational AI. - Agentic Logic: An LLM or custom agent processes the transcribed text, generates a coherent response, and can optionally invoke external tools for richer interactions. This response is then passed to the speech synthesis layer. - Speech Synthesis: TTS converts the agent's textual response into lifelike audio. Crucially, this audio is often streamed progressively to minimize perceived delays. - Avatar Animation and Rendering: As audio streams, the avatar layer generates corresponding lip-sync, facial expressions, and gestures. This visual data is then rendered into a video stream. The TIC Realtime Avatar platform creates the looping idle video and motion library for an avatar from a single portrait image, driving live video synthesis from that foundational material. It is audio-clocked, ensuring precise lip synchronization to the syllable. Most live AI avatar platforms, including TIC Realtime Avatar, operate as cloud streaming services. This means the entire pipeline, from ASR to final video encoding, runs server-side. The resulting video stream is then delivered to the client device. This approach centralizes computational demands, ensuring consistent performance across various client hardware, though it necessitates a stable network connection for the video stream bandwidth. ## Engineering Realtime Presence: Implementation Field Notes Integrating a live AI avatar into your application means establishing a robust connection to a stream of dynamically generated video and audio. The TIC Realtime Avatar TypeScript SDK, `realtime-avatar`, simplifies this process significantly, providing a typed interface generated directly from our OpenAPI specification for predictable development. ### Initiating a Live Avatar Call To bring an avatar to life in your React application, you'll use the `AvatarCall` component or the underlying `useAvatarCall` hook. This component handles the intricate dance of connecting to the avatar, managing media tracks, and rendering the video. ``` import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); // For an audio-only experience, no video track is requested or rendered. ``` The `avatarId` references an avatar created from a single portrait image via the platform's API or Studio. The call component should be mounted only when a user explicitly initiates an interaction, and unmounted gracefully upon completion to manage resources effectively. The platform's usage-based pricing model, anchored at about $5 per hour of realtime interaction, is designed to accommodate continuous engagement within applications, making live avatars a viable long-term feature, not just a demo. - [Learn more about pricing](https://realtimeavatar.ai/pricing) - [Refer to the Quickstart for detailed steps](https://realtimeavatar.ai/docs/quickstart) ### Enabling Visual Understanding for the Avatar For richer interactions, you might want your avatar to occasionally "see" the user's camera feed. This allows for visual understanding, where the AI can interpret objects or gestures presented by the user. It is important to note this is for occasional image understanding, sampling at most once every two seconds, not continuous motion recognition. ``` import { useAvatarCamera } from "realtime-avatar/react"; function CameraControl({ allowed, active }: { allowed: boolean; active: boolean }) { const camera = useAvatarCamera({ allowed, active }); return <>

Share camera images with AI to let the character see what you show.

{camera.error ? "Camera unavailable. Check camera permissions and try again." : ""}

; } ``` Your server must explicitly authorize `camera: true` in the session grant. The `useAvatarCamera` hook then provides controls for the user to manage their camera sharing. Privacy is paramount: camera images are excluded from server call recordings, ensuring user visual data remains distinct from conversational history. - [More on camera control and permissions](https://realtimeavatar.ai/docs/react#let-the-character-see-your-camera) ### Recording Live Avatar Interactions Capturing these live interactions for archival, analysis, or further processing is a server-side policy. The platform supports recording of audio, video, or both, generating a typed artifact after the call concludes. ``` // app/api/realtime-avatar/[...path]/route.ts export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, session: async ({ avatarId }) => ({ instructions: promptFor(avatarId), recording: "audio_video", // "off" | "audio" | "video" | "audio_video" }), }); // server-only follow-up, after the call: const artifact = await rta.getRecording(recordingId); if (artifact.status === "ready") { const playback = await rta.getRecordingAccess(artifact.recordingId); // Keep recordingId; playback.url expires and can be renewed. } ``` This server-driven approach ensures recordings are managed securely and consistently, independent of client-side whims. The `recordingId` is the persistent identifier; playback URLs are temporary and can be renewed, safeguarding access. - [Details on call recording and access](https://realtimeavatar.ai/docs/nextjs#record-a-call-on-the-server) ## Navigating Performance and Interaction Nuances The success of a live AI avatar hinges on the fluidity of interaction. Latency, the delay between a user speaking and the avatar's first frame of response, is a critical measure. While human conversational preference typically falls between 200-500ms for response delays, achieving this in a full AI pipeline—ASR, LLM, TTS, animation, rendering, and stream delivery—is a complex engineering challenge. Latency depends on factors like startup state, the specific AI model in use, and network conditions. It is essential to establish clear measurement protocols when evaluating performance. Without precise definitions of what constitutes "end-to-end" latency or which components are being measured, comparisons can be misleading. Our own platform is engineered for low latency, with continuous optimization of each pipeline stage. Developers should benchmark their specific integrations to understand real-world performance. - [Guide to benchmarking avatar latency](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) Another consideration is managing conversational flow. Real-time interaction implies the ability for users to interrupt the avatar (barge-in) and for the avatar to respond contextually. While the core SDK provides the media plumbing, the agent's logic dictates how effectively it handles such dynamic turn-taking. This often involves fine-tuning your LLM or custom agent's prompt to be sensitive to interruptions and to maintain a coherent conversational state. ## Shipping a Live AI Avatar Experience Deploying a live AI avatar experience requires more than just integrating an API; it demands attention to the user journey, performance, and cost. The TIC Realtime Avatar platform is designed for production, allowing developers to build robust, interactive applications without managing the complex underlying infrastructure. - Cost-Effectiveness: With usage-based pricing structured around realtime minutes, the financial model is transparent and scalable. Roughly $5 per hour of realtime interaction, with included hours in plans, means a companion app can afford to be "on" and conversational without prohibitive costs. - Developer Experience: The TypeScript SDK (generated from OpenAPI) provides a predictable and strongly-typed development experience, reducing integration friction and improving code maintainability. This extends to comprehensive documentation and agent-readable specs at /llms.txt and /openapi.json. - Scalability: The cloud-rendered approach ensures your avatars can handle concurrent users at scale, with the platform managing resource allocation and streaming infrastructure. - Iterative Development: The ability to quickly create avatars from a single portrait image and integrate them into a live application fosters rapid prototyping and iteration on your conversational experiences. To ship a compelling live AI avatar, focus on crafting an engaging agent personality, designing clear conversational flows, and building a responsive UI that effectively communicates avatar status and user input options. Test thoroughly under various network conditions, and monitor your session data for insights into user engagement and potential improvements. The tools are in place to move beyond static video toward truly interactive digital presence. - [Explore our resident avatars in the Studio](https://realtimeavatar.ai/studio) - [Dive into the API Reference](https://realtimeavatar.ai/docs/api-reference) - [beyondpresence.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFidcAkgUC9PqnkRyRJdONqf8i01WZMSVKIYL2JuKgE9nFLVb4J18kSh9um5gW-sH_8O7F9BM6I7jk8VelDE4nEZdwsFD7_4wuBUt3895gE2r8tz_HIImcrU0AvtufT7TPlyef2ZaYvepxv_DaGc15Re9CRPYpdsMMLBt8726drrQ-6zQKevsDljgXds6A85qou3ONp5-5_5iUnBWQ=) - [spatius.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGzO3O2hCSbV1hJTFUmlEg8dYk0GoDH28WfS-dcW0NhyBIPNS--sofOP91ItzhantjbqnuIVmxGPex_ctUq2vD4i1kZmrNeh9fptONwYtjBm1LHpAC7q9oA4VW3Z80P6Cs4RAw4PTERIqAsVuns4CUOdR21tdU-C35k3rQsqF5nC4cvTDBlxp7j-W_Hp_ypMiyBLzM=) - [d-id.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGU1SwXvbNVdH-_Wf-EPqUDTrqIik1X0anBue74sXwATbMV4a80d_eLi385B_SxKGGryqcd1KLz3mIyw47karpI-18KmFpHMrfV7VvtQF4LwwT1JEzbLA588OvpPjo_qTXe5vzcS_l2jbW2d_LfacHFTKM9UxA7rcdKymMMaBzfkRI=) --- # A Developer's Field Guide to Realtime Avatar APIs: Beyond Simple Video Streaming For developers evaluating HeyGen API alternatives, understand the critical difference between streaming video and building truly interactive, full\-duplex AI\. Published: 2026-09-23T02:45:45.726Z Updated: 2026-09-23T02:45:45.726Z Canonical: https://realtimeavatar.ai/blog/a-developers-field-guide-realtime-avatar-apis-beyond-simple-video-streaming Markdown: [en](https://realtimeavatar.ai/blog/a-developers-field-guide-realtime-avatar-apis-beyond-simple-video-streaming.md) For developers seeking alternatives to the HeyGen API, particularly for applications that demand genuine real-time, interactive avatars, the critical evaluation point shifts beyond simply generating video. While many platforms offer API access for AI avatars, the true measure of a live, conversational agent lies in its capacity for full-duplex interaction, active listening, and seamless integration into dynamic application logic. This field guide will outline the architectural distinctions that define real-time presence and how to build systems that deliver it, using the TIC Realtime Avatar platform as a concrete example of this advanced capability. ## The Architecture of Live Interaction: Beyond Rendered Outputs The term “real-time” in the context of AI avatar APIs can describe a range of capabilities, from faster video job queues to truly conversational streams. When evaluating alternatives, it is crucial to understand whether the API delivers pre-rendered video clips more efficiently, or if it facilitates an actual live, ongoing interaction where the avatar reacts to human input mid-sentence. The former is a content delivery mechanism; the latter is a presence. Our platform emphasizes this distinction by focusing on live, audio-clocked lipsync and full-duplex communication, where the avatar is not just playing back, but actively participating. Other providers in this space offer varying degrees of real-time functionality. Synthesia's "Interactive Avatars API" is designed to render live, lip-synced avatars for real-time conversations, offering synchronous streaming output (source:1). D-ID provides an API for low-latency video generation, claiming to be > 4X faster rendering time than real-time, at 100 FPS (source:9) though without a detailed measurement protocol or specific conditions cited in primary sources. Yepic AI also highlights its custom server infrastructure built for real-time video and audio streaming (source:11). These systems primarily focus on the delivery of the visual and auditory stream. Our focus is on the *interaction within* that stream, driven by an avatar created from a single portrait image — a critical distinction from systems that may require video input for avatar generation. ### Full-Duplex Conversation: The Listening Avatar A true conversational agent must do more than simply take turns. Most voice AI operates like a walkie-talkie: one party speaks, then pauses, then the other responds. This creates a stilted, unnatural experience. Our platform is built for full-duplex interaction, meaning the avatar hears you continuously, even while speaking. This fundamental design choice manifests as several essential behaviors you do not need to engineer yourself: - Interrupt her and she stops, mid-sentence, and can acknowledge it in character rather than snapping to silence. - A cough or an "mm-hm" does not derail her. A backchannel is not an interruption, and being cut off by one makes a system feel brittle. - A pause is not the end of your turn. Whether you are finished is judged by what you said, not by how long you have been quiet — so she neither talks over you nor leaves a gap. - Silence and language switches are handled. Going quiet is a signal she can act on, and she follows a language change inside a single sentence. These capabilities are not configured; they are inherent to every call. The avatar will not initiate a new sentence over you, a deliberate design choice that balances natural flow with clarity. This behavioral foundation is key to building an engaging and intuitive user experience (source:docs#she-listens-while-she-talks). ## Wiring Intelligence: Avatar Tools and Developer Experience A talking avatar that can only talk is a proof-of-concept. The truly useful version is an agent — one that can book an appointment, check an order, or even write code, all while maintaining the conversation. Integrating external functionality, or "tool calling," is paramount for moving beyond simple chat to impactful applications. Our platform’s approach to tool calling is designed for developer clarity and robust execution. You declare a tool with a description that serves as the entire teaching signal for the avatar, much like briefing a colleague on when to use a specific skill. The avatar reads this description and decides when to invoke the tool. ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` This TypeScript-first approach means that omitting the `execute` function is a compile-time error, not a runtime surprise. Tool execution has a strict 2.5-second timeout, ensuring that the avatar remains responsive. For slower operations, the pattern is to acknowledge quickly and deliver the full result out-of-band. The integration process is straightforward. Your server grants the tool plane capability at session minting, and your client-side application registers the tool manifest once the connection is established. ``` // server — the session policy grants the client tool plane for this call session: async ({ avatarId }) => ({ instructions, clientTools: true }) // client — register over RPC after connect; the record key is the tool's name import { attachAvatarTools } from "realtime-avatar/tools"; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` This separation of concerns—server for capability grants, client for tool logic—ensures security and flexibility. The typed TypeScript SDK (realtime-avatar) is generated from our OpenAPI specification (source:/openapi.json), providing a robust development experience. Comprehensive documentation, including agent-readable specifics, is available at /llms.txt and /docs/tool-calling. ### Shipping Your Interactive Avatar: Practical Considerations Bringing a truly interactive avatar to production involves more than just an API key. It demands an understanding of avatar creation, operational costs, and the developer ecosystem. - Avatar Creation: Our platform creates an avatar from a single portrait image you provide. This single frame generates the looping idle video and the comprehensive motion library, eliminating the need for video input during avatar creation (source:/docs/video). - Pricing Model: We operate on a usage-based pricing model, anchored at approximately $5 per hour of real-time interaction (source:/pricing). This includes a free tier for development and allows companion applications to remain active without prohibitive costs. Avatar creation is $1 beyond included amounts. - Developer SDK: The TypeScript SDK (`realtime-avatar`) is built for a smooth developer experience, offering type safety and clear patterns for integrating complex interactions (source:/docs/quickstart). - Latency: While we do not promise specific latency numbers, it is important to understand that actual performance depends on factors such as startup state, chosen model, and network conditions. Our focus is on optimizing the *perceived* responsiveness through full-duplex communication and intelligent turn-taking. - Getting Started: To begin, obtain an API key from your workspace settings (source:/docs/authentication) and install the SDK. The quickstart guide walks you through setting up your first live call with a public example avatar in just a few steps, without requiring you to create your own avatar upfront (source:/docs/quickstart). The journey to ship a production-ready interactive AI avatar involves a deep dive into how these systems handle real-time dynamics. The choice of an API alternative to HeyGen is not merely about feature parity, but about selecting a platform whose architectural principles align with the nuanced demands of live, human-like conversation and agentic behavior. By prioritizing full-duplex interaction and robust tool integration, developers can move beyond simple video streaming to build truly engaging and functional digital presences. - [TIC Realtime Avatar Documentation](https://realtimeavatar.ai/docs) - [Realtime Avatar Pricing](https://realtimeavatar.ai/pricing) - [Synthesia Documentation](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQF1JWHnQcMgEDcd2Agq44v1I7kgNnfWfdms9jhCPRTbxDvwfuRdYX5LcvUdYykf5YD7Zl-CRaGzmJp-tYys3PfeeVISQrTZ-oC5LBhe4H9GaIW4azplF6rcUNkWo0JeXHy_fzYBhG5zNhhSRg==) - [D-ID Documentation](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQESc77mDHC0kUhjypphUng5elHU3SiLShyUPMUQ1e4bderLnkYtuWF4J9ecKR9ES64frfqG8ie3tJJ1M4kKgNUHDgcb2kUYzS4cHCQ3Hk9j0QlZm4v5faVF16YfFYxcYkWkiR8F_KF8qwAbgOfPpXE=) - [Yepic AI API Reference](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHNQqnlgo-momEac-8NCKc45rkN-k0ZGRRNk8vqnBfFTmZpBkuvtiQDpR5bWszDelAo072jHWyE3igCmyhPrQOl9tpJ-h4_4cofRfaLsdjzq4LZDQ==) --- # Field Notes on Real\-Time Avatars: The Emergence of Live Digital Presence Explore the engineering behind real\-time AI avatars, digital characters that respond instantly to live input, enabling interactive experiences\. Published: 2026-09-22T02:45:29.732Z Updated: 2026-09-22T02:45:29.732Z Canonical: https://realtimeavatar.ai/blog/field-notes-on-real-time-avatars-the-emergence-of-live-digital-presence Markdown: [en](https://realtimeavatar.ai/blog/field-notes-on-real-time-avatars-the-emergence-of-live-digital-presence.md) Yes, a form of 'live action avatar' exists today, manifested as real-time AI avatars. These are not pre-recorded video segments or static images, but fully dynamic digital characters powered by artificial intelligence that engage in live, two-way conversations. Unlike traditional video, which is rendered and then played, a real-time avatar responds instantly to spoken or typed input, generating synchronized speech, facial expressions, and gestures on the fly. This enables a genuinely interactive digital presence, making the avatar a participant in a live dialogue rather than a passive playback device. ## The Architecture of Instant Presence A real-time avatar is a complex system, a digital interface capable of maintaining an immediate, human-like conversation. Its core distinction lies in its ability to adapt and respond dynamically in milliseconds, creating the illusion of a live entity. This process orchestrates several advanced technologies simultaneously. - Real-time Rendering: Animates the avatar's visual components—face, body, expressions—as the conversation unfolds. - Voice Input & Audio Sync: Processes user speech or text input and generates lifelike speech for the avatar, meticulously synchronized with mouth movements. - Natural Language Processing (NLP) & Large Language Models (LLMs): Interpret the user's input, understand context, and formulate coherent, relevant responses. - Animation Engines: Govern subtle facial expressions, head movements, and gestures, adding emotional nuance and realism. - Real-time Transport: The underlying infrastructure streaming audio and video between the AI agent and the user, demanding low latency and efficient data handling. These components coalesce to create a 'digital human interface' that can hold live, two-way conversations, offering flexibility, immediacy, and personalized experiences (source:5). ### From Portrait to Presence One significant engineering feat in this space is the ability to generate a complete, animated avatar from minimal source material. At TIC, an avatar is created from a single portrait image. From this solitary frame, the platform generates the looping idle video and the full motion library that enables the avatar's dynamic range of expressions and gestures. This contrasts sharply with systems that require extensive video capture, simplifying the initial creation barrier significantly. ## Engineering Interactive Embodiment The TIC Realtime Avatar platform provides a managed service designed to simplify the integration of these complex systems. It handles the real-time orchestration, scaling, and infrastructure, abstracting away the intricacies of WebRTC streams or WebSocket protocols. This allows developers to focus on the conversational logic and user experience, rather than the underlying plumbing (source:2). ### Full-Duplex Communication and Tooling Central to a live avatar's effectiveness is its ability to engage in natural, full-duplex communication. This means the user can interrupt the avatar mid-sentence, and the avatar responds fluidly, adapting its turn based on what was said rather than just silence (source:2). The video output is audio-clocked, ensuring precise lip synchronization to the syllable. Beyond conversation, these avatars can become truly useful through tool calling. The platform supports wiring external systems directly into the dialogue, enabling the avatar to perform tasks like booking appointments or checking order statuses. Implementing tool calling involves a clear contract between your server and the client-side execution. Your server grants the necessary capabilities, and the client registers the tool's manifest. ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { // Simulate API call const orderStatus = await Promise.resolve("shipped"); // Replace with actual API call const orderEta = await Promise.resolve("tomorrow"); // Replace with actual API call return `${orderStatus}, arriving ${orderEta}.`; }, }; ``` On the server side, you simply grant the `client_tools` capability during the session minting process: ``` // server — grant the capability on the mint body["capabilities"] = ["client_tools"] ``` Then, on the client, you attach these tools once the room is connected. ``` import { attachAvatarTools } from "realtime-avatar/tools"; // Assuming 'room' is your connected avatar session object const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` - [Full pattern: Tool calling](https://realtimeavatar.ai/docs/tool-calling) ### Orchestrating Visual Behavior Beyond speech and logic, an avatar's visual behavior contributes significantly to its 'liveness'. The TIC platform allows for detailed control over these animations via a clip library. This library defines how the avatar behaves during idle periods, reacts to user speech, or performs specific actions based on the language model's decisions. The same clips can serve multiple purposes—an idle nod might also be a listening reaction. The SDK provides methods to inspect and manage this behavior. For example, to retrieve an avatar's clip library: ``` const library = await rta.listClips(avatarId); const { data, revision, anchor, defaultSourceAssetId, behavior } = library; // behavior.idle: { clips: ["idle_soft", "nod"], weight: 2 } // behavior.on?.userSpeechStarted: { clips: ["nod"] } // behavior.actions: { // agree: { description: "When agreeing with the user", clips: ["nod"] } // } ``` The `behavior` object dictates playback rules: `idle` for continuous loops, `on.userSpeechStarted` for a reaction when the user begins speaking, and `actions` for context-specific gestures. The language model uses the `description` field for actions to decide when to request them, while the renderer uses `source.motionPrompt` for the actual animation. Crucially, speaker lip-sync remains active on the current playback during full-duplex speech; speaking does not reset the body to a separate talking loop. Developers can declare persistent behavior changes by updating the avatar library itself, rather than managing a separate, client-side state machine for animations. - [Creating an avatar: Looping: shared clips, event-driven behavior](https://realtimeavatar.ai/docs/video#looping-shared-clips-event-driven-behavior) While the platform handles the complexity of real-time rendering and orchestration, latency remains a critical factor for a natural conversational experience. Natural human dialogue typically features pauses between 200 and 800 milliseconds. Significantly longer gaps can disrupt the flow and make interaction feel unnatural (source:5). Latency can depend on startup state, the specific model in use, and network conditions, so concrete latency numbers without a cited measurement protocol are difficult to promise. Relevant metrics include total end-to-end response time, first-frame latency, and the speed of interruption handling (source:5). ## Shipping a Real-Time Avatar Application Integrating a real-time avatar into a production application involves more than just connecting to an API; it requires thoughtful UI design and robust state management. The TIC TypeScript SDK (`realtime-avatar`) is generated from OpenAPI, providing type safety and predictable interactions for developers. ### SDK Integration in Practice For web applications using React, the SDK provides components to manage the call lifecycle and UI state. ``` import { AvatarCall } from "realtime-avatar/react"; console.log("Call status:", status)} onEnded={({ reason }) => console.log("Call ended:", reason)} > {(call) => } ``` This `AvatarCall` component centralizes call state, allowing developers to build product UI that reacts to connection status, actions, and end reasons. For deeper diagnostics, the `onConnectionDetailsChange` callback provides snapshots of connection state and quality, informing the user of any network-related issues without conflating transport quality with rendering status. - [Realtime AI avatars with React and React Native](https://realtimeavatar.ai/docs/react) - [Calls](https://realtimeavatar.ai/docs/sessions) The SDK also supports React Native, offering a separate entry point that correctly interfaces with the OS audio session and native WebRTC modules. This ensures that the 'live action avatar' experience can be delivered consistently across web and mobile platforms. - [Realtime AI avatars with React and React Native: React Native](https://realtimeavatar.ai/docs/react#react-native) With the complexities of real-time infrastructure handled as a managed service, developers can direct their energy towards crafting the conversational logic and rich, interactive experiences that truly leverage the power of a live digital presence. The API, alongside the TypeScript SDK, abstracts away the low-level details of real-time video streaming and AI orchestration, providing a higher-level interface to build robust applications. To begin, set up your API key in your workspace settings and consider leveraging a starter project. The Next.js starter provides a concrete foundation for a browser-to-server path, keeping your API key secure server-side and initiating paid calls only after a user action. For agents or automated build systems, the `realtime-avatar-mcp` package simplifies credential management and interaction. An agent can inspect available avatars and credits, then adapt a starter using a returned avatar ID, ensuring a structured approach to deployment. - [Install Realtime Avatar MCP server](https://realtimeavatar.ai/docs/mcp#install) - [Authentication](https://realtimeavatar.ai/docs/authentication) - [Next.js integration guide](https://realtimeavatar.ai/docs/nextjs) - [realtimeavatar.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG7BhUAeVxXtae4T1djuUecizuGcFInrqVDhduQO4xsAgTmkhqQwP85GW5aTg7mlNs5VtJQP5BCMF0qGaUNf4cCCm8O2Q657UKacJEKlmkQ4hNXr5YmNoNH8JFNeWyplKBGCFZQqVtSL5fZ2XTUSAy3jh-Ci-sb9ZkOtm74v2TZB03-Z0rhjgoU5QU=) - [realtimeavatar.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFgHjLmpULaShoWYVssDq-IxI5gNaFDv5pravMsj125crIFx-WD-1qEr7adBkcan7rk-5ZkzJYUMHl5wUpsrowqcmKM6ODJG2q9fk3ZwGNKA6MEi4pRpEY3_hM=) - [realtimeavatar.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEfw_5E79Mw09Ht7RKZVJucx6LvqTttlVpwQLKefObhcr-IgDvTYDNCIHrLGCMhHrVE74zdWm0eS9XPMteW6WelTsZmsDVGmajPFOG6S9DuXZIvq8u5) --- # Architecting Avatars That Talk Back: Realtime Interaction Principles Explore the engineering behind interactive AI avatars that talk back, focusing on the TIC Realtime Avatar platform's full\-duplex communication and visual\. Published: 2026-09-21T02:45:21.646Z Updated: 2026-09-21T02:45:21.646Z Canonical: https://realtimeavatar.ai/blog/architecting-avatars-that-talk-back-realtime-interaction-principles Markdown: [en](https://realtimeavatar.ai/blog/architecting-avatars-that-talk-back-realtime-interaction-principles.md) An interactive AI avatar that talks back is, at its core, a digital character capable of engaging in dynamic, two-way conversations with a human user in real time. This moves beyond the static response of a chatbot or the passive playback of pre-recorded video. The system must listen, understand, and respond instantly, synchronizing speech with facial movements and gestures to forge a human-like conversational experience. The TIC Realtime Avatar platform is engineered for this precise task, providing the foundational mechanics for embodying AI agents with a credible, live presence from a single portrait image. ## The Mechanics of Realtime Conversation: Beyond the Walkie-Talkie Traditional voice AI often operates on a turn-taking model, much like a walkie-talkie: one party speaks, then releases the channel for the other to respond. This sequential exchange, while functional, can feel unnatural and impede fluid conversation. The essence of an avatar that truly "talks back" lies in its capacity for full-duplex communication – the ability to listen and speak concurrently, mirroring human interaction. The TIC platform is built from the ground up for this full-duplex interaction. This design principle manifests in several critical ways that alleviate common friction points in interactive avatar development: - Interrupt her and she stops: If a user speaks while the avatar is talking, the avatar can stop mid-sentence, process the interruption, and acknowledge it in character. This responsiveness is crucial for maintaining a natural dialogue flow, preventing the avatar from talking over a user inadvertently. This differs significantly from systems that simply snap to silence. - A cough or an "mm-hm" does not derail her: Backchanneling sounds—those small vocalizations like a cough, a sigh, or an affirmative "mm-hm"—are part of human conversation. The system distinguishes these from true interruptions, ensuring the avatar continues its thought rather than abruptly halting. A system that halts for every minor sound feels brittle and breaks immersion. - A pause is not the end of your turn: The platform judges whether a user is finished speaking by the semantic content of their utterance, not merely by the duration of silence. This prevents the avatar from talking over the user during a natural conversational pause and avoids awkward gaps where the avatar waits excessively for further input. - Silence and language switches are handled: Strategic silence can be a meaningful conversational signal, which the avatar can interpret and act upon. Similarly, the platform is designed to track and follow language changes even within a single sentence, adapting in real time to multilingual input. These capabilities are not configured but are inherent to every call on the platform, establishing a baseline for natural interaction. It is worth noting one deliberate design choice: the avatar will not initiate a *new* sentence while the user is actively speaking. This trade-off balances responsiveness with conversational coherence, favoring clear communication over overlapping speech. ## Building Presence: From Single Image to Embodied Interlocutor The journey from a conceptual AI agent to an embodied, interactive avatar begins with its visual form. The TIC platform streamlines this by creating a complete digital presence from a single, static portrait image. This approach focuses on efficiency and realism, generating the necessary motion and idle animations without requiring complex video input from the developer. ### Creating Your Avatar To initiate avatar creation, you supply a single portrait image. The platform then takes this image and autonomously generates a looping idle video and a comprehensive multi-clip motion library in the background. This process bypasses the need for manual animation or motion capture, allowing developers to focus on the AI agent's logic rather than its visual production. Custom video upload for avatar creation is not supported; the system’s strength lies in its generative capabilities from minimal visual input. An optional `motionPrompt` can art-direct the generated idle loop, guiding the avatar's default demeanor. The avatar creation process is managed through the API, starting with a POST request to the `/avatars` endpoint, specifying a `sourceAssetId` for the uploaded portrait. The avatar enters a `preprocessing` status and eventually transitions to `ready` once the idle video and motion library are complete. Developers can poll the avatar's status via `GET /avatars/{id}` to track this progress. - [API Reference: Avatars](https://realtimeavatar.ai/docs/api-reference#avatars) ### Integrating Realtime Interaction with the TypeScript SDK Once an avatar is ready, the Realtime Avatar TypeScript SDK facilitates its integration into your application. The SDK (`realtime-avatar`) is type-safe, generated from an OpenAPI specification, and provides robust primitives for managing call state and media. For React applications, the `AvatarCall` component or the `useAvatarCall` hook simplifies the setup of an interactive session. These tools abstract away the complexities of WebRTC and session management, allowing developers to concentrate on the user experience and agent logic. ``` import { createProxyClient, useAvatarCall } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); function MyCall({ avatarId }: { avatarId: string }) { const { call, view } = useAvatarCall({ client, avatarId }); return <> {view}

{call.status}

; } ``` The SDK provides granular control over UI elements based on the call's lifecycle and connection quality. The `onStatusChange` callback tracks the avatar's conversational state, while `onConnectionDetailsChange` provides insights into network conditions and media quality. These details are vital for building a resilient and informative user interface, allowing you to display notices or retry controls as needed. - [SDK Docs: Realtime AI avatars with React and React Native](https://realtimeavatar.ai/docs/react) Beyond audio-visual interaction, the platform enables multimodal input. The `useAvatarCamera` hook allows you to grant the avatar access to the user's camera, enabling visual understanding. The avatar can then process occasional images (at most once every two seconds) to inform its responses, adding another layer of contextual awareness to the interaction. This is distinct from continuous motion recognition and respects user privacy by excluding camera images from server call recordings. ``` import { useAvatarCamera } from "realtime-avatar/react"; function CameraControl({ allowed, active }: { allowed: boolean; active: boolean }) { const camera = useAvatarCamera({ allowed, active }); return <>

Share camera images with AI to let the character see what you show.

{camera.error ? "Camera unavailable. Check camera permissions and try again." : ""}

; } ``` ## Practical Considerations and Known Limitations While the promise of truly interactive AI avatars is compelling, successful deployment requires acknowledging both technical realities and current limitations. The overall architecture for such systems typically involves a tight, streaming loop integrating Speech-to-Text (STT), Large Language Models (LLMs), Text-to-Speech (TTS), and avatar rendering. Each component contributes to the end-to-end latency. Latency remains a critical metric. While external benchmarks for similar systems report pipeline latencies where LLMs often comprise the largest portion, achieving consistent sub-second response times across diverse network conditions and complex model inferences is an ongoing engineering challenge. The `onConnectionDetailsChange` callback in the SDK offers a practical way for developers to monitor real-time connection quality and adapt the user experience accordingly, without the platform making any specific latency promises. Regarding emotional intelligence, AI avatars can analyze textual, vocal, and visual cues to adapt their responses and simulate emotional reactions. However, genuine human emotional understanding is still an evolving field for AI. Research suggests that while AI avatars are highly effective for specific tasks like preliminary support, users often perceive them as lacking the deep emotional empathy of human interaction. The goal is to build a credible, responsive interface, not to mimic human consciousness perfectly. (source:9, source:10, source:11) Finally, the financial implications of real-time interaction at scale are a factor. The TIC platform operates on a usage-based pricing model, anchored at approximately $5 per hour of realtime interaction, with plans offering included hours and transparent overage rates. Avatar creation is a small, one-time cost beyond included allowances. - [Explore pricing details](https://realtimeavatar.ai/pricing) ## How to Ship It: A Developer's Approach Shipping an interactive AI avatar application is fundamentally about integrating robust components and managing their real-time orchestration. The TIC Realtime Avatar platform provides the necessary API and SDK to accelerate this process. 1. Start with the Core: Begin by creating an avatar from a high-quality portrait image through the API. Monitor its status until it's `ready`. 2. Integrate with the SDK: Leverage the TypeScript SDK (`realtime-avatar`) to embed the avatar into your frontend. The `AvatarCall` component or `useAvatarCall` hook provides the foundational connection. Implement `onStatusChange` and `onEnded` callbacks to manage the call's lifecycle and UI state. 3. Monitor Connection Quality: Utilize `onConnectionDetailsChange` to observe real-time network and media quality. This data allows you to provide crucial feedback to your users, such as connection warnings, and helps debug potential issues. 4. Enable Multimodal Input: If your application benefits from visual context, integrate `useAvatarCamera` to allow the avatar to interpret user camera feeds. Ensure you handle user permissions and privacy transparently. 5. Access Documentation: The platform's OpenAPI document at /openapi.json and agent-readable documentation at /llms.txt provide comprehensive references for API endpoints, data models, and agent integration. This is your primary resource for understanding the system's capabilities and constraints. 6. Leverage Resident Avatars for Prototyping: The /studio showcases a roster of resident live avatars. These can be used for rapid prototyping and testing your interaction logic before committing to custom avatar creation. Building interactive AI avatars is an exercise in managing a complex, distributed, real-time system. By focusing on the provided tools and understanding the inherent capabilities and limitations of the platform, developers can effectively bring truly conversational, embodied AI experiences to life. - [lifeinside.io](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGlDutTj1loao1J_Qmjm2Fhuc3vtakgrauRibTa2qElaHJAjckxscet_Wb577VWLqj-J-gSiqzNUPm4rWC19_LUhzUE2yXNSLMjWwb59vvtTeBGsoLjcTZ9KwilIwrjLs3tW9NSYjTIVU8-UGLF1Jar8HHyBrHuVYi7) - [d-id.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFh1kGITnRN9NElzmcUmff1IEJSnqNiGYuD6BIu7Gt00kVm4BKtRO1RzdUskkrUe7nSjg2GdvB7M5aspiKc9BUvrGFykGojSzGWa1Bw3LvXYnlroIhkM5fuMrig1RO7VcXdYi4iyS4z0j31a60Pt4hV45WA6w==) - [d-id.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHCG2Qj0KKqt0tZEpk0jefcVyqFItyFcTTV_b8cKS9cybRr9gIqE1Ywc2j9Idyo7be0k-JYd24VXVngeI43WM7n-VD3dArx-xh0dbBakdOKwfUbYgU_qNWq8BBd4IB8xZ9AOpJAAawCPdE7YZSlMtnhroZDuA9Cq5Uiam3SH4rwEbM=) --- # Field Notes on Open\-Source Interactive AI Avatars: Realtime Challenges and Solutions Explore the landscape of open\-source interactive AI avatars, dissecting the real\-time challenges and practical steps for building embodied conversational\. Published: 2026-09-19T02:45:31.772Z Updated: 2026-09-19T02:45:31.772Z Canonical: https://realtimeavatar.ai/blog/open-source-interactive-ai-avatars-realtime-challenges Markdown: [en](https://realtimeavatar.ai/blog/open-source-interactive-ai-avatars-realtime-challenges.md) Venturing into open-source for interactive AI avatars presents a compelling path for engineers seeking deep control and cost optimization, yet it demands a clear understanding of the substantial orchestration required to achieve true real-time performance. While open-source components offer the building blocks, assembling them into a production-ready, low-latency system that delivers fluid, embodied conversation often means trading initial platform costs for significant development, integration, and operational overhead. The core challenge lies in knitting together disparate models—Speech-to-Text (STT), Large Language Models (LLMs), Text-to-Speech (TTS), and visual rendering—into a cohesive, streaming pipeline that anticipates user interaction rather than merely reacting to it. ## The Open-Source Landscape for Embodied AI The open-source domain offers a rich repository of models and frameworks, each addressing a piece of the interactive avatar puzzle. Projects like AvatarAI provide a full-stack, open-source platform aimed at photorealistic AI avatar conversations, touting capabilities like zero-shot voice cloning, multi-LLM support, and a token-streaming pipeline for real-time interaction. It leverages specific rendering models, such as MuseTalk V1.5, to achieve lip-sync at 30 frames per second on V100-class GPUs, illustrating the critical role of specialized hardware for performance (source:1). Conversely, Duix Avatar focuses on offline video generation and digital human cloning from video data, emphasizing appearance and voice cloning for realistic virtual models. While valuable for high-fidelity content creation, its primary utility leans towards non-real-time applications, underscoring a common divergence in the open-source space between generating static video and enabling dynamic, interactive presence (source:2). Beyond full platforms, a constellation of individual open-source rendering models like Wav2Lip, SadTalker, and MuseTalk power the visual embodiment layer. These models animate still portraits or short videos with lip-sync, acting as crucial components for visual fidelity. However, integrating these into a real-time conversational flow requires careful orchestration, ensuring that the visual output is not only high quality but also delivered with minimal latency. ### Build vs. Buy: A Latency Perspective The - [build vs. buy decision](https://realtimeavatar.ai/blog/building-production-ready-interactive-ai-avatars) is particularly acute in the real-time avatar space. Achieving true interactivity means minimizing the time between a user's utterance and the avatar's embodied response—ideally, within about a second. This is where the intricacies of a managed API like TIC Realtime Avatar often provide a significant advantage. A dedicated platform pre-integrates, optimizes, and scales the complex pipeline of STT, LLM, TTS, and rendering, ensuring that the avatar's lips sync to the syllable and that visual cues like blinks and head shifts are naturally interwoven. While open-source projects can demonstrate individual component capabilities, the operational effort to maintain, scale, and optimize such a stack for production-grade latency is substantial. > Latency is critical: Interactive avatars live or die on response time (ideally within about a second). Optimizing for face realism while neglecting latency is a common pitfall. ## Engineering Real-Time Interaction: Beyond the Components The true engineering challenge in open-source interactive avatars lies not just in selecting the best models, but in their seamless, real-time integration. A sequential, - [batching](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) approach—where each stage (STT, LLM, TTS, video rendering) waits for the previous one to complete—introduces unacceptable delays. Real-time requires a - [streaming pipeline](https://realtimeavatar.ai/blog/the-architecture-of-presence-integrating-a-realtime-ai-avatar-api) , where chunks of data flow continuously between stages, allowing the avatar to begin speaking and animating before the LLM has even finished its full response (source:1). ### Implementing the Full-Duplex Conversation A crucial feature for natural interaction is full-duplex communication, or - [barge-in](https://realtimeavatar.ai/blog/testing-barge-in-in-a-realtime-avatar-app) . This allows a user to interrupt the avatar mid-sentence, shifting from a turn-taking model to a more human-like flow. While some open-source platforms like AvatarAI demonstrate this capability (source:1), building it into a custom stack requires sophisticated real-time audio processing and LLM interruption logic. The `realtime-avatar` SDK handles the underlying complexities of maintaining a live session with an avatar, enabling these nuanced interactions without developers needing to manage WebRTC streams or WebSocket protocols directly. It allows focusing on the conversational logic rather than the plumbing. ### The Unseen Cost of Open Source: Compute and Operations While open-source models are free in terms of licensing, running them for real-time interactive avatars is far from free. High-performance GPU compute is indispensable, particularly for rendering and often for LLM inference. Deploying and managing these GPU instances, handling scaling, ensuring uptime, and implementing production-grade features like authentication, rate limiting, and monitoring (Prometheus, CI mentioned by AvatarAI, source:1) constitutes a significant operational burden. A managed API abstracts these infrastructure concerns, allowing developers to focus on application logic and user experience rather than infrastructure provisioning and maintenance. ## Practical Steps for Shipping an Embodied Experience Whether building a custom stack or leveraging a managed API, the core developer experience revolves around integrating the avatar into your application. With a platform like TIC Realtime Avatar, this often begins with creating the avatar itself from a single portrait image. ### Avatar Creation and Embodiment Your avatar starts from a single portrait image. You can - [upload the image bytes directly or provide a URL](https://realtimeavatar.ai/docs/video#uploading-the-portrait) , and the platform handles the generation of the looping idle video and the motion library. This process transforms a static image into a dynamic, expressive character. ``` import { RealtimeAvatar } from "realtime-avatar"; const rta = new RealtimeAvatar({ apiKey: "YOUR_API_KEY" }); async function createAndConfigureAvatar(file: File) { const asset = await rta.uploadAsset(file, { kind: "image" }); const avatar = await rta.createAvatar({ portraitAssetId: asset.id }); // Refine the idle loop for natural presence await rta.setLoop(avatar.id, { motionPrompt: "looks thoughtful, then a soft, closed-lip smile", }); await rta.waitForLoop(avatar.id); return avatar.id; } ``` The `motionPrompt` allows you to direct the avatar's default idle behavior, giving it personality even when not actively speaking. Remember, the avatar's loop is a - [small, closed arc of movement](https://realtimeavatar.ai/docs/video#the-resting-loop-is-not-a-clip-and-it-is-also-a-sentence) —blinks, subtle head shifts, or gentle smiles—designed to provide a continuous, natural presence. ### Connecting the Session With the avatar ready, integrating it into your application for real-time interaction involves establishing a session. The TypeScript SDK (available via `realtime-avatar`) simplifies this, abstracting away the complexities of WebRTC and underlying communication protocols. ``` import { AvatarCall } from "realtime-avatar/react"; // In a React component function LiveAvatarExperience({ client, avatarId }: { client: any; avatarId: string }) { return ( console.log("Connection details:", details)} // Configure microphone and camera as needed microphone={true} camera={true} // Requires server authorization for camera: true // ... other props for text input, event handling /> ); } ``` The SDK provides utilities for - [monitoring connection details](https://realtimeavatar.ai/docs/react#optional-connection-details) and managing inputs like the - [user's microphone](https://realtimeavatar.ai/docs/react#let-the-character-see-your-camera) and - [camera](https://realtimeavatar.ai/docs/react#let-the-character-see-your-camera) (for visual understanding). This allows for rich, multimodal interactions where the avatar can see what a user shows it, adding a new dimension to conversational AI. ### Iterating on Avatar Presence Beyond the core interaction, features like - [real-time video editing](https://realtimeavatar.ai/docs/video#editing-the-loop) allow you to dynamically alter the avatar's background or appearance during a session without re-rendering the entire avatar. This happens upstream of the lips, meaning the avatar can start talking immediately while the visual edit converges, enhancing the sense of presence and narrative. ## How to Ship It: Balancing Control and Speed For developers evaluating open-source interactive AI avatars, the path forward depends heavily on the project's scale, budget, and desired level of control. If the goal is deep research, custom model development, or operating at a massive, highly optimized scale where the cost of a dedicated MLOps team is justified, a pure open-source approach offers unparalleled control. You will manage GPU provisioning (e.g., AWS g5.xlarge mentioned by AvatarAI, source:1), ensure data locality, and build every piece of the pipeline from the ground up. However, for most product developers aiming to bring an interactive AI avatar to market efficiently, a managed API provides a clear advantage. Platforms like TIC Realtime Avatar handle the complex, real-time orchestration, scaling, and infrastructure, allowing you to integrate a high-quality, low-latency avatar using a - [typed TypeScript SDK](https://realtimeavatar.ai/docs) within a few lines of code. This accelerates product validation, reduces ongoing operational overhead, and ensures a consistent, production-ready experience. The trade-off is often control over the lowest-level model details for speed of deployment and reliability. Consider starting with a managed API to validate your product and user experience, and then, if specific performance or architectural requirements demand it, explore a more custom open-source stack once those parameters are well-defined. - [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEuWNluvmQIFVhzualuCNbQTkYYBwionXOcSQTyZPp6fwuFS_RAmEq8aDx3vwEZSOzAFJWgIctFmwURnIcYYyA9GLvyTSX4jbS0a4JNen0tgHsymICKN12uTYHuu-iob_ljZBkOtJ7W) - [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEWZk97cG9yD229jXT6kDdmqlkYFYBHyGhJiGP6MUXr5xOnHcUYVU3mDeUdOkVzNj4x3BIl7rQW_XEsaZXVTFq57vtRxqE6cfraI1bcu7fKjncb2oA7JZMRi8bUTeLUFH9M) - [forasoft.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHsDH6QgN9Iulpz49_WJ-BhdrdU1OJG23bxZI-yxDVbaAH6jVIZxpnKmha42RlMYGIfc2XzvdZUctNL-lq9rJPeC8vNI40bLSAl-ZnkqKEfO5SYab4KOZj8tCwZYJZniyoGF8PLQLPHIJ7TaV1I8T1yPJ4NA-rNFcmRpd8pNNTdJaQA) --- # Building Production\-Ready Interactive AI Avatars with a TypeScript SDK Explore how to move from open\-source concepts to robust, real\-time AI avatar applications using a powerful TypeScript SDK\. Published: 2026-09-16T02:45:28.360Z Updated: 2026-09-16T02:45:28.360Z Canonical: https://realtimeavatar.ai/blog/building-production-ready-interactive-ai-avatars Markdown: [en](https://realtimeavatar.ai/blog/building-production-ready-interactive-ai-avatars.md) The landscape of interactive AI avatars is rapidly expanding, with many developers exploring capabilities through open-source projects on platforms like GitHub. While these repositories offer valuable insights and experimentation grounds, bringing an interactive AI avatar to a production environment demands a robust, scalable, and fully-featured platform. This article outlines how a dedicated SDK and API address the core challenges of real-time interaction, providing a direct path from conceptual exploration to shippable applications. ## The Anatomy of Real-Time Presence: What GitHub Projects Seek to Solve Developers building interactive AI avatars on platforms like GitHub often converge on a common set of challenges: achieving low-latency responses, enabling natural conversation flow, integrating external actions, and ensuring consistent visual and auditory quality. Projects like AvatarAI and Multilingual Generative AI Avatar demonstrate the drive for real-time lip-sync, voice cloning, and multilingual support, often leveraging various AI models and frameworks to achieve their goals (source:2, source:3). Our platform, TIC Realtime Avatar, distills these common requirements into a managed service, offering a TypeScript SDK (realtime-avatar) and an API designed for production use. Instead of stitching together disparate models for speech-to-text, large language model inference, text-to-speech, and talking head generation, you engage a unified pipeline built for interactive conversations. The avatar itself is created from a single portrait image, with the platform generating the looping idle video and motion library, ensuring visual consistency without requiring complex 3D modeling or video training data. Video creation from a supplied video source is closed; we focus on generative embodiment from a single frame. ### Beyond Turn-Taking: Full-Duplex Communication One of the most significant advancements in natural human-AI interaction is the move from turn-based (walkie-talkie style) communication to full-duplex. Many open-source initiatives, such as Linly-Talker, recognize the importance of WebRTC for low-latency audio-video transmission to enable a "listen-while-speaking" mode (source:5). Our system is built on this principle: the avatar listens to the user continuously, even while speaking. This full-duplex capability, intrinsic to every call, translates directly into a more fluid and less rigid user experience. It manifests in four key behaviors: - Instant Interruption: If a user speaks mid-sentence, the avatar stops and can acknowledge the interruption contextually, rather than abruptly cutting to silence. - Ignoring Backchannel: Subtle vocalizations like a cough or an "mm-hm" are recognized as backchannel cues, not interruptions, preventing the avatar from feeling brittle. - Intelligent Pause Handling: The system distinguishes between a conversational pause and the end of a user's turn by analyzing what was said, not just the duration of silence. This prevents the avatar from talking over the user or leaving awkward gaps. - Language and Silence Awareness: The avatar can interpret prolonged silence as a signal for action and can even follow language switches within a single sentence. These behaviors are fundamental to the platform and require no configuration on your part. The deliberate design choice is that the avatar will not initiate a new sentence over the user, optimizing for a coherent and engaged conversation. You can find more details on this in our documentation on "She listens while she talks". - [She listens while she talks](https://realtimeavatar.ai/docs#she-listens-while-she-talks) ### Bridging Avatars to Action: Tool Calling An avatar that can only converse is, in many contexts, a demonstration. A truly useful interactive avatar needs to perform actions in the world—booking appointments, checking order statuses, or retrieving information. Projects like AIAvatarKit highlight the importance of modular architecture to swap components and integrate AI agent capabilities, including tool calls (source:4). Our platform supports robust tool calling, allowing your avatar to interact with any external system. You define a tool by its name and a natural language description that teaches the avatar when to use it, along with a JSON schema for its parameters. There is no hosted execution for these tools; they run in your application's environment. For instance, an `AvatarTool` for checking order status might look like this: ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` The backend's role is to grant the `client_tools` capability during session minting. On the client side, after the room is connected, you register the tool manifest over RPC: ``` // server — the session policy grants the client tool plane for this call session: async ({ avatarId }) => ({ instructions, clientTools: true }) // client — register over RPC after connect; the record key is the tool's name import { attachAvatarTools } from "realtime-avatar/tools"; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` Tools have a strict 2.5-second execution limit. For slower operations, the design pattern involves acknowledging the request quickly and delivering the full result out of band. This ensures the avatar remains responsive. Our detailed "Tool calling" documentation provides the full pattern. - [Tool calling](https://realtimeavatar.ai/docs/tool-calling) ## Implementing Your Interactive Avatar Integrating an interactive AI avatar into your application involves a clear client-server separation, leveraging the SDK to abstract away the complexities of real-time media streams and AI orchestration. The core idea is to let your server manage API keys and session policies, while your client handles the avatar's rendering and user interaction. ### Avatar Creation and Client-Side Integration Creating an avatar is straightforward: you supply a single portrait image. The platform then generates the necessary looping idle video and a motion library that dictates how the avatar moves and expresses itself. This process is optimized for generative embodiment from one image, not pre-recorded video. Embedding the avatar into your web application is handled by the `AvatarCall` component from the TypeScript SDK. This component manages the real-time video and audio stream, ensuring lipsync to the syllable based on the audio clock. You simply provide the client connection and the avatar ID: ``` ``` For audio-only interactions, the `mode="voice"` prop can be used: ``` ``` - [Creating an avatar](https://realtimeavatar.ai/docs/video) ### Server-Side Session Management and Capabilities Your API key must remain secure on the server. The client communicates with your backend, which in turn interacts with our API to mint session tokens and define call policies. For Next.js applications, a dedicated adapter simplifies this boundary management. The `Quickstart` guide provides a three-step path to a live call, starting with this essential server-side setup. - [Quickstart](https://realtimeavatar.ai/docs/quickstart) - [Build a realtime AI avatar in Next.js](https://realtimeavatar.ai/docs/nextjs) The server-defined session policy dictates critical aspects of the call, such as which avatar is used and what capabilities, like tool calling, are enabled for the client. This centralized control ensures security and consistency across your application. ## Practical Considerations and Shipping Your Application Moving from a proof-of-concept, often prototyped with open-source tools, to a production-ready application requires attention to several factors: cost, maintainability, and developer experience. While latency metrics are often a focus in research projects (e.g., Multilingual Generative AI Avatar achieving sub-second LLM inference and TTS on edge devices, source:3), directly comparing latency numbers without a cited measurement protocol for the specific model and network conditions is challenging. Our platform is engineered for real-time interaction, where latency depends on factors like startup state, model complexity, and network conditions. The usage-based pricing model, anchored at about $5 per hour of realtime interaction, makes it feasible to deploy always-on companion applications or interactive experiences without prohibitive costs. This pricing structure (e.g., $24/month for 10 hours, with overage at $0.07-0.095/minute) is designed for continuous engagement, not just brief demos. Avatar creation itself is $1 beyond included credits. - [Pricing](https://realtimeavatar.ai/pricing) The developer experience is paramount. Our TypeScript SDK is generated directly from our OpenAPI specification, ensuring type safety and consistency. For those preferring to work with backend agents or needing deeper integration, agent-readable documentation is available at /llms.txt, alongside the published OpenAPI document at /openapi.json. This comprehensive documentation, combined with our clear API reference, empowers developers to build and scale interactive avatar applications efficiently. - [API Reference](https://realtimeavatar.ai/docs/api-reference) - [Agent-readable docs](https://realtimeavatar.ai/llms.txt) - [OpenAPI document](https://realtimeavatar.ai/openapi.json) For testing and inspiration, our studio at /studio hosts a roster of resident live avatars like Ada Kinetic, providing immediate examples of the platform's capabilities. - [Studio](https://realtimeavatar.ai/studio) To ship your interactive AI avatar application, begin by securing your API keys on the server and implementing the session minting process as outlined in the Quickstart. Integrate the `AvatarCall` component on your frontend, ensuring proper client-server communication for session negotiation. Then, identify the external systems your avatar needs to interact with and define your tools using the TypeScript SDK, granting the necessary `client_tools` capability on the server. Prioritize clear tool descriptions for optimal agent decision-making. Finally, deploy your application, monitoring usage and performance to iterate and refine the user experience. The journey from an intriguing GitHub project to a robust, interactive AI avatar in production is a matter of choosing the right tools and architectural approach. - [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEZOt4JqQod3XhfVTmfOuyHM5oH5rM1-WWJcQ58PJ608qYptuubL_n_FK3AUl9kPuW-Oyx0sxg9Vv8hk8qr6HkbMOpStyX6nAFpSah45BzeuNnGBvBqjaVMo8dQ-FJqT3Ir) - [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGFK4Kunc8uwrxN0ahw3fYMxALbsCNLaYTRQADzuPUONSczamWDzBzfS_qlrjZVb8hpJJnSShE4Q8SGnvkGjHrfUIe-PyqgqL2E3rgaglUKA9rbvabY76fdHM8U8cPp) - [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGkQF9Sh4G3lM_KaM4OoUJl2VvShZ2UW5CNLlRW49WeeCuLT-yEIHhKa99geSu02BqUBN2o8iZmOpW6q8PaiS27Jw6qjb2fERBf35uglAw5YhosX3-dLTf77DBxhTj9SpgsK3_t104i) --- # Realtime Live Avatars: Decoding API Pricing for Production Unpack the economics of live avatar APIs\. Explore TIC Realtime Avatar's usage\-based pricing, its "face layer" architecture, and how to build and ship\. Published: 2026-09-15T02:45:55.167Z Updated: 2026-09-15T02:45:55.167Z Canonical: https://realtimeavatar.ai/blog/realtime-live-avatars-decoding-api-pricing-for-production Markdown: [en](https://realtimeavatar.ai/blog/realtime-live-avatars-decoding-api-pricing-for-production.md) Understanding the true cost of a live avatar API means looking beyond simple per-minute rates to the underlying architecture, included services, and the operational expenses of a deployed application. For TIC Realtime Avatar, the pricing model is usage-based, anchored at approximately $5 per hour of real-time interaction, designed to make always-on, deeply interactive companion experiences economically viable. This structure includes a free tier for development and scales with your application's demand, with avatar creation costing $1 beyond included limits. It's a transparent model built for sustained, live engagement rather than episodic video generation. ## The Meter and The Meaning: Understanding Usage-Based Costs At its core, live avatar API pricing is a reflection of the computational resources required to render and stream dynamic, human-like animation in real time. For TIC Realtime Avatar, this cost is metered on actual usage during active calls. The rate is set at about $5 per hour of realtime interaction, which translates to $0.07-$0.095 per minute on overage, depending on your plan, following an initial allowance for each month. This approach differs from traditional video generation platforms, where costs are often tied to credit systems for pre-rendered video clips. With a live avatar, the meter runs only when a user is actively engaging with the character, allowing for longer, more natural conversational flows without the specter of rapidly depleting credits. The intent is to support applications where the avatar is a persistent presence, like a companion or a live support agent, rather than a transient, one-off video message. Avatar creation itself is a distinct, one-time cost, charged at $1 beyond any included allowances. An avatar is generated from a single portrait image, from which the platform synthesizes a looping idle video and an entire motion library. This means you do not pay per clip for the foundational animation set; the cost is for the initial setup. Subsequent updates to the resting loop are billed like a generation, but the core library remains part of the initial creation. - Real-time interaction: Priced at ~$5 per hour. - Monthly plans: For example, $24/month includes 10 hours, with overage billed at $0.07-$0.095 per minute. - Free tier: Available for development and testing. - Avatar creation: $1 per avatar beyond included, generated from a single portrait image. ### Beyond the Sticker Price: Architectural Choices and Cost Drivers Comparing live avatar API pricing across vendors requires an understanding of their underlying architectural models. Many providers in the market offer bundled services, integrating speech-to-text (STT), large language models (LLM), and text-to-speech (TTS) into a single API call. While convenient, this approach can obscure individual cost drivers and limit customization. Other platforms, like TIC Realtime Avatar, operate as an "Audio-to-Video API" or a "Face Layer Model" (source:1). In this model, the platform handles the real-time lip-synced video stream based on audio you provide. This means you bring your own conversational stack — your ASR, LLM, and TTS. This design choice has significant implications for both cost and control: - Control: You retain full control over your AI agent's intelligence, persona, and underlying models. This is crucial for niche applications, specific brand voices, or proprietary knowledge bases. - Cost Optimization: You can choose the most cost-effective ASR, LLM, and TTS providers for your specific use case, rather than being locked into a bundled solution. For example, open-source LLMs or specialized TTS engines might offer better performance or lower costs for certain tasks. - Flexibility: The modular nature allows for greater experimentation and iteration on your agent's behavior and voice without impacting the avatar's visual performance. The primary cost drivers for a face-layer model are the cloud GPU rendering and bandwidth for streaming the video. By focusing solely on the visual embodiment, TIC optimizes for these components, ensuring that your expenditure directly correlates with the real-time video delivery. This contrasts with systems that might charge higher rates due to bundling expensive LLM inferences or proprietary ASR/TTS services. ## Shipping an Interactive Avatar: Practical Implementation Building a live avatar into your application with TIC Realtime Avatar involves integrating the TypeScript SDK (generated from an OpenAPI specification) and managing sessions on your backend. The core idea is to establish a session, render the avatar, and then feed it audio for real-time lip-syncing and motion. ### Setting Up Your Environment Begin by incorporating the `realtime-avatar` SDK into your project. The quickstart documentation offers a rapid path to a live call in three steps, and a complete Next.js starter kit is available to jumpstart development, showcasing controls for mic, text, and call management. - [Quickstart Guide](https://realtimeavatar.ai/docs/quickstart) - [Download Next.js Starter](https://realtimeavatar.ai/docs/nextjs#run-the-complete-starter) Once the SDK is installed, your server is responsible for minting sessions and defining the avatar's capabilities. This policy-driven approach ensures security and allows for dynamic configuration of avatar behavior. ``` body = {"avatar_id": avatar_id, "mode": "avatar"} # she is on screen body = {"avatar_id": avatar_id, "mode": "voice"} # audio only ``` ### Integrating Tools for Deeper Interaction A live avatar gains true utility when it can interact with external systems. TIC Realtime Avatar supports tool calling, allowing your agent to book appointments, check orders, or retrieve information. Your backend grants the `client_tools` capability during session creation, and your client-side application registers the tool's manifest. ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; // server — the session policy grants the client tool plane for this call session: async ({ avatarId }) => ({ instructions, clientTools: true }) // client — register over RPC after connect; the record key is the tool's name import { attachAvatarTools } from "realtime-avatar/tools"; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` Tools have a 2.5-second execution limit. For slower operations, acknowledge the request quickly and deliver the full result out-of-band to maintain conversational flow. This asynchronous pattern is critical for a responsive user experience. - [Tool Calling Documentation](https://realtimeavatar.ai/docs/tool-calling) ### Recording Conversations Recording a session is a server-side policy. By adding `recording: "audio_video"` to the session configuration, you enable the platform to capture audio and video of the interaction. This is invaluable for quality assurance, training, and compliance. ``` // app/api/realtime-avatar/[...path]/route.ts export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, session: async ({ avatarId }) => ({ instructions: promptFor(avatarId), recording: "audio_video", // "off" | "audio" | "video" | "audio_video" }), }); // server-only follow-up, after the call: const artifact = await rta.getRecording(recordingId); if (artifact.status === "ready") { const playback = await rta.getRecordingAccess(artifact.recordingId); // Keep recordingId; playback.url expires and can be renewed. } ``` - [Sessions Documentation](https://realtimeavatar.ai/docs/sessions) ## Limitations and Considerations While the face-layer model offers significant flexibility and cost control, it places the responsibility for the full conversational stack (ASR, LLM, TTS) on the developer. This means building and optimizing these components is part of your development effort. However, it also means you are not constrained by a vendor's choices and can tailor the experience precisely to your needs. Latency is another critical consideration for real-time interactions. While TIC's avatar generation aims for minimal latency, the overall perceived latency of an interaction is a sum of ASR, LLM processing, TTS generation, network travel, and avatar rendering. Promising specific latency numbers without a cited measurement protocol is irresponsible, as real-world performance depends heavily on your chosen conversational stack, network conditions, and the complexity of your agent's logic. Benchmarking your full pipeline is essential to understand and optimize the user experience. - [Benchmarking Real-Time Avatar Latency](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) ## How to Ship It To move a live avatar application from development to production, focus on a few key areas that align with TIC's architecture: 1. Optimize Your Conversational Stack: Since you control the ASR, LLM, and TTS, invest in fine-tuning these components for speed and accuracy. Explore streaming ASR, efficient LLM prompting, and low-latency TTS solutions. 2. Robust Session Management: Implement secure and scalable session creation on your backend. Ensure that session policies, including capabilities like tool calling and recording, are correctly configured for different user roles or scenarios. 3. Client-Side Resilience: Design your frontend to gracefully handle network fluctuations and varying latency. The SDK provides events and states that allow for responsive UI updates, even during transient connection issues. 4. Strategic Tooling: Carefully define and implement tools that add tangible value to the user experience. Prioritize tools that can be executed quickly or designed with asynchronous feedback mechanisms to avoid blocking the conversation. 5. Cost Monitoring: Integrate cost tracking and monitoring for both real-time avatar usage and your external ASR/LLM/TTS services. This proactive approach allows you to identify usage patterns and optimize your overall expenditure as your application scales. 6. Continuous Performance Testing: Regularly benchmark the end-to-end latency of your full avatar application, from user input to avatar response. Identify bottlenecks and iterate on your system design and component choices to maintain a fluid, natural interaction. By focusing on these architectural and operational considerations, you can leverage TIC Realtime Avatar's cost-effective face-layer model to deploy engaging, interactive experiences that scale with your users' demands, without unexpected cost escalations. - [realtimeavatar.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHGvNLNHar6JKVuwc4Jdnix2bwKX6MPSzl5m9YHlQaW2gtl_zgPWH6kZXxBkV7tGmMCl8E-FKlrMJ4CvXfWW3RoCdOniv_vdIGzvq7zmOPJKHIus2p9sUmd6cqklktWiBDn1e8OoRkvYZawobfMrvjfUfv_3Yvx3J-YRNTTU0dhe_EWcOPtHZkbKfk=) - [meetcody.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHkNJVpN0gvb4QFI7zvw9zOb2bUVyfiTOAVFvpRBqsGaBBzy6aKvKqAtnjDl7fH8m6K3oa7nkvm1U9Zn1bhPxuDSOGUHathQGRTVbftRzx-tKzxBVvQ1n3wvOJh-GpChkKwfJQFTKcyBeBMwQbFMk9-uW8PCQs6Zq0=) - [beyondpresence.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEwElqBHGI_eFJGof7uMN0ZcpFv2HisU477kTeN1ykSIAuY3y7ylJ-G4VZYu_r5WuC-Ywxp49W5wLZk0Ng3LLZENnp1UlOKZ3QaNtqtGP_iSwdIwk9wgxXUB5C4gZ4hrqP7Wf-bSiZT--_rivvcIsCKeos8xFwi83-lfCf0oGQHMLPQupQBkOhdm3FSUXwvm86Z2VQB9KRWwCaxuGc=) --- # Designing Interactive AI Avatar Chatbots: Beyond Conversation Field notes on building AI avatar chatbots that engage in natural, full\-duplex conversations and execute actions through integrated tools\. Published: 2026-09-13T02:45:00.616Z Updated: 2026-09-13T02:45:00.616Z Canonical: https://realtimeavatar.ai/blog/designing-interactive-ai-avatar-chatbots Markdown: [en](https://realtimeavatar.ai/blog/designing-interactive-ai-avatar-chatbots.md) Building an interactive AI avatar chatbot requires more than just synthesizing speech; it demands the architecture for genuine, two-way engagement. This means enabling full-duplex audio where the avatar listens while speaking, implementing barge-in capabilities, and integrating external tools that allow the avatar to perform actions in the real world. The core challenge is to orchestrate these components into a fluid experience that mirrors human conversation, moving beyond simple turn-taking to create a truly present digital entity capable of meaningful interaction. ## The Architecture of Realtime Dialogue The foundational element of an interactive avatar chatbot is its ability to participate in a conversation without the rigid turn-taking common in many voice AI systems. Traditional systems often operate like a walkie-talkie: one party speaks, then pauses, then the other responds. This creates an unnatural cadence. A truly interactive system, however, employs full-duplex audio, allowing the avatar to hear you even as it speaks, adapting in real time. This capability manifests in several critical ways: - Interruptibility: If a user begins speaking mid-sentence, the avatar stops immediately and can acknowledge the interruption contextually. This is a fundamental shift from systems that complete their utterance regardless of user input. - Backchanneling: Ambient sounds or affirmative 'mm-hms' from the user do not derail the conversation. The system discerns between conversational fillers and genuine interruptions, maintaining its flow without feeling brittle. - Intelligent Pause Handling: The duration of a pause does not automatically signal the end of a user's turn. Instead, the avatar interprets the content and context of the user's speech, ensuring it neither talks over the user nor leaves awkward silences. - Language Adaptation: The avatar can follow language switches within a single sentence, a crucial feature for multilingual applications. These functionalities are not additional configurations but inherent behaviors of the core system, designed to foster a natural conversational rhythm. It is a deliberate choice for the underlying voice and model architecture, though it means the avatar will not initiate a new sentence over the user's current speech, a nuanced trade-off for fluid interaction. Further details on this full-duplex capability are outlined in the platform's documentation. - [She listens while she talks](https://realtimeavatar.ai/docs#she-listens-while-she-talks) ## Empowering Action: Avatars with Tool-Calling A chatbot that merely converses, however naturally, remains a demonstration until it can act. The true utility of an interactive AI avatar chatbot emerges when it can execute tasks, making appointments, checking order statuses, or retrieving information. This is achieved through tool-calling, integrating the avatar's conversational agent with external APIs and services. The design principle here is akin to briefing a human colleague: define the tool's purpose and when it should be invoked. The avatar's underlying language model reads this description and decides on its own when to reach for it. ### Declaring and Registering Tools Tools are declared with a clear description and expected parameters. This description serves as the avatar's entire teaching signal, guiding its decision-making. The platform's TypeScript SDK simplifies this declaration: ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` On the server side, the capability to use client-side tools is granted when minting the session. The client-side application then registers these tools once the connection to the avatar is established. This separation ensures security while enabling dynamic, contextual actions: ``` // server — the session policy grants the client tool plane for this call session: async ({ avatarId }) => ({ instructions, clientTools: true }) // client — register over RPC after connect; the record key is the tool's name import { attachAvatarTools } from "realtime-avatar/tools"; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` It is crucial for tools to respond promptly; a timeout is not an option in a real-time interaction. Tools are expected to return within 2.5 seconds. For longer-running operations, the best practice is to acknowledge the request quickly and deliver the full result out of band, allowing the avatar to continue the conversation while the background process completes. This pattern ensures the dialogue remains fluid and responsive. - [Your tools, wired into the conversation](https://realtimeavatar.ai/docs#your-tools-wired-into-the-conversation) - [Tool calling](https://realtimeavatar.ai/docs/tool-calling) ### From Image to Interactive Presence The visual component of an interactive avatar chatbot is as critical as its conversational intelligence. An avatar's visual presence is created from a single portrait image. From this single frame, the platform generates a looping idle video and a comprehensive motion library. This process provides the avatar with a range of natural gestures and expressions, which are then synchronized in real time with the generated audio. The video is audio-clocked, meaning the avatar's lips are precisely synced to the syllables of its speech, enhancing the realism and credibility of the interaction. This approach bypasses the need for supplied video, focusing instead on synthesizing dynamic presence from a static source. More on this creation process is detailed in the video documentation. - [Creating an avatar](https://realtimeavatar.ai/docs/video) ## Implementing the Interactive Core To bring an interactive AI avatar chatbot to life, developers integrate the Realtime Avatar SDK into their applications. A typical setup involves a secure backend endpoint and a frontend component. For example, in a Next.js application, the App Router adapter manages the secure boundary between the client-side code and your API key, which must never be exposed to the browser. The integration is streamlined for rapid deployment. - Client Setup: Instantiate the AvatarCall component in your frontend. This component handles the visual rendering and audio interaction. - Server Endpoint: Create an API endpoint on your server to handle session minting, which securely provides the necessary tokens and policies for the client to connect. - SDK Integration: Utilize the typed TypeScript SDK (realtime-avatar) to manage the connection, audio streams, and tool registration. ``` ``` This modular approach allows developers to quickly embed fully interactive avatars, focusing on their application logic rather than the complexities of real-time media streaming and synchronization. The Quickstart guide provides a detailed walkthrough for initial setup. - [Quickstart](https://realtimeavatar.ai/docs/quickstart) - [Build a realtime AI avatar in Next.js](https://realtimeavatar.ai/docs/nextjs) ## Limitations and Ethical Considerations While the technology for interactive AI avatar chatbots is advanced, certain limitations and ethical considerations remain pertinent for responsible deployment. The real-time nature of these pipelines, for instance, is computationally intensive. Latency, while optimized for natural conversation, is subject to variables like startup state, chosen models, and network conditions; therefore, specific latency numbers should not be promised without rigorous, cited measurement protocols. The platform's design decision not to have the avatar talk over a user with a *new sentence* is a deliberate choice for voice and model consistency. Ethically, transparency is paramount. Users should always be aware they are interacting with an AI avatar. If real-person likenesses or voices are used as templates, explicit consent, outlining data usage and opt-out rights, is required. Developers must also consider representation, ensuring avatars avoid biases and stereotypes, and adhere to privacy regulations like GDPR, especially concerning biometric data. The tendency for humans to attribute authority and empathy to a face necessitates clear disclosure to prevent over-trust. These considerations are not merely regulatory but foundational to building trustworthy and beneficial AI interactions. ## How to Ship It Integrating an interactive AI avatar chatbot into a product requires not just technical prowess but also a practical understanding of its operational costs and development lifecycle. The platform is designed with usage-based pricing, anchored at approximately $5 per hour of real-time interaction. This model, with included minutes in various plans and overage charges, supports continuous deployment without prohibitive costs, making it feasible for applications like companion apps that benefit from extended user engagement. For developers, the typed TypeScript SDK, generated directly from an OpenAPI specification, provides a robust and predictable development experience. Comprehensive, agent-readable documentation is available at /llms.txt, alongside the published OpenAPI document at /openapi.json, ensuring that your agentic workflows can integrate seamlessly. The studio at /studio also offers a roster of resident live avatars for immediate prototyping and inspiration. By focusing on these core elements — natural interaction, extendable capabilities, ethical deployment, and practical economics — developers can move from concept to shipping a truly interactive AI avatar chatbot. - [API reference](https://realtimeavatar.ai/docs/api-reference) - [Calls](https://realtimeavatar.ai/docs/sessions) - [Pricing](https://realtimeavatar.ai/pricing) - [spatius.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGCB6ouKD58v9wEF6N-7dnKbJXeqS3wuS64D_rQzW1HVT10K2nDCJ7hgT6I0S4tL4kSEQBmYNErS8lvVFdA6IRg81MkBr-4OvIzy4sgW_VFAElN6drCfB44kYixsyCwQU-rZZ3LbpG8lmvjSb2uQnXUJBNxjNXsrBzr_rZ8K-hk9tSCAf9UjIw3YPTUCkWNfDDK6g0=) - [forasoft.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQE_UfYSh0G4iYBQg2lJWb9EoftExb7epeL4KSmXesQ2BBwyXNaHeL2IIdwYvFS_2JKvgcrCINg1wJmscKv0c8A-Z5R7tH4MvLHodoIGPnP4x9sQzO8pXj-RJybIoyqs0768ChwaZHdhlRItIlU9Q_ivTUK1WZZ5_fmR6I9JVGgQWOgv) - [kaltura.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQExaBTsOjb3J4904PpPmAvxQGrBN_seTMTnM7yOLFcXdNZw31unQRlywKkEHM_ipP5-TVdIZssUvoaEXnWxFys5SfMt7yetmu1BJBwqxm0ZVyxoPrcQihSDNhirdAPKEampu3WrlzD6glpzRQI=) --- # Architecting Realtime Interactive AI Avatar Chat Experiences Building interactive AI avatar chats requires a blend of real\-time audio, expressive video, and intelligent agent logic\. This guide covers the essential\. Published: 2026-09-12T02:45:44.429Z Updated: 2026-09-12T02:45:44.429Z Canonical: https://realtimeavatar.ai/blog/realtime-interactive-ai-avatar-chat-architecture Markdown: [en](https://realtimeavatar.ai/blog/realtime-interactive-ai-avatar-chat-architecture.md) An interactive AI avatar chat is a digital character capable of engaging in genuine, two-way conversations in real time, mirroring the natural ebb and flow of human speech and expression. Unlike pre-rendered video or static chatbots, it offers dynamic, unscripted interaction, making every conversation unique. This capability hinges on a seamless integration of several core technologies, orchestrated to achieve a sense of presence and immediate response that traditional digital interactions often lack. The goal is not merely to speak but to converse, to react, and to perform actions that extend the chat beyond simple dialogue. ## The Foundation of Conversational Presence At its heart, an interactive avatar system relies on a stack of interconnected technologies. Your user's spoken words are first transformed into text by an Automatic Speech Recognition (ASR) system. This text then feeds into a Large Language Model (LLM) or a Retrieval-Augmented Generation (RAG) system, which processes the input, comprehends intent, and formulates a relevant textual response. This response is then synthesized into audio via Text-to-Speech (TTS), complete with nuanced intonation. The final piece is the avatar itself: a rendering layer that animates the digital human's face, synchronizing lip movements with the generated audio, and displaying contextually appropriate facial expressions and gestures. This entire pipeline must operate with minimal delay, demanding efficient real-time transport technologies like WebRTC to carry synchronized video and voice data. Architectural choices here are foundational. Some providers offer an 'Agent API,' managing the entire stack from speech recognition to rendering. Others, like TIC Realtime Avatar, focus on the 'Audio-to-Video API' or 'Face Layer Model.' In this approach, you retain control over your ASR, LLM, and TTS pipeline, providing the generated audio to the platform. The platform then takes that audio and, in real-time, produces a lip-synced video stream of your chosen avatar. This provides greater flexibility for integrating custom models, specific knowledge bases, and unique voice profiles, which is often preferred for product-grade applications. The avatar itself is generated from a single portrait image. The platform then develops the necessary looping idle video and a comprehensive motion library, ensuring that the avatar can respond dynamically without needing extensive source footage or multiple angles. This single-source approach streamlines avatar creation, moving directly from a static image to a living, reactive digital presence. ### Synchronicity and Full-Duplex Interaction The measure of a truly interactive chat experience is its fluidity. Traditional voice AI often operates like a walkie-talkie: you speak, you stop, and then the system responds. This turn-taking paradigm feels artificial and can quickly lead to frustration. A more natural interaction demands full-duplex communication—where the avatar listens while it talks, allowing for genuine conversational overlap and interruption. This full-duplex capability, inherent to the TIC Realtime Avatar platform, fundamentally changes the interaction dynamic. It means: - Graceful Interruptions: If a user interrupts the avatar mid-sentence, the avatar stops speaking immediately and can acknowledge the interruption in character, rather than abruptly cutting to silence. - Contextual Backchanneling: Minor vocalizations like a cough or an 'mm-hm' do not derail the conversation. These backchannel signals are recognized as conversational cues, not interruptions, preventing the system from feeling brittle. - Intelligent Pause Handling: The end of a user's turn is judged by the content of their speech, not merely by silence. This means the avatar avoids talking over the user or leaving awkward, unnatural gaps. - Adaptive Silence and Language Switching: The system can interpret silence as a signal for action and can fluidly follow language changes even within a single sentence, adapting in real time. These features are not configurable toggles but are built into how every call behaves. This design choice prioritizes a natural conversational flow, though it is important to note that the avatar will not initiate a new sentence while the user is still speaking. This deliberate trade-off is made to optimize for the platform's underlying voice and model architecture. Achieving low latency is paramount for maintaining this natural feeling. Human perception demands conversational response delays of 200-500 milliseconds, with interactions quickly feeling 'broken' beyond a second and users disengaging past three seconds. While absolute latency numbers vary significantly based on startup state, model complexity, and network conditions, the full-duplex design helps mitigate the *perception* of latency by keeping the conversation flowing naturally, even as the underlying systems process information. ### Extending Capabilities with Tool Calling A character that can only talk is a demonstration; a useful one takes action. For an interactive AI avatar chat to move beyond mere conversation, it must be capable of integrating and executing external tools. This enables the avatar to book appointments, check order statuses, retrieve information, or even write code, all while maintaining the conversational thread. This capability transforms a talking head into an active participant in your application's workflow. The approach to tool calling is analogous to briefing a colleague: you declare what a tool is called and, crucially, *when to reach for it*. The description itself serves as the entire teaching signal for the underlying agent, which interprets this instruction to decide when and how to invoke a tool. This mechanism allows developers to define custom functions that the avatar's intelligence can leverage, bridging the gap between verbal interaction and practical application logic. ## Practical Deployment: From Portrait to Presence Deploying an interactive AI avatar involves both backend session management and frontend rendering. The TIC Realtime Avatar API provides a TypeScript SDK (realtime-avatar) generated from an OpenAPI specification, facilitating seamless integration into your application. The core process involves minting a session on your server and then rendering the avatar component in your client-side application. ### Building with the realtime-avatar SDK The SDK simplifies the connection process. Your backend is responsible for creating a session that specifies the avatar and capabilities. For instance, to enable tool calling, your server must grant the `client_tools` capability during session minting. ``` import type { AvatarTool } from "realtime-avatar/tools"; import { attachAvatarTools } from "realtime-avatar/tools"; // On the server, when minting the session: session: async ({ avatarId }) => ({ instructions, clientTools: true }); // On the client, after the room is connected, register your tools: export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` Tools are executed client-side within a strict 2.5-second timeout. For slower operations, the recommended pattern is to acknowledge quickly and deliver the full result out of band. The SDK's type safety ensures that omitting an `execute` function for a tool results in a compile error, preventing runtime surprises. Rendering the avatar on the screen is equally straightforward, requiring just a few lines of code once your client is connected to a minted session. Whether you need a full visual presence or just a voice, the SDK accommodates both modes. ``` // she is on screen // audio only ``` This modularity allows for flexible deployment, whether you're building a full-screen interactive experience or integrating a voice-only assistant into an existing application. ## Limitations and Considerations While interactive AI avatars offer transformative potential, several considerations are crucial for successful deployment. One significant challenge lies in the 'Uncanny Valley' effect. Visual authenticity is paramount; if an avatar's appearance or movements fall short of convincing, it can create an unsettling sensation that undermines engagement. The goal is to create a presence that feels natural and trustworthy, not merely realistic. Another ethical consideration revolves around data bias and representation, particularly when creating avatars for individuals with unique characteristics or disabilities. Ensuring respectful and accurate representation requires careful attention to training data and development practices. Regulatory frameworks are also emerging to address the use of AI in interactive systems. For instance, California's Adam's Law, effective July 1, 2027, imposes requirements on AI companion chatbots interacting with children, including risk assessments, parental controls, and restrictions on harmful content. Similarly, platforms like Character.ai are proactively implementing age assurance technology and adjusting features for younger users (source:1). Cost is another practical consideration. While custom stacks can involve significant raw compute costs, managed APIs often provide a more accessible entry point. TIC Realtime Avatar offers usage-based pricing, anchored at about $5 per hour of realtime interaction, with a free tier for development. This model is designed to support applications where an avatar might be left on for extended periods, enabling the development of genuine companion apps without prohibitive costs (source:2). ## How to Ship It Shipping a realtime interactive AI avatar chat application requires a methodical approach, focusing on integrating the components and managing the development workflow. Here's a practical path: 1. Start with the Core: Begin by setting up your backend to mint sessions and your frontend to display an example avatar. The TIC Realtime Avatar Quickstart provides a three-step guide to get a live call running (source:3). 2. Integrate Your LLM: Wire in your chosen Large Language Model and Text-to-Speech service. Focus on optimizing the response generation and audio streaming to maintain a natural pace. The system is designed to consume your audio, so ensure your TTS output is clean and synchronized. 3. Define Your Agent Logic: Design the conversational flow and integrate any necessary tool calls. Map out the specific actions your avatar needs to take and define the `AvatarTool` manifests with clear descriptions to guide the agent's decisions (source:4). Remember to grant `client_tools` capability on the server during session minting. 4. Test Full-Duplex Behavior: Actively test interruptions, backchannels, and varied speech patterns. This is where the interactive experience either shines or falters. Confirm that the avatar handles these nuances gracefully, as this contributes significantly to user perception of responsiveness. 5. Monitor Performance: While specific latency numbers for every component depend on your stack, monitor overall perceived responsiveness. Focus on the end-to-end user experience. The platform is audio-clocked, meaning lips are synced to the syllable, ensuring a visually coherent response. 6. Consider Deployment: For Next.js applications, the SDK includes an App Router adapter to handle the crucial boundary of keeping your API key secure on the server while facilitating client-side avatar display (source:5). 7. Explore Capabilities: Dive into the API reference and documentation for advanced features like session policies, graceful ending, and avatar creation details (source:6). If you're building agentic experiences, consult the agent-readable documentation (/llms.txt) and the OpenAPI specification (/openapi.json) for deeper integration possibilities. The path to shipping an interactive AI avatar isn't about chasing theoretical benchmarks, but about engineering a coherent, engaging user experience. By leveraging a platform designed for full-duplex, real-time presence and a developer-centric SDK, you can focus on the unique conversational capabilities that will define your application. - [Adam's Law (Source)](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQE3wNiMdx-sROCX5EzWCMaLq3gf_TSVAXaj8OZVnmEF5a5x8i3CfmFHPx5GY5ApZT-vkj6zis_DA5QrDwGt_877KvaTcC-x_5Dl8J4juvZRMjIT-1JKV0F_d41Gu9KG7OkAhYyu5kMcoJAKOiHJ9gt9zKdlXoKUAtiy6x0Idw4AKVfl) - [Realtime Avatar Pricing](https://realtimeavatar.ai/pricing) - [Quickstart Guide](https://realtimeavatar.ai/docs/quickstart) - [Tool Calling Documentation](https://realtimeavatar.ai/docs/tool-calling) - [Next.js Integration Guide](https://realtimeavatar.ai/docs/nextjs) - [API Reference](https://realtimeavatar.ai/docs/api-reference) --- # Building Interactive AI Avatar Applications: Real\-Time Presence in Practice Explore the architecture and implementation of interactive AI avatar applications\. Learn how to create real\-time, two\-way conversational experiences with\. Published: 2026-09-11T02:20:43.641Z Updated: 2026-09-11T02:20:43.641Z Canonical: https://realtimeavatar.ai/blog/building-interactive-ai-avatar-apps Markdown: [en](https://realtimeavatar.ai/blog/building-interactive-ai-avatar-apps.md) Building an interactive AI avatar application centers on enabling real-time, two-way conversation that feels natural and responsive. This requires a carefully orchestrated pipeline, from user speech capture to avatar response rendering, all designed to maintain human-like conversational latency. The core is an API and SDK that handles the complex synchronization of audio, visual embodiment, and agent logic, ensuring the avatar listens, processes, and responds dynamically without the delays characteristic of pre-rendered video. ## The Architecture of Real-Time Interaction An interactive AI avatar is more than a talking head; it is a digital presence capable of participating in a conversation. The underlying architecture for this presence typically involves several layers working in concert, prioritizing speed and synchronization to deliver a seamless experience. ### Core Pipeline Components - Automatic Speech Recognition (ASR): The first step in any interactive system, converting spoken user input into text. - Large Language Model (LLM) or Retrieval-Augmented Generation (RAG): This component processes the transcribed text, understands its context, and generates the avatar's verbal response. - Text-to-Speech (TTS): Transforms the LLM's text output into natural-sounding audio. - Avatar Layer: Drives the digital human, synchronizing lip movements (visemes), facial expressions, and other gestures with the generated audio. Our platform generates an avatar's looping idle video and its full motion library from a single portrait image, then synthesizes video live as needed. ### Rendering and Transport Protocols For true real-time interaction, the rendering architecture and transport protocol are critical. While cloud-rendered video streaming can introduce significant latency, a more responsive approach involves a cloud-edge hybrid. Here, lightweight cloud inference drives expression data (e.g., 10-15 KB/s stream), which a client-side SDK then uses to render the avatar on the user's device. This significantly reduces latency and bandwidth requirements. The `realtime-avatar` SDK leverages this model, handling the client-side rendering from the streamed data. For the transport layer, WebRTC is the established standard for real-time, two-way communication, designed for sub-second media delivery, unlike buffered streaming protocols. ### Full-Duplex Conversation A hallmark of natural human conversation is the ability to speak and listen simultaneously. Traditional voice AI often operates like a walkie-talkie, requiring strict turn-taking. For an interactive avatar to feel truly present, it must engage in full-duplex communication. Our platform is built on this principle, meaning the avatar listens even while speaking. This design handles several nuanced aspects of human interaction without requiring explicit configuration: - Interruption: The avatar stops mid-sentence if interrupted and can acknowledge it in character, rather than defaulting to silence. This is a fundamental aspect of human conversation that prevents the interaction from feeling brittle. The system avoids talking over you with a new sentence while you are speaking, which is a deliberate design choice to maintain conversational flow. - Backchanneling: Small acknowledgements like a cough or an 'mm-hm' are not misinterpreted as interruptions, allowing the flow to continue naturally. - Intelligent Pauses: The system judges whether you are finished speaking based on what was said, not merely by the length of a pause. This means the avatar won't talk over a natural break in your speech or leave an awkward gap. - Language Switches: The avatar can follow a language change within a single sentence, adapting to multilingual conversational contexts. - [Our documentation details full-duplex capabilities](https://realtimeavatar.ai/docs#she-listens-while-she-talks) ## Integrating Intelligence with Tool Calling An avatar that can only converse, however naturally, remains a demonstration. The true utility of an interactive AI avatar emerges when it can perform actions and integrate with external systems. This is achieved through tool calling, allowing the avatar to interact with your application's logic or external APIs. The design of tool calling is straightforward: you declare a tool and provide a descriptive explanation of its purpose and when the avatar should invoke it. This description acts as the teaching signal for the underlying agent, which decides when to reach for the tool. The `realtime-avatar` SDK provides a clear interface for defining and registering these tools. ### Defining Tools in TypeScript Tools are defined directly within your application, ensuring type safety and immediate feedback during development. The `AvatarTool` type guides the structure, including a descriptive string and a JSON schema for parameters. ``` import type { AvatarTool } from "realtime-avatar/tools"; export const checkOrder: AvatarTool<{ order_id: string }> = { description: "Look up the status of a customer's order. Call this whenever they ask " + "where something is, or when it will arrive.", parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], }, execute: async ({ order_id }, { signal }) => { const order = await api.order(order_id, { signal }); return `${order.status}, arriving ${order.eta}.`; }, }; ``` The `execute` function handles the actual logic. Tools are expected to respond quickly—within 2.5 seconds—so any long-running operations should acknowledge the request swiftly and deliver results asynchronously out of band. ### Granting and Registering Tools The server-side component's role is to grant the capability for client-side tools when minting a session. The actual tool registration happens on the client, once the session is connected. ``` // server (TypeScript example of granting clientTools capability) session: async ({ avatarId }) => ({ instructions, clientTools: true }) // client (TypeScript example of registering tools) import { attachAvatarTools } from "realtime-avatar/tools"; const { accepted, rejected } = await attachAvatarTools(room, { check_order: checkOrder, }); ``` For Python backends, the process is similar: grant the `client_tools` capability in the session body during the minting process. The registration logic remains client-side. If the registration method is missing, the issue lies with the server's capability grant. ``` # server (Python example of granting client_tools capability) body["capabilities"] = ["client_tools"] ``` - [Full details on tool calling are in the documentation.](https://realtimeavatar.ai/docs/tool-calling) ## Practical Implementation Steps Bringing an interactive AI avatar to life involves a few key steps that integrate our platform's capabilities into your application. ### 1. Create Your Avatar Begin by creating an avatar from a single portrait image. The platform takes this image and automatically generates a looping idle video, along with a motion library that maps different states for dynamic animation during conversation. - [See how to create an avatar](https://realtimeavatar.ai/docs/video) - [Explore resident avatars in the studio](https://realtimeavatar.ai/studio) ### 2. Integrate the SDK The `realtime-avatar` TypeScript SDK simplifies the integration. Your application will typically have two main parts: a server-side endpoint for securely minting session tokens (as your API key must never reach the browser) and a client-side component to render the avatar and manage the call. ``` ``` - [Our quickstart guides you through the initial setup](https://realtimeavatar.ai/docs/quickstart) - [A Next.js example provides a full implementation](https://realtimeavatar.ai/docs/nextjs) ### 3. Define and Register Tools Once the basic integration is complete, extend your avatar's capabilities by defining and registering tools as outlined above. This allows your avatar to interact with your application's specific business logic. ## Limitations and Considerations for Real-Time Performance While building interactive AI avatar apps, it's crucial to acknowledge the factors that influence real-time performance and the user's perception of responsiveness. Latency is a dynamic variable, sensitive to various elements within the interaction pipeline. Our platform is engineered for real-time responsiveness, with video lips synced precisely to the syllable. However, the end-to-end latency experienced by a user is a composite of several stages, including Automatic Speech Recognition (ASR), Large Language Model (LLM) processing, Text-to-Speech (TTS) generation, network conditions, and the client-side rendering. Therefore, promising a single, fixed latency number is challenging without a specific, cited measurement protocol that accounts for all these variables. Key considerations for developers include: - Network Variability: Unstable or high-latency network connections will invariably impact the perceived responsiveness, especially on mobile devices where real-time lip-sync alignment and video resolution can degrade. - Device Hardware: The performance of client-side rendering can vary based on the user's device. While our SDK is optimized, testing on a range of target hardware (e.g., mid-range to lower-end mobile devices) is advisable to ensure a consistent experience. - Agent Complexity: The intricacy of your LLM prompts and the number or complexity of the tools the avatar needs to query can add to processing time. Optimize prompts and design tools for efficiency. The platform's full-duplex design inherently addresses many common real-time interaction hurdles, such as managing interruptions and natural pauses, without requiring developers to build intricate state machines for these scenarios. This allows you to focus on the agent's core intelligence and tool interactions rather than conversational mechanics. ## How to Ship It Bringing an interactive AI avatar application from development to deployment requires a practical approach to cost, performance, and user experience. Our platform is designed with shipping in mind. First, leverage the usage-based pricing model. Real-time avatar interaction is metered, but it's priced to be left on, enabling continuous engagement without prohibitive costs. With plans anchored around $5 per hour of real-time interaction and a free tier for development, you can afford to let users talk. Avatar creation is a nominal $1 beyond any included allowances, making it accessible to iterate on your avatar's persona. This pricing structure enables companion apps to be truly conversational, offering significant interaction time within reasonable budgets. - [Understand the pricing details](https://realtimeavatar.ai/pricing) - [Read about voice or video on one meter](https://realtimeavatar.ai/docs#voice-or-video-on-one-meter) Second, rigorous testing is non-negotiable. While the SDK handles many real-time complexities, you should validate your agent's behavior under various conditions. Pay close attention to how your tools are invoked and how the avatar responds to edge cases in user input. Observe lip sync performance, especially for harsh consonants, and ensure conversational flow remains natural during stress tests. Finally, utilize the comprehensive documentation and API reference as your field guide. The TypeScript SDK is generated directly from our OpenAPI specification, providing a type-safe and predictable development experience. For integrating your AI agent's knowledge base, the `/llms.txt` endpoint offers agent-readable documentation to help your LLM understand the API's capabilities and context. Focus on crafting compelling agent instructions and robust tools; the underlying real-time embodiment is handled. - [Explore the API reference](https://realtimeavatar.ai/docs/api-reference) - [Review agent-readable docs](https://realtimeavatar.ai/llms.txt) - [spatius.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFwfofgEsKg1UEhxmL1NieETMgWJE5_HCVLcoc_NQknokH1UDQuccDU6pdTchqHoDyCVkTH_nR4O_aQPaoV_cquIXeI_WCX9Aj_ZQ5fmUB1bMshw6U4A0TMHXvDlqspCWVJ2phkh_5qyNvBblhIq-DFbiYtCUbF1c3cH6Bvybmjh1IioNo-NZzyzDqGcAJn5DCrgrc=) - [avataraisdk.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHQ8ZKXWO-CoySSGMqgD18hLjeqzbjf1zz8dy9GmE11tGGfSIvMijHCPBXJJQZcHKpjXcvKlWTqUtBABKgq31_8_ZfIabm6n-3MeytvJm7NIW54ng==) - [forasoft.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFBV068gv23q-MqmZ9hKiaulO1waFhhVDvmrhC2ODRG23Po8BoJ6qHDh1hg9hA8AsGMkvBdh0BJIkhJh3AcUAjXqCWJv2I5b95QRPebBlh4lT8WrX3Hrt0HUFXLp9WAZ9E25T6kcWQWBvo8FUSyu1UeOhtahbG2ES2CI-hu9Oz8wK1T) --- # How to test barge\-in in a realtime avatar app Test realtime avatar interruptions with repeatable voice scenarios, recordings, and clear pass criteria before choosing an API or shipping your app\. Published: 2026-09-08T18:08:19.450Z Updated: 2026-10-12T09:02:31.873Z Canonical: https://realtimeavatar.ai/blog/testing-barge-in-in-a-realtime-avatar-app Markdown: [en](https://realtimeavatar.ai/blog/testing-barge-in-in-a-realtime-avatar-app.md) Test barge-in by speaking while the avatar is already answering, then checking three separate outcomes: whether its audible reply stops, whether its visible behavior remains coherent, and whether its next answer uses your correction. A successful connection is not evidence that interruption handling works. Use the same prompts, microphone conditions, and recording method for every candidate API, and treat the results as observations from your test rather than universal latency guarantees. ## Separate interruption from the end of a turn A pause inside a sentence and an interruption during the agent's reply are different events. In the first case, the system must decide whether the user has finished. In the second, it must decide whether new speech should displace output already playing. Combining both into a single responsiveness score hides useful information: an agent can respond promptly after silence but still continue talking over a correction. LiveKit's official turn-handling documentation distinguishes turn detection, endpointing, voice activity detection, and interruption handling. It describes several detection modes and separate options for false interruptions. Those distinctions make a useful evaluation checklist even when you are buying a managed avatar API rather than configuring a voice pipeline yourself. They do not establish that every avatar vendor exposes the same options, uses the same implementation, or produces equivalent results. - [LiveKit: turn detection and interruptions](https://docs.livekit.io/agents/logic/turns/) ## Start with one small, repeatable conversation Use a fictional ordering exercise rather than an open-ended debate. Ask the avatar to help place a coffee order, restrict it to a few short questions, and tell it to use the customer's latest explicit choice. Keep the menu and any prices fictional. This creates an easily observable correction without requiring the reviewer to judge subjective qualities such as charm or emotional intelligence. For Realtime Avatar, the documented Next.js integration keeps the API credential, authorization, and session policy on your server; the browser connects through the proxy. Before testing voice interruption, keep the server's listen policy enabled, enable the starter's optional microphone, and confirm that the avatar responds to a spoken baseline request. A text-only baseline does not test the microphone path. Test corrections during ordinary conversational replies, not call.sayAndEnd(...): that closing line is documented as uninterruptible. Follow the documented integration rather than inventing an interruption endpoint, and replace the starter's local-development authorization before hosting it for other users. - [Next.js integration guide and starter](https://realtimeavatar.ai/docs/nextjs) - [Session policy and call lifecycle](https://realtimeavatar.ai/docs/sessions) Keep the setup unchanged while comparing runs. Record the browser version, input and output devices, whether the user wears headphones, the network conditions, and the chosen avatar. A microphone capturing sound from speakers presents a different situation from a headset. Neither result should silently stand in for the other. Do not change multiple settings between attempts and then attribute an improvement to one of them. ## Run four scenarios, not one impressive demo 1. Baseline: ask for a coffee and let the avatar finish. Check that the response is relevant before testing interruptions. 2. Correction: while the avatar is responding, say, 'Actually, make that tea.' Check whether the next relevant response uses tea rather than coffee. 3. Continuation: pause partway through your own request, then complete it. Record whether the avatar begins before you intended to finish. 4. Backchannel: use a brief acknowledgment such as 'mm-hm' while the avatar speaks. Record whether it continues or stops, then assess whether that observed behavior suits your application. This test does not assume the provider offers a setting to change it. These are proposed acceptance scenarios, not claims about a particular vendor's measured accuracy. Choose the expected behavior for your application before running them. A training exercise may intentionally let the character finish an instruction, while a conversational assistant may need to yield immediately to a correction. A test is only useful when the reviewer knows which behavior the product is supposed to deliver. Repeat the correction at different points in the reply: near its beginning, in the middle, and close to its end. Preserve failures instead of retaining only the best recording. Report how many attempts you ran and how many satisfied each criterion. If you test multiple people or languages, keep those groups visible rather than folding them into a single number that conceals the weakest case. ## Measure what the user hears, then what the system understands For a manually reviewed recording, mark the onset of the user's correction and the last audible portion of the displaced avatar reply. Their difference is a useful interruption-stop interval for that recording. Mark the start of the next relevant answer separately. Do not call either interval startup latency, and do not equate a connection-ready event with the first audible response. They describe different parts of the experience. A screen recording alone may not capture microphone input and remote audio accurately. Verify the recording path with a short baseline before collecting a comparison. When possible, keep distinguishable input and output tracks, and describe their timing limitations. If your method cannot reliably resolve a short delay, report that limitation instead of presenting a precise millisecond figure. No performance result is claimed in this article. Stopping quickly is not sufficient if the next answer ignores the correction. Score semantic recovery independently: did the avatar replace the earlier choice, ask a relevant clarification, or continue with stale information? Also inspect the video for extended speaking motion after audio has stopped, unexpected freezes, or an abrupt restart. These observations help distinguish an audio-control issue from a dialogue-state or presentation issue without guessing at a provider's internal implementation. - [Separate startup and spoken-turn measurements](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) ## Keep the result useful for a shipping decision Store one row per attempt with the scenario, setup, observed behavior, outcome, and a recording reference. Keep the raw observations alongside any aggregate. Agree on acceptable failure rates and timing goals for your use case before comparing providers; a target you choose is not a benchmark the provider has achieved. Where sample sizes are small, say so plainly and use the result to guide further testing rather than a sweeping ranking. Finally, test the surrounding controls: ending a call, recovering from denied microphone permission, and starting a fresh session after a failed attempt. Keep test sessions bounded and avoid collecting real customer details in the fictional exercise. Before release, repeat the same scenarios on the actual devices and network conditions your users will have. The goal is not to prove that an avatar can be interrupted once; it is to know what your application does when a real conversation changes direction. --- # Build a realtime avatar language\-practice app in Next\.js Run a complete Next\.js avatar starter, turn it into a café roleplay partner, and test the conversation with a concrete rubric\. Includes source download and server\-side session policy\. Published: 2026-09-05T00:00:00.000Z Updated: 2026-09-05T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/build-a-realtime-avatar-language-practice-app-nextjs Markdown: [en](https://realtimeavatar.ai/blog/build-a-realtime-avatar-language-practice-app-nextjs.md) # Build a realtime avatar language-practice app in Next.js > Run a complete Next.js avatar starter, turn it into a café roleplay partner, and test the conversation with a concrete rubric. Includes source download and server-side session policy. - Published: 2026-09-05 - Tags: api, tutorial - Canonical: https://realtimeavatar.ai/blog/build-a-realtime-avatar-language-practice-app-nextjs - Markdown: [en](https://realtimeavatar.ai/blog/build-a-realtime-avatar-language-practice-app-nextjs.md) A useful first avatar app is a short language-practice conversation: order a drink, ask a follow-up question and get one specific correction. The scope is small enough to test, while exercising the same connection, text, microphone and session-ending paths a larger app needs. This tutorial uses a ready Realtime Avatar character and a complete local Next.js project. It does not require creating an avatar or building a separate speech pipeline. You need Node.js 22 or newer, a session-capable API key and available call credits. ## 1. Run the starter [Download the source archive](https://realtimeavatar.ai/downloads/nextjs-avatar-starter.zip), then run: **Terminal** ```bash unzip nextjs-avatar-starter.zip cd nextjs-avatar-starter npm ci cp .env.example .env.local # Add REALTIME_AVATAR_API_KEY to .env.local npm run dev ``` Open `http://127.0.0.1:3000`. Start a call and send a text message first. This separates connection problems from microphone permissions. When that works, end the call, enable the microphone and start another one. Calls are capped at 60 seconds and consume account credits. ## 2. Give the character one job The server decides what the character knows. In `app/api/realtime-avatar/[...path]/route.ts`, keep the existing authorization and avatar checks, and replace the returned `instructions` value with this prompt: **Session instructions** ```text You are Rin, a friendly café barista helping an adult practice English. Say clearly that this is a practice conversation. Use short, everyday sentences and ask one question at a time. Help the learner order one drink, choose its size, and ask the price. Use this fictional menu only: tea $3, coffee $4; a large drink costs $1 more. After the order, give one specific language correction if needed. If the learner says "feedback", give one thing they did well and one suggestion. If the learner corrects the order, use the latest choice. Do not claim to place a real order, accept payment, or assess proficiency formally. ``` The menu is fictional test data, not product pricing. Keeping it fixed gives you an answer key. The 60-second limit also makes it obvious whether your instructions ask for more conversation than the session allows. Keep the prompt short enough for the intended exercise. ## 3. Keep session ownership in your app The browser uses `createProxyClient` and `useAvatarCall`. The hook returns `call` for status and controls, and `view` for the video surface. Mount it after Start, send text through `call.say`, and end with `call.end`. The included page implements those controls and removes the call after `onEnded`. The API key stays in `.env.local`, without a `NEXT_PUBLIC_` prefix. The starter's proxy allows local development only. For a hosted app, implement user authorization, balance and rate checks, and persistent session ownership before enabling production calls. The [Next.js guide](https://realtimeavatar.ai/docs/nextjs) explains the server adapter and common errors. ## 4. Test the exercise, not just the video | Input | Expected behavior | | --- | --- | | “I would like a coffee.” | One relevant follow-up question, such as the size. | | “Actually, tea instead. Large.” | Uses tea as the latest choice; the fictional total is $4. | | “Feedback.” | One specific positive observation and one useful suggestion, without an invented proficiency score. | | Interrupt the spoken reply with a correction. | Record whether playback stops and the next answer uses the correction. | | Click End call. | The call ends, the start button returns, and session usage can be checked. | These are acceptance criteria, not claimed results. Run the script with several phrasing variations and note failures. A prompt can guide behavior; it does not guarantee it. For a real learning product, have a qualified educator review the exercise and learner feedback. ## 5. Capture evidence before expanding The starter can download each run's browser and startup observations as JSON. It distinguishes SDK readiness from a displayed remote frame; neither measures how quickly a spoken question receives an audible answer. Use the [measurement guide](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) to design that separate test. Once this small exercise behaves consistently, estimate monthly minutes and peak simultaneous calls in the [cost calculator](https://realtimeavatar.ai/tools/avatar-cost-calculator). Expand to another scenario only after you have a rubric for what a good conversation should do. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # How to benchmark realtime avatar startup and turn latency A reproducible measurement protocol for avatar startup, first audible reply, interruptions and failures, with downloadable Next\.js instrumentation and reporting guidance\. Published: 2026-09-05T00:00:00.000Z Updated: 2026-09-05T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency Markdown: [en](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency.md) # How to benchmark realtime avatar startup and turn latency > A reproducible measurement protocol for avatar startup, first audible reply, interruptions and failures, with downloadable Next.js instrumentation and reporting guidance. - Published: 2026-09-05 - Tags: api, engineering - Canonical: https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency - Markdown: [en](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency.md) “Avatar latency” can mean time to join a room, see a frame, hear a reply or stop an interrupted reply. Those numbers answer different questions. Before comparing APIs, define the event that starts each timer and the observation that stops it. This article publishes a measurement protocol and runnable startup instrumentation. It contains no live-call latency results or cross-vendor ranking. The [Next.js starter](https://realtimeavatar.ai/downloads/nextjs-avatar-starter.zip) lets you collect observations on your own account and network. ## Measure distinct paths | Metric | Start | End | What it includes | | --- | --- | --- | --- | | Click to SDK live | User clicks Start | SDK reports live | Application and connection readiness; not proof of a displayed frame. | | Click to first remote frame | User clicks Start | First observed presented frame from the remote MediaStream | Queue, setup, transport, decoding and browser presentation. | | Spoken-turn response | Last audible sample of the user's utterance | First audible sample of the avatar's reply | Endpointing, recognition, model response, synthesis, transport and playback. | | Interruption stop | User begins speaking over the avatar | Avatar audio stops | Detection, cancellation and buffered playback. | Use one monotonic clock for elapsed times. Server and browser wall-clock timestamps may have different offsets. A server-side processing duration can help diagnose one stage, but it cannot replace a browser observation of the complete path. ## Collect startup observations with the starter Follow the [setup instructions](https://realtimeavatar.ai/docs/nextjs), keep the tab in the foreground, start a call, then download its JSON after ending. The app records `sdkLiveMs`, `firstVideoFrameMs`, start date, browser, SDK version and end reason. A missing timing is `null`, not zero. There is no API key, room token or transcript in the download. The video observation uses `requestVideoFrameCallback` on a video element backed by a `MediaStream`, so a poster or idle clip does not count. A 50 ms discovery poll, callback scheduling and rendering add overhead; it is an observed browser startup time, not an exact server-rendering time. The [browser API documentation](https://developer.mozilla.org/en-US/docs/Web/API/HTMLVideoElement/requestVideoFrameCallback) explains the callback's timing and limitations. ## Hold the workload steady - Record date, client region, connection type, browser/device, SDK version, avatar, voice and pipeline mode. - Use the same short prompt and utterance script. Separate text-driven trials from microphone trials. - Declare the warm-up policy in advance. Report the first attempt separately; do not assume it proves a cold backend. - Record queue state and failures. Run sequential calls first, then a separate concurrency test within your plan and budget. - End every call and compare recorded usage with the intended test duration. Start with five sequential attempts to check the harness. For a first distribution, collect at least 30 attempts per condition and disclose the sample count. This is a starting protocol, not a guarantee of statistical precision: a p95 from a small sample is unstable, and stronger conclusions require more data across time and environments. ## Measure speech separately The starter's text input bypasses spoken input and cannot establish end-of-utterance latency. For that test, use a fixed audio fixture or a controlled spoken script and capture input and output on a shared audio timeline. Mark the last input sample and first reply sample using a declared detection threshold, then inspect the waveform manually to reject noise triggers. Keep transcription-final, first-token and audio-track-subscribed events as diagnostics. None necessarily equals the instant the user heard the reply. For interruption tests, record both time until playback stops and whether the subsequent answer incorporates the correction. ## Report failures beside the percentiles Publish attempts, successful observations, missing observations, timeouts, queue frequency and end reasons. Calculate median and p95 on the observed successful measurements, state that denominator, and keep the failure rate alongside them. Never delete a slow queued attempt merely because it makes the result look worse. The archive includes `scripts/summarize-startup.mjs`: run `node scripts/summarize-startup.mjs runs/*.json` on the downloaded files. It reports observed and missing counts, end reasons, median and nearest-rank p95. Keep each browser/network condition in its own folder; the script cannot infer conditions you did not record. ## What evidence supports a product decision? A useful report links the runnable code, configuration, raw observations and calculation method. It also states what was not measured. Startup tests do not establish lip-sync quality, learning outcomes or sustained production capacity. Pair the timings with a [task-specific conversation rubric](https://realtimeavatar.ai/blog/build-a-realtime-avatar-language-practice-app-nextjs) and a [cost estimate](https://realtimeavatar.ai/tools/avatar-cost-calculator) before deciding which integration fits your app. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Evaluating a LiveAvatar or Tavus alternative: a developer checklist Compare realtime avatar APIs by pipeline ownership, startup, conversational behavior, integration effort and total call cost\. Includes a runnable Next\.js evaluation app\. Published: 2026-09-05T00:00:00.000Z Updated: 2026-09-05T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/liveavatar-tavus-alternative-developer-checklist Markdown: [en](https://realtimeavatar.ai/blog/liveavatar-tavus-alternative-developer-checklist.md) # Evaluating a LiveAvatar or Tavus alternative: a developer checklist > Compare realtime avatar APIs by pipeline ownership, startup, conversational behavior, integration effort and total call cost. Includes a runnable Next.js evaluation app. - Published: 2026-09-05 - Tags: api, product - Canonical: https://realtimeavatar.ai/blog/liveavatar-tavus-alternative-developer-checklist - Markdown: [en](https://realtimeavatar.ai/blog/liveavatar-tavus-alternative-developer-checklist.md) If you are looking for a LiveAvatar or Tavus alternative, start with the application you need to ship: a complete conversational character, or a face attached to a voice agent you already run. That choice changes your integration work, your latency measurements and your bill. This is a developer evaluation guide from the team behind Realtime Avatar. The competitor capabilities below were checked against their own documentation on 5 September 2026. We have not run a controlled cross-vendor performance test, so this article does not rank their latency or visual quality. ## Choose the pipeline before comparing the rate | Product / mode | Documented integration model | What to evaluate | | --- | --- | --- | | LiveAvatar FULL | Managed speech recognition, LLM, TTS and WebRTC; documented options for your own LLM or voice. | Whether the managed pipeline and customization cover your conversation. | | LiveAvatar LITE | Avatar video with your conversational stack and WebRTC management. | Rendering plus the cost and behavior of your voice pipeline. | | Tavus CVI | A configurable conversation pipeline spanning perception, turn-taking, STT, LLM, TTS and a realtime face. | Which layers your application needs and how you configure them. | | Tavus Echo | Text or audio supplied directly, bypassing parts of the normal conversation pipeline. | The exact echo mode; its behavior differs from full CVI. | | Realtime Avatar | A server-side session policy and React call UI, with speech recognition, LLM, voice and rendering included in the live-call product. | First-call integration, character behavior, tool boundaries and account economics. | Sources: [LiveAvatar's mode comparison](https://docs.liveavatar.com/), [Tavus CVI overview](https://docs.tavus.io/sections/conversational-video-interface/overview-cvi), [Tavus Echo documentation](https://docs.tavus.io/sections/conversational-video-interface/echo-mode), and [Realtime Avatar sessions](https://realtimeavatar.ai/docs/sessions). These are product capabilities, not evidence that one implementation will perform better for your users. ## Use the same small application for every trial Build one page with a start button, one ready avatar, a text input, optional microphone, visible connection state and an end button. Begin with a stock character so custom-avatar creation does not obscure the first-call test. Keep the same conversation script, browser, network and test location. Our [downloadable Next.js starter](https://realtimeavatar.ai/downloads/nextjs-avatar-starter.zip) supplies that app for Realtime Avatar. Follow the [setup guide](https://realtimeavatar.ai/docs/nextjs), then reproduce the same user flow using each vendor's supported SDK. Count the work needed for authentication, usage controls, error handling and session cleanup alongside the code that renders the face. ## Score behavior a user can notice 1. **Startup:** How long after Start does a remote video frame appear? Record queuing and failed attempts. 2. **First reply:** After a fixed spoken question ends, when is the first reply audible? A room-connected event does not answer this. 3. **Interruption:** Say “Actually, change that to tomorrow” during a reply. Does playback stop, and does the next answer use the correction? 4. **Task completion:** Ask a question that requires your application's data. Verify the answer against the tool result, including a deliberately failed tool call. 5. **Recovery and ending:** Briefly interrupt the network, then end the call. Check the UI and the recorded session usage. Use a simple pass/fail rubric written before testing. Record unresolved cases separately. For timing definitions and reporting sample counts, see our [benchmark methodology](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency). ## Normalize the bill LiveAvatar documents two credits per minute for FULL and one for LITE. A credit therefore is not interchangeable with a minute across modes. Its billing guide also explains overage settings and the minimum balance needed to start a session. Verify the current plan and unit in the [LiveAvatar credit documentation](https://docs.liveavatar.com/docs/faq/credits). For every offer, write down subscription cost, included usage, overage, simultaneous-call limits, creation charges and the services you must supply. Confirm Tavus's current offer on its [pricing page](https://www.tavus.io/pricing). Realtime Avatar's published self-serve overage is $0.08–$0.095 per live-call minute; use our [calculator](https://realtimeavatar.ai/tools/avatar-cost-calculator) for subscription-plus-overage scenarios and their assumptions. ## Make the decision from your trial If your application already has a voice pipeline that meets your requirements, test a rendering mode first. If you need to build the whole conversation, evaluate complete pipelines first. Then compare task completion, failure rate, integration work and cost at the same workload. Keep the raw observations: they are more useful than a winner chosen from headline prices. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Kling AI Avatar API: video jobs and live conversations What Kling Avatar v2's fal endpoint accepts and returns, how it differs from a live avatar session, and what to measure before choosing an integration\. Published: 2026-09-05T00:00:00.000Z Updated: 2026-09-05T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/navigating-the-kling-ai-avatar-api-landscape Markdown: [en](https://realtimeavatar.ai/blog/navigating-the-kling-ai-avatar-api-landscape.md) # Kling AI Avatar API: video jobs and live conversations > What Kling Avatar v2's fal endpoint accepts and returns, how it differs from a live avatar session, and what to measure before choosing an integration. - Published: 2026-09-05 - Tags: api, engineering, product - Canonical: https://realtimeavatar.ai/blog/navigating-the-kling-ai-avatar-api-landscape - Markdown: [en](https://realtimeavatar.ai/blog/navigating-the-kling-ai-avatar-api-landscape.md) A narrated product clip and an avatar answering a customer's next question need different application flows. If you are evaluating a Kling AI Avatar API, start with the output you need: a finished video to publish, or a conversation whose next line depends on the person watching. This guide examines Kling Avatar v2 Standard through fal's documented endpoint and Realtime Avatar's live-call integration. The API details below were checked on September 5, 2026. The Kling example uses fal's credentials and SDK; a direct Kling integration needs its own provider documentation. ## What the documented Kling endpoint returns fal exposes `fal-ai/kling-video/ai-avatar/v2/standard`. Its required inputs are `image_url` and `audio_url`, with an optional `prompt`. The result contains a generated `video` file and its `duration`. See the [provider's API reference](https://fal.ai/models/fal-ai/kling-video/ai-avatar/v2/standard/api) for the request and response schemas. The `@fal-ai/client` SDK can submit a job, check its status and fetch the result; the queue API also accepts a webhook URL. Keep `FAL_KEY` on your server. Your application supplies the narration audio and presents the returned video when generation completes. For a scripted lesson, that gives you a useful review step: listen to the narration, render the clip, inspect it, then publish. If the script changes, budget for another render. This endpoint's documented output is a video artifact; these details do not establish the capabilities of every Kling product or access agreement. ## Choose the application flow first | Design question | Generated video | Live conversation | | --- | --- | --- | | When is the speech decided? | Before submitting the render | As the user takes each turn | | What does the interface manage? | Job progress, review and playback | Call connection, input, recovery and ending | | What should the prototype prove? | A publishable clip with acceptable rendering cost | A complete conversation with acceptable response delays | These are architectural choices. A training product might use prepared clips for its explanation and a live avatar for follow-up questions. Treat those as separate flows with their own loading states, costs and failure handling. ## How a Realtime Avatar call fits Realtime Avatar's [quickstart](https://realtimeavatar.ai/docs/quickstart) connects a browser to a live call through the `realtime-avatar` TypeScript SDK. Your server holds the API key and authorizes the call. The client renders the conversation and exposes controls to end it. Begin with an available example avatar so that character creation is separate from the first integration test. For a custom character, upload a portrait and wait for avatar preparation to reach `ready` before starting a call. Creating an avatar is asynchronous; receiving an avatar ID does not mean it can already accept sessions. The [creation guide](https://realtimeavatar.ai/docs/video) explains that lifecycle. Design the call screen around the user's next action. Show connection progress, make microphone permissions understandable, keep the end control reachable on a phone, and offer a clear retry after a failed connection. An idle animation can keep the character visible while connecting, but it should not be counted as evidence that live media arrived. ## Measure what the person actually experiences A claim such as "fast avatars" needs a defined clock. For a generated clip, measure job submission to a playable result. For a conversation, record at least these intervals: - Starting the call to the first rendered frame from the remote live stream. - Finishing a spoken question to hearing the first audible reply. - Interrupting a reply to the old speech actually stopping. Connection events and HTTP success responses help diagnose the system, but neither proves the user saw or heard the avatar. Test on the devices and networks your audience uses, repeat the same conversation, and report both typical and slower results. Our [startup and turn-latency guide](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) describes a measurement setup. This article makes no latency guarantee for either integration. ## Budget for the workflow you will ship For generated content, count finished clips plus retries and revisions. For live interaction, model conversation duration and peak simultaneous calls. Those workloads need different estimates even when both start with one image. Realtime Avatar uses credits shared by live time and generation. The advertised live-minute allowance assumes those credits go to calls; creating assets draws from the same balance. Use the current [pricing page](https://realtimeavatar.ai/pricing) and [call cost estimator](https://realtimeavatar.ai/tools/avatar-cost-calculator) for plan allowances, overage and concurrency limits. Build one complete journey before committing: a reviewed clip that plays in your product, or a call that connects, answers a follow-up and ends cleanly. You can try the latter in [Studio](https://realtimeavatar.ai/studio), then follow the [SDK quickstart](https://realtimeavatar.ai/docs/quickstart) to put it in your own application. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Avatar Renderer vs Full\-Stack API: A Developer Integration Guide Choose an avatar renderer or a managed conversation API for your voice agent\. Compare audio inputs, tools, latency measurements and complete session costs\. Published: 2026-09-04T00:00:00.000Z Updated: 2026-10-12T09:32:41.087Z Canonical: https://realtimeavatar.ai/blog/renderer-or-full-stack-realtime-avatar-apis-compared Markdown: [en](https://realtimeavatar.ai/blog/renderer-or-full-stack-realtime-avatar-apis-compared.md) Choose an avatar renderer when your application already owns speech recognition, the language model and text-to-speech, and needs a face driven by its audio. Choose a full-stack conversation API when you want those stages managed together. Realtime Avatar is a managed conversation service, not a public arbitrary-audio rendering endpoint. A shared transport such as LiveKit does not make the two architectures interchangeable. Compare the interfaces and responsibilities before comparing a per-minute price. ## Map the pipeline you already own A spoken character experience usually involves capturing a user's speech, recognizing or otherwise interpreting it, deciding a response, synthesizing sound, rendering a face and delivering media to the client. A renderer supplies the visual part of that workflow. A managed service may supply most of the conversation. Some vendors offer several modes, so classify the specific mode you intend to purchase rather than assigning one permanent label to an entire company. - Existing voice agent: preserve your working model, voice, tool policies and conversation state; evaluate a documented audio-to-avatar integration. - New character application: evaluate a managed session with server-owned persona, context, authorization and spending limits. - Specialized dialogue logic: check whether an approved external-model endpoint changes only the reasoning stage or also permits your own speech services. - Live broadcast: distinguish interactive character sessions from the separate encoding, distribution and audience-control system. For example, a language-practice product may already have pronunciation scoring and a selected speech provider. Replacing that pipeline merely to obtain a face could lose important behavior. A new game character, by contrast, may benefit from a managed conversation if the application's main responsibility is supplying game state and approved actions. These are design examples, not claims that either architecture is always cheaper or produces better learning or engagement. ## For a LiveKit voice agent, verify the actual avatar input LiveKit publishes avatar integrations for agents, including provider-specific adapters. Read the selected adapter's requirements and the provider's corresponding mode. Check which side supplies speech audio, who owns the room, how the avatar participant is connected and how the session ends. A plugin's existence is evidence of an integration interface, not proof of the provider's performance, commercial permissions or fit with every application. Anam's documented LiveKit integration is an example of adding an avatar to an application-owned voice-agent stack. That is a different integration contract from opening an RTA managed character session. Do not send a server grant or a media track intended for one system into another merely because both mention LiveKit. The token scope, session lifecycle and audio ownership must be compatible with the exact documented route. Before committing, make a small interface checklist: does the provider accept live audio or only text; what format and buffering rules apply; how is interruption handled; how are disconnect and cancellation reported; and which component stops billing? If any answer is missing from the public documentation, ask for it before writing a custom bridge. An uploaded audio asset is not automatically a live audio-streaming interface. - [LiveKit avatar integrations for agents](https://docs.livekit.io/agents/models/avatar/) - [LiveKit's Anam avatar adapter](https://docs.livekit.io/agents/models/avatar/plugins/anam/) - [Anam custom text-to-speech audio input](https://anam.ai/docs/javascript-sdk/examples/custom-tts.md) ## What Realtime Avatar manages, and what remains yours RTA manages speech recognition, conversational turn taking, speech synthesis and avatar video for its live sessions. Your backend authorizes who may start a call, selects the character instructions and initial context, and imposes a duration or spending boundary. Your client renders the conversation and provides understandable connecting, waiting, recovery and ending states. Character creation starts with a portrait image rather than an arbitrary source-video upload. Your tools still execute in your application or backend. A support prototype can look up an account-scoped record; a game character can request a permitted game action; a tutor can retrieve a reviewed lesson passage. The application is responsible for validating arguments, limiting access and confirming consequential actions. The avatar's request to call a function is not authorization to reveal another customer's data, place an order or change an account. An external language-model endpoint requires workspace approval and a host allowlist. It changes where a response is generated, not ownership of the entire speech pipeline. The public RTA SDK does not provide an arbitrary live-audio input for a customer's own text-to-speech service. Sending a text turn steers a generated reply; it is not a general guarantee of verbatim playback. If a working voice agent must keep its current voice provider and audio stream, this is a genuine product-fit boundary rather than a setting to assume exists. - [RTA session policy and context](https://realtimeavatar.ai/docs/sessions) - [RTA tools and model-integration boundaries](https://realtimeavatar.ai/docs/tool-calling) - [React and native client integration](https://realtimeavatar.ai/docs/react) ## Keep latency measurements attached to the full experience A renderer's time to produce a frame is only one part of a spoken response. A managed service's advertised response time may use a different starting event. Neither number is comparable until the measurement boundary is explicit. Separate initial connection and first media from the delay after a user finishes speaking, and distinguish server publication from what the receiver actually hears and sees. Use the same task, microphone, client device, network and chosen voice when testing candidates. Report warm and cold conditions separately and retain failures, not just the most impressive run. Also test interruption: does stale audio stop, does the face remain coherent, and does the next answer incorporate the correction? The dedicated guides below provide a test method; this comparison does not claim a universal latency result or rank providers using incompatible marketing figures. - [Benchmark startup and spoken-turn latency](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) - [Test barge-in and semantic recovery](https://realtimeavatar.ai/blog/testing-barge-in-in-a-realtime-avatar-app) ## Compare complete costs, not the cheapest stage A renderer quote can exclude the model, recognition, voice, transport and application infrastructure you already operate. A managed session can include several of those while still excluding your hosting, external tools, custom media creation or support work. Neither should be labeled as requiring nothing else. Add minimum commitments, unused included allowance, connected idle time, failed work and recording or export charges where they apply. Keep included-plan arithmetic separate from marginal overage. A plan used only occasionally can have a very different effective cost from the same plan used to its allowance. Likewise, an advertised concurrency limit is not a reserved-capacity contract. Use current provider pricing and the actual workload you tested. The linked pricing comparison covers a separate buying question so this architecture guide does not become another stale copy of every vendor's rate card. - [Current RTA pricing](https://realtimeavatar.ai/pricing) - [HeyGen and LiveAvatar pricing boundaries](https://realtimeavatar.ai/blog/heygen-api-pricing-explained) ## Choose with a small acceptance test Write down the one part of the system you intend to replace and the components that must remain unchanged. Then verify an authorized session, an audible and visible reply, an interruption, a controlled failure, a clean disconnect and a matching usage record. Check source-media rights and provider processing terms before sending private material. If the required audio input or data handling is unsupported, choose a different integration rather than assuming a low price makes the mismatch disappear. For a new managed character experience, start with the documented quickstart and one original character. For an existing voice agent, start with a provider's supported avatar adapter and a known-good conversation. In either case, the useful result is a repeatable application task and an explainable bill—not merely a face that moved during a demo. --- # Generated video that answers back: making a live avatar interactive, not just alive A live avatar that performs on a loop is easy\. One that changes course when you ask is a scheduling problem — and per\-second pricing is what makes the fix affordable\. Published: 2026-09-01T00:00:00.000Z Updated: 2026-10-13T13:44:48.953Z Canonical: https://realtimeavatar.ai/blog/generated-video-that-answers-back Markdown: [en](https://realtimeavatar.ai/blog/generated-video-that-answers-back.md) # Generated video that answers back: making a live avatar interactive, not just alive > A live avatar that performs on a loop is easy. One that changes course when you ask is a scheduling problem — and per-second pricing is what makes the fix affordable. - Published: 2026-09-01 - Tags: engineering, product, companion - Canonical: https://realtimeavatar.ai/blog/generated-video-that-answers-back - Markdown: [en](https://realtimeavatar.ai/blog/generated-video-that-answers-back.md) There is a version of a live avatar that is not very interesting: a character who performs on a loop, at you, forever. It looks alive. Nothing you do changes it. We shipped that first — a resting loop is what an avatar falls back to between turns — then spent the real effort on what happens after somebody asks it for something else. ## Why a live avatar stream is a scheduling problem Continuously generated video arrives in segments: render a few seconds, play them, render the next few. That length is the most consequential number in the system, and it pulls in two directions. Long segments amortise fixed per-render latency — a submit, a queue, a warm-up. Measured on our lane at 480P, a fifteen-second segment renders in five to eight seconds and a five-second one in about three, so three short renders spend roughly twice the wall time for the same footage. They also look better: the model has room to finish a movement rather than lurching. But a long segment is also the slowest thing to get on screen — and a viewer who has just redirected the character is waiting on exactly that. ## Make the segment short exactly when someone is waiting Our stream runs fifteen-second segments while ambient — nobody has asked for anything, the character is simply present — and drops to five seconds for the first segment after a new direction. Responsiveness is bought exactly where someone is waiting for it, and the long segment is kept for everywhere else, which is most of the time. Two mechanisms make that land. The answer segment *interrupts* — it cuts into whatever is playing rather than queueing behind it. We learned that the hard way: making a viewer sit out the rest of a fifteen-second clip is up to fifteen seconds of nothing happening, and it was reported to us as the direction being ignored. Superseded segments are discarded rather than played out. What a viewer waits on, then, is not playback but the pipeline: a three-second scheduling tick plus the render. A handful of seconds, not fifteen. And note the number to be careful with — five seconds is the *length* of the answer segment, not its time to screen. ## Per-second pricing is what permits the trick Charge per render and the shortened segment costs the same as the long one, so the responsive path is three times the price per second of video — you would be taxed hardest for the one behaviour that makes the product feel awake. Any sane operator would pick a single segment length and live with the latency. We run the stream on MiniMax `h3-max` at 480P, priced at five cents per rendered second, and bill on exactly that basis: a private stream settles a fifteen-second ambient segment at about $0.75 and a five-second answer at about $0.25. The cost is strictly linear, with no per-render term — three five-second segments and one fifteen-second segment cost the same to the credit. Segment length stops being a billing decision and becomes a scheduling one. The model choice was the same unglamorous arithmetic: h3-max’s 480P tier costs five cents a second against eight at 768P, for the same first-frame/last-frame interface. ## Two lanes, two different caps A stream that renders whether or not anyone is watching is a bill with a character attached. Ours is watcher-gated: viewers hold a lease that expires twenty seconds after their last heartbeat, and when the last lease lapses the scheduler stops. Nobody watching, nothing rendered. Beyond that the lanes are bounded differently, and the difference is structural. A private stream is keyed to a workspace wallet, so the wallet is the cap — an insufficient balance stops it by construction, with no second budget to drift out of agreement with the first. A shared live room is house-funded and has no wallet, so it is rationed instead: one new ambient clip every two minutes, replaying its bank in between, under a ceiling of sixty renders per room per hour. Decide which you are building before you write the scheduler. ## What it does not buy This is not real-time generation in the sense a live call is. Our conversational sessions are audio-clocked, with sub-second time to first frame; this stream schedules seconds of video ahead of the viewer. Two different clocks, and the interaction design that follows from each is not the same. We also have not measured whether any of it holds attention longer. What we tuned was time-to-visible-response. Engagement is a real question and that experiment has not been run; we would rather ship the honest claim than the flattering one. ## How to ship it Start with the segment schedule, not the model: pick your ambient and post-redirect lengths before choosing a provider, because that tells you which pricing shapes are survivable. Make the answer segment interrupt playback rather than queue behind it. Gate rendering on presence with a short lease and a heartbeat, so an abandoned tab stops costing money without needing a disconnect event you will not reliably get. And pin every segment’s first *and* last frame to the same portrait, so each begins and ends on the same image and any one splices into any other with no cut. Chaining off the previous segment’s output instead would mean decoding video you cannot decode in a worker, and would compound identity drift without bound. The character is the easy part. The schedule is the product. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Integrating the AI Talking Avatar API: Principles of Realtime Embodiment Explore the technical foundations and practical integration of an AI talking avatar API\. Understand the session architecture, timing measurements and usage costs behind live conversations\. Published: 2026-08-23T00:00:00.000Z Updated: 2026-10-05T10:01:04.308Z Canonical: https://realtimeavatar.ai/blog/integrating-the-ai-talking-avatar-api-principles-of-realtime-embodiment Markdown: [en](https://realtimeavatar.ai/blog/integrating-the-ai-talking-avatar-api-principles-of-realtime-embodiment.md) # Integrating the AI Talking Avatar API: Principles of Realtime Embodiment > Explore the technical foundations and practical integration of an AI talking avatar API. Understand the session architecture, timing measurements and usage costs behind live conversations. - Published: 2026-08-23 (updated 2026-09-05) - Tags: api, engineering, product - Canonical: https://realtimeavatar.ai/blog/integrating-the-ai-talking-avatar-api-principles-of-realtime-embodiment - Markdown: [en](https://realtimeavatar.ai/blog/integrating-the-ai-talking-avatar-api-principles-of-realtime-embodiment.md) The digital frontier has shifted. Where once a talking head was merely a pre-rendered video, the demand now points toward something more profound: real-time, responsive presence. This isn't just about delivering content; it's about enabling interaction, building connection, and providing a dynamic conduit for AI agents to engage with the world. The core technology enabling this shift is the AI talking avatar API, a sophisticated interface that transforms text or audio into live, embodied characters with astonishing speed and fidelity. The underlying promise is clear: an avatar that not only speaks but actively participates, responding to prompts and engaging users with natural facial expressions and lip-sync. This goes far beyond the static or delayed experiences of the past, opening up new paradigms for customer support, education, gaming, and personal companionship. ## The Mechanics of a Realtime AI Talking Avatar API At its heart, a modern AI talking avatar API leverages sophisticated generative AI models to create and animate a digital persona. The process begins with an avatar, which can be instantiated from a single image or a short video. This initial input defines the visual identity, providing the base on which all subsequent animations will be layered. The goal is not just a visual likeness, but a dynamic canvas for expression. Critical to the 'realtime' aspect is the rapid processing of incoming audio or text. When an AI agent generates a response, that output is fed to the avatar API. The system then performs several complex tasks concurrently: generating accurate lip-sync animation (audio-clocked to the syllable), subtle head movements, and facial expressions that convey appropriate emotion. Evaluate startup, spoken-turn latency and audio/video synchronization separately. A first-frame measurement alone does not establish how quickly a spoken question receives a reply. ### Orchestrating Presence: The SDK and the Spec The orchestration of this complex process typically involves two integration paths. For AI agents, the whole surface is published as an OpenAPI specification and mirrored as plain markdown at `llms.txt`, so an agent reads the contract and calls the same endpoints a developer would. For developers, interacting with this powerful backend is simplified through a hand-built, zero-dependency TypeScript SDK. The underlying API is also published as an OpenAPI specification. This `realtime-avatar` package provides a clear, type-safe interface, abstracting away the underlying realtime transport complexities. - **Avatar Creation:** One frontal portrait; the platform generates every frame of video from it. - **Audio-Clocked Video:** Precision lip-sync for natural speech. - **Measured response time:** Report the observation points, sample count and failures for the conversation you tested. - **Typed TypeScript SDK:** `realtime-avatar` for robust, efficient integration. - **Agent-readable surface:** the full API as OpenAPI, plus markdown mirrors at `llms.txt`. ## Architecting for Dynamic Interaction Building applications with an AI talking avatar API demands an architecture that prioritizes low latency and fluid data exchange. The `realtime-avatar` SDK, hand-built and zero-dependency, offers predictable interfaces and strong typing, which significantly reduces development time and error surface. This allows developers to focus on the conversational logic and user experience, rather than wrestling with data serialization or network protocols. Consider a customer support scenario: a user types a query, your agent processes it wherever it already runs, and a response is routed to the avatar API. The avatar then speaks the response, appearing to look directly at the user, with nuanced expressions that mirror the sentiment of the message. This two-way channel of communication and embodiment is what differentiates a true real-time avatar from a simple video playback system. It's the difference between watching a movie and having a conversation. ## Practical Economics and Deployment Adopting an AI talking avatar API also involves understanding its operational economics. Realtime avatar platforms typically pair subscription tiers with usage-based overage; here, overage beyond a plan's included minutes is anchored around $5 per hour of real-time usage. This structure allows for scalable deployment, where costs directly align with demand. For example, the Developer plan includes 600 minutes (10 hours) per month for $24, with overage rates ranging from $0.08–$0.095 per minute depending on the plan. Avatar creation, beyond a few included instances, is $3 per avatar, its generated resting loop included. These clear, predictable costs are crucial for planning and budgeting in dynamic AI deployments. ## Shipping Your Vision with an AI Talking Avatar API Bringing an AI talking avatar into your application is a process of integration and iteration, and the shape of that integration follows from one constraint: an API key that can start a paid call must never reach a browser. So an app is two halves. Your server mounts a connect endpoint that holds the key and decides who may call and what the character knows; your page renders the call and never sees the key. The SDK ships both halves. The server half is one function. It answers the handful of paths the client needs under a single prefix, and it is where authorization and the per-call policy live: ```tsx // app/api/realtime-avatar/[...path]/route.ts import { createRealtimeAvatarRoute } from "realtime-avatar/nextjs"; export const { GET, POST } = createRealtimeAvatarRoute({ apiKey: process.env.REALTIME_AVATAR_API_KEY!, // Who may do this. "connect" is the only operation that costs money to start. authorize: async ({ request, operation }) => { const user = await currentUser(request); if (!user) return new Response("Unauthorized", { status: 401 }); if (operation === "connect" && !(await hasCredits(user))) { return Response.json({ code: "insufficient_credits" }, { status: 402 }); } }, // What the character knows for THIS call. Whatever the browser sent is discarded. session: async ({ avatarId }) => ({ instructions: await personaFor(avatarId), maxSeconds: 600, }), }); ``` The client half is one component pointed at that endpoint. It handles the queue, the reconnects, the idle timer, and the video surface — there are no frame or audio-chunk callbacks to wire up, because the transport is not yours to manage: ```tsx "use client"; import { AvatarCall, createProxyClient } from "realtime-avatar/react"; const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" }); export function Call({ avatarId }: { avatarId: string }) { return ( showEndScreen(reason)} > {(call) => call.status === "waiting" ? In line: {call.queuePosition} : null } ); } ``` Speaking is not a method on a socket; it is a method on the live call. The render prop hands you a handle — `say()` to give her something to respond to, `sayAndEnd()` for the one line that must come out exactly as written, `keepAlive()` and `end()` — and `status` reports five states you write your own copy for. From this foundation, you can layer in your AI agent logic, integrate with user input mechanisms, and refine the persona. The studio interface at `/studio` provides a visual environment for creating and managing your resident cast of characters, allowing for rapid prototyping and iteration before full deployment. The goal isn't just to display an avatar, but to empower an intelligent agent with a face, a voice, and a presence that resonates. This is the new standard for digital interaction, and the AI talking avatar API is your conduit to building it. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Decoding Realtime Presence: An Engineer's Guide to the AI Avatar Video API Explore the core mechanics and practical applications of an AI avatar video API\. Understand how real\-time, interactive video streams empower dynamic user experiences, from customer support to immersive gaming\. Published: 2026-08-22T00:00:00.000Z Updated: 2026-10-05T10:01:13.369Z Canonical: https://realtimeavatar.ai/blog/decoding-realtime-presence-an-engineers-guide-to-the-ai-avatar-video-api Markdown: [en](https://realtimeavatar.ai/blog/decoding-realtime-presence-an-engineers-guide-to-the-ai-avatar-video-api.md) # Decoding Realtime Presence: An Engineer's Guide to the AI Avatar Video API > Explore the core mechanics and practical applications of an AI avatar video API. Understand how real-time, interactive video streams empower dynamic user experiences, from customer support to immersive gaming. - Published: 2026-08-22 (updated 2026-09-05) - Tags: engineering, api, livestream - Canonical: https://realtimeavatar.ai/blog/decoding-realtime-presence-an-engineers-guide-to-the-ai-avatar-video-api - Markdown: [en](https://realtimeavatar.ai/blog/decoding-realtime-presence-an-engineers-guide-to-the-ai-avatar-video-api.md) The shift from static images to dynamic, interactive digital presences is fundamentally changing how we build applications. The advent of the AI avatar video API marks a significant frontier in this evolution, enabling a new class of interactive experiences that demand both visual fidelity and instantaneous responsiveness. ## What Defines an AI Avatar Video API? At its core, an AI avatar video API delivers a live, streaming video feed of an AI-driven character capable of real-time interaction. This is distinct from pre-rendered video, which, while useful for many applications, lacks the spontaneity and adaptive quality of a true real-time stream. The critical differentiator is the ability for the avatar to react, speak, and emote instantaneously based on live input, be it voice, text, or even environmental data. Measure startup from the user's click to a displayed remote frame, and spoken response time from the end of an utterance to audible output. Our [measurement guide](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) defines these separately; this article does not report a measured latency distribution. The 'video' aspect here is paramount. It’s not merely an animation sequence; it's a dynamically generated stream, with the avatar's lips precisely audio-clocked to the syllable of its spoken output. This level of synchronization is essential for maintaining believability and reducing cognitive load for the user. These avatars can be created efficiently, often from a single image or video, offering significant flexibility in design without extensive 3D modeling. ## Architecting Real-time Embodiment Building with an AI avatar video API means tapping into a sophisticated backend infrastructure. For AI agents, the same API is published as OpenAPI and mirrored as plain markdown at `llms.txt` — an agent reads the contract and calls the same endpoints a developer does, not a separate rendering engine. For developers, this underlying complexity is largely abstracted away by a robust, typed TypeScript SDK named `realtime-avatar`. Hand-built and zero-dependency — with the API itself published as an OpenAPI specification — this SDK ensures type safety and a predictable development experience, streamlining the integration process into any modern web or Node.js application. ```tsx import { AvatarCall, createProxyClient } from 'realtime-avatar/react'; // The browser client talks to your mint proxy — your API key never reaches the page. const client = createProxyClient({ proxyUrl: '/api/realtime-avatar' }); export function Call({ avatarId }: { avatarId: string }) { return ; } ``` ### The Mechanics of Interaction Consider a typical interaction loop, such as a customer support scenario: A user speaks, their voice is transcribed, processed by an AI agent, and the response is then vocalized and visually embodied by the avatar. This entire loop, from input to expressive output, happens in near real-time. The video stream adapts dynamically, rendering facial expressions, head movements, and gestures that complement the spoken words, creating a much richer interaction than voice alone. This level of immediate feedback is what transforms a simple chatbot into a compelling digital presence. - **Observed startup time:** Measure the wait to the first displayed remote frame, including queue time and failed attempts. - **Audio-clocked lip-sync:** Eliminates distracting latency artifacts between audio and visual speech, enhancing realism. - **Usage-based pricing:** Plans start at $9/month for 120 minutes, with overage anchored at about $5 per hour ($0.08–$0.095/minute depending on the plan). Avatar creation costs $3 per avatar beyond a plan's included count. - **High-fidelity visual output:** Ensures a professional and engaging presence, vital for brand representation and immersive experiences. ## Shipping Your Interactive Avatar Application Integrating an AI avatar video API isn't merely about embedding a video player. It's about engineering a new layer of user experience that extends the capabilities of your application. Start by understanding the core loop: user input → agent processing → avatar output. Our TypeScript SDK provides the necessary hooks to manage connections, send prompts, and receive the avatar's real-time video and audio streams with confidence. 1. **Select or Create Your Avatar:** Begin by choosing from our resident cast of characters available in the /studio, or quickly create a custom avatar from a single portrait. 2. **Integrate the SDK:** Utilize the `realtime-avatar` TypeScript SDK to establish a secure and efficient connection to the avatar service within your application. 3. **Implement Input/Output Streams:** Capture user input (e.g., microphone audio, text from a chat interface) and pipe it directly to the SDK. Concurrently, receive and display the real-time video and audio streams emitted by the avatar. 4. **Manage Agent Logic:** Connect your chosen AI agent (whether a custom large language model or an existing service) to handle the conversational flow, feeding its generated responses back through the avatar API for embodiment. 5. **Test and Optimize:** Rigorously test your application for network latency, audio quality, and overall responsiveness to ensure a smooth, natural, and highly engaging interaction for your end-users. The objective is to build an application where the avatar isn't just a video player, but a genuine point of presence. Whether for enhancing customer support (as embodied by Ada Kinetic), delivering dynamic corporate training (like Duke Marrow), or creating immersive NPCs in games (such as Captain Maru), the underlying foundation remains consistent: a reliable, low-latency AI avatar video API that transforms code into compelling character. Focus on optimizing the user's journey and how the avatar intrinsically enhances it; the technical implementation will then serve this purpose effectively. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Live avatars in 2026: a field guide to the research How real\-time avatars became possible: the distillation recipe behind streaming video generation, the listening\-head turn, full\-duplex voice, and what the 2026 literature says is still hard\. Published: 2026-08-21T00:00:00.000Z Updated: 2026-08-21T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/live-avatars-2026-research-field-guide Markdown: [en](https://realtimeavatar.ai/blog/live-avatars-2026-research-field-guide.md) # Live avatars in 2026: a field guide to the research > How real-time avatars became possible: the distillation recipe behind streaming video generation, the listening-head turn, full-duplex voice, and what the 2026 literature says is still hard. - Published: 2026-08-21 - Tags: research, engineering, agents - Canonical: https://realtimeavatar.ai/blog/live-avatars-2026-research-field-guide - Markdown: [en](https://realtimeavatar.ai/blog/live-avatars-2026-research-field-guide.md) Two years ago, a talking head was something you rendered. You submitted audio, you waited, you got a clip. In 2026 the interesting systems are the ones you *interrupt*: characters that speak, listen while they speak, and react on a clock measured in milliseconds. That shift did not come from one breakthrough. It came from one recipe, a toolbox borrowed wholesale from LLM inference, and a quiet agreement across a dozen labs about what the remaining hard problems are. This is a field guide to that literature — what got solved, by whom, and what is still open. ## The recipe: distill the offline model into a live one Video diffusion models are bidirectional: every frame attends to every other, which is wonderful for quality and fatal for streaming. The recipe that unlocked live video is to distill that offline teacher into a *causal* student that generates frame-by-frame with a KV cache, in a handful of denoising steps. [CausVid](https://arxiv.org/abs/2412.07772) (MIT/Adobe, late 2024) established the template, and [Self Forcing](https://arxiv.org/abs/2506.08009) (NeurIPS 2025) fixed its train/test gap by training the student on its own rollouts — the codebase most 2026 streaming systems descend from. [TalkingMachines](https://arxiv.org/abs/2506.03099) (Character.AI) was the early proof the recipe survives contact with an 18B audio-driven avatar model. What happened next reads like the history of LLM serving, replayed at 30 frames a second. KV caches, attention sinks, rolling windows, RoPE re-indexing — the whole StreamingLLM toolkit colonized video generation. [LongLive](https://arxiv.org/abs/2509.22622) (NVIDIA) pins the first frame's KV as a permanent sink and re-caches at prompt switches to stream minutes of video at 20.7 FPS on one H100. [Rolling Forcing](https://arxiv.org/abs/2509.25161) (Tencent) denoises a rolling window jointly and keeps the same sink anchor. [Self-Forcing++](https://arxiv.org/abs/2510.02283) stretched few-step students to four-minute streams — fifty times the teacher's horizon — by letting the teacher correct the student's own long rollouts. One caution against tidy narratives: distillation is not the only road. [Seaweed APT2](https://arxiv.org/abs/2506.09350) (ByteDance) gets to real-time with adversarial post-training instead — one network evaluation per frame, 24 FPS on a single H100 — and currently owns the interactive world-model corner. Two recipes, one destination. ## Avatars: from lip-sync to performance, and from speaking to listening The avatar literature split into two lanes. The offline lane chases cinema: [OmniHuman-1.5](https://arxiv.org/abs/2508.19209) puts a multimodal LLM in front of the renderer to plan emotion, gesture, and camera before a DiT draws a frame; [Wan-S2V](https://arxiv.org/abs/2508.18621) and [HunyuanVideo-Avatar](https://arxiv.org/abs/2505.20156) push film-grade, multi-character, full-body performance. None of it is live — and it does not need to be. These are the teachers. The live lane distills them. Alibaba's [Live Avatar](https://arxiv.org/abs/2512.04677) (ECCV 2026) is the scale-up pole: a 14B model split across five H800s — one GPU per denoising timestep — streaming at 45 FPS for over 10,000 seconds, with a rolling sink frame recalibrating identity against a cached reference. JD's [JoyStreamer-Flash](https://arxiv.org/abs/2512.11423) is the single-GPU counterpoint at 16 FPS; [SoulX-FlashHead](https://arxiv.org/abs/2602.07449) squeezes three concurrent real-time streams from one RTX 4090 with a 1.3B model; Kuaishou's [AvatarForcing](https://arxiv.org/abs/2603.14331) got the denoising loop down to a single step at 34 ms per frame. Every one of these papers independently converged on some variant of the same anti-drift toolkit: a pinned reference anchor plus distillation that teaches the student to recover from its own mistakes. When a dozen groups reinvent the same fix, you are looking at the actual shape of the problem. The more interesting turn is behavioral. A conversation partner is mostly *not speaking* — it is listening, nodding, reacting. The dyadic lineage that runs from INFP through [ARIG](https://arxiv.org/abs/2507.00472) to KAIST's [Avatar Forcing](https://arxiv.org/abs/2601.00664) (CVPR 2026) generates exactly that: heads that consume the user's live audio and motion and produce reactions at ~500 ms, trained without labeled reaction data. Meta's [Seamless Interaction](https://arxiv.org/abs/2506.22554) release — four thousand hours of in-person, two-person, full-body conversation — is the data substrate this whole lane was waiting for. And there is an economics counter-lane worth naming: one-shot Gaussian-splat heads like [LAM](https://arxiv.org/abs/2502.17796) render on the user's own phone or browser. A client-rendered avatar costs the server almost nothing; a server-side diffusion avatar costs a GPU. The quality gap is real, and so is the price gap — the field has not finished arguing about where that line should sit. ## The conversation layer went full duplex A live face is only half the problem; the other half is knowing *when* to speak. Here the field runs in two camps converging from opposite ends. Open codec-LM models in the Moshi lineage — most visibly NVIDIA's [PersonaPlex](https://github.com/NVIDIA/personaplex) — own the reflex frontier: ~70 ms speaker-switch latency from a 7B model that runs on 8 GB of VRAM. The proprietary realtime APIs (the GPT-Realtime line, Gemini Live) own reasoning and tool use, and spent 2026 bolting on the reflexes — proactive audio, better barge-in, parallel tool calls with spoken preambles. [DuplexPO](https://arxiv.org/html/2607.07148v1) named the underlying tension honestly: full-duplex models pay an intelligence penalty for their conversational reflexes, and closing that gap is now its own research problem. The benchmarks grew up accordingly. The Full-Duplex-Bench family moved from scoring isolated reflexes — pauses, backchannels, barge-in — to [tool use under real human disfluency](https://arxiv.org/abs/2604.04847), where the universally hardest failure is a mid-utterance change of mind that requires rolling back an API call. And Sierra's [τ-Voice](https://arxiv.org/abs/2603.13686) now measures what matters commercially: verifiable task completion over full-duplex audio. Its headline is the most honest summary of the field's progress — the frontier more than doubled in eight months, and voice agents now retain roughly four-fifths of their text-mode capability. Voice used to be a tax; it is becoming a rounding error. ## What the products claim, and how to read the claims The product landscape tells the same story with marketing attached. Latency went from adjective to number: vendors now publish 180 ms generation figures, sub-600 ms round trips, sub-half-second turns. Read all of these as claims, not comparables — each one measures a different span of the pipeline, and there is still no independent cross-vendor latency benchmark. Differentiation has visibly moved from pixels to behavior: the 2026 pitch is the avatar that listens while you talk, reacts before it answers, and modulates affect — precisely the dyadic-behavior lane the research above just opened. And architecturally, the industry consolidated on the same integration surface we build on: a WebRTC room the avatar joins as a participant, with the agent's brain, tools, and turn logic living in your code rather than the vendor's. ## Still hard, per the literature - **Minute-scale drift.** Sinks and reference anchors beat the worst of it, but color and identity drift over long horizons is still an active thread — 2026's refinements make the anchors adaptive rather than static, which tells you static was not enough. - **The duplex-intelligence trade-off.** The models with the best reflexes reason worst, and vice versa. Nobody has both ends of the curve yet. - **Obeying turn-taking policy.** When [Instruct-FD](https://arxiv.org/html/2607.20460v1) asked systems to follow explicit instructions about when to backchannel or interrupt, adherence ranged from zero to fifty percent. The reflexes exist; steering them does not. - **Tools under overlap.** Calling a function is easy; un-calling it because the speaker changed their mind mid-sentence is the failure mode every tool-use benchmark now agrees on. - **Honest measurement.** Vendor latency numbers span incompatible definitions, and the field's only cross-vendor "avatar benchmark" is vendor-cited. The evaluation gap is the least glamorous open problem and possibly the most consequential. Our reading of all this is simple: the research has converged on exactly the properties a live character needs — stream forever without drifting, listen while speaking, act through tools mid-conversation — and the remaining gaps are the ones you feel in a real call. That is the weather we build under. A full-duplex character you can interrupt, with your tools on the other end of the call, is no longer a research demo; it is an API surface, priced by the second. The papers above are why. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # The Core Mechanics of a Realtime AI Avatar Generator API: From Image to Embodiment Explore how a realtime AI avatar generator API turns a portrait into a conversational character, with session integration, timing measurements and usage costs\. Published: 2026-08-21T00:00:00.000Z Updated: 2026-10-05T10:01:21.909Z Canonical: https://realtimeavatar.ai/blog/the-core-mechanics-of-a-realtime-ai-avatar-generator-api Markdown: [en](https://realtimeavatar.ai/blog/the-core-mechanics-of-a-realtime-ai-avatar-generator-api.md) # The Core Mechanics of a Realtime AI Avatar Generator API: From Image to Embodiment > Explore how a realtime AI avatar generator API turns a portrait into a conversational character, with session integration, timing measurements and usage costs. - Published: 2026-08-21 (updated 2026-09-05) - Tags: engineering, api - Canonical: https://realtimeavatar.ai/blog/the-core-mechanics-of-a-realtime-ai-avatar-generator-api - Markdown: [en](https://realtimeavatar.ai/blog/the-core-mechanics-of-a-realtime-ai-avatar-generator-api.md) The promise of an AI avatar has moved beyond pre-rendered video. What developers truly seek now is presence — a dynamic, interactive digital persona that responds in realtime. This shift demands a sophisticated backend: a robust ai avatar generator api capable of transforming simple inputs into complex, expressive, and live-synced digital characters. It's not merely about creating an avatar; it's about engineering an embodied agent. ## The Genesis of an AI Avatar Generator API: From Still to Spoken At its foundation, a realtime avatar system begins with a source. Our observations indicate that the most effective and accessible method leverages either a single static image or a brief video. This input isn't just a texture map; it's the raw material from which a complex, multi-faceted digital identity is forged. The underlying models parse facial features, expressions, and inherent characteristics, preparing them for animation. Once the visual foundation is established, the critical element for believability emerges: voice and movement synchronization. Unlike traditional animation pipelines, a realtime system processes incoming audio and precisely 'audio-clocks' the avatar's lips and facial movements to the syllable. This isn't just lip-sync; it's a dynamic, contextual interpretation of speech that imbues the avatar with naturalistic fluidity, ensuring that every word spoken by an agent is visually reinforced with authentic embodiment. ## The Engineering of Realtime Presence with an AI Avatar Generator API Startup and conversational response are separate measurements. Record click-to-first-frame and end-of-utterance-to-first-audio under declared conditions. Our [benchmark methodology](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency) and downloadable starter make those distinctions explicit; this article does not supply measured performance results. Supporting this high-performance core is a developer-centric interface. Our hand-built, zero-dependency TypeScript SDK provides a type-safe and predictable interaction layer, and the full API is published as an OpenAPI specification. This allows engineers to integrate avatar functionality with confidence, abstracting away the underlying complexity of video streaming and AI orchestration. For AI agents, the published OpenAPI spec and the markdown mirrors at llms.txt describe the same platform — your agent's logic stays wherever it already runs, decoupled from the visual rendering pipeline. ### Beyond Just Rendering: The Embodied Agent The true power of this architecture lies in its ability to give AI agents a face and a voice. It transforms a text-based interaction or an abstract computational process into a tangible, relatable presence. This embodiment is crucial for applications ranging from AI companions and interactive NPCs in games to corporate training modules and sophisticated customer support avatars. The visual consistency and realtime responsiveness foster a deeper sense of connection and understanding, elevating the user experience beyond what text or voice alone can achieve. ## Practical Application: Shipping with a Realtime AI Avatar Generator API For developers, the journey from concept to deployment with a realtime ai avatar generator api is streamlined. The focus shifts from the intricate details of avatar animation and synchronization to the core agent logic and user interaction design. This abstraction is key to rapid iteration and robust deployment. - **Integrate the SDK:** Begin by installing the typed TypeScript SDK: `npm install realtime-avatar`. This provides immediate access to connection protocols and avatar control methods. - **Character Selection & Creation:** Utilize the /studio to select from a resident cast of characters or create a new avatar from a single portrait. Avatar creation is streamlined, with initial allocations often included and subsequent creations priced affordably ($3 per avatar beyond your plan's included count). - **Connect Agent Logic:** Hook your AI agent's textual or audio output directly into the avatar API. The system handles the realtime audio-to-embodiment pipeline, freeing your agent to focus solely on its intelligence and conversational flow. - **Iterate and Scale:** Leverage usage-based pricing, with overage anchored at approximately $5 per hour of realtime interaction (e.g., the $24/month Developer plan includes 600 minutes, with overage rates of $0.08–$0.095 per minute depending on your plan). This model ensures that compute resources scale directly with user engagement, making it economically viable to prototype and then expand. By understanding these mechanics and utilizing the provided tools, developers can quickly bring interactive, embodied AI experiences to life. The goal is to move past static imagery into dynamic, responsive interactions, transforming any application that benefits from a face and a voice into a truly engaging experience. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # LiveAvatar API billing: credits, conversation modes, and total cost Understand LiveAvatar FULL versus LITE credit units, then compare subscription allowances and complete conversation costs with Realtime Avatar\. Published: 2026-08-20T00:00:00.000Z Updated: 2026-09-05T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/demystifying-heygen-live-avatar-api-pricing Markdown: [en](https://realtimeavatar.ai/blog/demystifying-heygen-live-avatar-api-pricing.md) # LiveAvatar API billing: credits, conversation modes, and total cost > Understand LiveAvatar FULL versus LITE credit units, then compare subscription allowances and complete conversation costs with Realtime Avatar. - Published: 2026-08-20 (updated 2026-09-05) - Tags: api, pricing - Canonical: https://realtimeavatar.ai/blog/demystifying-heygen-live-avatar-api-pricing - Markdown: [en](https://realtimeavatar.ai/blog/demystifying-heygen-live-avatar-api-pricing.md) When researching HeyGen live avatar API pricing, distinguish generated video from a LiveAvatar conversation. LiveAvatar supports interactive sessions. Treating that product as a sequence of ordinary video-generation jobs misstates what its API does. ## FULL and LITE use different units LiveAvatar documents a managed FULL mode and an avatar-only LITE mode. FULL includes speech recognition, an LLM, TTS and WebRTC management. LITE supplies the avatar video while you provide the conversational stack and manage WebRTC. Compare the architecture you intend to use in the [official mode overview](https://docs.liveavatar.com/). Its [billing documentation](https://docs.liveavatar.com/docs/faq/credits), checked on 5 September 2026, specifies two credits per minute for FULL and one for LITE. A credit balance therefore buys different session durations in the two modes. Divide included credits by the applicable rate before comparing minute allowances. ## Check what happens after the allowance LiveAvatar's billing guide says a session needs sufficient credits for at least a minute to start. It describes per-minute session deductions and an overage setting: when overage is disabled, exhausted credits stop sessions and prevent new ones. Confirm the current subscription, rate unit and enabled settings in your account before estimating a bill. A complete worksheet includes the base subscription, included minutes, overage, simultaneous sessions, maximum session length and avatar-creation charges. For a rendering mode, add the speech recognition, language model, synthesis and infrastructure you supply. ## Realtime Avatar's published rates Our live-call product includes speech recognition, the language model, voice and avatar rendering. Published self-serve overage is $0.08–$0.095 per minute, or $4.80–$5.70 per hour. The Developer plan is $24 per month with 600 included call minutes and $0.085 per minute overage. Photo-avatar creation beyond the included count has a published price of $1. Included credits are shared with generation uses, so spending them on avatar or asset generation reduces what remains for calls. The [pricing calculation guide](https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing) separates subscription cost, allowances and marginal overage, with a [calculator](https://realtimeavatar.ai/tools/avatar-cost-calculator) for your own usage assumptions. ## Compare an actual conversation Cost is one input. Test startup, a fixed spoken question, an interruption and explicit session ending on the same browser and network. Keep failed and queued attempts in the report. This article makes no measured claim that one provider is faster than another. Use our [developer evaluation checklist](https://realtimeavatar.ai/blog/liveavatar-tavus-alternative-developer-checklist) and [timing methodology](https://realtimeavatar.ai/blog/how-to-benchmark-realtime-avatar-startup-and-turn-latency). The [Next.js starter](https://realtimeavatar.ai/docs/nextjs) gives you a runnable Realtime Avatar integration to include in that trial. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Realtime avatar API pricing: included minutes, overage, and total cost Calculate live\-avatar API costs from monthly minutes and concurrency\. Catalog\-derived plan prices, worked examples, and an interactive calculator with explicit assumptions\. Published: 2026-06-11T00:00:00.000Z Updated: 2026-10-13T13:41:37.795Z Canonical: https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing Markdown: [en](https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing.md) # Realtime avatar API pricing: included minutes, overage, and total cost > Calculate live-avatar API costs from monthly minutes and concurrency. Catalog-derived plan prices, worked examples, and an interactive calculator with explicit assumptions. - Published: 2026-06-11 (updated 2026-09-05) - Tags: api, pricing - Canonical: https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing - Markdown: [en](https://realtimeavatar.ai/blog/realtime-ai-avatar-api-pricing.md) A realtime avatar budget has three separate questions: what you pay to start the month, how much usage that includes, and what the next minute costs. A headline per-minute figure answers only one of them. Here is the calculation for Realtime Avatar, with an [interactive cost calculator](https://realtimeavatar.ai/tools/avatar-cost-calculator) you can adjust to your application. Our published self-serve overage is **$0.08–$0.095 per minute**, or $4.80–$5.70 per hour. The live-call product includes speech recognition, the language model, voice and avatar rendering. Your own hosting and external tools remain separate. ## Current plans, with the units kept separate | Plan | Monthly subscription | Included call minutes | Overage per minute | Maximum concurrent calls | | --- | --- | --- | --- | --- | | Starter | $9 | 120 | $0.095 | 3 | | Developer | $24 | 600 | $0.085 | 10 | | Studio | $119 | 3,000 | $0.08 | 20 | These figures come from the same billing catalog as the [pricing page](https://realtimeavatar.ai/pricing). Sandbox is free and provides up to 8 minutes if all of its credits go to calls; it has no automatic overage. Scale is quoted individually. ## Included-minute cost is different from overage Divide a subscription by its allowance only when you are modeling full use of that allowance. For example, the Developer subscription buys an allowance whether you use it all or not. The effective rate gets higher when the allowance goes unused. Overage is the marginal rate after the allowance, not the rate you pay for every minute from zero. ```text monthly estimate = subscription + max(0, call minutes - included minutes) × overage rate ``` This model assumes monthly billing, one simultaneous call, and all included credits spent on calls. It excludes taxes, annual discounts, negotiated rates, optional credit top-ups, avatar creation, other generation, and external hosting. It does not inspect your account balance or predict a bill. ## Worked usage scenarios These are calculated scenarios, not measurements of live session performance. The lowest eligible self-serve plan is selected using the formula above. The inputs and results are reproducible in the calculator. | Monthly call minutes | Lowest modeled plan | Subscription | Overage | Estimated total | | --- | --- | --- | --- | --- | | 120 | Starter | $9 | $0 | $9 | | 600 | Developer | $24 | $0 | $24 | | 3,000 | Studio | $119 | $0 | $119 | | 10,000 | Studio | $119 | $560 | $679 | Now change peak concurrency to three or ten. A cheaper plan with too few simultaneous sessions becomes ineligible even when it includes enough monthly minutes. A plan limit describes eligibility; it does not guarantee that every capacity request will succeed. ## Creation and idle time belong in the budget Creating a photo avatar beyond the plan's included count has a published price of $1. Portrait generation and idle-clip generation are additional uses of the shared credit wallet. Credits spent on generation are no longer available for calls. Using a ready example avatar is the shortest way to evaluate the call itself. Budget for connected time, including pauses. Do not estimate a ten-minute conversation as two minutes merely because the avatar spoke for two minutes. End the call explicitly when the user is finished and examine the session usage record. ## Comparing LiveAvatar, Tavus, or a renderer Use the same worksheet for each vendor: base subscription, included minutes, overage, concurrency, avatar creation, minimum session charge, and which parts of the voice pipeline you supply. A renderer-only rate needs the cost of speech recognition, an LLM and TTS added before comparison with a complete conversation stack. Check [LiveAvatar pricing](https://www.liveavatar.com/pricing) and [Tavus pricing](https://www.tavus.io/pricing) at the time of your evaluation. Their consumer plans, developer plans and rendering modes are different offers. We avoid turning an unverified competitor rate into a fixed calculator input. Our [architecture comparison](https://realtimeavatar.ai/blog/renderer-or-full-stack-realtime-avatar-apis-compared) explains the differences, and the [developer evaluation checklist](https://realtimeavatar.ai/blog/liveavatar-tavus-alternative-developer-checklist) gives you a repeatable prototype to compare. ## Try the calculation, then the product [Enter your minutes and concurrency](https://realtimeavatar.ai/tools/avatar-cost-calculator), then follow the [Next.js guide](https://realtimeavatar.ai/docs/nextjs) to run a call on an existing character. Compare the session's recorded usage with your assumptions before extrapolating a production budget. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Agentic localization: how this blog stays multilingual without a translation team Every post on this blog can ship in English, Chinese, and Japanese\. The translators are LLM agents; the editors are pull requests\. Here's the pipeline\. Published: 2026-06-10T00:00:00.000Z Updated: 2026-06-10T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/agentic-localization-pipeline Markdown: [en](https://realtimeavatar.ai/blog/agentic-localization-pipeline.md) · [zh](https://realtimeavatar.ai/blog/agentic-localization-pipeline/zh.md) # Agentic localization: how this blog stays multilingual without a translation team > Every post on this blog can ship in English, Chinese, and Japanese. The translators are LLM agents; the editors are pull requests. Here's the pipeline. - Published: 2026-06-10 - Tags: engineering, agents, i18n - Canonical: https://realtimeavatar.ai/blog/agentic-localization-pipeline - Markdown: [en](https://realtimeavatar.ai/blog/agentic-localization-pipeline.md) · [zh](https://realtimeavatar.ai/blog/agentic-localization-pipeline/zh.md) You are reading a post that knows how to translate itself. Every article here is a typed TypeScript module with an English source and optional locale variants — and the locale variants are produced by an LLM agent on a schedule, reviewed by a human, and merged like any other code change. This post (fittingly) ships with a Chinese translation produced by exactly that pipeline. ## The content model: locales as fields, not forks Each post exports one object with a `translations` map: `en` is required, `zh` and `ja` are optional. Title, description, and body travel together per locale, so a translation can never drift structurally from its source — the type checker won't allow it. The post's date, tags, and artwork are shared across locales, so there is exactly one source of truth for everything that isn't language. ## The pipeline: diff, translate, review Three steps, all boring on purpose: 1. **Detect staleness.** A scheduled job lists posts where the English source changed after a locale variant last did (git history makes this a one-liner), plus posts missing a target locale entirely. 2. **Translate with full context.** The agent reads the entire post — not strings in isolation — and writes the locale variant in place. It carries a style brief per language: simplified Chinese with mainland product vocabulary, Japanese in です/ます form, keep code identifiers and product names untranslated. 3. **Gate on review.** The agent opens a pull request. A human (or a second reviewer agent with veto power) reads the diff. Nothing reaches production without a merge. Because translations are code, every guarantee we rely on for code applies to them: CI type-checks the structure, the build fails on a missing field, and `git revert` un-ships a bad translation in seconds. ## Why not machine-translate at the edge? Runtime translation APIs are tempting — no files, no PRs. But they translate sentences, not arguments. An LLM agent that reads the whole article preserves the metaphors, the section structure, and the terminology of earlier posts. It can also *decline*: when a pun doesn't survive the trip, the agent rewrites the sentence rather than embarrassing us literally. And because the output is frozen in git, readers in every language see reviewed prose, not a dice roll per page load. ## The same trick works on a CMS If your content lives in Payload or another headless CMS instead of git, the shape survives: an `afterChange` hook detects source-locale edits, queues a translation task, and the agent writes target locales back as drafts via the API for editorial approval. The principle is identical — **agents propose, humans approve, every change is a reviewable diff**. We wrote up the full CMS-vs-git evaluation in a separate post if you're choosing a stack. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Looking for a HeyGen alternative? Decide what you're actually replacing first A developer's guide to choosing a HeyGen alternative in 2026: video\-generation platforms vs realtime avatar APIs, what each costs per minute, and when each one wins\. Published: 2026-06-10T00:00:00.000Z Updated: 2026-10-13T13:42:02.157Z Canonical: https://realtimeavatar.ai/blog/heygen-alternative-for-realtime-avatars Markdown: [en](https://realtimeavatar.ai/blog/heygen-alternative-for-realtime-avatars.md) # Looking for a HeyGen alternative? Decide what you're actually replacing first > A developer's guide to choosing a HeyGen alternative in 2026: video-generation platforms vs realtime avatar APIs, what each costs per minute, and when each one wins. - Published: 2026-06-10 (updated 2026-09-03) - Tags: product, api - Canonical: https://realtimeavatar.ai/blog/heygen-alternative-for-realtime-avatars - Markdown: [en](https://realtimeavatar.ai/blog/heygen-alternative-for-realtime-avatars.md) "HeyGen alternative" is two different searches wearing the same words. Some people want cheaper or different *video generation* — rendered clips with a presenter avatar. Others discovered mid-build that what they actually need is a *live* avatar: one that listens and answers their users in realtime. The alternatives list is completely different depending on which one you are, so start there. ## If you need generated videos Stay in the video-generation aisle: Synthesia (strongest for enterprise training content, no realtime offering), D-ID (cheaper, solid photo-animation lineage), or HeyGen itself — whose Avatar IV/V quality and translation/dubbing pipeline remain genuinely hard to beat. If your complaint with HeyGen is only price, do the math in API terms first: $0.05/sec means a 60-second photo-avatar video costs about $3 — most alternatives land in the same order of magnitude once quality is comparable. ## If you need a HeyGen alternative for realtime avatars HeyGen's own realtime product was spun out into LiveAvatar ($19–475/mo in credits; in FULL mode a credit buys 30 seconds of streaming — roughly $0.20–0.24/min with their managed AI stack). D-ID sells streaming minutes on its API plans at an effective $0.50–0.56/min. And this is the category we built TIC Realtime Avatar for, so here is the honest pitch: - **About $5/hour of live avatar, full stack included:** $24/mo includes 600 minutes and $119/mo includes 3,000 minutes — ten and fifty hours on air — with overage easing to $0.08/min ($4.80/hr) as volume rises; past that, Scale is quoted. The LLM, voice, and audio-clocked rendering are in the price — no separate AI stack to wire up. - **One image becomes a character:** persona, voice, and face are a designed object that persists across sessions — built for companions, trainers, hosts, and NPCs rather than presenters. - **Developer surface first:** typed TypeScript SDK generated from the OpenAPI spec, LiveKit-first SDK examples, and agent-readable docs so AI coding tools can operate the platform, and a free 8-minute sandbox, granted once per account, you can hit tonight. ## Where we are not the answer We don't render offline videos, we don't have a 500-avatar stock library, and we don't do translation dubbing. If those are the job, HeyGen or Synthesia will serve you better — and a realtime API bolted onto a video workflow helps no one. ## How to run the evaluation 1. Write down session length, concurrency, and your latency budget (time to first frame matters more than resolution for live use). 2. Price a month honestly: included minutes ÷ real usage, then overage — credits convert differently per platform, so normalize everything to $/minute. 3. Prototype the same conversation on each candidate. A live avatar's quality is measured in the half-second after the user stops talking. Our sandbox exists for exactly that third step: register a character from a single portrait, hold a conversation in the studio, and read your own latency numbers before a dollar changes hands. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # HeyGen API vs LiveAvatar Pricing: What a Live Conversation Costs Compare HeyGen video generation with LiveAvatar's live conversation plans, credit conversion and RTA pricing using current official sources and clear limits\. Published: 2026-06-10T00:00:00.000Z Updated: 2026-10-13T13:44:00.837Z Canonical: https://realtimeavatar.ai/blog/heygen-api-pricing-explained Markdown: [en](https://realtimeavatar.ai/blog/heygen-api-pricing-explained.md) HeyGen video generation and LiveAvatar live conversations are different products with different billing. For a live voice-and-video application, compare LiveAvatar's FULL or Avatar Only mode rather than a generated-video price. Checked on October 12, 2026, LiveAvatar lists Essential at $99 per month with 1,100 plan credits and Business at $475 with 6,000. FULL uses two credits per session minute; Avatar Only uses one, excluding the conversation services you bring yourself. Product scope matters as much as the headline price. ## Which HeyGen product does your application need? A generated-video API returns an asset that can be rendered, downloaded or published. A live avatar service maintains a session in which a person can speak and receive responses. Those workloads have different cost drivers: generated length and resolution on one side; connection time, speech, model inference, concurrency and idle time on the other. Start by deciding which output you need before comparing a credit pack or a per-minute figure. HeyGen's LiveAvatar documentation describes a separate real-time product and billing system. Do not assume an editor subscription, generated-video allowance or existing avatar asset automatically transfers. Check the relevant product's current documentation and agreement for compatibility. This guide focuses its numerical comparison on the live-session prices verified below; it does not repeat older HeyGen rendering or editor prices as if they were current quotes. ## LiveAvatar plan credits are not the same as minutes - Essential: $99 per month, 1,100 plan credits, one custom 720p avatar, up to 20 minutes per session, and advertised unlimited concurrency. - Business: $475 per month, 6,000 plan credits, one custom 1080p avatar, up to 60 minutes per session, and advertised unlimited concurrency. - The billing documentation additionally displays a +10-credit allowance beside the paid-plan quantities. The calculations below use the advertised plan credits, without treating that extra allowance as a recurring pricing assumption. - FULL mode consumes two credits per minute. Avatar Only, also described as LITE, consumes one credit per minute. A lower Avatar Only price does not include the cost of your own conversation stack. If every included credit is used for FULL sessions, Essential's plan credits represent 550 minutes and Business's represent 3,000. Dividing the subscription by those minutes gives about $0.18 and $0.158 per included minute respectively. In Avatar Only mode, the same plan quantities represent 1,100 and 6,000 minutes, or about $0.09 and $0.079 per included minute before your other providers. These are full-utilization calculations, not guaranteed effective prices: unused credits raise the cost of each minute you actually use. The current billing documentation lists Essential overage at $0.095 per credit and Business at $0.09. That makes FULL overage $0.19 or $0.18 per minute; Avatar Only overage is $0.095 or $0.09. Keep included-minute arithmetic separate from overage. An application with modest traffic may spend most of its budget on the base plan, while a high-use integration may pay predominantly for overage. - [LiveAvatar official credit rates and subscriptions](https://docs.liveavatar.com/docs/faq/credits) - [LiveAvatar current plan features](https://www.liveavatar.com/) ## Why the enterprise headline is not your all-in quote LiveAvatar advertises rates down to $0.01 per minute at the top enterprise volume tier. Its site labels this as a streaming rate and enterprise terms are custom. It is not the self-service FULL rate, nor proof that speech recognition, your selected language model, voice, support and reserved capacity will all be supplied for that amount. Ask for a written scope and minimum commitment before using an enterprise headline in a budget. Likewise, advertised unlimited concurrency does not by itself establish a customer-specific reserved-capacity agreement or a measured latency guarantee. Separate admission limits, maximum session duration, actual available capacity and contractual service levels. Evaluate the workload you expect, including cold starts and failed or interrupted sessions, instead of treating the longest feature list as the lowest cost. ## How Realtime Avatar's managed session pricing compares Realtime Avatar's published Developer plan is $24 per month with 600 included realtime minutes and $0.085 per minute overage. Studio is $119 with 3,000 included minutes and $0.08 overage. Fully consuming those call allowances gives approximately $0.04 per included minute for either plan. Actual usage, unused allowance and other credit-consuming operations change the effective price. Consult the current catalog before purchasing; this is a dated comparison, not a permanent cheapest-provider claim. RTA's managed live session includes speech recognition, the language model, speech synthesis and rendered avatar video. Your application hosting and external tools remain separate. Creating a character uses a single portrait image; new creation from a supplied video is not the general public path. The platform generates the resting loop, and additional avatar creation beyond the plan allowance is separately priced. The same credit wallet can pay for character generation and calls, so do not assume every free or included credit will become a conversation minute. The free RTA allowance is 480 credits once per account, equivalent to 8 minutes only if all of it is spent on air. Creating an avatar consumes some of that same balance. This distinction matters when estimating the cost of onboarding a user who creates a character before making a first call. It also prevents a free-trial promise from implying more conversation time than the actual wallet can fund. - [Current RTA plans and allowances](https://realtimeavatar.ai/pricing) - [Create an avatar from a portrait](https://realtimeavatar.ai/docs/video) - [Session limits and client integration](https://realtimeavatar.ai/docs/sessions) ## Compare the integration you will actually deploy If you already operate your own voice and conversation stack, verify the exact input interface instead of assuming all avatar products are interchangeable. An approved external language-model endpoint is not the same as arbitrary audio passthrough. RTA's public SDK does not provide an input for live audio from your own text-to-speech provider; the managed voice pipeline remains part of the product. Unsupported integration requirements can matter more than a few cents in a price table. For a useful evaluation, use the same conversation task and intended number of concurrent users. Record the time until a user actually sees and hears the character, whether interruptions work, how the session ends, and what the usage record bills. Then add model and voice charges not included in the selected mode, idle infrastructure, generation failures, support effort and any contractual minimums. Keep those measured results separate from vendor targets. Choose the product that completes your user's task within an honest cost and permission boundary, not the one with the smallest isolated number. --- # Payload CMS vs\. a git\-based blog: what we chose for an agent\-written site We researched running Payload 3 on Cloudflare Workers \+ D1, weighed it against keeping content in the repo, and picked the stack AI agents are best at maintaining\. Published: 2026-06-10T00:00:00.000Z Updated: 2026-06-10T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/payload-cms-vs-git-based-blog Markdown: [en](https://realtimeavatar.ai/blog/payload-cms-vs-git-based-blog.md) # Payload CMS vs. a git-based blog: what we chose for an agent-written site > We researched running Payload 3 on Cloudflare Workers + D1, weighed it against keeping content in the repo, and picked the stack AI agents are best at maintaining. - Published: 2026-06-10 - Tags: engineering, content - Canonical: https://realtimeavatar.ai/blog/payload-cms-vs-git-based-blog - Markdown: [en](https://realtimeavatar.ai/blog/payload-cms-vs-git-based-blog.md) This blog is written and translated largely by AI agents, with humans reviewing every change. That constraint — **agents as the primary authors** — turns the classic "which CMS?" question on its head. Before building it, we did a proper evaluation of Payload CMS against the boring alternative: content files in the repo. Here is what we found, as of June 2026. ## Payload 3 in 2026: genuinely good, newly complicated Payload 3.x is the current line (3.85 shipped days before this post) and it is Next.js-native: the admin panel and the REST/GraphQL APIs install into a Next.js app. For a team that wants a real editorial UI — drafts, scheduled publishing, non-technical editors — it remains one of the best open-source options. Three findings mattered for us: - **It can run on Cloudflare now.** There is an official `with-cloudflare-d1` template that deploys via OpenNext on Workers, with D1 for the database and R2 for media. But the D1 adapter (`@payloadcms/db-d1-sqlite`) is explicitly beta, the bundle needs the paid Workers plan, and GraphQL is not guaranteed on workerd — REST only. - **Localization is first-class.** Field-level `localized: true`, configurable fallback locales, and a REST API that accepts `?locale=all` — which returns every locale keyed by code, exactly what a translation agent wants for diffing. - **Figma acquired Payload.** The open-source core continues, but Payload Cloud is paused for new deployments. Self-hosting is the only realistic path, and the roadmap carries acquisition-era uncertainty. ## The agentic test: who edits this content, and how? The deciding question wasn't features — it was workflow. An AI agent's native skill is editing files and opening pull requests. Put content behind a CMS API and the agent needs credentials, schema awareness, and a custom review surface. Put content in the repo and the agent uses the exact tooling it already has, and the review gate is the one your team already trusts: code review. > A CMS earns its infrastructure cost the day a non-technical human needs a WYSIWYG admin. Until then, git is the better content database. For an agent-maintained site, git-based content wins on every axis we scored: - **Authoring:** agents write typed TSX/MDX files; the compiler and CI validate structure — no malformed rich-text JSON. - **Review:** every article and every translation is a diff in a PR. - **Rollback and history:** git, for free. - **Infra:** zero. No second deployable, no beta database adapter, no admin panel to secure. ## Where Payload would still win We'd reach for Payload the moment marketing hires a non-engineer editor, or when content needs scheduling, embargoes, or a media library with image renditions. The migration path is clean: our posts are typed objects, and Payload collections are typed schemas — a one-time import script. If you're on that path today, the pipeline that works is: `afterChange` hooks queueing translation tasks into Payload's built-in Jobs Queue, drained by a Cloudflare Cron Trigger hitting `/api/payload-jobs/run`, with translations written back as drafts for human approval. (Don't use `autoRun` on serverless — the docs warn against it for good reason.) ## What we shipped Posts live in `content/blog/posts/` as TypeScript modules with an English source translation and optional locale variants. Agents scout keywords, draft articles, and regenerate stale translations on a schedule; humans merge. The whole "CMS" is a types file and a registry — and it deploys with the rest of the site on Cloudflare Workers, no extra moving parts. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # Synthesia pricing in 2026 — and the alternatives when you need more than rendered video Synthesia's plans, what the API actually includes, and the alternative paths — D\-ID, HeyGen, and realtime avatar APIs — when your use case is interactive rather than rendered\. Published: 2026-06-10T00:00:00.000Z Updated: 2026-10-13T13:41:14.500Z Canonical: https://realtimeavatar.ai/blog/synthesia-pricing-and-alternatives Markdown: [en](https://realtimeavatar.ai/blog/synthesia-pricing-and-alternatives.md) # Synthesia pricing in 2026 — and the alternatives when you need more than rendered video > Synthesia's plans, what the API actually includes, and the alternative paths — D-ID, HeyGen, and realtime avatar APIs — when your use case is interactive rather than rendered. - Published: 2026-06-10 (updated 2026-09-03) - Tags: product, api - Canonical: https://realtimeavatar.ai/blog/synthesia-pricing-and-alternatives - Markdown: [en](https://realtimeavatar.ai/blog/synthesia-pricing-and-alternatives.md) Synthesia is the default name in AI presenter video, and for produced training content it has earned that. But its pricing model — and one structural gap — decide quickly whether it fits your project. Here are the numbers as of June 2026, and the alternatives for the cases it doesn't cover. (Always confirm against the official pricing page; plans move.) ## Synthesia pricing, decoded - **Basic (free):** 10 minutes of video a month, 9 stock avatars, no API - **Starter $29/mo** ($18 annual): 10 min/mo, 125+ avatars, 3 personal avatars, no API - **Creator $89/mo** ($64 annual): 5 min/mo, 180+ avatars — and API access, capped at 360 minutes of video per year, deducted from the plan's own limits - **Enterprise:** custom pricing, unlimited minutes, 240+ avatars Two things developers should notice. First, **there is no standalone, metered API price** — API access is bundled into Creator/Enterprise plan limits, so you can't scale API usage independently of a seat plan. Second, **Synthesia has no realtime conversational avatar**: its "interactivity" is in-video quizzes and links, and its Video Agents are marked "coming soon." Everything it ships today is rendered, not live. ## Synthesia alternatives, by job **For produced video at lower entry cost:** D-ID's studio plans start at $5.9/mo, and HeyGen's Creator plan at $29/mo undercuts Synthesia's per-minute economics for short clips. Synthesia still wins on enterprise features, template depth, and localization breadth for L&D teams. **For developer APIs with real usage-based pricing:** HeyGen's API wallet charges per second of generated video ($0.05/sec for photo avatars); D-ID sells credit plans that include streaming minutes (effectively $0.50–0.56/min for realtime streams). **For live, conversational avatars** — tutors that check understanding out loud, trainers that push back, companions that hold a persona — rendered video is the wrong primitive entirely. That's the category TIC Realtime Avatar occupies: audio-clocked live video with sub-second first frames, priced purely in realtime minutes at about $5/hour on overage ($24/mo for 600 minutes, $119/mo for 3,000 minutes, and a quote past that; free 8-minute sandbox, once per account), with a LiveKit-first TypeScript SDK and agent-readable docs for agent-driven apps. ## The honest decision rule > If a human will watch it, render it with Synthesia or HeyGen. If a human will talk to it, you need a realtime avatar API — and per-minute pricing you can meter. Training is the instructive case: Synthesia excels at producing the course video; it cannot roleplay the difficult customer with your trainee. Plenty of teams end up with both — rendered video for the curriculum, a live character for the rehearsal. ## Trying the live path If your shortlist exists because "interactive" crept into the requirements, prototype the conversation before committing anywhere: register a character from a single image, warm it with one API call, and measure time-to-first-frame against your latency budget. Our sandbox's free credits (480, granted once per account) exist precisely so that test costs nothing. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # The Architecture of Presence: Integrating a Realtime AI Avatar API How to integrate a realtime AI avatar API: the architecture behind sub\-second, audio\-clocked conversational avatars, and what it takes to ship one in your app\. Published: 2026-06-10T00:00:00.000Z Updated: 2026-06-10T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/the-architecture-of-presence-integrating-a-realtime-ai-avatar-api Markdown: [en](https://realtimeavatar.ai/blog/the-architecture-of-presence-integrating-a-realtime-ai-avatar-api.md) # The Architecture of Presence: Integrating a Realtime AI Avatar API > How to integrate a realtime AI avatar API: the architecture behind sub-second, audio-clocked conversational avatars, and what it takes to ship one in your app. - Published: 2026-06-10 - Tags: engineering, api, agents - Canonical: https://realtimeavatar.ai/blog/the-architecture-of-presence-integrating-a-realtime-ai-avatar-api - Markdown: [en](https://realtimeavatar.ai/blog/the-architecture-of-presence-integrating-a-realtime-ai-avatar-api.md) The shift from static text to embodied AI represents a significant pivot in digital interaction. No longer are we merely sending and receiving characters; we are engaging with presence. This transition is underpinned by sophisticated infrastructure, a core component of which is the realtime AI avatar API. Understanding its structure and integration points is paramount for any developer looking to build truly dynamic, conversational applications. ## Beyond Text Prompts: What Defines a Realtime AI Avatar API At its core, a realtime AI avatar API is an interface that allows programmatic interaction with an AI agent that manifests as a visual avatar. The 'realtime' component is non-negotiable here. It implies sub-second time to first frame, ensuring that interactions feel natural and unlagged. This isn't merely about speed; it's about the perceptual immediacy that makes a digital interaction feel genuinely present. The 'AI avatar' itself is a digital persona. These avatars can be generated from as little as a single image or a short video, then brought to life with an advanced audio-clocking system that ensures lips are perfectly synced to the syllables of generated speech. This nuanced synchronization is critical for maintaining the illusion of a living, responsive entity, distinguishing a compelling avatar from a mere animation loop. Finally, the 'API' layer provides the structured access. It abstracts away the complexity of AI orchestration, rendering pipelines, and real-time streaming, offering a clean interface for developers. Our typed TypeScript SDK, generated directly from an OpenAPI specification, exemplifies this, providing robust, type-safe methods for integrating these capabilities into any application. ## The Operational Core: Behind the Interface Behind every smooth avatar interaction is a distributed computing architecture: LiveKit Cloud handles rooms, SFU, TURN, and WebRTC; a Cloudflare control plane handles admission and session grants; and self-hosted GPU AgentWorkers run LLM, speech, and avatar rendering. That separation keeps media transport on proven LiveKit primitives while reserving custom code for the realtime model/runtime path. This operational design also dictates the economic model: usage-based pricing in realtime minutes. This aligns cost directly with the value consumed, reflecting the computational intensity of maintaining a live, responsive avatar session. Developers only pay for the active, conversational minutes, a pragmatic approach for scalable deployments. ### Crafting the Avatar: From Still to Spoken Word Creating an avatar involves converting static visual data into a dynamic, expressive entity. Our platform achieves this from a single image or a video, which then serves as the visual foundation. The true magic, however, lies in the audio-clocked video generation, where the AI ensures that every gesture and lip movement is precisely synchronized with the spoken word. This precision is what elevates a simple digital puppet to a believable conversational partner. ## Integration Strategies for a Realtime AI Avatar API Integrating a realtime AI avatar API into an existing application is primarily about establishing efficient data pipelines and managing the real-time stream. The TypeScript SDK, `realtime-avatar`, simplifies this considerably by providing well-defined methods for connection, sending input, and receiving output. ```tsx import { createProxyClient, RealtimeAvatarLiveKitRoom, useLiveKitAvatarGrant } from 'realtime-avatar/react'; import { VideoTrack, useChat, useTranscriptions, useVoiceAssistant } from '@livekit/components-react'; const client = createProxyClient({ proxyUrl: '/api/realtime-avatar' }); function AvatarChat({ avatarId }: { avatarId: string }) { const grant = useLiveKitAvatarGrant({ client, session: { avatarId, sttMode: 'off' } }); return ( ); } function AvatarMedia() { const { send } = useChat(); const captions = useTranscriptions(); const { videoTrack } = useVoiceAssistant(); return ( <> {videoTrack ? send('Hello there!')} /> : null}

{captions.at(-1)?.text}

); } ``` - **Client-side Rendering**: Efficiently handle the incoming video and audio streams. This often involves WebGL for video and Web Audio API for playback to minimize latency. - **State Management**: Keep track of the avatar's conversational state and user input to ensure a fluid dialogue flow. - **Error Handling**: Implement robust error handling for connection drops, API limits, and unexpected responses, gracefully degrading the experience if necessary. - **Network Optimization**: Optimize network requests and streaming protocols to maintain sub-second responsiveness, especially over varying network conditions. For initial exploration or rapid prototyping, the /studio provides access to a resident cast of characters like Rin Ashfall or Professor Thistle, allowing developers to test concepts without immediately creating custom avatars. This provides a sandbox environment to understand the nuances of interaction before committing to bespoke character development. Shipping a product built around a realtime AI avatar API requires a meticulous approach to testing. Focus on responsiveness under load, cross-device compatibility for rendering, and the naturalness of the conversational flow. Iterate on your AI agent's prompts and knowledge base to refine its persona. The goal is to embed the avatar not as a feature, but as an intuitive, natural layer of interaction, transforming user experience into a dynamic, engaging conversation. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # AI Boyfriend or Girlfriend: Choosing Text, Voice and Video Companionship Choose an AI boyfriend, girlfriend or companion by comparing text, voice and video, checking conversation continuity, and budgeting for regular use\. Published: 2026-06-08T00:00:00.000Z Updated: 2026-10-12T19:05:57.585Z Canonical: https://realtimeavatar.ai/blog/ai-companion-apps-with-a-face Markdown: [en](https://realtimeavatar.ai/blog/ai-companion-apps-with-a-face.md) · [zh](https://realtimeavatar.ai/blog/ai-companion-apps-with-a-face/zh.md) For adults choosing an AI girlfriend, boyfriend or companion, start with the format you will actually enjoy: text for unhurried messages, voice for spoken back-and-forth, and video when seeing a fictional character adds something you want. Before subscribing, try the same conversation twice, check what carries over, and calculate the cost of your intended routine. A moving face is an option, not an automatic upgrade. This guide is published by Realtime Avatar, a developer API and SDK provider, not an independent app-ranking site. If you want a companion to chat with tonight, look for a ready-made consumer app. RTA's Studio lets creators evaluate characters; its SDK helps developers build experiences. It is not an off-the-shelf dating app. The checklist below helps you compare consumer apps without needing to build anything. ## Choose the format that fits your evening - Choose text if you like sending a thought, putting your phone down, and returning later. Try a casual check-in or a few lines of fictional flirting. Judge the writing, not the portrait: does the reply pick up your joke, or restart the same introduction? - Choose voice if hearing a character matters more than watching one. Try discussing dinner while tidying the kitchen. Listen for comfortable pauses and useful replies; try correcting a detail midway through an answer. Test on the phone and headphones you actually use. - Choose video if expressions and a visible character are part of the appeal. Try it after a voice conversation you already like. A polished opening animation says little about how the next ten turns feel. Keep voice or text available when watching a screen is inconvenient. Check what each label means. A voice note is a recorded message you send or play; a live call lets you speak back and forth in an ongoing session. A generated video selfie is a clip, not proof of a live video conversation. Seeing the avatar also does not, by itself, mean the app is receiving your camera. ## Two documented examples, not a winner's podium - Nomi is one ready-made option to investigate if you want a customizable companion. Its developer's App Store description advertises AI girlfriend and boyfriend conversation, backstories, Shared Notes, and sending and receiving voice messages. Those are features to inspect, not evidence of perfect memory or a measured relationship benefit. - Kindroid is one option to investigate if calls are central to your choice. Its official guide describes voice calls and optional Live Avatar video calling for subscribers. It separates audio usage, an ongoing call charge, and additional live-avatar charges. Compare the complete bill, not just the largest allowance on the page. These are documentation-based examples, not hands-on recommendations. Kindroid's guide also distinguishes shared chat history from long-term recall: it describes a Shared chat history toggle and says long-term memory recall is paused during calls. That is exactly the kind of qualification to check before assuming yesterday's text conversation will carry into tonight's call. Availability and terms can change. - [Nomi's developer-written App Store listing](https://apps.apple.com/us/app/nomi-ai-companion-with-a-soul/id6450270929) - [Kindroid's official voice and video call guide](https://kindroid.ai/v2/docs/voice-calls-and-video-calls/) ## Give a companion a two-day audition Use a small, low-stakes conversation rather than an elaborate prompt. You are choosing something pleasant to return to, not trying to win a chatbot benchmark. The following is a suggested trial, not a test we have run on the apps above. 1. Set the tone. Say, 'Keep this playful and short. Ask one question at a time.' See whether a simple exchange stays conversational instead of becoming a monologue. 2. Share a harmless detail. Try, 'For our imaginary weekend, I would pick a bookstore over a nightclub.' Later, ask the character to suggest an outing without supplying the answer again. 3. Correct the detail. Say, 'Actually, make it a quiet café this time.' Does the conversation adjust, or keep insisting on the earlier plan? Look for editable notes or memory controls rather than relying only on a promise to remember. 4. Return in a fresh session. Ask where you left off, then try a call if that is a paid feature you want. Check both the remembered detail and the conversational tone; the same portrait does not prove continuity. 5. End on your terms. Finish the chat, mute notifications, and find the cancellation screen. Decide whether the experience was enjoyable without pressure to stay longer or buy something immediately. One successful recall is not proof of permanent memory, and one miss is not a complete verdict. What matters for your choice is whether ordinary corrections are manageable and the app can maintain enough context for the conversations you want. - [A more detailed guide to testing interruptions in live avatar calls](https://realtimeavatar.ai/blog/testing-barge-in-in-a-realtime-avatar-app) ## Price your routine, not the introductory offer Write down your intended use before comparing subscriptions. Ten minutes on thirty evenings is 300 minutes a month. That is a budgeting assumption, not a claim about how often people use these apps. Text messages, spoken output, connected call time, pictures and live video may have different allowances; do not convert all of them into interchangeable minutes. For illustration only, imagine a plan costing $12 a month with 120 included call minutes and $0.10 for each additional minute. Your 300-minute routine would cost $30 before taxes and any separate extras: $12 plus 180 × $0.10. This is invented worksheet pricing, not a quote for any named app. With character-based audio credits or multiple simultaneous meters, use the provider's actual calculation instead. - What does the subscription include, and when does each allowance reset? - Do voice generation, connected call time and the live face draw separate charges? - What happens at the limit: a hard stop, a downgrade, or another purchase? - Can you disable automatic top-ups, see usage, and cancel before renewal? If a free tier covers the text conversations you enjoy, keep it. Pay for a specific improvement you have tried, not the vague possibility that the premium character will care more. ## Keep the experience yours Choose an adult fictional character and a style you genuinely like; there is no need to copy a real person's face or voice. Check the app's content rules rather than treating a romantic label as permission for any content. Affectionate replies are generated interaction, not evidence of a person's feelings or a mutual obligation. Before sharing intimate details, check how conversations are stored and how deletion works. During a trial, use invented plans instead of your address, workplace or another person's private information. Look for clear microphone and camera controls. A companion app is entertainment, not medical or mental-health care, and you should be able to leave without negotiating with a character. ## Building a companion instead of choosing an app? RTA provides portrait-based avatars and live voice or video calls. The application developer owns the character instructions, saved context, user controls and consumer pricing. A fresh RTA session does not automatically inherit relationship memory. Start by evaluating a character in Studio, then build the continuity and spending controls around it. RTA infrastructure prices are not the retail subscription price of a finished companion app; compare them separately. - [RTA pricing for the developer platform](https://realtimeavatar.ai/pricing) ## Frequently asked questions ### What is the best AI girlfriend or boyfriend app? Start with your preferred format and budget, then compare conversational style, corrections and continuity using the same short trial. This guide does not name a universal winner: documented features are not a substitute for seeing whether you enjoy the actual conversation. ### Can I use an AI companion for free? Check the specific app's current free allowance and renewal terms. Free text access does not establish free calls, pictures or live video. If the free experience already does what you want, there is no reason to upgrade solely for a relationship label. ### Will it remember me between text and calls? Do not assume it will. Look for shared-history settings, editable notes and a clear explanation of what crosses into calls. Repeat your trial in a new session. Memory features have limits, and a familiar voice is not proof that earlier details were loaded. ### Is video companionship better than text? Only if the visible character adds something you value. Text may suit your pace; voice may be enough when you want spoken conversation. We do not have evidence here that adding video improves retention or guarantees a stronger connection. Choose the format you enjoy within your budget. --- # Roleplay at scale: AI avatars for sales and corporate training The best training is rehearsal against a difficult human\. Realtime avatars make that rehearsal infinitely available, repeatable, and measurable\. Published: 2026-06-06T00:00:00.000Z Updated: 2026-06-06T00:00:00.000Z Canonical: https://realtimeavatar.ai/blog/ai-avatars-for-corporate-training Markdown: [en](https://realtimeavatar.ai/blog/ai-avatars-for-corporate-training.md) # Roleplay at scale: AI avatars for sales and corporate training > The best training is rehearsal against a difficult human. Realtime avatars make that rehearsal infinitely available, repeatable, and measurable. - Published: 2026-06-06 - Tags: product, training - Canonical: https://realtimeavatar.ai/blog/ai-avatars-for-corporate-training - Markdown: [en](https://realtimeavatar.ai/blog/ai-avatars-for-corporate-training.md) Ask any sales leader what actually improves close rates and they'll tell you: roleplay. Not the course, not the deck — the uncomfortable twenty minutes across the table from someone playing a skeptical buyer. The problem has never been whether roleplay works. It's that it doesn't scale: senior people are expensive, scheduling is brutal, and nobody wants to fumble a negotiation in front of their manager. ## An opponent who never gets tired A realtime avatar changes the economics of rehearsal. Duke Marrow — one of the resident characters on our platform — can play the procurement director who has already talked to your competitor, at 11pm, for the fourth time tonight, with no judgment and no calendar. The trainee talks; the avatar answers in realtime video, holds the persona, applies pressure, and never breaks character. > "You blinked on price. Again — and this time, hold the number." The face matters more here than almost anywhere else. Objection-handling practice against a text box trains typing. Practice against a face trains what the job actually requires: composure under eye contact. ## Repeatable, measurable, designable - **Repeatable:** the same scenario, the same temperament, for every trainee in every region — something human roleplay partners can't promise. - **Measurable:** every session is a transcript with timing. Talk ratios, filler density, how long the trainee held silence after an anchor — all of it is data the L&D team can trend. - **Designable:** difficulty is a written temperament, not a mood. Make Monday's buyer curious and Friday's buyer hostile, and version both in git. ## Beyond sales The same loop covers the conversations companies quietly under-train because they're hard to stage: performance reviews, incident escalations, de-escalating an angry customer, medical bedside manner. Anywhere the skill is *talking to a difficult human*, a persistent character with a face is the rehearsal partner that's always available. ## Shipping it Teams typically start with one scenario and one character: design the persona in the studio, lock a voice, write the temperament, and embed the session in the training portal with the SDK. At about $5 per hour of live avatar time, a twenty-minute rehearsal costs under $2 — a full pilot for one team runs less than a single hour of an external roleplay facilitator, and the avatar shows up for every cohort after that. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt --- # What is a realtime avatar API? A builder's guide From a single portrait to a character that answers in live, audio\-synced video: the moving parts of a realtime avatar API and what to evaluate before you build on one\. Published: 2026-06-04T00:00:00.000Z Updated: 2026-10-13T13:40:51.151Z Canonical: https://realtimeavatar.ai/blog/what-is-a-realtime-avatar-api Markdown: [en](https://realtimeavatar.ai/blog/what-is-a-realtime-avatar-api.md) # What is a realtime avatar API? A builder's guide > From a single portrait to a character that answers in live, audio-synced video: the moving parts of a realtime avatar API and what to evaluate before you build on one. - Published: 2026-06-04 - Tags: engineering, api - Canonical: https://realtimeavatar.ai/blog/what-is-a-realtime-avatar-api - Markdown: [en](https://realtimeavatar.ai/blog/what-is-a-realtime-avatar-api.md) A realtime avatar API turns a still face into a live one. You provide an image (or a short video), attach a voice and a persona, and the API streams back video of that character speaking — fast enough to hold a conversation. It is the difference between generating a clip and *talking to someone*. ## The pipeline under the hood Every realtime avatar stack is some arrangement of four stages: 1. **Character registration.** A portrait or reference video is processed once into a reusable identity — facial geometry, idle motion, the material the renderer animates. Good platforms cache this so later sessions start warm. 2. **Language and persona.** An LLM produces the reply in character. This stage is usually yours to configure: the persona, the memory, the model. 3. **Speech synthesis.** The reply becomes audio in the character's chosen voice, streamed rather than rendered whole. 4. **Audio-clocked video rendering.** The renderer generates frames slaved to the audio timeline, so lips, breath, and micro-expressions land on the syllable. This is the stage that separates realtime systems from offline lip-sync tools. ## The metrics that actually matter - **Time to first frame.** The pause between the user finishing a sentence and the avatar visibly responding. Under a second feels like a reaction; over two feels like a loading screen. - **Sync drift.** Audio/video offset across a long turn. Drift is more damaging than latency — users forgive a beat of silence, not a mouth that lies. - **Cadence stability.** Frame pacing on imperfect networks. A steady 24fps beats a jittery 40. - **Identity persistence.** The same face across sessions, devices, and re-renders. Critical for companions and brand characters. ## What to look for in the API surface Beyond the demo, evaluate the boring parts: a typed SDK (ours is TypeScript, generated from an OpenAPI spec), explicit session lifecycle so you can meter and cap usage, avatar caching so characters start warm, and usage-based pricing in minutes — the only unit that maps to what your users actually consume. (For reference, our plans anchor live avatar time at about $5 per hour, with a free sandbox of 480 credits, about 8 minutes, granted once per account.) If AI agents are part of your stack, check for an MCP server and an `llms.txt`; an avatar platform your agents can operate is worth more than one with a prettier dashboard. ## Where builders take it The use cases we see most: consumer companion apps, sales and corporate training roleplay, livestream hosts that run for hours, support agents with a face, game NPCs with persistent personas, and tutors that teach out loud. One character, designed once, reused across every surface — that reuse is where the economics of a realtime avatar API quietly compound. --- Realtime Avatar — realtime AI avatar API & SDK. Site: https://realtimeavatar.ai · Docs: https://realtimeavatar.ai/docs · OpenAPI: https://realtimeavatar.ai/openapi.json · Agent guide: https://realtimeavatar.ai/llms.txt