Docs

Realtime AI avatars with React and React Native

Embed live AI avatar calls in React or React Native with AvatarCall, useAvatarCall, connection states, and a server-side session proxy.

Every server adapter mounts the same proxy, so the client half is the same everywhere: one component pointed at your route.

The component

For an existing browser React app, install the SDK and its browser media peers. The downloadable Next.js starter already declares all three.

npm install --save-exact realtime-avatar@0.20.0 zod@4.4.3 @livekit/components-react@2.9.24 livekit-client@2.22.3
import { AvatarCall, createProxyClient } from "realtime-avatar/react";

const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" });

<AvatarCall client={client} avatarId={avatarId} />

// Audio only — no video track is requested, so nothing renders and
// nothing decodes.
<AvatarCall client={client} avatarId={avatarId} mode="voice" />

Build product UI from call state

Use the SDK's call status, actions and end reasons for your interface. The standard call flow does not require Room, Participant, Track or WebRTC objects.

<AvatarCall
  client={client}
  avatarId={avatarId}
  onStatusChange={(status) => console.log("Call status:", status)}
  onEnded={({ reason }) => console.log("Call ended:", reason)}
>
  {(call) => <button onClick={call.end}>End call</button>}
</AvatarCall>

Call state, transport quality and actual media presentation answer different questions. A connected room or an available track does not prove that a frame appeared on the user's screen. Keep detailed diagnostics out of the default product flow.

LiveKit owns transport recovery. RTA owns session preparation, capacity and fresh grants. Your product decides which notice or retry control to show. RTA's optional quality policy callback describes its policy result; it is distinct from the provider's native connection quality. See Callsfor correlation and historical diagnostics.

Optional connection details

SDK 0.16.0 adds onConnectionDetailsChange when your product needs current connection facts. It receives a snapshot and subsequent changes;null clears a retired source. Keep its state scoped to one call. Omitting the callback adds no reporting listeners, timer or stats polling.

import { useState } from "react";
import { AvatarCall, type AvatarCallProps, type AvatarConnectionDetails } from "realtime-avatar/react";

function CallWithDetails(props: Pick<AvatarCallProps, "client" | "avatarId">) {
  const [details, setDetails] = useState<AvatarConnectionDetails | null>(null);
  return <>
    <AvatarCall {...props} onConnectionDetailsChange={setDetails} />
    {details && <output>Local connection: {details.localQuality}</output>}
  </>;
}

The snapshot contains connectionState, localQuality, and separate audio/video publisher quality and stream state. The publisher fields come from the selected tracks, which may have different publishers. An absent source is null; an existing source can report unknown quality. These facts do not establish rendered video or audible audio, and the SDK does not upload them. Your product decides which notices or user actions they warrant.

Native and custom integrations pass this same callback to their existing SessionLifecycleRoomBridge. Both client entries export the same AvatarConnectionDetails type. There is no extra wrapper or separate native observer. Continue using call status and end reasons for the call lifecycle.

The hook underneath it

AvatarCall is a default surface over useAvatarCall. Reach for the hook when you want the call's state but your own layout — a custom control bar, your own video element, a call that lives inside something else.

import { createProxyClient, useAvatarCall } from "realtime-avatar/react";

const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" });

function MyCall({ avatarId }: { avatarId: string }) {
  const { call, view } = useAvatarCall({ client, avatarId });
  return <>
    {view}
    <p role="status">{call.status}</p>
    <button onClick={() => call.end()}>End call</button>
  </>;
}

Mounting the call component starts connection. Mount it only after the user clicks a start button, and unmount it after onEnded. Keep connection, queue, recovery and end states visible. The complete Next.js starter demonstrates that lifecycle with text messages and an optional microphone.

Let the character see your camera

Your server must authorize camera: true when starting the call. This grants permission; capture stays off until the user chooses to share. Mount one useAvatarCamera control inside the existing room, withallowed from the grant and active from your live-call and foreground state. Do not also drive the room's video prop.

import { useAvatarCamera } from "realtime-avatar/react";

function CameraControl({ allowed, active }: { allowed: boolean; active: boolean }) {
  const camera = useAvatarCamera({ allowed, active });
  return <>
    <p>Share camera images with AI to let the character see what you show.</p>
    <button style={{ minHeight: 44 }} disabled={!camera.available}
      aria-pressed={camera.enabled} onClick={() => void camera.toggle()}>
      {camera.pending ? "Cancel camera request" : camera.enabled ? "Stop sharing camera" : "Share camera"}
    </button>
    <p role="status">{camera.error ? "Camera unavailable. Check camera permissions and try again." : ""}</p>
  </>;
}

Repeated on/off toggles keep the same session and microphone. CallingsetEnabled(false) also cancels a pending permission request in the SDK; any track returned later is stopped without publishing. Stop sharing onvisibilitychange when hidden and on pagehide. Returning to the page must not automatically enable the camera. For a local preview, pass the returned participant andpublication to LiveKit's VideoTrack, with camera source and muted playback.

The default capture is front-facing, 640×360 at 5 frames per second. Visual understanding samples at most once every two seconds using Gemini 2.5 Flash-Lite; this is occasional image understanding, not continuous motion recognition. Camera images are excluded from server call recordings, although spoken descriptions can remain in recorded audio or conversation history.

React Native

A separate entry point, because the media stack is not the browser's: it wraps LiveKit's React Native room and needs an audio session the OS has granted.

import {
  RealtimeAvatarLiveKitRoom,
  AvatarVideoSurface,
  registerGlobals,
} from "realtime-avatar/react-native";

registerGlobals();

registerGlobals from the same entry point must run once at app start — it installs the WebRTC globals React Native does not ship.

RealtimeAvatarLiveKitRoom owns the OS audio session by default. Only call useRealtimeAvatarAudioSession(connected) yourself when the room hasmanageAudioSession={false}; keep one owner. Import useAvatarCamera from realtime-avatar/react-nativefor the same camera API. Use a native Pressable and Text for the control and LiveKit's native VideoTrack for the preview.

import { useEffect, useState } from "react";
import { AppState, Pressable, Text, View } from "react-native";
import { useAvatarCamera } from "realtime-avatar/react-native";

function NativeCameraControl({ allowed, active }: { allowed: boolean; active: boolean }) {
  const [foreground, setForeground] = useState(AppState.currentState === "active");
  const camera = useAvatarCamera({ allowed, active: active && foreground });
  const { setEnabled } = camera;
  useEffect(() => {
    const listener = AppState.addEventListener("change", (state) => {
      if (state === "active") setForeground(true);
      if (state === "background") {
        setForeground(false);
        void setEnabled(false);
      }
    });
    return () => listener.remove();
  }, [setEnabled]);
  return <View>
    <Text>Share camera images with AI to let the character see what you show.</Text>
    <Pressable accessibilityRole="button" disabled={!camera.available}
      style={{ minHeight: 44, justifyContent: "center" }} onPress={() => void camera.toggle()}>
      <Text>{camera.pending ? "Cancel camera request" : camera.enabled ? "Stop sharing camera" : "Share camera"}</Text>
    </Pressable>
    <Text>{camera.error ? "Camera unavailable. Check camera permissions." : ""}</Text>
  </View>;
}

Treat iOS's transient inactive during a permission prompt differently from background. Stop capture in the background and require a new user action to share again. Pass false for activewhen the call ends or is held.

Camera sharing reuses the existing LiveKit React Native and WebRTC native modules; it does not require expo-camera. The native binary must already contain those modules and camera permission declarations (iOS camera usage description and Android CAMERA permission). Otherwise ship a new native build. A JavaScript-only update may use OTA only when its runtime fingerprint matches the installed build and store review/release policy permits it. Update your privacy disclosure for camera images sent to an AI provider; an OTA cannot add native permissions. SDK 0.20.0 also requires a compatible Zod 4 peer, which is JavaScript-only.

Next