# Avatar API tool calling compared: client tools, deadlines, limits

Where Realtime Avatar, Anam, Tavus, Runway, Simli and LiveAvatar run tool handlers: in the page, on your server or in your agent, with deadlines \(Nov 2026\)\.

Published: 2026-11-18T07:50:44.003Z
Updated: 2026-11-18T07:50:44.003Z
Canonical: https://realtimeavatar.ai/blog/avatar-api-tool-calling-compared
Markdown: [en](https://realtimeavatar.ai/blog/avatar-api-tool-calling-compared.md)

For an app that wants tool handlers running in the browser page (React or any other front end) and the result spoken back, the closest fits besides Tavus are Realtime Avatar and Anam, as of November 2026. Realtime Avatar client tools run in your page and return their result to the model, with a fixed 2.5-second deadline per call. Anam client tools also run in the page, through registerToolCallHandler, and return a result when awaitResult is true, with a timeout you set from 1 to 600 seconds (10 by default). Tavus itself delivers tool calls to your frontend by default, or to your server over HTTPS with a timeout of up to 60 seconds. Anam webhook tools and Runway server tools run on your server. Runway client tools run in the browser but do not return results to the conversation. With Simli or HeyGen LiveAvatar LITE, the vendor renders the face and your tools live in your own agent.

Every vendor below was read from its own docs on 16 November 2026 (links at the end), asking three things: where the handler runs, whether its result goes back to the model so the avatar can say it, and how long the avatar waits.

## Where tool handlers run, vendor by vendor (November 2026)

The placement decides your security model as much as your latency. An in-page tool runs in the user's browser, so a same-origin fetch carries the signed-in user's session and your existing route authorizes it. A server tool is called by the vendor's cloud, so your endpoint must authenticate the vendor and work out which user the call is for. An agent tool lives in an agent you run, next to a vendor that only draws the face.

- Realtime Avatar client tools: the handler runs in your page, registered over RPC with attachAvatarTools from realtime-avatar/tools after the room connects; the result returns to the model; each call has 2.5 seconds, fixed and not tunable; your server grants the tools per call at mint with clientTools: true (realtimeavatar.ai/docs/tool-calling, read 16 November 2026).
- Realtime Avatar server steering: your backend runs the tool with its own credentials and returns the result, and the page hands it to the avatar through sendTurn with per-turn instructions. There is no hosted executor either way.
- Anam client tools: the handler runs in the browser, registered with registerToolCallHandler in the @anam-ai/js-sdk JavaScript SDK; the return value goes back to the LLM only when awaitResult is true, and with the default (false) the call is fire-and-forget; toolTimeoutSeconds ranges from 1 to 600 seconds, default 10 (anam.ai client tools docs, read 16 November 2026).
- Anam webhook tools: they run server-side and call your HTTPS endpoint; with awaitResponse true (the default) the LLM uses the response; Anam asks for responses ideally under 1 second and at most 5, with a 60-second hard timeout.
- Tavus, app-message delivery (the default): the call arrives in your frontend as a conversation.tool_call event over the call's data channel, your app runs the logic and sends conversation.tool_result with the matching tool_call_id; there is no hard timeout on this path, and app messages are capped at 4 KB (Tavus tool calling and tool delivery docs, read 16 November 2026).
- Tavus, API delivery: Tavus sends an HTTPS request to your endpoint and a 2xx response body becomes the result; timeout up to 60 seconds, default 10; HMAC signing is available. On both channels, the tool's on_resolve setting decides whether the avatar speaks the result, summarizes it, absorbs it silently or ignores it.
- Runway Characters (its avatar API) client tools: defined with clientTool and handled with useClientEvent in @runwayml/avatars-react; they run entirely in the browser and do not return results to the conversation, so they suit UI actions (Runway docs, read 16 November 2026).
- Runway Characters server tools: backend_rpc tools answered by your Node process through @runwayml/avatars-node-rpc, which joins the session; results feed back into the conversation; timeoutSeconds defaults to 4, range 1 to 8; one handler connection per session.
- Simli: adds a lip-synced face to a realtime agent you run, through its LiveKit Agents, Pipecat, JavaScript SDK, Python SDK or raw WebRTC pathways, so on those paths your tools are defined and executed in your agent (Simli docs overview).
- HeyGen LiveAvatar LITE: you handle speech recognition, the LLM and the voice, for example through its LiveKit, Pipecat or Agora plugins, or connect a hosted voice agent such as OpenAI Realtime or ElevenLabs through a connector, and LiveAvatar renders the video, so tools live in that agent. LITE costs 1 credit per minute against 2 for FULL mode, where LiveAvatar's managed agent runs speech recognition, the LLM and the voice and you can still bring your own LLM (LiveAvatar LITE docs).

Two assumptions are worth correcting. Tavus is not limited to callbacks to your server: its default delivery puts the tool call in your frontend. And a timeout you can raise is not free: while a tool runs, the user is waiting on a live video call, which is why Realtime Avatar keeps its deadline fixed.

## Realtime Avatar client tools in practice

Realtime Avatar ships TypeScript, React and React Native SDKs. Your server opts a call in when it mints the session with clientTools: true (on the wire, capabilities: ["client_tools"]), so the page cannot grant itself tools. After the room connects, the page declares its tools and the manifest is registered over RPC; nothing about a tool appears in the mint request. The example is a SaaS page, such as a Next.js App Router client component, where the avatar looks up the signed-in user's subscription, plus a slow tool that answers inside the deadline and finishes on screen.

```
// Server: the connect route opts this call in. Part of the call policy.
session: async () => ({ instructions, clientTools: true }),

// Page: after room.connect(). The key is the name the avatar calls.
// buildRenewalQuote and showOnScreen stand for your own app code.
import { attachAvatarTools, type AvatarTool } from "realtime-avatar/tools";

const check_subscription: AvatarTool<{ detail: "plan" | "renewal" | "seats" }> = {
  description:
    "Look up the signed-in user's subscription. Call this when they ask which plan " +
    "they are on, when it renews, or how many seats they have.",
  parameters: {
    type: "object",
    properties: { detail: { type: "string", enum: ["plan", "renewal", "seats"] } },
    required: ["detail"],
  },
  execute: async ({ detail }, { signal }) => {
    // Same-origin request: the browser sends the user's session cookie,
    // so your route authorizes it like any other request from this page.
    const res = await fetch("/api/me/subscription?detail=" + detail, { signal });
    if (!res.ok) throw new Error("subscription lookup failed: " + res.status);
    return await res.json();
  },
};

const quote_renewal: AvatarTool<{ plan: string }> = {
  description: "Prepare a renewal quote for a plan the user names.",
  parameters: {
    type: "object",
    properties: { plan: { type: "string" } },
    required: ["plan"],
  },
  execute: async ({ plan }) => {
    void buildRenewalQuote(plan).then(showOnScreen); // may take longer than 2.5 s
    return "Started the quote. It will appear on screen in a moment.";
  },
};

const { accepted, rejected } = await attachAvatarTools(room, {
  check_subscription,
  quote_renewal,
});
if (rejected.length > 0) console.warn("Tools the avatar does not have:", rejected);

```

Read the return value: a tool in rejected is a tool the avatar simply does not have, which looks exactly like the model ignoring it. The description is the teaching signal, so say what the tool does and when to reach for it.

### The 2.5-second rule

A tool has 2.5 seconds. Realtime Avatar's docs call that a conversational floor, not a tunable: past a couple of seconds of dead air a video call stops feeling live. After the deadline the platform abandons the call and tells the model it failed. The abort is cooperative: every execute receives an AbortSignal, but a handler that ignores it keeps running and can still commit a side effect; only its result is discarded. Four habits follow from that.

- Pass the signal to fetch, as check_subscription does, so an abandoned lookup stops instead of finishing for nobody.
- Make slow writes idempotent, or check signal.aborted right before you write, so a refund or booking abandoned at the deadline is not committed twice when the user asks again.
- Do not call a model inside a tool; it will not fit the deadline. Return an acknowledgement inside the deadline and deliver the real answer on screen when it lands, which is what quote_renewal does.
- Fetch what you already know before the call starts. Supply the user's plan, open tickets or recent orders as context when your server starts the call, and the avatar needs no tool for them at all.

## When the tool belongs on your server

With server steering, your backend runs the agent loop. The page posts the user's turn text to your route (a typed message, or your own speech recognition: in video calls listen: false turns the platform's recognition off so your client supplies every turn's text). Your server decides a tool is needed and runs it where your data and credentials already are. It returns the result, and the page passes it to the avatar with sendTurn and per-turn instructions, so the result shapes that one reply in the character's own voice.

```
// Server: your /api/turn route runs the tool with its own credentials
// and returns the steer text. The credentials never reach the page.
export async function POST(req: Request) {
  const { text, orderId } = await req.json();
  const userId = await currentUserId(req); // your own auth
  const result = await tools.checkOrderStatus({ userId, orderId });
  return Response.json({
    text,
    steer: [
      "Answer using ONLY these facts:",
      JSON.stringify(result),
      "One or two spoken sentences. Do not read the JSON aloud.",
    ].join("\n"),
  });
}

// Page: send the turn, then hand the RESULT to the avatar for this one reply.
const { text, steer } = await fetch("/api/turn", {
  method: "POST",
  headers: { "content-type": "application/json" },
  body: JSON.stringify({ text: userText, orderId }),
}).then((r) => r.json());
await session.sendTurn(text, { instructions: steer });

```

Use server steering when a tool needs secrets the page must never hold. For a string that must be exact, such as a confirmation number or a required disclosure, skip the model: sendClosingTurn speaks your text verbatim and then ends the call. To have your own model write every reply, Realtime Avatar can stream from an OpenAI-compatible /chat/completions endpoint you host, enabled per workspace on request; without it you get 403 external_llm_not_enabled.

## Auditing what the avatar did

Register a transcript webhook when your server mints the call, and after the call ends Realtime Avatar POSTs the two-sided transcript plus the tool calls the model acted on, signed with HMAC-SHA256. Each tool_calls entry has name, call_id, arguments, ts, ok, result or error, and duration_ms. Store it next to the transcript: a reply grounded in a lookup is only auditable if the lookup is in the history.

- Arguments and results are truncated to 2,000 characters each.
- At most 200 tool calls are recorded; tool_calls_truncated: true marks an overflow.
- An entry with no ok field produced nothing the model saw.
- A tool round that belongs to a turn the user barged in over is not reported, so those calls are absent even though they ran.
- duration_ms shows which tools run close to the 2.5-second line.

## Which to pick

- Handlers in your React page, results spoken back, lookups that finish fast: Realtime Avatar client tools, or Anam client tools with awaitResult set to true. Tavus app-message delivery is the same pattern if you are staying on Tavus.
- Tools that can take longer than 2.5 seconds, such as a slow backend or a call into another model: Anam client tools (timeout up to 600 seconds), Tavus (API delivery up to 60 seconds; app messages have no hard timeout) or Runway server tools (up to 8 seconds) fit better, because Realtime Avatar's deadline is fixed.
- Tools that should run on your backend: Anam webhook tools (60-second hard limit; Anam asks for under 1 second, 5 at most), Tavus API delivery with HMAC signing, Runway server tools through a Node RPC handler you run (8 seconds at most), or Realtime Avatar server steering, where your server runs the tool and the page relays the result.
- UI-only actions such as navigating or opening a modal: Runway client tools, or Anam client tools left fire-and-forget.
- You already run a LiveKit Agents or Pipecat agent with its own tools: keep it and add a face-only renderer such as Simli or LiveAvatar LITE. Realtime Avatar does not offer a renderer-only mode that lip-syncs audio from your own voice agent.

### Honest limits of Realtime Avatar tool calling

- No hosted executor and no hosted function registry: you host every handler, in the page or on your server.
- The 2.5-second deadline is not configurable, and aborts are cooperative, so a handler that ignores its signal can still commit a side effect.
- Your own model endpoint is per workspace on request, not self-serve.
- It cannot be the face of an existing LiveKit Agents, Pipecat or OpenAI Realtime agent.

To try the client-tool example, Realtime Avatar's free Sandbox includes 480 credits once per account, no card: 8 realtime minutes, or your first avatar plus 2 minutes of calls. Starter is $9 a month with 120 minutes included, then $0.095 a minute, and Developer is $24 a month with 600 minutes, then $0.085 a minute (Realtime Avatar pricing page, November 2026).

## FAQ

### Which realtime avatar APIs besides Tavus run tool handlers in the page and speak the result?

It depends on your slowest tool. If every tool answers well inside 2.5 seconds, Realtime Avatar client tools and Anam client tools (awaitResult true) both run in the page and return results to the model. If some tools are slow, Anam's configurable client-tool timeout (1 to 600 seconds) fits better. Runway's client tools cannot return results, so they suit UI actions only.

### Is there an avatar SDK where the tool handlers run right in my React page instead of on my server?

Yes: Realtime Avatar client tools, Anam client tools, and Tavus too, whose default app-message delivery sends conversation.tool_call to your frontend and takes conversation.tool_result back. Callbacks to your server are its API delivery option.

### How do I make a live AI avatar call my own API mid-conversation, like checking an order status?

Declare a tool such as check_order with a JSON Schema for order_id and a description that says when to use it. With Realtime Avatar its execute function in your page fetches your route; with a webhook vendor you register your endpoint URL and the vendor calls it.

### Should tools run in the browser or on my server with a 2 to 3 second deadline?

Run a tool in the browser when the page can already reach the data with the user's own session, which skips building vendor authentication into your API. Run it on your server when it needs secrets or touches several systems. Neither is faster in general; it depends on where your API sits relative to users and the vendor, so measure it, for example with duration_ms in Realtime Avatar's transcript webhook.

### Which avatar APIs let a Next.js support agent look up the logged-in user's subscription mid-conversation?

In-page tools make this simple because the handler fetches a route handler that reads the user's session: Realtime Avatar, Anam client tools with awaitResult true, or Tavus app messages. The check_subscription example above is exactly that.

### Simli function calling through LiveKit or Vapi is getting complicated. Which avatar API has built-in tool calling to my own functions?

Realtime Avatar, Anam, Tavus and Runway each run the conversation themselves and send tool calls to your own code, in the page or on your server, so you do not run a separate voice agent. The trade-off with Realtime Avatar is that it runs the conversation itself; it cannot put a face on the agent you already built. If that agent is the asset, keep Simli.

- [Tool calling docs: client tools, the 2.5-second deadline, server steering](https://realtimeavatar.ai/docs/tool-calling)
- [React SDK docs](https://realtimeavatar.ai/docs/react)
- [Realtime Avatar vs Tavus](https://realtimeavatar.ai/compare/tavus)
- [Realtime Avatar vs Anam](https://realtimeavatar.ai/compare/anam)
- [Building a customer support avatar](https://realtimeavatar.ai/use-cases/customer-support-avatar)
- [All-in cost per minute and per hour](https://realtimeavatar.ai/pricing#all-in-cost)

- [Anam client tools and events (read 16 November 2026)](https://anam.ai/docs/personas/tools/client-tools)
- [Anam webhook tools (read 16 November 2026)](https://anam.ai/docs/personas/tools/webhook-tools)
- [Tavus tool calling (read 16 November 2026)](https://docs.tavus.io/sections/onboarding-guide/tool-calling-examples)
- [Tavus tool delivery: app messages and API calls (read 16 November 2026)](https://docs.tavus.io/sections/conversational-video-interface/pal/llm-tool-delivery)
- [Runway Characters client tools (read 16 November 2026)](https://docs.dev.runwayml.com/characters/tools/client-tools/)
- [Runway Characters server tools (read 16 November 2026)](https://docs.dev.runwayml.com/characters/tools/server-tools/)
- [Simli docs overview (read 16 November 2026)](https://docs.simli.com/overview)
- [HeyGen LiveAvatar LITE mode overview (read 16 November 2026)](https://docs.liveavatar.com/docs/lite-mode/overview)
