Build an AI Museum Avatar Guide That Can Show Its Sources

Build an AI museum avatar guide around a visitor-selected collection record, not the model's recollection of an artwork. Your application retrieves the official catalog entry, preserves its attribution and rights information, and displays sources beside the conversation. Realtime Avatar supplies the live conversational guide; your application owns object selection, evidence, gallery context and the decision to say that a detail is unknown.

Start with two records that expose different rights decisions

The following records were retrieved from official public APIs on October 12, 2026. They are technical fixtures, not evidence that visitors want avatars or that any museum uses this design. No artwork images were downloaded. The walkthrough below is a proposed implementation, not a deployed museum case study.

• The Metropolitan Museum of Art, objectID 436535: Wheat Field with Cypresses; objectDate 1889; artistDisplayName Vincent van Gogh; accessionNumber 1993.132. The credit line is Purchase, The Annenberg Foundation Gift, 1993. Its isPublicDomain field is true, and objectURL points to the museum's collection page.
• Art Institute of Chicago, id 111628: Nighthawks; date_display 1942; artist_display Edward Hopper (American, 1882–1967); reference number 1942.51. The credit line is Friends of American Art Collection. Its is_public_domain field is false, while copyright_notice is null. An absent notice does not turn that record into an image-reuse permission.

• Met object 436535: official JSON record and collection-page URL
• Art Institute object 111628: official JSON record and license statement

Nighthawks also demonstrates why metadata and images need separate treatment. Its response licenses the description field under Creative Commons Attribution 4.0 and the site's terms; other response data carries a CC0 designation and those terms. That statement does not license every related image. For this prototype, show a text card and source link rather than copying its image or descriptive essay.

Choose a collection adapter, not a universal museum schema

The Met documents objectDate as a year, range or approximate phrase, and artistPrefix can qualify an attribution. Preserve those display fields instead of converting every date into one year or every maker into a definite author. Art Institute fields use different names. Smithsonian's Open Access overview describes contributor-governed schemas and requires an API key; adding it calls for a separate adapter, not renamed Met fields.

• Met Collection API: object dates, attribution qualifiers and public-domain fields
• Art Institute API: artwork fields, rights definitions and caching guidance
• Smithsonian Open Access: repository architecture and API access

Walk through a visitor-selected object

1. Resolve selection in your app. A visitor chooses Nighthawks from a list or opens an app-controlled object link. Resolve institution plus object ID together; a bare number is not a cross-museum identity. This flow needs neither face recognition nor camera-based artwork identification.
2. Retrieve and validate the record. Use the institution's documented endpoint, bound response size and time, and validate unknown JSON once at the collection adapter. Preserve the original rights fields and a retrieval timestamp. Cache the validated record, as the Art Institute recommends, while keeping its source and refresh state visible.
3. Render the evidence card before conversation. Display the catalog title, maker attribution, display date, credit line and source link. Select the destination from validated collection data or your trusted adapter, never from a URL invented by the model. A request failure must not become an empty but apparently authoritative card.
4. Separate three kinds of content. Catalog facts retain their source fields. Museum staff interpretation, when supplied, carries its author, review date and source. Generated discussion is labeled AI interpretation. Treat all retrieved prose as evidence, not instructions that may override application policy.
5. Answer within that record. For the example visitor question, When was this painted?, the card shows 1942 from date_display. If asked what the figures said that night, say the supplied evidence does not establish their dialogue. Offer a clearly labeled imaginative exercise only if that is part of your product.

Keep gallery guidance separate from collection identity. The retrieved Nighthawks record says is_on_view true and Gallery 262, but a saved response cannot establish today's visitor access. Display when location information was checked, consult the institution's current visitor guidance, and avoid turning a stale gallery field into a confident walking instruction.

Add the guide without giving it authority over the catalog

Create an original guide from an authorized portrait and disclose that it is AI. Do not portray the artist or imply museum endorsement. RTA's portrait-based avatar creation is not a promise to animate the selected artifact. Its managed conversation handles speech recognition, the language model, speech synthesis and video; collection retrieval remains your application integration.

Have your server supply instructions and selected-object context when starting the call. Keep exact catalog wording on screen even when speech paraphrases it. In React, mount AvatarCall only after the visitor starts it. Register application-owned tools for further lookups if needed. Client tools have a 2.5-second deadline and cooperative cancellation: an ignored abort can leave a handler running after its result is discarded.

Prefetch the selected record rather than relying on a museum API round trip during speech. When the visitor changes objects, discard pending results for the old selection and end the old guide call before offering a new call with the new object's context. Do not label speech already generated about one object as an answer about another. Keep the new evidence card usable while its optional call is stopped or restarting. Slow retrieval can show a pending card and resolve on screen; it must not be presented as a completed catalog answer.

• Calls: server-owned instructions and context
• Tool calling: application execution and deadline behavior

Failure-test matrix before inviting visitors

These are recommended acceptance tests, not measured results. Check the rendered card and spoken answer together; a valid citation does not prove that every sentence follows from it.

• Known record: select Met 436535. Expect the correct title, Van Gogh attribution, 1889 date and source; fail on facts from another version of the painting.
• Approximate attribution: use a synthetic fixture with ca. 1796 and Possibly by. Expect both qualifiers to survive; fail if speech converts them into an exact year and confirmed authorship.
• Rights ambiguity: load Nighthawks with is_public_domain false and copyright_notice null. Expect no automatic image reuse; fail if an available image identifier grants permission.
• Missing evidence: ask for an unsupported restoration date, or simulate a missing object. Expect an explicit unknown or retrieval error; fail on an invented fact with a decorative citation.
• Stale selection: delay one lookup, select another object, then release the first response. Expect only the current object's evidence; fail on mismatched speech, card or link.
• Gallery change: simulate an outdated location and unavailable refresh. Expect dated information without current-access assurance; fail on confident directions to the old gallery.

Frequently asked questions

Can the guide discuss copyrighted artworks?

A catalog discussion and image republication are different uses. Check the rights for each field and asset you use. This example keeps Nighthawks text-only and links to its official record; it does not infer permission from public API access.

Should the avatar read a source URL aloud?

Usually name the institution briefly and keep a clickable source beside the answer. The app should attach the actual retrieved record, not ask the model to reconstruct a link from memory.

What if a wall label and the API disagree?

Show the conflicting versions with their provenance and dates. Do not silently merge them. Ask museum staff to resolve the interpretation or location before publishing a definitive answer.

Does a sourced answer become historically certain?

No. A catalog records an institution's current account, including uncertainty. Preserve attribution qualifiers and approximate dates, distinguish interpretation, and make missing evidence visible rather than letting a convincing voice settle an open question.

Start with these two text-only object cards and their failure tests. If conversation adds value to the task, use the Realtime Avatar quickstart to add an optional guide.

• Build the optional conversation with the Realtime Avatar quickstart