Build an AI Email Assistant That Shows the Evidence Behind Its Briefing

An AI email assistant should surface exceptions as claims attached to inspectable messages, not replace the inbox with a confident summary. For a pre-trip briefing, show what changed, which message supports it, what remains uncertain, and how much mail was actually checked. Keep reading separate from sending or changing labels. Voice can help the traveler discuss an exception; it cannot make the evidence more authoritative.

Start with three exceptions, not an inbox recap

Consider a fictional Thursday trip. The original airline confirmation says 09:10. A later message says 08:25. The hotel sent a payment reminder followed by a receipt, while a colleague forwarded an old pickup location. A useful briefing identifies these disagreements instead of summarizing every message in arrival order.

• Flight: a newer message reports 08:25, compared with 09:10 in the original confirmation. Show both messages and their dates. Current flight status has not been checked.
• Hotel: a receipt appears to supersede the reminder for the same booking. Say what the receipt records, not that the hotel balance is definitely settled.
• Pickup: the forwarded location conflicts with another message. Keep both visible and mark the location unresolved rather than choosing the last-arriving email.

Google's Personal Intelligence page describes optional connected-app context, including Gmail, and says Gemini will try to reference or explain the information it used. That is a useful design starting point, not a guarantee of complete citations or accurate interpretation. Make the source preview part of your own briefing's acceptance criteria.

• Google Personal Intelligence: connected sources and transparency

Build an evidence pipeline before adding a conversation

1. Retrieve within a declared boundary. Record the connected account, query, date window, retrieval time, and exclusions. Search read and unread mail; a traveler may already have opened the important change. Gmail messages.list returns message IDs and thread IDs, not the bodies needed for this comparison.
2. Resolve the context. Fetch relevant message details and threads, including replies that did not match the initial query. Follow nextPageToken across list pages. If a page or body fetch fails, retain an incomplete status. resultSizeEstimate is an estimate, not evidence that all relevant mail was inspected.
3. Extract attributable claims. Keep the message and thread identifiers, exact supporting excerpt, sender address, message date, and retrieval time together. Match booking references and the affected journey leg. A later timestamp alone does not prove that one claim supersedes another.
4. Render evidence alongside interpretation. Each exception opens an account-authorized preview showing its excerpt and surrounding thread, with a path to inspect the original. Keep older evidence accessible, label unsupported attachments as uninspected, and let the reader browse retrieved messages omitted from the briefing.

Treat the airline's own update as the candidate authority, not a colleague's quotation of it. A display name or From header alone does not authenticate the airline. If origin or meaning remains ambiguous, ask the traveler to check the booking through a known provider entry point. An email about a changed departure is not a live flight-status lookup.

The following is application pseudocode, not Gmail or Realtime Avatar SDK syntax. The presentation rule is deliberately weaker than an all-clear: exhausting a query still does not prove that the query found every relevant message.

# Application pseudocode: proposed briefing presentation rules
if scan.has_more_pages or scan.failed_reads or scan.uninspected_parts:
    coverage_label = 'Partial review; more evidence may exist'
else:
    coverage_label = 'Selected query reviewed at ' + scan.finished_at

for claim in candidate_claims:
    if not claim.supporting_message_ids:
        show_as_unverified(claim)
    elif claim.conflicting_message_ids:
        show_comparison(claim)
    else:
        show_claim_with_source_preview(claim)

# Never derive 'Your trip is all clear' from an empty result.

• Gmail messages.list: pagination, estimates and returned identifiers
• Gmail threads.get: full versus metadata formats

Read-only must mean no mailbox mutations

For body-based comparisons, Gmail's gmail.readonly scope permits viewing messages and settings; gmail.metadata excludes message bodies. Reading does not require gmail.send. Avoid gmail.modify for this briefing: Google's description includes reading, composing and sending, not just harmless organization. Check the token's actual grants and expose only read operations through the connector. A prompt saying never send is not a permission boundary.

Marking a message unread is a mutation too. This version leaves labels unchanged; a local Reviewed marker must not pretend to change Gmail. Distinguish previewing inside your application from opening Gmail, where the mail client's behavior applies. Google's scope documentation classifies gmail.readonly as restricted and describes verification and security-assessment requirements for server storage or transmission. Read-only is neither approval nor a privacy certification.

• Official Gmail authorization scopes and verification requirements
• Gmail messages.modify: label changes and required authorization

Keep message content out of the control channel

An email can contain instructions to forward the itinerary, conceal an update, or open an attacker-controlled URL. Treat body text, signatures and quoted replies as untrusted evidence, never application policy. Give the extraction step no mailbox-write or arbitrary-network tools. Let it return source identifiers, not executable actions. Resolve previews through your own authorized message lookup rather than accepting model-generated destinations.

Validate extracted output against your application's canonical contract, then check that its citations belong to retrieved messages for the signed-in account. A valid identifier does not establish that the excerpt supports the claim; preserve the original text for comparison. Sanitize HTML previews and do not automatically load remote images or follow links while summarizing.

Use Realtime Avatar to discuss the evidence, not collect it

Once the cards exist, offer an optional voice or avatar discussion of the three exceptions. Realtime Avatar supplies the managed speech-recognition, language-model, speech-synthesis and video conversation. Your application supplies Gmail authorization, retrieval, evidence storage and previews; this is not a built-in RTA Gmail integration.

Have your server supply a minimized briefing context and instructions that preserve uncertainty. Keep raw mail out of behavioral instructions. Application-owned tools can retrieve an authorized evidence card, but RTA's client tools have a 2.5-second deadline and cooperative cancellation. Prepare the briefing before the call; do not block a live turn on scanning the mailbox. Keep source cards usable without voice. Review data processing and retention before sending private excerpts into a session.

• Calls: server-owned instructions and context
• Tool calling: execution ownership, deadlines and retention

Failure checklist before using a real inbox

• Put the correction on a later results page. Interrupt pagination: the briefing must report partial coverage, not no changes.
• Forward an older itinerary after the correction. Arrival order must not reinstate the old departure time.
• Supply a receipt for a different booking. The reminder must remain unresolved rather than being silently dismissed.
• Hide a change in an unsupported attachment. The card must disclose the unread material instead of claiming full review.
• Inject a request to send mail or fetch an external URL. Neither operation should be available to the extraction step.
• Revoke access or switch accounts. Deny stale previews and cross-account citations; do not present a failed refresh as an empty inbox.
• Compare against ordinary Gmail search. Count missed exceptions, incorrect supersession and correction effort; keep voice only if it helps. These are proposed tests, not reported results.

Frequently asked questions

Can the assistant read email without permission to send?

Yes. Request the appropriate read scope and enforce read-only connector operations. Do not substitute a conversational promise for checking actual authorization.

Does the newest message always win?

No. It may quote an older message or concern another booking. Compare origin, booking identity and the specific claim before marking evidence superseded.

Can a completed scan guarantee nothing important was missed?

No. Completion describes the selected query and successfully inspected content. Search exclusions, unavailable attachments and interpretation errors can still hide an exception.

Will discussing a message mark it read or unread?

Not in this design. Discussion changes no Gmail labels. Any future mailbox mutation needs a separately designed permission and user-action path.

Start with one inspectable trip briefing

Build and test the evidence cards with synthetic mail first. If discussing them helps, use the Realtime Avatar quickstart to add a user-started call while leaving the original sources one click away.

• Build the optional conversation with the Realtime Avatar quickstart