Jose Nicholas Francisco

Real-Time Voice AI Stack for Agents: Architecture Guide

real-time voice AI latency budget25 août 2026

One-second turn deadline

A production voice agent must begin speaking within about one second of the user's utterance ending, or the exchange feels broken.

Éléments favorables

Jose Nicholas Francisco·25 août 2026

Extrait original

It has about a second to start talking before the exchange feels broken.

Contexte

A production voice agent chains transport, speech recognition, a language model, speech synthesis, and orchestration into a single real-time exchange, and each layer draws from the same shared clock.

Ouvrir l’original · Paragraphe 1 Source du texte original

perceived latency thresholds

Real-Time Voice AI Stack for Agents: Architecture Guide

perceived latency thresholds25 août 2026

600–800 ms perceived willingness drop

Perceived willingness to continue the interaction begins to decline after 600 ms and drops significantly between 700 and 800 ms.

Éléments favorables

Jose Nicholas Francisco·25 août 2026

Extrait original

Perceived willingness begins to drop after 600 ms and steps down significantly from 700 to 800 ms.

Contexte

Transport, recognition, endpointing, the LLM's first token, TTS first byte, and the return network hop each take a slice of that budget. This guide breaks down what each layer costs and where you can cut to keep your voice AI stack inside it.

Ouvrir l’original · Paragraphe 2 Source du texte original

production latency SLOs

Real-Time Voice AI Stack for Agents: Architecture Guide

production latency SLOs25 août 2026

P95 under 1,000 ms target

A realistic production per-turn latency target is P95 under approximately 1,000 ms; beyond a couple of seconds, callers begin to disengage, and percentile-based SLOs—not averages—should be used and calibrated against user studies and real caller feedback.

Éléments favorables

Jose Nicholas Francisco·25 août 2026

Extrait original

Target a P95 turn latency under about 1,000 ms, and treat anything beyond a couple of seconds as the point where callers start to disengage. Set percentile SLOs, not averages, and calibrate against user-study research like this IEEE study on perceived response delay in voice interfaces, then confirm the specific thresholds against your own caller feedback.

Ouvrir l’original · Paragraphe 97 Source du texte original

voice agent function calling

Voice Agent Function Calling: Real-Time Data Lookup Guide

voice agent function calling12 sept. 2026

Function calling enables direct answers instead of call transfers

Function calling allows a voice agent to directly answer questions like 'Where’s my order?' rather than transferring the caller to a human agent.

Éléments favorables

Jose Nicholas Francisco·12 sept. 2026

Extrait original

Function calling is how a voice agent answers "where's my order?" instead of transferring the caller to someone who can.

Contexte

One order-status lookup on the inbound telephony agent pattern takes two round trips, three function calls, and a spoken line for every way it can fail.

Ouvrir l’original · Paragraphe 1 Source du texte original

latency tolerance in voice interactions

Voice Agent Function Calling: Real-Time Data Lookup Guide

latency tolerance in voice interactions12 sept. 2026

Perceived willingness drops significantly after 600–800 ms of silence

Ratings of perceived willingness drop noticeably at 600 ms and become statistically significant between 700 and 800 ms, based on the 2013 gap-tolerance study by Roberts and Francis.

Éléments favorables

Jose Nicholas Francisco·12 sept. 2026

Extrait original

Ratings of perceived willingness drop off noticeably at 600ms in the classic 2013 gap-tolerance study from Roberts and Francis, and the difference turns statistically significant between 700 and 800ms.

Contexte

Dead air is what the caller hears while your handler works, and it costs you fast. InjectAgentMessage is what covers that silence in the reference pattern.

Ouvrir l’original · Paragraphe 22 Source du texte original

function selection accuracy

Voice Agent Function Calling: Real-Time Data Lookup Guide

function selection accuracy12 sept. 2026

Tool count degrades selection accuracy: 98% at 10 tools → 88% at 100

The HumanMCP study measured a decline in function selection accuracy from 98% with 10 tools to 88% with 100 tools.

Éléments favorables

Jose Nicholas Francisco·12 sept. 2026

Extrait original

The HumanMCP study measured a drop from 98% at 10 tools to 88% at 100.

Contexte

When the toolset grows, selection accuracy falls. Add functions one at a time and re-check selection accuracy after each. One lookup, three failure lines, and a filler function make a complete first release.

Ouvrir l’original · Paragraphe 72 Source du texte original

session limits in voice agents

Voice Agent Function Calling: Real-Time Data Lookup Guide

session limits in voice agents12 sept. 2026

Two-hour session limit applies regardless of in-flight lookups

Voice agent sessions expire after two hours; an in-flight function call does not extend the session, and the system emits MAXIMUM_SESSION_LENGTH_APPROACHING at 1h 55m and MAXIMUM_SESSION_LENGTH_REACHED at 2h.

Éléments favorables

Jose Nicholas Francisco·12 sept. 2026

Extrait original

Yes, at two hours, and an in-flight lookup won't hold the session open. You get a Warning with MAXIMUM_SESSION_LENGTH_APPROACHING at 1 hour 55 minutes, then an Error with MAXIMUM_SESSION_LENGTH_REACHED at the two-hour limit .

Contexte

KeepAlive doesn't extend it, and the docs don't cover a pending FunctionCallRequest at close. Finish or abandon it, then reconnect with agent.context .

Ouvrir l’original · Paragraphe 83 Source du texte original