subtopic

Latency budget

Learn the target: a natural reply starts within about 1 second after the caller stops speaking (good agents reach ~600–900 ms). Add up the budget: end-of-turn wait + STT final + LLM first token + TTS first audio + network. Ways to cut it: streaming everywhere, a fast small LLM, short replies, servers close to users and providers (India region), keeping connections warm, speaking a filler while a tool runs.

Builds on: 1.2 How LLMs work