subtopic

Caching, model routing and fall-backs

Cut cost and latency safely: cache exact repeated questions (and optionally similar ones with embeddings) in Redis with an expiry; route easy requests to a smaller/cheaper model and hard ones to a stronger model; fall back to a second provider when the first fails or is slow; set per-user and per-day budgets. Measure before/after on the eval set.

Builds on: 3.6 Run it reliably, 3.7 Evaluate