Streaming, batching, retries and rate limits

Learn what real apps need when calling an LLM API: streaming (show the reply word by word so users don't wait), batching (process the 120 test emails in parallel with a limit, or with the provider's batch API at lower cost), error handling (timeouts, retry with increasing waits on rate-limit and server errors), respecting rate limits (requests and tokens per minute), and keeping API keys in environment variables or a secrets manager. Also learn to switch providers (OpenAI, Gemini, Claude, Ollama) behind one function, e.g. with LiteLLM, so you can compare models without rewriting code.