Observability & Cost
Seeing what your LLM feature actually did, what it cost, and how slow it was for the person waiting.
5 topics
Tracing LLM Applications
A request that fans out into retrieval, several model calls, and a dozen tool invocations is unreadable in logs — it needs a trace, and OpenTelemetry now has conventions for exactly this.
core20 minUnit Economics of an LLM Feature
What one user, one task, and one plan actually cost you — the analysis that decides whether an AI feature is a product or a subsidy.
core20 minLatency Budgets
Where the seconds actually go in an LLM request, which of them the user perceives, and the handful of techniques that move each one.
core20 minCaching Strategies for LLM Apps
Four distinct caches — prompt prefix, exact response, semantic, and embedding — each solving a different problem, and each with its own way of going wrong.
core20 minMonitoring Quality in Production
Quality regressions in an LLM feature arrive silently — no errors, no alerts, just answers getting slightly worse — so you have to monitor for them deliberately.
core20 min