Working with the API
The request/response surface you actually integrate against, and the four things that decide its cost and latency.
8 topics
The Messages API Shape
One stateless endpoint that takes a system prompt, an alternating message list, and a tool list, and returns typed content blocks with a stop reason.
core20 minStreaming & Server-Sent Events
Why every serious LLM feature streams, what the event sequence actually looks like, and the plumbing problems that appear the moment you put a server in the middle.
core25 minStructured Outputs
Constraining the model to a JSON schema so the response is a typed object you can rely on, rather than prose you have to parse and pray over.
core20 minReasoning, Effort & Thinking
Models that spend tokens thinking before answering, and the one dial that trades quality against cost and latency across your whole product.
core20 minPrompt Caching
Reusing the processed prefix of a prompt across requests for roughly a tenth of the price — and the byte-level rule that decides whether it works at all.
core20 minToken Accounting & Cost
Turning the usage object into a cost per request, a cost per user, and an answer to "why did the bill triple?"
core20 minRate Limits & Retries
Provider limits are per-minute token budgets, not just request counts — and the retry strategy that works is the one that doesn't stampede.
core20 minBatch & Async Processing
Half price for work nobody is waiting on, and the job architecture that any long-running LLM task needs anyway.
deep15 min