Production & Ops
Run AI at scale: serving, observability, cost control, and safe iteration.
This track assumes you already know
- 1
Serve and scale
- How Inference Serving Works Batching, scheduling, and memory management. Why throughput and latency pull against each other, and what a serving system is actually doing. Advanced Architecture
- Self-Hosting Models Running open-weight models on your own hardware. When it makes sense, what it actually costs, and the arithmetic to do first. Production
- Latency Optimization Perceived speed is dominated by time to first token, and most of that is prefill. Where the milliseconds actually go. Production
- 2
Observe and control
- Observability for LLM Systems When a user reports a bad answer next week, you need to reconstruct what happened. What to log, and the one field everyone omits. Production
- Cost Monitoring and Attribution Token spend is easy to measure and easy to ignore until the bill arrives. What to track, and which dimension actually explains your costs. Production
- Reliability Patterns Providers rate-limit, time out, and return malformed output. The patterns that keep a system up when its dependency is unreliable. Production
- 3
Iterate safely
- Prompt Versioning Prompts are code that changes behaviour. Treat them accordingly, and keep the eval result attached to the version. Production
- A/B Testing AI Features Nondeterministic output and subjective quality break the usual experiment design. What to measure instead, and why offline evals come first. Production
- Migrating Between Models New model, better benchmarks, worse output. Prompts are tuned to models, and migration is a real project rather than a config change. Production
- Statistical Significance: Why One Comparison Deceives Model A scored 84%, model B scored 86%. Is B better, or did it get lucky on your test set? Significance testing separates a real difference from noise. Evaluation