#evals
3 posts. Other tags: #llm · #agents · #nodejs · #typescript · #tutorial · #fintech · #healthcare
LLMs in production: the hard part is the contract around the model
What I learned putting OpenAI, Anthropic and Gemini behind an accounting product: schemas, deterministic checks, confidence thresholds and a human review queue.
Three ways LLMs fail in production, and what to do about each
API errors, broken output and answers that look right but aren't. Each one needs a different defense.
Building an LLM feature, part 4: test and evaluate it
Unit tests for the deterministic parts, a golden set for the model, and an eval script that reports accuracy, coverage and confident mistakes and can fail CI.