AI Governance & Making LLMs Predictable in Production
Most LLM-powered systems hit a token limit and do the dumbest possible thing: they chop from the front. Take the first N characters, summarize it, discard the rest. Sometimes the logic is slightly smarter: random sample, drop from the middle, oldest-first. But the result is the same. You might silently lose the fact that the player killed the town’s mayor three sessions ago, while keeping five lines of small talk about the weather.<br> More
I’ve been working on AI-powered JIT compilation recently, which in turn led me to test several “autonomous” AI coding agents for research. The pattern across the board is strikingly consistent: impressive demos masking a disappointing reality.
One agent took 9 hours to fix a simple configuration bug, blindly claiming “Fixed!” multiple times without ever actually validating its work. Another wasted 11 commits rewriting documentation when I had explicitly asked for a bug fix.<br> More
Wednesday morning: I had game rules scattered across eight design documents, each making conflicting assumptions about how fights resolve in LAIRD 2. No single source of truth. No clear way to sync them. So I designed a tool I called “claude-interrogate“.
On Wednesday afternoon: I had built a full production MCP server that reads design docs, interviews me Socratically, and generates formatted specs in my voice.<br> More
Every LLM integration starts with the same white lie: “I’ll just add some guardrails.” But that wont solve the core problem. Deterministic LLM behavior is impossible if you treat the model as a black box.
The naive fix is temperature=0, but that collapses into repetitive, lifeless output. I needed variance. I also needed control. These requirements appeared mutually exclusive until I changed my mental model.<br> More
