What the demo doesn't tell you

No — and I want to be clear, because the easy version of this story is that someone fell for a slick pitch. The demo was genuinely good. An agent read a customer's support ticket, pulled the account history, reasoned about the problem, resolved it — end to end, no human — while the room watched. When it finished, the board applauded. I'm not a sentimental man and I felt the future arrive in my chest. I shipped it.

Eleven weeks later I turned it off. Not in a crisis, not after a disaster that made the news — quietly, the way you put down a project everyone can see isn't working but no one wants to name. The reason is the whole essay: it worked about ninety percent of the time, and the other ten percent was unpredictable, occasionally catastrophic, and impossible to see coming.

Because of what a demo is, structurally. It's the happy path, under good lighting, with data someone cleaned the night before. The median case at its best. And the median case was never the question. You don't lose sleep over the ticket the agent nails. You lose sleep over the one where the account data was stale, the customer asked something sideways, and the agent — sounding exactly as confident as it did on stage — did something wrong with total conviction.

A demo tells you the agent can be right. It doesn't tell you the agent won't be wrong. Those are different facts — and the second is the only one that matters in production.

My ten percent wasn't random noise to shrug off. It was the long tail — the weird tickets, the edge accounts, the malformed inputs that arrive at three in the morning when no human is watching. A demo has no long tail, because you don't demo the long tail. But production is the long tail. Production is nothing but edge cases wearing trench coats, lined up, waiting their turn. My mistake — and it was mine — was treating the agent as a feature I could ship, rather than a system I had to operate.

I stopped talking about "the agent" almost entirely. The model — the clever part the demo was selling — turned out to be the smallest, most interchangeable piece. What I'd been missing was everything around it. That's where Databricks came in, and the unglamorous truth is the work that fixed it had almost nothing to do with intelligence and almost everything to do with discipline.

The worst answers had come from the agent improvising on thin or stale data — it was reasoning over a half-remembered version of the truth. An agent is only ever as trustworthy as the data under its feet, and mine had been quietly lying to it. The eval harness sounds boring and is completely essential: I could change something and know whether it helped, before a customer found out for me. And observability meant that when it got something wrong — it still does, everything does — I could see where, instead of shrugging at a black box. The ten percent didn't vanish. It shrank, became visible, and stopped being catastrophic. That's the most you can honestly ask of any system that makes decisions.

The version I run now never got a standing ovation. It shipped quietly, to a narrow slice of tickets, with a human in the loop on anything it wasn't sure about, and it earned more responsibility one boring, measured week at a time. By every demo metric it's less impressive than the thing the board applauded. It's also still on.

The companies that win with agents this decade won't have the cleverest model — models are becoming a commodity, everyone will have a good one. The winners will be the ones who did the unglamorous work of turning a clever thing into a dependable one.