Buy the receipt before you buy the wall

Two entries from the same day’s capture. One is a vendor-neutral agent control plane. The other is a Show HN for monitoring ML models after deployment, and its author states the problem plainly: a model can keep running normally while something underneath it is starting to go wrong. A few lines over, trajectory-aware routing that cuts agent cost. A category is forming in public.

The fear behind it is the right fear. Silent failure is the characteristic agent failure. Nothing throws, the exit code is zero, the run completes, the summary is confident, and the wrong thing is now committed or sent. If the only signal you have is that the process finished, you do not have a signal.

The vault’s observability notes start somewhere much less impressive than a control plane. One trace, following one request from input to result: messages, model steps, tool calls, retrieved documents, permissions, retries, errors, outputs, state transitions, with identifiers stable enough that a person can reconstruct the run without trusting a chat summary. The minimum record includes a field most dashboards skip. Task outcome and acceptance result. Not completed. Did it do the thing, judged against something that can say no.

The notes are also direct that observability is not an eval. It supplies the real failures that make evaluation credible. And a dashboard is not the goal by itself. Start from error analysis of actual runs, and make every metric answer an operating question: did this complete correctly, what failed, what changed, what did it cost, who acts next.

The buying order usually runs the other way. Get the panel, then work out what belongs in it. A wall of green tiles is a screensaver with a subscription.

Whether the agent is running was never the hard question. Whether anyone would notice it had stopped being right is.