Pick the model for the job, not the leaderboard

A developer posted a benchmark for model routers this week under a blunt title: the routers optimize the wrong axis. I have not run it, and the project is small, but the complaint stuck with me because it names something the whole race gets wrong. We keep asking which model is best, as if best were a single number you could sort a column by.

A router, plainly, is the thing that decides which model gets a given request. Send everything to the biggest model and you pay for a bulldozer to hang a picture. Send everything to the cheapest and you hand your hardest thinking to the model least able to do it. So a router chooses. The only question that matters is what it chooses on.

Most routers choose on one axis. Price, or a benchmark rank, or raw speed. And that is the mistake, because the right model is a function of the job, not a leaderboard. The vault’s routing matrix is built entirely around that: one model for the long multi-day build, another for fast tool execution, another when the context is enormous. The durable rule underneath it is to spend the expensive model where judgment lives, the planning and the checking, and let cheap models grind the loop in between. Never hand project ownership to a mini model to save a nickel.

Having said that, I do not want to talk you out of routing. A sane default router beats hand-picking every single call, and most of the time you should not be thinking about this at all. That is the point of it.

Still, watch the axis. A router tuned for the cheapest token will, by design, route your one genuinely hard step to your weakest model, at the exact moment fit mattered more than price. Cheaper is easy to measure. Right is the thing you actually wanted.