The good argument for local is not the electricity bill

Two local AI demos went by this week and both closed on money. One is a desktop box with an image and video stack preinstalled, generating a few hundred images and then left running overnight on videos, where the pitch is that the only cost was electricity instead of paying by the second. The other brings four kinds of model onto a laptop with one install, speech to text, embeddings, a language model, text to speech, where the pitch is not configuring four separate runtimes.

The cost framing is the weaker half of both arguments, and the vault’s notes on local stacks say why in a line. Local is not free. It moves cost out of a visible monthly bill and into hardware, power, cooling, maintenance, slower iteration, lower peak capability, and the hours of whoever keeps it running. A hosted bill is a meter you can read. A machine in the corner is a cost you have to go looking for.

The second demo does say the real argument, almost in passing. Since the models are just files sitting on the machine, there is no risk of being rate limited, re-priced, or having something deprecated out from under you.

That is not a money argument. That is a question of whether the thing you built last quarter still runs next quarter. The vault lists reasons to self host in an order I think is right: privacy for sensitive files first, then repetitive narrow loops that are cheap to verify, then fallback capacity for when a provider throttles or changes the rules.

So before buying hardware, the question is not what the tokens cost. It is which parts of your work would simply stop if the service you lean on doubled its price or dropped a model on a Tuesday. Those are the parts worth running on files you own. The rest of it is a bill, and a bill you can read is not the enemy.