The default reasoning effort was set for the benchmark, not for you

Simon Willison spent the weekend with Qwen 3.8 27B, an Apache 2 licensed model small enough to run on a good laptop, and found that it ships with reasoning effort set to the highest of its three levels by default. He calls that a hilarious default. His first attempt at the pelican riding a bicycle drawing took twenty one minutes and burned 22,276 reasoning tokens to produce 3,223 tokens of output. He rates the result the best he has gotten out of a local model. Asked whether it was worth twenty one minutes, he says absolutely not. The same prompt with reasoning turned off finished in a bit over two minutes.

There is a reason a lab picks that default and it is not carelessness. The top setting is what the benchmark table was built on, and on a leaderboard nobody pays for the tokens or watches the clock.

The vault’s research notes are blunt about when the extra thinking earns its keep. Reasoning compute pays when the task has a verifier, when tools feed results back, when the answer is worth real money, or when the person can wait. Send easy work down the cheap path and escalate only when more thinking could change the outcome. Judge it on utility per dollar and per second instead of benchmark score. Those notes also point out that reasoning benchmarks only compare when the inference budget is stated, which headline numbers rarely bother to do.

So the default is not wrong so much as it is answering a question that is not yours. It was tuned for the score. You are paying in minutes on your own hardware, and the switch between those two lives in one line of the model card.