The ceiling moved and the reason for short skills stayed put

Two talks published from the same conference on September 9, and they pull in opposite directions. Laurie Voss of Arize reran an instruction following benchmark called IFScale, where the model has to work a list of exact words into a business report. A year ago, per their replication, frontier models fell apart somewhere around 200 to 300 rules. On the current crop they had to push the test to a 10,000 word vocabulary to find a ceiling at all, and they put the new bend near 2,000 rules, up to 5,000 for the best of them. The advice follows: stop sharding your skill files to fit the model. If the job needs 300 rules, put in 300 rules. The whole experiment cost 29 dollars.

Same day, Ajay Prakash of LinkedIn describes the opposite build. Over 600 playbooks behind one internal MCP server, used by more than 8,000 people a day, with two rules for authors: a playbook does one specific task, and a big playbook is broken into small ones that reference each other. Agents find them through three meta tools, search, get schema, execute, because listing everything at once degraded the system past a few dozen entries.

Voss is right about the number and honest about the caveat. It is a proxy task, and the models that stopped forgetting now fail four different ways, one of which is a polished report that quietly gives up halfway through.

The vault’s skill notes never justified short skills by the ceiling. They justify them by progressive disclosure, so many skills can exist without the context window becoming a manual, and by testability, since a skill earns its place when it can be verified against real runs. The same page records that curated skills helped unevenly in one benchmark and self-generated ones did not help on average.

So the compression problem is gone, as Voss says. The reason to keep a skill small was never only that the model would drop rule 201. It was that a person has to be able to find the rule that is wrong.