AI safety frameworks are measuring the wrong thing as inference spending redefines model capability

Noam Brown has identified a structural gap in the safety architecture that the AI industry has spent years building. The preparedness frameworks and responsible scaling policies that labs treat as their primary governance instruments share a common assumption: that model capability is a property you can measure at a fixed point and then regulate accordingly. Brown’s argument is that this assumption no longer holds.

The problem is test-time compute. As inference spending has become a meaningful lever for extracting capability from a given model, the question “how capable is this model” has lost a stable answer. A system evaluated at one inference budget can perform at a substantially different level when more resources are directed at it during deployment. If the evaluation happens at one spend level and the deployment happens at another, the safety threshold set during evaluation may not correspond to the system that users and institutions actually encounter.

Brown puts the core tension plainly: the current frameworks assess a model’s capability without accounting for how much compute is applied at inference time. What they treat as a fixed ceiling is, in practice, a floor that scales with spending. That is not a minor calibration issue. It means the risk profile of a deployed system can diverge from its evaluated profile, and the divergence grows as inference costs fall and organizations have more economic reason to scale up at runtime.

The preparedness frameworks and responsible scaling policies, they don't really account for the amount of test time compute. They just say, 'Okay, well, what's the capability of the model? The problem is we're in a world now where the capability of the model is a function of how much money you put into it.' Noam Brown

The implications run in several directions at once. Safety evaluations are expensive and time-consuming, which is part of why frameworks anchor them to a model as trained rather than a model as run. But if a system’s effective capability at deployment is meaningfully higher than its capability at evaluation, then the thresholds that trigger additional scrutiny or restrictions may never be reached on paper, even as the system crosses them in practice. The framework would be working as designed while failing at its purpose.

There is also a competitive dimension. Labs operating under responsible scaling policies commit to specific capability thresholds above which they will not deploy without additional safeguards. If those thresholds are defined in terms of a model’s base capability rather than its capability at realistic inference spend, a lab can remain technically compliant while deploying systems that, under the old framing, would have required a harder look. That is not an accusation of bad faith. It is a description of what happens when a governance framework is built around a variable that has since stopped being stable.

Brown’s critique does not offer a remediation plan, and it would be overreaching to suggest one from a single observation. What the observation does is name a gap that the frameworks themselves have not addressed publicly. Responsible scaling policies are living documents at several labs, revised as understanding improves. Whether test-time compute enters that revision process, and how evaluators would even construct a principled budget for capability assessments, are open questions the field has not answered.

What is clear is that the premise Brown is challenging sits at the foundation of current AI governance practice. If capability is now a function of inference spending rather than a fixed model property, then the unit of analysis that safety frameworks are built around needs to change. The alternative is a growing distance between what the frameworks certify and what the systems actually do.