Liquid AI's architecture search found that hand-tuned gating is a dead end

Ramin Hasani, a founder at Liquid AI, has made a claim that cuts against one of the more settled convictions in modern sequence modeling: that sophisticated, hand-crafted gating is a feature, not a bug. According to Hasani, an automated architecture search process the company calls AFMD reached the opposite conclusion. When the search was allowed to explore the space without prejudice, the complex gating mechanisms that define models such as Mamba and gated delta nets were not selected. They were eliminated.

The result Hasani describes is not a modest refinement. It is a structural verdict on a class of design choices that the research community has spent significant effort developing and defending. Gating mechanisms in sequence models have generally been treated as a hard-won solution to the problem of selective state updates, a way of giving a model fine-grained control over what information it retains across long sequences. The underlying assumption is that this control requires architectural complexity. Hasani’s account of the AFMD search says that assumption does not hold when efficiency is the objective criterion.

What the search returned instead, by Hasani’s account, was a double gated convolution. That is a considerably simpler structure than the mechanisms it displaces. The implication is that the field has been adding machinery that does not survive contact with a sufficiently thorough search over the possible design space. The hand-tuning that produced Mamba-style gating was a form of expert intuition applied to a space too large to search manually. AFMD, as Hasani describes it, searched that space at a scale where intuition cannot follow, and the answer it returned was simpler than the question suggested it would be.

Turns out all of this has to go away if you want to get to the most efficient form of format of architecture and it became the double gated convolution that actually came out of this um this massive search space like AFMD Ramin Hasani

This kind of result has a precedent in the history of neural architecture search more broadly. Human designers tend to import assumptions from prior generations of models, and those assumptions accumulate as constraints that narrow the search implicitly, even when no one intends to constrain it. Automated search methods that are genuinely unconstrained have repeatedly found designs that human researchers would not have proposed, sometimes simpler and sometimes structurally alien. Hasani’s claim fits that pattern, though the specific conclusion, that gating of the Mamba variety is unnecessary rather than suboptimal, is a sharper claim than the usual finding of marginal improvement through search.

The word Hasani uses is worth sitting with. He says all of the gating complexity “has to go away.” That is not the language of a tradeoff. It is the language of a search that found the complexity was not load-bearing. If the double gated convolution achieves equivalent or superior efficiency without the gating apparatus, then the gating apparatus was solving a problem the architecture did not actually have, at least not in the form the hand-tuned designs assumed.

What makes this claim consequential is not only the architectural specifics. It is what the AFMD result says about methodology. If automated search at sufficient scale can overturn design choices that a significant portion of the research community has converged on, then the field’s reliance on expert intuition as the primary driver of architecture development deserves examination. The AFMD process, as Hasani describes it, treated the design space as genuinely open rather than as a space to be explored around a set of fixed priors. The outcome suggests that the fixed priors were doing more work than the researchers holding them realized.

Liquid AI has not yet published results that would allow independent assessment of these claims, and the account here rests entirely on Hasani’s own description. The specific mechanics of AFMD, the scale of the search, and the precise performance characteristics of the double gated convolution relative to Mamba-style architectures are not detailed in what Hasani has said publicly. Those are the questions a skeptical reader will want answered before treating this as a settled finding rather than a reported claim from a party with an interest in the result. What can be said is that the claim is specific enough to be testable, and the methodology it describes, systematic search over a large architecture space with efficiency as the objective, is a legitimate approach. Whether the double gated convolution holds up under that scrutiny is the question the claim invites.