Liquid AI's automated architecture search quietly eliminated the hand-tuned gating that rivals still rely on

Liquid AI’s AFMD automated architecture search arrived at an unexpected conclusion: the hand-tuned gating mechanisms built into popular efficient architectures like Mamba and gated delta nets are not necessary. The search returned a simpler structure, which Hasani calls the double gated convolution, as the optimal efficient form. The implication is that a lot of careful human engineering in competing models was, in retrospect, overhead.

Turns out all of this has to go away if you want to get to the most efficient form of format of architecture, and it became the double gated convolution that actually came out of this massive search space like AFMD. Ramin Hasani

The broader Liquid AI story runs from theory to deployment. The company published the closed-form solution to liquid neural network neuron-interaction equations in Nature Machine Intelligence in November 2022, giving the research a concrete academic anchor. More recently, the Apollo App ships a 1-billion-parameter hybrid attention-plus-gated-convolution model that runs on-device on an iPhone for private local data search and classification. Hasani describes the use case plainly: on-device, fast enough for basic tasks, private by design.

On the commercial side, Hasani says fine-tuning a Liquid AI device model up to cloud quality costs “between tens of dollars to let’s say like low thousands of dollars.” The company also claims over one million downloads per week on HuggingFace and the number five spot on that platform’s US downloads leaderboard. None of those claims have been independently verified, but they sketch a picture of a lab moving from architecture research toward broad distribution.