Quadratic attention cost makes linear attention the only viable approach for extremely long context windows.

Many AI-native companies will have trillions of tokens of internal proprietary data within 18 months (by approximately January 2028).

“I think that people don't fully comprehend the size of the of the knowledge workspaces they will deal with in 18 months in 18 months many companies would maybe trillions of tokens which of internal company of internal company data proprietary data I'm talking about like maybe trillions it sounds exaggerated but I don't think it's an impossibility if they're really AI native.”

Liquid AI solved the liquid neural network neuron-interaction equations in closed form for the first time, resulting in a paper published in Nature Machine Intelligence in November 2022.

“We solved the liquid neural network kind of interaction of neurons with each other in closed form for the first time. this became a nature machine intelligence paper published in November of 2022.”

LTX Video's variable tokens architecture allows models to allocate more tokens to challenging areas of a scene such as physics.

“We finally were able to crack variable tokens architecture.”