Kimi K3 dropped the positional embedding its own inventor built, and that detail deserves more attention than it has received

Simon Mo has flagged something that cuts against the normal logic of architectural progress: the Kimi K3 model removed rotary positional embedding, and the engineer who made that call is the person who originally invented it.

That pairing is worth sitting with. RoPE, shorthand for rotary positional embedding, has been a fixture of Transformer architecture for long enough that most practitioners treat it as settled infrastructure rather than a live design choice. Positional embeddings exist to give a model its sense of sequence, its understanding of which token comes before which. RoPE in particular became widely adopted because it handled that task efficiently and generalized well across context lengths. It is not a component most teams would put on the chopping block.

Which makes the Kimi K3 decision unusual on its face. Removing a core architectural component from a model at this level of development is a meaningful bet. Teams inherit prior design decisions precisely because those decisions carried the field forward, and unpicking one of them requires both confidence that something better is available and a deep enough understanding of why the original choice was made to know exactly what you are giving up.

The detail Mo highlights is that the person who made this removal is not simply someone who knows RoPE well from using it. It is the original inventor. That framing reframes the story entirely. This is not a team discarding a tool they inherited without fully understanding it. It is the person who built the tool, who understands its foundations, tradeoffs, and limits better than anyone, deciding that it no longer belongs in the architecture.

They removed a uh rotary positional embedding. So rope has always been there for a lot of the Transformers model and guess who removed it is the inventor of rope Simon Mo

There is a specific kind of credibility that attaches to that kind of reversal. When a builder concludes that what they built should be retired, the conclusion carries weight that outside critics cannot easily replicate. It is not a competitive team arguing that a rival’s approach is flawed. It is the original designer working through the logic far enough to arrive at a different answer. That is a different signal.

What Mo does not spell out, and what the claim alone cannot settle, is what replaced RoPE in K3’s architecture, or what the measured tradeoffs looked like in practice. Those are the natural follow-on questions. A removal at this level is not self-explanatory, and the performance implications would need to be weighed against whatever the new approach provides. But the significance of the decision does not depend on having those answers in hand. The architectural choice itself, taken by the person most qualified to make it, is the news.

For practitioners watching how frontier model development is evolving, the Kimi K3 case is a concrete example of something that gets discussed more abstractly: that the accumulated design conventions of the Transformer era are not permanent. They were built to solve problems as those problems were understood at a particular moment. As the problems change, as context lengths grow, as inference requirements shift, the solutions that made sense in earlier conditions are being revisited. When the person who introduced a given solution is the one doing the revisiting, that is a signal worth tracking carefully.

Mo presents this as a notable fact rather than a detailed technical account, and the claim should be read in that register. It is a sharp observation about who made a significant architectural call, not a full technical writeup of what K3’s architecture now looks like or why the change was made. But precise claims with clear attribution are often more useful than exhaustive ones. This one points at something real: the architecture of large language models is still genuinely in motion, and the people with the longest view of why things were built the way they were are among those doing the rebuilding.