A protein language model appears to have learned convergent evolution on its own terms
Alex Rives describes a finding inside ESMC that no one explicitly trained the model to produce: a single internal feature that activates across evolutionarily distant protein families sharing the same structural motif. The implications for what these models are actually learning deserve careful attention.
Protein language models were built to predict structure and function. What Alex Rives describes finding inside ESMC is something more unexpected: evidence that the model may have organized its internal representations around a biological concept that evolution itself discovered more than once.
The motif in question is the nucleophilic elbow, a structural arrangement found across protein families that arrived at it through entirely separate evolutionary paths. Convergent evolution, in the biological sense, is the phenomenon by which unrelated lineages independently settle on the same solution to a shared problem. It is one of the more striking patterns in the history of life, and it is not the kind of thing a model learns by being told to look for it.
Rives reports that ESMC appears to have developed a single internal feature corresponding to this motif, and that the feature activates consistently when the model encounters the nucleophilic elbow regardless of which evolutionary lineage a given protein comes from. In his words, “the model has a kind of a single feature for this nucleophilic elbow and it’s activating across these very evolutionarily diverse families.”
The model has a kind of a single feature for this nucleophilic elbow and it's activating across these like very evolutionarily diverse families Alex Rives
That is a precise and consequential observation. A single feature, not a family of features tuned to each lineage separately, fires across organisms that share a structural solution but not a common ancestor for that solution. The model was not handed a taxonomy of convergent motifs. It was trained on sequences, and something in that training produced a representation that collapses evolutionary distance in favor of functional geometry.
The field of mechanistic interpretability in large language models has spent considerable effort trying to identify whether sparse, human-interpretable features emerge inside these systems at all. The answer, in the text domain, has increasingly been yes: features corresponding to syntax, sentiment, and entity type appear in models trained only on next-token prediction. What Rives describes suggests the same phenomenon extends into the biology domain, and perhaps more sharply, because the concept that has been internalized, convergent evolution toward a shared functional motif, is one that required centuries of comparative biology to formalize in humans.
The finding raises a harder question about what protein language models are doing when they generalize. Standard accounts treat generalization as interpolation across training distribution. If a single feature activates across evolutionarily independent lineages that share only a functional geometry, the model may be doing something closer to abstraction: grouping proteins not by ancestry but by what they do and how their shape accomplishes it. That is not the same as interpolation, and it is not what the training objective explicitly rewards.
None of this settles what ESMC understands, or whether the language of understanding is even the right frame. What Rives has identified is a structural fact about the model’s internals that does not have an obvious alternative explanation. A single feature, consistent activation, evolutionarily diverse inputs. The geometry of the finding is clean even if its implications are not yet fully traced. That is usually where the more durable scientific questions begin.