Bilinear sparse featurizers (BSFs) produce richer features than SAEs, avoid SAE pathologies, and are likely to become the standard interpretability tool for LLM residual streams.

Bilinear sparse featurizers (BSFs) produce richer feature representations than SAEs and avoid SAE pathologies.

“Features are much richer than we may have otherwise seen and they don't suffer from some of the same pathologies sapes suffer from.”