What is Qwen?

Some Qwen models show benchmark improvement when trained with RL on literally random rewards, raising questions about base model quality or evaluation validity.

“We showed that some of the Qwen models actually will improve in if you train them with RL on literally random rewards.”

A Qwen paper found RL training is approximately 17x more compute-heavy than distillation for small models.

“The Quen paper showed that the RL was like kind of 17 times more compute.”

OmniDraft transfers speculative decoding draft models across different model families (Llama 3, Vicuna, Qwen) by using an n-gram cache to bridge tokenizer differences, enabling cross-family inference acceleration.

“It's called omnidraft so basically it enables you to transfer the draft model train for speculative decoding across different families of networks and they I guess they show it on Llama 3, Vikuna and Quen models and they come up with some with an with an interesting approach and Graham cache the difference the main the main difference is across families of models we have different tokenizers and how do you make sure that one tokenizer corresponds to another one and engram cache basically bridges that gap.”

A Bittensor subnet built a model that outperformed Qwen's best 35B model, but Qwen shipped a superior 35B update just before the Bittensor launch, illustrating the speed disadvantage decentralized teams face against centralized incumbents.

“Last year we did produce a model that was better than Quen's best model at the 35 billion range. And just before we launched it, they launched another 35 billion model that was better than ours.”

Qwen maintains a daily-updated preview API endpoint, enabling near-continuous iteration that gives it outsized influence over the research ecosystem.

“They have an endpoint which you can use for a preview version and they've updated this endpoint daily.”

Open reasoning models including Qwen exhibit repetition meltdowns in roughly 0.01% or more of samples, where paragraphs repeat 50 to 100 times before the model recovers.

“0.01% or even more. So, like one in a thousand or one in ten thousand samples from Qwen have an mental break where the model will just repeat paragraphs like 50 or 100 times.”

Alibaba's Qwen 30 billion parameter model is reported to perform on par with GPT-4o.

“There was a release today of a quen you know 30 billion parameter model which is performing as good as GPT40.”

US open-source models from Nvidia (Nemotron) and Google (Gemma) are not sufficient substitutes for Chinese open-weight models like Deepseek, Qwen, and Moonshot, leaving startups avoiding paid APIs with no reliable fallback.

“The models from Nvidia, the Neatron family, that the models from Google's Gemma family aren't sufficient to replace what we have from Deepseek, from Zippu, from Moonshot, from Quinn, and we're going to end up in a place where a lot of people are betting on rolling their own, especially startups that don't want to pay open margin for them, and they're just not going to have something to fall back on.”