hn • r/hackernews
Comment on: Kimi Linear: An Expressive, Efficient Attention Architecture (2025)
My expertise lies in deep learning theory, and yes, the "intelligence" is coming primarily from scaling up, among other things. There are good reasons for this, but essentially it comes down to taking advantage of a narrow statistical trick, where a very well-crafted model/optimizer pair that has a strong implicit bias toward simplicity can exhibit progressively increasing performance with respect to model size. Marcus Hutter's lab has shown that you can phrase this in terms of Solomonoff induction, so this bias is truly universally effective. An effective bias can continue to improve performa