Compared with Kimi K3, they replace the full-attention Gated MLA layers with G… · @Chonghan_Liu · TwKit

@Chonghan_Liu

Compared with Kimi K3, they replace the full-attention Gated MLA layers with Gated DeepSeek Sparse Attention (Gated DSA) layers for more efficient long-context modeling, that’s all.

曝光 436 · 评论 0 · 点赞 5 · 书签 1