https://niyunsheng.github.io/tags/kv-cache/Recent content in KV Cache on Yunsheng NiHugo -- 0.154.5en-usContent is licensed under CC BY-NC-SA 4.0.Sun, 02 Aug 2026 09:30:00 +0800https://niyunsheng.github.io/absorbed-mla-vs-naive-mla/Sun, 02 Aug 2026 09:30:00 +0800https://niyunsheng.github.io/absorbed-mla-vs-naive-mla/Matrix absorption rewrites MLA to attend directly on the compressed latent. A side-by-side PyTorch implementation plus a FLOPs breakdown shows the token-linear cost is bit-for-bit identical while core attention gets 3.4x more expensive — which is exactly why dense causal training keeps the naive form and sparse attention (DSA) training uses the absorbed one.