Absorbed MLA vs. Naive MLA: Same Linear FLOPs, 3.4x Core Attention
Matrix absorption rewrites MLA to attend directly on the compressed latent. A side-by-side PyTorch implementation plus a FLOPs breakdown shows the token-linear cost is bit-for-bit identical while core attention gets 3.4x more expensive — which is exactly why dense causal training keeps the naive form and sparse attention (DSA) training uses the absorbed one.