https://niyunsheng.github.io/series/open-model-architectures/Recent content in Open Model Architectures on Yunsheng NiHugo -- 0.154.5en-usContent is licensed under CC BY-NC-SA 4.0.Tue, 18 Aug 2026 10:00:00 +0800https://niyunsheng.github.io/deepseek-mla-dsa-hca-csa/Tue, 18 Aug 2026 10:00:00 +0800https://niyunsheng.github.io/deepseek-mla-dsa-hca-csa/MLA carried DeepSeek from V2 through V3.2 and was adopted across the industry — then V4 dropped it. V3.2 keeps MLA and bolts a lightning indexer on top: dense KV cache, sparse read. DeepSeek-V4 throws that layout away — every layer is a 128-token sliding window plus a compressed long-range branch, and layers alternate between CSA (m=4, indexer-selected) and HCA (m’=128, dense over blocks). Walking both through the reference implementations in huggingface/transformers.https://niyunsheng.github.io/adaln-zero-dit-sd3-hunyuan/Mon, 02 Feb 2026 23:30:00 +0800https://niyunsheng.github.io/adaln-zero-dit-sd3-hunyuan/From DiT to Hunyuan Video, adaLN-Zero remains the gold standard for conditioning. Here’s how this zero-initialized module works and why it persists in the era of Flow Matching.