Context Parallelism on Yunsheng Nihttps://niyunsheng.github.io/tags/context-parallelism/Recent content in Context Parallelism on Yunsheng NiHugo -- 0.154.5en-usContent is licensed under CC BY-NC-SA 4.0.Sat, 21 Mar 2026 15:41:55 +0800Loss Reduction in Distributed Traininghttps://niyunsheng.github.io/loss-reduction-dp-cp/Sat, 21 Mar 2026 15:41:55 +0800https://niyunsheng.github.io/loss-reduction-dp-cp/An analysis of how Data Parallelism (DP) and Context Parallelism (CP) affect loss reduction, and how to maintain mathematical equivalence between distributed and single-device training for LLM and Video DiT models.