https://niyunsheng.github.io/tech/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/tags/bank-conflict/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/categories/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/tags/cuda/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/tags/gemm/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/categories/kernels/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/progressive-cuda-gemm-optimization-from-memory-bound-to-swizzling/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/tags/shared-memory/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/tags/swizzling/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/tags/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/2026-03-22T08:00:00+08:00https://niyunsheng.github.io/tags/cp/2026-03-21T15:41:55+08:00https://niyunsheng.github.io/categories/distributed-system/2026-03-21T15:41:55+08:00https://niyunsheng.github.io/tags/dp/2026-03-21T15:41:55+08:00https://niyunsheng.github.io/tags/llm/2026-03-21T15:41:55+08:00https://niyunsheng.github.io/loss-reduction-in-distributed-training/2026-03-21T15:41:55+08:00https://niyunsheng.github.io/tags/video-generation/2026-03-21T15:41:55+08:00https://niyunsheng.github.io/computing-global-gradient-norm-in-distributed-training-tp-dp_shard-dp_replicate-ep-and-pp/2026-03-21T07:02:02+08:00https://niyunsheng.github.io/tags/ep/2026-03-21T07:02:02+08:00https://niyunsheng.github.io/tags/fsdp/2026-03-21T07:02:02+08:00https://niyunsheng.github.io/tags/gradient-clipping/2026-03-21T07:02:02+08:00https://niyunsheng.github.io/tags/hsdp/2026-03-21T07:02:02+08:00https://niyunsheng.github.io/tags/pp/2026-03-21T07:02:02+08:00https://niyunsheng.github.io/tags/tp/2026-03-21T07:02:02+08:00https://niyunsheng.github.io/demystifying-flashattention-forward-backward-and-triton-implementation/2026-03-15T22:00:00+08:00https://niyunsheng.github.io/tags/flash-attention/2026-03-15T22:00:00+08:00https://niyunsheng.github.io/tags/triton/2026-03-15T22:00:00+08:00https://niyunsheng.github.io/tags/dit/2026-02-25T22:00:00+08:00https://niyunsheng.github.io/tags/flow-matching/2026-02-25T22:00:00+08:00https://niyunsheng.github.io/tags/rope/2026-02-25T22:00:00+08:00https://niyunsheng.github.io/series/2026-02-25T22:00:00+08:00https://niyunsheng.github.io/the-devil-in-the-details-engineering-tricks-for-sota-video-models/2026-02-25T22:00:00+08:00https://niyunsheng.github.io/tags/training-stability/2026-02-25T22:00:00+08:00https://niyunsheng.github.io/categories/video-generation/2026-02-25T22:00:00+08:00https://niyunsheng.github.io/series/video-generation-theory/2026-02-25T22:00:00+08:00https://niyunsheng.github.io/deep-dive-into-triton-gemm-optimization-from-naive-tiling-to-hopper-tma/2026-02-11T00:39:00+08:00https://niyunsheng.github.io/tags/swizzing/2026-02-11T00:39:00+08:00https://niyunsheng.github.io/tags/tiling/2026-02-11T00:39:00+08:00https://niyunsheng.github.io/tags/tma/2026-02-11T00:39:00+08:00https://niyunsheng.github.io/tags/flops/2026-02-09T23:20:00+08:00https://niyunsheng.github.io/tags/h200/2026-02-09T23:20:00+08:00https://niyunsheng.github.io/tags/mbu/2026-02-09T23:20:00+08:00https://niyunsheng.github.io/tags/mfu/2026-02-09T23:20:00+08:00https://niyunsheng.github.io/tags/recomputation/2026-02-09T23:20:00+08:00https://niyunsheng.github.io/tags/roofline/2026-02-09T23:20:00+08:00https://niyunsheng.github.io/roofline-analysis-of-llms-on-h200-performance-modeling-and-recomputation-strategies/2026-02-09T23:20:00+08:00https://niyunsheng.github.io/categories/system-optimization/2026-02-09T23:20:00+08:00https://niyunsheng.github.io/tags/classifier-free-guidance/2026-02-03T23:30:00+08:00https://niyunsheng.github.io/tags/ddpm/2026-02-03T23:30:00+08:00https://niyunsheng.github.io/from-ddpm-to-flow-matching-the-evolution-of-generative-trajectories/2026-02-03T23:30:00+08:00https://niyunsheng.github.io/from-dit-to-hunyuan-the-evolution-of-adaln-zero-in-generative-models/2026-02-02T23:30:00+08:00https://niyunsheng.github.io/beyond-theoretical-flops-analyzing-mfu-hfu-and-attention-overhead-in-transformers/2026-01-31T09:30:00+08:00https://niyunsheng.github.io/tags/hfu/2026-01-31T09:30:00+08:00https://niyunsheng.github.io/series/3d-sparse-attention/2026-01-26T23:30:00+08:00https://niyunsheng.github.io/tags/sparse-attention/2026-01-26T23:30:00+08:00https://niyunsheng.github.io/tags/visualization/2026-01-26T23:30:00+08:00https://niyunsheng.github.io/visualizing-3d-attention-bridging-the-gap-between-1d-sequences-and-3d-space/2026-01-26T23:30:00+08:00https://niyunsheng.github.io/tags/a100/2026-01-25T23:21:35+08:00https://niyunsheng.github.io/tags/b300/2026-01-25T23:21:35+08:00https://niyunsheng.github.io/gpu-network-constants/2026-01-25T23:21:35+08:00https://niyunsheng.github.io/categories/hardware/2026-01-25T23:21:35+08:00https://niyunsheng.github.io/tags/infiniband/2026-01-25T23:21:35+08:00https://niyunsheng.github.io/tags/nvidia/2026-01-25T23:21:35+08:00https://niyunsheng.github.io/archives/https://niyunsheng.github.io/explore/https://niyunsheng.github.io/guestbook/https://niyunsheng.github.io/search/