Topics / GPU kernel fusion

Performance topic

GPU kernel fusion

Fuse GPU kernels so intermediates stay in registers or shared memory instead of round-tripping through global memory.

766proven techniques
24pitfalls: tried, then slower
231repositories
226from papers

Examples from real commits

pytorch/executorch 4.5× Decode packed Q5_K tiles in registers into a BF16 tile and feed them directly to tl.dot, so no BF16 weight is materialized in HBM. proof: commit → Plonky3/Plonky3 3.8× Fuse two sumcheck rounds into one pass: compute a bivariate round polynomial over 4-row blocks, then fold both variables and sum the next pair from the just-written rows. proof: commit → linkedin/Liger-Kernel 2× Fuse the vocabulary GEMM with per-token NLL and entropy in one online-softmax pass on Hopper WGMMA, keeping logits in SMEM and registers. proof: commit →

Languages

  • CUDA207
  • Python171
  • C++105
  • C15
  • Rust14
  • Zig12

Hardware

  • NVIDIA GPU444
  • GPU172
  • CPU43
  • AMD GPU34
  • x8630

Fields

  • GPU481
  • graphics86
  • math53
  • networking41
  • compilers40

Where these come from

Repositories with the most techniques on this topic in the database:

ddalcu/mlx-serve · 18linkedin/Liger-Kernel · 14alibaba/rtp-llm · 12hao-ai-lab/FastVideo · 11sgl-project/sglang · 11harleyszhang/rapid_llm · 10NVIDIA/cudnn-frontend · 10xLLM-AI/xllm · 91CatAI/1Cat-vLLM · 9PaddlePaddle/FastDeploy · 7

Ask Claude for the full answer

Each technique comes with its bottleneck, measured speedup, trade-offs, conditions and the commit or paper behind it, served to Claude through MCP.

claude mcp add --transport http adtomic https://www.adtomic.co/mcp

Connect Claude Pricing

Related topics

Arena and pool allocationI/O batchingFast math and approximationSIMD vectorizationBetter algorithms