Skip to content
Draft
Show file tree
Hide file tree
Changes from 106 commits
Commits
Show all changes
161 commits
Select commit Hold shift + click to select a range
1604db4
p5 starter
KJLdefeated Sep 1, 2026
b95ba80
ci: fix black line-length drift in vllm_runtime/flash_attn; pin black…
KJLdefeated Sep 1, 2026
e6373c2
[DSv4][P1-S0] Start kit for the P1 work package (mHC + RMSNorm)
zhangj1an Sep 3, 2026
3e836d4
[DSv4][P1-S0] Pin unfused as canonical; fix alpha to Megatron's 3 sca…
zhangj1an Sep 3, 2026
48bf9eb
[DSv4][P1-S0] Correct the P1 task split to match issue #2
zhangj1an Sep 3, 2026
39c4aea
[DSv4][P1-S0] Use the real sub-issues of #2 (#14-#21)
zhangj1an Sep 3, 2026
0a9c889
[DSv4][P5-5] Shared Expert MLP: strict CUDA + Triton kernels (#64)
jyizheng Sep 4, 2026
42aa927
fix(p5-5): use libdevice exp in the Triton SwiGLU core
jyizheng Sep 4, 2026
d1ace11
fix(p5-5): source Triton sigmoid from torch.sigmoid
jyizheng Sep 4, 2026
bef20ee
fix(p5-5): forbid FMA contraction in the Triton kernels via libdevice…
jyizheng Sep 4, 2026
e4f8c14
feat: add deterministic clamp swiglu weighted
Sep 6, 2026
4c7f1b3
fix: expose optional p_s in P5 bindings
Sep 6, 2026
b0784e6
fix: register P5 forward binding
Sep 6, 2026
0e88933
perf: recompute P5 SwiGLU intermediates in backward
Sep 7, 2026
85a3211
perf: optimize P5 SwiGLU state and packed layout
Sep 7, 2026
da6b81b
perf(p5-5): det_gemm / tl.dot performance profiles alongside the stri…
jyizheng Sep 9, 2026
7abd313
test(p5-5): per-profile oracle tolerances for the performance backends
jyizheng Sep 9, 2026
7a491ff
feat(moe): P5.4 MXFP8/MXFP4 grouped GEMM kernel
Bignonia7 Sep 10, 2026
fd2a631
Merge pull request #389 from hsiuyee/feat/p5-task2-clamp-swiglu-weighted
KJLdefeated Sep 12, 2026
ec3a781
Merge branch 'dsv4-p5-dev' into ajiao/p5-5-shared-expert-mlp
KJLdefeated Sep 12, 2026
5d1d6fb
Merge pull request #387 from jyizheng/ajiao/p5-5-shared-expert-mlp
KJLdefeated Sep 12, 2026
06bf98d
[DSv4][P5-3] shared_grouped_lora_delta: three provider backends
ScottNi0809 Sep 16, 2026
6231d2b
[DSv4][P5-3] benchmark: torch-native vs Triton vs CUDA
ScottNi0809 Sep 16, 2026
be2eef8
[DSv4][P5-3] tests and invariance sweep for both deterministic backends
ScottNi0809 Sep 17, 2026
5d3d54e
feat(moe): add P5-4 SM90 WGMMA compute kernel
Bignonia7 Sep 17, 2026
b9570be
feat(p5): mxfp8_act_quant CUDA + Triton kernels, bit-exact with the P…
ryankert01 Sep 18, 2026
fa6a287
test(p5): assert mxfp8_act_quant byte-equality at production shapes a…
ryankert01 Sep 18, 2026
ea76186
refactor(p5): simplify the mxfp8_act_quant wrappers, providers, and b…
ryankert01 Sep 18, 2026
273f486
Merge pull request #422 from Bignonia7/dev5
KJLdefeated Sep 19, 2026
e74af0d
fix(p5): tolerate a missing envs module in the CUDA mxfp8_act_quant w…
ryankert01 Sep 19, 2026
c17f921
test(p5): generate the 2**31-element input directly in bf16
ryankert01 Sep 19, 2026
344f676
[DSv4][P5-4] MXFP8×MXFP4 Routed Expert grouped GEMM: wgmma triton kernel
Bignonia7 Sep 20, 2026
74ae44d
Merge pull request #419 from ryankert01/p5-1-mxfp8-act-quant
KJLdefeated Sep 21, 2026
da5ba25
Merge pull request #416 from ScottNi0809/p5-3-shared-grouped-lora-delta
KJLdefeated Sep 24, 2026
b1fb7ed
Merge branch 'dsv4-p5-dev' into p5-4-mxfp8-mxfp4-grouped-gemm
KJLdefeated Sep 24, 2026
78813ea
Merge pull request #399 from Bignonia7/p5-4-mxfp8-mxfp4-grouped-gemm
KJLdefeated Sep 24, 2026
59bef32
moe grouped gemm
KJLdefeated Sep 24, 2026
dfc0e71
[DSv4][P1-2] fp32_gemm_rms
Sep 24, 2026
4788564
dsv4-refactor
KJLdefeated Sep 24, 2026
9133d68
add fused moe mlp & triton
KJLdefeated Sep 29, 2026
f7560d2
moe triton kernel dsv4 shared/routed fc1+swiglu, fc3
KJLdefeated Oct 3, 2026
6565815
Rebase MHC H Aggregate onto dsv4-p1-dev, as during the time of writin…
zhangj1an Oct 6, 2026
f988217
update sum order for dPRE to be ascending order rather than warp tree
zhangj1an Oct 7, 2026
f47158e
Merge branch 'dsv4-p5-dev' into p5-4-wgmma-triton
Bignonia7 Oct 8, 2026
f082687
Complete MHC h_aggregate P1 integration with Triton backend
nodeeeeee Oct 12, 2026
029d95c
Merge pull request #438 from Bignonia7/p5-4-wgmma-triton
KJLdefeated Oct 19, 2026
2f1376f
Merge dsv4 dev P1-P7 branch to one dev branch
KJLdefeated Oct 30, 2026
9c58c64
Merge pull request #366 from nodeeeeee/main
Flink-ddd Oct 30, 2026
5874b90
Merge pull request #443 from Junyifannnn/fable/dsv4-p1-2-fp32-gemm-rms
KJLdefeated Oct 30, 2026
5b99c55
Merge test-dsv4-cuda into dsv4-p1-dev and resolve layout conflicts
Flink-ddd Oct 30, 2026
a2132bb
Merge pull request #383 from RL-Align/dsv4-p1-dev
Flink-ddd Oct 30, 2026
88afc0b
Merge test-dsv4-cuda into dsv4-p5-dev and resolve layout conflicts
claude Oct 30, 2026
18216a4
Merge pull request #368 from RL-Align/dsv4-p5-dev
KJLdefeated Oct 30, 2026
830b902
merge from p1=p
KJLdefeated Oct 30, 2026
4f0d9df
Merge branch 'main' into test-dsv4-cuda
maxiaosong1124 Oct 30, 2026
1dd053f
Merge branch 'main' into test-dsv4-cuda
Flink-ddd Oct 30, 2026
7123ed8
feat(p7-t02): add sampling-params normalization and fail-closed gate
Endeavor-19 Nov 1, 2026
3a6275f
feat(p7-t02): add M10 input schema, pre-checks and stub provider
Endeavor-19 Nov 1, 2026
289fc9f
feat(p7-t02): wire M10 scoring through the strict linear_logp and cap…
Endeavor-19 Nov 1, 2026
5701b13
docs(p7-t02): add sampling and M10 implementation contract
Endeavor-19 Nov 1, 2026
8a57bf4
test(p7-t02): run sampling and M10 suites in CI
Endeavor-19 Nov 1, 2026
e25127b
fix(p7-t02): close automated review findings in the gate and M10 pre-…
Endeavor-19 Nov 1, 2026
806d9db
fix(p7-t02): harden M10 versioning, provenance and output guards
Endeavor-19 Nov 1, 2026
ae67c6f
fix(p7-t02): make malformed records fail closed and pin layer provenance
Endeavor-19 Nov 1, 2026
437191b
fix(p7-t02): tighten fallback validation and align scoring result views
Endeavor-19 Nov 1, 2026
31ca466
fix(p7-t02): validate fallback entries and record weight-lock verific…
Endeavor-19 Nov 1, 2026
ce9e3a0
fix(p7-t02): require explicit sampling fields and guard raw byte order
Endeavor-19 Nov 1, 2026
02823be
fix(p7-t02): reject non-canonical layer values and unknown fallback n…
Endeavor-19 Nov 1, 2026
a013c9f
fix(p7-t02): require real vocab size and reject untrusted provenance
Endeavor-19 Nov 1, 2026
2ad56da
fix(p7-t02): canonicalize direct layers and narrow gate status mapping
Endeavor-19 Nov 1, 2026
26337b6
Merge branch 'main' into test-dsv4-cuda
inaniloquentee Nov 2, 2026
d28a3e3
Merge branch 'test-dsv4-cuda' into feat/p7-t02-sampling-m10-contract
Flink-ddd Nov 2, 2026
b8536f5
Merge pull request #648 from Endeavor-19/feat/p7-t02-sampling-m10-con…
maxiaosong1124 Nov 2, 2026
ca5a42d
add design docs
KJLdefeated Nov 5, 2026
2b6f595
Merge branch 'test-dsv4-cuda' of https://github.com/RL-Align/RL-Kerne…
KJLdefeated Nov 5, 2026
c2de274
style: apply pre-commit formatting across the DSv4 branch
KJLdefeated Nov 9, 2026
900f01d
fix(dsv4): resolve Phase 0 blockers for the CUDA SM90 path
KJLdefeated Nov 9, 2026
9df841a
Merge origin/main into test-dsv4-cuda
KJLdefeated Nov 9, 2026
d1bfa2b
feat(dsv4): probe DeepGEMM batch invariance and settle D3 (Phase 1)
KJLdefeated Nov 9, 2026
4d6e033
feat(dsv4): probe DeepGEMM FP8 backward kernels
KJLdefeated Nov 9, 2026
06e68f7
feat(dsv4): add per-token-group FP8 activation quantization for DeepGEMM
KJLdefeated Nov 9, 2026
5252ccb
feat(dsv4): pin DeepGEMM and add the DeepGEMM FP8 shared-expert provider
KJLdefeated Nov 9, 2026
ec257d4
feat(dsv4): add CUDA kernels for the shared expert's clamped SwiGLU
KJLdefeated Nov 9, 2026
5fbf9b1
docs(dsv4): add CLAUDE.md handoff notes for the DSv4 CUDA work
KJLdefeated Nov 9, 2026
02c8bd5
docs(dsv4): record Phase 0/4 status and the P5 shared-expert clamp de…
KJLdefeated Nov 9, 2026
ebc4f61
fix(dsv4): do not launch det_gemm kernels for empty shapes
KJLdefeated Nov 9, 2026
988538b
feat(dsv4): add a general DeepGEMM FP8 linear for the dense projections
KJLdefeated Nov 9, 2026
9380cee
feat(dsv4): add the indexer's stable per-row top-k (P2 T04)
KJLdefeated Nov 9, 2026
092024a
feat(dsv4): probe DeepGEMM's FP8 MQA-logits kernels for batch invariance
KJLdefeated Nov 10, 2026
895252a
feat(dsv4): add the indexer query RoPE + FP8 quantization (P2 T03)
KJLdefeated Nov 10, 2026
8b0ef05
feat(dsv4): probe DeepGEMM's BF16 GEMM for the unquantized projections
KJLdefeated Nov 10, 2026
c741be5
feat(dsv4): add the indexer's KV selection on DeepGEMM MQA logits
KJLdefeated Nov 10, 2026
72dd311
docs(dsv4): record the T02 compressor analysis and the parity decision
KJLdefeated Nov 10, 2026
1c90987
docs(dsv4): record the T02 compressor and T05 FlashMLA decisions
KJLdefeated Nov 10, 2026
345cb9e
feat(dsv4): add the indexer KV compressor (P2 T02, head 128)
KJLdefeated Nov 10, 2026
fd3f061
feat(dsv4): add the head-512 main-KV compressor (C4 and C128)
KJLdefeated Nov 10, 2026
1aa0652
feat(dsv4): pin FlashMLA and probe its SM90 sparse kernel (T05)
KJLdefeated Nov 10, 2026
d131e98
feat(dsv4): add the sparse attention core on FlashMLA (T05)
KJLdefeated Nov 10, 2026
e9b4745
feat(dsv4): add the attention query and output side kernels
KJLdefeated Nov 10, 2026
2742396
feat(dsv4): compose the batch-invariant DSv4 attention layer (T06, T07)
KJLdefeated Nov 10, 2026
b5b6d67
perf(dsv4): move the attention glue to CUDA and make decode graph-cap…
KJLdefeated Nov 10, 2026
497b6d9
build(dsv4): move to the vLLM fork's torch 2.13 / CUDA 13 stack
KJLdefeated Nov 10, 2026
690068a
docs(dsv4): record the verl / Megatron / vLLM integration findings
KJLdefeated Nov 10, 2026
b785c36
feat(dsv4): add the batch-invariant MoE router (P3)
KJLdefeated Nov 10, 2026
13b4513
feat(dsv4): add the MoE route plan, fixed-order combine and the EP=1 …
KJLdefeated Nov 10, 2026
f7dc3d1
feat(dsv4): EP MoE block with backward, SM90 routed backward, faster …
KJLdefeated Nov 10, 2026
d21e002
feat(dsv4): add the batch-invariant mHC with fused RMSNorm and dX bac…
KJLdefeated Nov 10, 2026
a12b6b2
feat(dsv4): assemble the DSv4 decoder layer and model forward
KJLdefeated Nov 10, 2026
78254e8
fix(dsv4): use plain RoPE on the sliding-window-only layers
KJLdefeated Nov 10, 2026
3f0ca16
feat(dsv4): add the batch-invariant selected-token logprob head (P7)
KJLdefeated Nov 10, 2026
d87db23
docs(dsv4): tick the indexer op composition (T03)
KJLdefeated Nov 10, 2026
37a3303
feat(dsv4): check P5's scale-residual range on the real FP4 experts
KJLdefeated Nov 10, 2026
ace607e
perf(dsv4): build the MoE route plan with a CUDA counting sort
KJLdefeated Nov 10, 2026
ddbade7
perf(dsv4): stage the mHC gemm_bwd rows with cp.async
KJLdefeated Nov 11, 2026
eb4d8d3
fix(dsv4): return FlashMLA's LSE, and propose the attention backward
KJLdefeated Nov 11, 2026
7389af8
docs(dsv4): add the per-operator performance table
KJLdefeated Nov 11, 2026
eb1f64e
perf(dsv4): run the per-group wo_a GEMMs concurrently at decode sizes
KJLdefeated Nov 11, 2026
86aff0d
feat(dsv4): wrap cuDNN's SM90 sparse attention backward (non-determin…
KJLdefeated Nov 11, 2026
822c868
feat(dsv4): add the attention layer backward (dL/dhidden, non-determi…
KJLdefeated Nov 11, 2026
db2485a
feat(dsv4): add LoRA adapters on the attention projections
KJLdefeated Nov 11, 2026
f3042b0
feat(dsv4): add LoRA to the MoE block (routed + shared expert)
KJLdefeated Nov 11, 2026
9a58311
feat(dsv4): add the full-model LoRA training forward / backward
KJLdefeated Nov 11, 2026
6c82a9e
feat(dsv4): MoE LoRA with expert parallelism
KJLdefeated Nov 11, 2026
b9eea9d
perf(dsv4): fuse the attention backward's RoPE / RMSNorm steps into o…
KJLdefeated Nov 11, 2026
3da74e1
perf(dsv4): move the attention compressor backward to one CUDA kernel
KJLdefeated Nov 11, 2026
4647df0
docs(dsv4): tick the finished design-doc items
KJLdefeated Nov 11, 2026
d3d6f08
perf(dsv4): run the grouped wo_a as one DeepGEMM fp8_einsum below 819…
KJLdefeated Nov 11, 2026
d7938d7
perf(dsv4): cache hc_head's padded mHC weight instead of padding per …
KJLdefeated Nov 11, 2026
892a250
refactor(dsv4): drop the P5 fused single-launch MoE path
KJLdefeated Nov 11, 2026
5804978
feat(dsv4): deferred MoE finite check, graph-capturable decode step
KJLdefeated Nov 11, 2026
a3b509b
docs(dsv4): write the SM90 batch-invariant execution profile
KJLdefeated Nov 11, 2026
b7bfeec
test(dsv4): probe DeepGEMM grouped BF16 GEMMs for per-expert MoE LoRA
KJLdefeated Nov 11, 2026
70f9d85
feat(dsv4): single-device layerwise exact-zero gate
KJLdefeated Nov 11, 2026
c6e6512
docs(dsv4): record the per-expert LoRA and chunked-training decisions
KJLdefeated Nov 11, 2026
8ec3d0a
Merge branch 'main' into test-dsv4-cuda
KJLdefeated Nov 11, 2026
e6300a9
style(dsv4): black the P5 MoE test and benchmark after the fused-path…
KJLdefeated Nov 11, 2026
d8908b5
chore: remove the P7-T05 Megatron training provider and its test doubles
KJLdefeated Nov 11, 2026
3850757
chore: remove the P2 / P6 start kits
KJLdefeated Nov 11, 2026
d85069a
chore: drop tests and benchmarks of legacy P1 / P5 code DSv4 does not…
KJLdefeated Nov 11, 2026
11ac30a
fix: make mypy pass on rl_engine (CI linting)
KJLdefeated Nov 11, 2026
14cf80b
docs(dsv4): one design doc for handoff; operator pages point to it
KJLdefeated Nov 11, 2026
1352359
docs(dsv4): note the unrun EP=2 LoRA check and the current regression…
KJLdefeated Nov 11, 2026
fd0e60d
perf(dsv4): prebuilt token-block map for the SM90 grouped MoE GEMMs
KJLdefeated Nov 12, 2026
6db09cc
test(dsv4): EP = 2 with LoRA: forward bitwise, LoRA dX to rounding
KJLdefeated Nov 15, 2026
7d0b7d9
feat(dsv4): guide and scripts for a verl + vLLM-BI end-to-end test on…
KJLdefeated Nov 15, 2026
0469a3e
feat(dsv4): 200-step run (T4) in the verl end-to-end guide, for 8 x H100
KJLdefeated Nov 15, 2026
5598067
docs(dsv4): shorten the verl end-to-end guide to T1, rollout speed, T4
KJLdefeated Nov 15, 2026
33b65bb
docs(dsv4): rename the verl guide's training run T4 to T2
KJLdefeated Nov 15, 2026
5ae50f3
feat(dsv4): vLLM rollout plugin, bitwise equal to Dsv4Model (no LoRA)
KJLdefeated Nov 15, 2026
f74765a
fix(dsv4): rollout compressors on RL-Kernel kernels; vLLM gate passes…
KJLdefeated Nov 15, 2026
604efcc
docs(dsv4): the vLLM rollout plugin, what it replaces and why; verl s…
KJLdefeated Nov 15, 2026
bdda3f9
perf(dsv4): keep the MoE forward's E4M3 weight expansion across calls
KJLdefeated Nov 16, 2026
7d45e6f
feat(dsv4): verl engine backend "rlkernel" for the DSv4 LoRA actor
KJLdefeated Nov 16, 2026
7b5ccff
feat(dsv4): verl GRPO on RL-Kernel end to end, rollout bitwise equal …
KJLdefeated Nov 16, 2026
e6a01f6
docs(dsv4): verl integration (section 10.1) and status
KJLdefeated Nov 16, 2026
b410ac0
feat(dsv4): workload knobs in the RL-Kernel verl launcher
KJLdefeated Nov 16, 2026
8b83f07
docs(dsv4): 8-GPU guide, RL-Kernel + verl (BI, LoRA) vs verl #7483
KJLdefeated Nov 16, 2026
682fdde
docs(dsv4): CLAUDE.md from the working notes; status of the 8-GPU guide
KJLdefeated Nov 16, 2026
ccf0c66
style(dsv4): black / isort on the vLLM plugin and its gate
KJLdefeated Nov 16, 2026
f7c208b
feat(dsv4): non-invariant rollout switch for the BI ablation
KJLdefeated Nov 16, 2026
bfc8fb3
docs(dsv4): the BI ablation in the 8-GPU guide, design doc and CLAUDE.md
KJLdefeated Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
23 changes: 23 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,29 @@ jobs:
tests/backends/rocm/test_rocm_paged_workspace.py \
tests/distributed/collectives/test_collective_ipc_errors.py

- name: Run P7 T05 Megatron Training Readback Tests (CPU-safe)
run: |
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q -rs \
tests/test_megatron_training_provider.py \
tests/test_megatron_training_readback.py \
tests/test_megatron_training_readonly.py \
tests/test_megatron_training_gate.py \
tests/test_megatron_training_recorded.py \
tests/test_megatron_training_evidence.py \
tests/test_megatron_training_upstream.py \
tests/test_megatron_training_export.py \
tests/test_megatron_training_gradients.py \
tests/test_megatron_training_ranks.py \
tests/test_megatron_training_gradient_ranks.py \
tests/test_megatron_training_pairing.py

- name: Run P7 T02 Sampling and M10 Tests (CPU-safe)
run: |
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q -rs \
tests/validation/sampling_params/test_sampling_gate.py \
tests/validation/m10/test_m10_precheck.py \
tests/validation/m10/test_m10_score.py

docs:
runs-on: ubuntu-latest
steps:
Expand Down
43 changes: 43 additions & 0 deletions .github/workflows/p2-start-kit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: P2 start-kit CPU conformance

on:
pull_request:
paths:
- "rl_engine/p2/**"
- "tests/p2/**"
- "scripts/p2_distributed_smoke.py"
- ".github/workflows/p2-start-kit.yml"
workflow_dispatch:

permissions:
contents: read

jobs:
cpu:
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
matrix:
python: ["3.10", "3.12"]
env:
OMP_NUM_THREADS: "1"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- run: python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
- run: python -m pip install numpy pytest ruff
- run: python -m ruff check rl_engine/p2 tests/p2 scripts/p2_distributed_smoke.py
- run: python -m ruff format --check rl_engine/p2 tests/p2 scripts/p2_distributed_smoke.py
- run: python -m pytest tests/p2 -q --junitxml=p2-results.xml
- run: python -m rl_engine.p2 conformance --tokens 513 --output p2-artifact
- run: python -m rl_engine.p2 verify p2-artifact
- uses: actions/upload-artifact@v4
if: always()
with:
name: p2-start-kit-py${{ matrix.python }}
path: |
p2-results.xml
p2-artifact/
retention-days: 14
43 changes: 43 additions & 0 deletions .github/workflows/p6-start-kit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: P6 start-kit CPU conformance

on:
pull_request:
paths:
- "rl_engine/p6/**"
- "tests/p6/**"
- "docs/archive/design/p6-start-kit.md"
- ".github/workflows/p6-start-kit.yml"
workflow_dispatch:

permissions:
contents: read

jobs:
cpu:
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
matrix:
python: ["3.10", "3.12"]
env:
OMP_NUM_THREADS: "1"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- run: python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
- run: python -m pip install numpy pytest ruff
- run: python -m ruff check rl_engine/p6 tests/p6
- run: python -m ruff format --check rl_engine/p6 tests/p6
- run: python -m pytest tests/p6 -q --junitxml=p6-results.xml
- run: python -m rl_engine.p6 conformance --device torch-cpu --hidden-size 4096 --output p6-artifact
- run: python -m rl_engine.p6 verify p6-artifact
- uses: actions/upload-artifact@v4
if: always()
with:
name: p6-start-kit-py${{ matrix.python }}
path: |
p6-results.xml
p6-artifact/
retention-days: 14
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,9 @@ dmypy.json
# Cython debug symbols
cython_debug/

# Vscode
.vscode

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
Expand Down Expand Up @@ -226,3 +229,6 @@ csrc/hip/*_hip.hip
.rlk-profile.json

/artifacts/

# Local DSv4 probe reports (regenerate with scripts/probe_dsv4_deepgemm_bi.py)
docs/validation/dsv4-deepgemm-bi-*.json
110 changes: 110 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# CLAUDE.md — DSv4 CUDA work on `test-dsv4-cuda` (PR #635)

Handoff notes for continuing the DSv4 SM90 batch-invariant work. The plan of record is
`docs/architecture/dsv4-cuda-design.md` (decisions D1-D7, phases, todo); read it first and
keep its checkboxes current. Last updated 2026-11-09 at `ec257d43` (pushed).

## Working with the user

- Reply in Traditional Chinese (technical terms may stay English); the user writes a mix.
- Work step by step: explain why, how and in what order, and confirm before starting a large
step. Commit only when asked; push only when asked.
- Commits: conventional style (`feat(dsv4): ...`, `fix(dsv4): ...`), `git commit -s` (DCO),
end with the Co-Authored-By line. Split pure formatting from logic changes.
- Goal right now: every DSv4 operator runs, is batch-invariant (bitwise), and is within
1.3x of the non-invariant path (naive PyTorch, or an efficient non-BI kernel, same shapes).

## Settled decisions (beyond the design doc's D1-D7)

- Integration target and formal #230 anchor: **verl** (rollout and actor recompute on the
same kernel set). The vLLM fork (`ISEEKYAN/vllm@1a6542c`) is the semantic reference for
rollout bytes; its golden is deferred.
- CUDA SM90 first; Triton paths deferred; ROCm out of scope.
- P3 and P7 are redone on this branch; the old P-dev branches are ignored. Merge main last.
- Dense FP8 GEMMs (q/kv/o/indexer projections and the shared expert) use **unmodified
DeepGEMM** pinned to `vllm-project/DeepGEMM@e21c821f` (D3; probe passed, forward and
backward). Routed experts stay on P5 `csrc/cuda/moe/sm90_fused_moe_mlp.cu` (DeepGEMM runs
FP4 only on SM100).
- Shared expert moved to DeepGEMM FP8 (forward and dX). The det_gemm BF16 backward stays as
a fallback/reference (`backward="bf16_det_gemm"`). Shared-expert weights are frozen: dX only.
- Keep `csrc/cuda/moe/mxfp8_mxfp4_grouped_gemm_wgmma.cu` although nothing references it.

## Done on this branch (newest last)

| Commit | What |
|---|---|
| `c2de2741`, `900f01d4`, `9df841a1` | Phase 0: pre-commit formatting; SM90 MoE build gating (`RL_KERNEL_ENABLE_MOE_SM90`, sm_90a only), fast-math error on ROCm, fp32_gemm_rms G8 N=31 fix, benchmark fixes; merge main |
| `d1bfa2b5`, `4d6e033e` | Phase 1: `scripts/probe_dsv4_deepgemm_bi.py` — DeepGEMM BI forward/dgrad, deterministic wgrad, perf, negative control |
| `06e68f7b` | `fp8_group_quant` (golden `rl_engine/kernels/ops/pytorch/quant/`, CUDA `csrc/cuda/quant/`) — byte-exact with vLLM `per_token_group_fp8_quant` (group 128, eps 1e-10, UE8M0); token-direction variant for wgrad |
| `5252ccb8` | `dsv4` extra pin + `rl_engine/kernels/ops/cuda/deepgemm.py` (`require_deepgemm`, fail-closed); `rl_engine/moe/backends/deepgemm_shared_expert.py` (profile `dsv4-deepgemm-fp8-v1`) |
| `ec257d43` | `csrc/cuda/activation/silu_and_mul_clamp.cu`: vLLM clamped SwiGLU (limit 10) fused with FP8 quant, plus backward; shared expert now 0.71-0.78x of eager BF16 |

## Next steps (agreed order)

1. Housekeeping: tick "Merge main" (Phase 0) and "DeepGEMM backward determinism" (Phase 4) in
the design doc; fix det_gemm launching an empty grid for M=0; decide whether the P5
oracle's shared-expert SwiGLU should clamp like the rollout.
2. A general DSv4 dense FP8 linear operator (fp8_group_quant + DeepGEMM + weight prep),
extracted from the shared-expert provider, for the attention and indexer projections.
3. Phase 2 (critical path): T04 stable top-k first (its comparator is shared with P3's
`stable_topk6`), then T03 indexer (needs a BI probe of DeepGEMM `fp8_mqa_logits`), T02 KV
compressor, T05 FlashMLA sparse wrapper (new dependency: pin + probe like DeepGEMM).
4. P3 router redo. Check first: D4 wants `det_gemm_fwd_fp32`, which is not in the default
build of `rl_engine._C`.

## Environment (TWCC-style cluster; paths are specific to this account)

- GPU nodes via Slurm (`normal2` = H200, SM90). Login nodes have no GPU. `/home` and `/work`
are shared; `/tmp` (and the Claude scratchpad) is node-local.
- `gh` is not installed: use `curl https://api.github.com/repos/RL-Align/RL-Kernel/...`.
- `ls` is aliased to `eza` (missing): use `command ls`.
- Python: conda env `cosq` (torch 2.9.0+cu128). Do not pip-install into it. Work venv:
`~/dsv4-work/venv` (`--system-site-packages` on cosq) with pytest, pre-commit, a real
`ninja` binary (cosq's `ninja` module has none), and `deep_gemm 2.6.1+e21c821`.
- CUDA for building `rl_engine._C`: `module load cuda/12.4` (this sets `CUDA_HOME`).
- CUDA for DeepGEMM's JIT: must be 12.8+. Conda toolkit at `/work/u4478697/cuda128`, with a
standard-layout symlink facade `/work/u4478697/cuda128-home` (conda puts headers/libs under
`targets/x86_64-linux`). Run DeepGEMM code with
`CUDA_HOME=/work/u4478697/cuda128-home PATH=/work/u4478697/cuda128-home/bin:$PATH`.
`module load cuda/12.4` breaks DeepGEMM's JIT (`require_deepgemm` now says so).
- DeepGEMM source: `~/dsv4-work/deepgemm-src/DeepGEMM` (pinned commit). Reinstall from a clean
tree (a stray `third-party/deep_jit/` makes the version `+local`, which the pin rejects):
`CUDA_HOME=/work/u4478697/cuda128-home pip install --no-build-isolation --no-deps .`
- Do not build in the main checkout: it holds the user's own `build/` and a py3.11 `_C.so`.
Build in a worktree (`git worktree add --detach ~/dsv4-work/<name> HEAD`), e.g.:

```bash
module load cuda/12.4
export PATH=~/dsv4-work/venv/bin:$PATH MAX_JOBS=16 RL_KERNEL_REQUIRE_EXT=1
python setup.py build_ext --inplace
export CUDA_HOME=/work/u4478697/cuda128-home PATH=/work/u4478697/cuda128-home/bin:$PATH
python -m pytest -q tests/ops/quant tests/ops/activation tests/test_deepgemm_shared_expert.py \
tests/ops/gemm/test_deepgemm_capability.py tests/build
python scripts/probe_dsv4_deepgemm_bi.py --output report.json # report is gitignored
```

Optional builds: `KERNEL_ALIGN_MOE_SM90=1` (P5 routed SM90 kernel),
`KERNEL_ALIGN_DET_GEMM_SM90=1` (fast det_gemm; the BF16 fallback is very slow without it).
- Existing worktrees under `~/dsv4-work/` (base, fix, merged, aq, act) are disposable.

## Known failures that predate this work

`tests/runtime/test_attention_dispatch.py` (2, from the main merge),
`tests/test_p5_lora_delta.py` Triton cases (4), `tests/test_p1_provider.py::test_golden_manifest_anchor`.

## Numerics lessons (each one bit us)

- CUDA PyTorch `tensor / python_scalar` multiplies by the reciprocal (not correctly rounded);
use `tensor / tensor` in goldens. CPU is exact, so CPU-only tests miss it.
- On CUDA, `torch.exp` is libdevice `expf`, same as vLLM; CPU `exp` differs in the last bit,
so rollout-parity goldens are defined on CUDA.
- In CUDA kernels, spell FP32 steps with `__fadd_rn`/`__fmul_rn`/`div.rn` to stop FMA
contraction; avoid `.ftz` paths; refuse `--use_fast_math`.
- vLLM's FP32-scale UE8M0 uses `exp2f(ceilf(log2f(s)))`; it equals bit-math ceiling for every
BF16/FP16 input but not for FP32, so the quant ops reject FP32 input.
- DeepGEMM's `per_token_cast_to_fp8` clamps amax to 1e-4 (vLLM uses eps 1e-10): not a golden.
- DeepGEMM SM90 wgrad: FP32 output with accumulate only, token count % 16 == 0 (pad zeros).
- det_gemm with M=0 leaves a pending CUDA error that a later, unrelated call reports.
- The P5 oracle's unweighted shared-expert SwiGLU has no clamp and one FP32 rounding; the
rollout clamps at 10 and rounds twice from BF16.
- Timing: use CUDA-graph replay for small shapes; eager timing measures host overhead.
Loading
Loading