Skip to content

[DSv4][CUDA] DSv4 testing control threads. - #635

Draft
KJLdefeated wants to merge 161 commits into
mainfrom
test-dsv4-cuda
Draft

KJLdefeated wants to merge 161 commits into
mainfrom
test-dsv4-cuda

Conversation

@KJLdefeated

@KJLdefeated KJLdefeated commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR adds a complete DeepSeek-V4 forward and LoRA-training path on Hopper (H100 / H200) whose forward is bitwise batch-invariant. Rollout and actor recompute can therefore produce identical logprobs on the same kernel set.

  • Every forward operator is byte-equal to a PyTorch golden and batch-invariant, and runs within 1.3x of its non-invariant baseline (most are faster).
  • LoRA training (attention + MoE adapters) back-propagates end to end; gradients match FP32 autograd per component. Backward is not required to be deterministic.
  • A layerwise gate shows 0 differing tokens at every layer between a continuous-batching rollout, the packed actor recompute and the training forward.
  • EP = 2 forward and dX are bitwise equal to EP = 1 (without LoRA; the LoRA EP check has not run yet).

Start here: docs/architecture/dsv4-design.md. It is the single design doc: guarantees, environment and build, how to run tests / benchmarks / probes, the module map, decisions, the reduction order of every module, performance, what the vLLM rollout must patch, legacy code, and open work. Each operator has a detail page under docs/operators/ linked from it.

Size and how to review

About 44.6k added lines over 256 files, but most of it is tests (7.4k), goldens (2.0k), benchmarks (1.9k), probe / check scripts (2.7k) and docs (4.0k). Reviewers only need the modules below; the arithmetic of each kernel is specified on its operator page and pinned by a byte-equality test against its golden, so the goldens, benchmarks and scripts do not need line-by-line review.

Suggested order:

  1. Design (30 min): docs/architecture/dsv4-design.md, sections 1, 3, 4, 5.
  2. Model composition, where batch invariance is assembled:
    • rl_engine/models/deepseek_v4/model.py: layer / model forward, forward_train, finite-check modes.
    • rl_engine/models/deepseek_v4/attention.py: Dsv4Batch (host-side sizes, no readback), compressor / indexer / FlashMLA wiring, canonical index order.
    • rl_engine/models/deepseek_v4/moe.py: route plan, dispatch, EP all-to-all, combine.
  3. Third-party pins and fail-closed checks (rule: a missing kernel is an error, never a fallback):
    • rl_engine/kernels/ops/cuda/deepgemm.py, flashmla.py, dsa_backward.py;pyproject.toml extra dsv4.
    • rl_engine/kernels/ops/cuda/fp8_linear.py (all dense FP8 projections),dsv4_attention.py (FlashMLA wrapper, output projection).
  4. Our CUDA kernels that fix a reduction order (read with the matching operator page):
    • csrc/cuda/mhc_dsv4/dsv4_mhc.cu (dsv4-mhc)
    • csrc/cuda/moe/sm90_fused_moe_mlp.cu, sm90_moe_mlp_backward.cu(dsv4-moe); largest kernel, FP8 WGMMA + TMA.
    • csrc/cuda/moe/moe_route_plan.cu, moe_slot_sum.cu, csrc/cuda/router/dsv4_router.cu
    • csrc/cuda/compressor/compress_fp8.cu, csrc/cuda/attention/dsv4_qo.cu,
      csrc/cuda/topk/stable_topk.cu, csrc/cuda/indexer/indexer_q_rope_fp8.cu
    • csrc/cuda/logprob_dsv4/dsv4_logprob.cu
  5. Training: rl_engine/models/deepseek_v4/lora.py, attention_backward.py, csrc/cuda/attention/dsv4_attn_bwd.cu.
  6. Gates: rl_engine/models/deepseek_v4/layerwise_gate.py and tests/models/deepseek_v4/.
  7. Build and shared files: build_tools/extensions.py and csrc/bindings/ops.cpp(SM90 MoE kernels behind KERNEL_ALIGN_MOE_SM90=1, sm_90a only), rl_engine/_C.pyi, .github/workflows/ci.yml.

Can be skimmed or skipped: rl_engine/kernels/ops/pytorch/** (goldens), benchmarks/, scripts/probe_dsv4_* / scripts/check_*, docs/validation/*.json, and the legacy modules listed in design doc section 11.

Dependencies

Dependency Pin Used for
DeepGEMM vllm-project/DeepGEMM@e21c821f (optional extra dsv4) FP8 / BF16 GEMMs, MQA logits, mHC TF32 GEMM
FlashMLA 1.0.0+b70aff3 (SM90 sources = vLLM's pin a8f794d1) sparse attention forward
cuDNN frontend 1.31.0 sparse attention backward (training only)

All are optional at install time and checked fail-closed when a DSv4 op is constructed.

Testing

H200, torch 2.13 / CUDA 13.0, KERNEL_ALIGN_MOE_SM90=1 build:

python -m pytest -q tests/ops/logprob_dsv4 tests/ops/mhc_dsv4 tests/models tests/ops/router \
    tests/ops/moe tests/ops/compressor tests/ops/indexer tests/ops/topk tests/ops/attention_dsv4 \
    tests/ops/quant tests/ops/activation tests/ops/gemm/test_dsv4_fp8_linear.py \
    tests/ops/gemm/test_deepgemm_capability.py tests/test_deepgemm_shared_expert.py \
    tests/test_sm90_fused_moe_mlp.py tests/test_sm90_moe_backward.py tests/build
# 1217 passed, 72 skipped, 0 failed (80 s)
PYTHONPATH=. python scripts/check_dsv4_layerwise_gate.py           # GATE PASS, 0 diff at every layer
  • EP = 2 (torchrun --nproc-per-node 2 scripts/check_dsv4_moe_ep.py): forward and dX bitwise equal to EP = 1 (verified earlier on this branch). The same check with LoRA adapters (--lora) is written but has not run yet.

KJLdefeated and others added 30 commits September 1, 2026 11:49
…/isort config in pyproject

Signed-off-by: KJLdefeated <linkai0508@gmail.com>
Freezes the contract, the reference answers and the checker for the P1
work package so all six sub-tasks (P1-D1..P1-D6) can start in parallel
against the same frozen math and the same golden bytes. No GPU kernels.

- rl_engine/mhc/reduction.py: the two pinned reduction trees (serial
  ascending left fold for long reductions, (a0+a1)+(a2+a3) for the four
  streams). Every P1 accumulation goes through here.
- rl_engine/mhc/contract.py: LayerContract, ResidualBatch, controller
  and norm params, GradBoundary, checkpoint fingerprints.
- rl_engine/mhc/oracle.py: FP32 reference for hc_split_sinkhorn,
  fp32_gemm_rms, mhc_post, mhc_pre/h_aggregate, rmsnorm_residual and the
  fixed-K GEMM, forward and backward, plus the block composition.
- rl_engine/mhc/provider.py: MHCProvider protocol, oracle-backed
  reference, fail-closed stub, capability gate.
- rl_engine/mhc/fixtures.py + tests/fixtures/p1/golden_hashes.json:
  seeded cases and the CI anchor for the golden bytes.
- scripts/check_p1.py: the acceptance command.
- 53 CPU tests; every hand-written backward is cross-checked against an
  autograd graph built from the same frozen formulas.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kcu8vXve5YMZ5wqCCKRfvJ
…lars

Follow-up after reading the Megatron mHC source
(megatron/core/transformer/hyper_connection.py and
megatron/core/fusions/fused_mhc_kernels.py, dev).

Megatron's native and fused paths disagree with each other: the fused
kernel uses TF32 MMA, split_k=16 at K>=16384, and runtime autotuning,
and computes sqrt(s/K) where native computes sqrt(s)/sqrt(K). Three of
issue #2's explicit bans (Split-K off, TF32 off, fixed reduction tree)
are violated by that path at exactly the production shape. So Megatron
supplies the formulas and constants for this contract, never the
reduction order.

Changes:

- fusion: 'unfused' is now CANONICAL and documented as such. Train/infer
  byte-equality needs every boundary hashable on its own, which only the
  unfused decomposition gives. v1 does not reuse TE's
  TEFusedResidualRMSNorm -- it refuses to expose the intermediate (raises
  on any forward hook), collapsing two boundaries into one. Fusion stays
  legal; changing the reduction layout or a downcast point does not.
- alpha: Megatron holds three learnable scalars (alpha_pre/post/res)
  broadcast over the PRE/POST/COMB segments, not 24 independent gains.
  Forward was already equivalent; backward was not, so dAlpha is now
  three scalars with the segment fold pinned like every other reduction.
- design doc: new section on what Megatron actually does and why it is
  not the byte reference; D1/D6/D8 rewritten with the source evidence;
  open questions updated (Miles must also run unfused -- needs their
  sign-off).
- the fused/unfused test now says what it actually proves (oracle
  self-consistency, not that a real fused kernel matches).

54 tests pass; check_p1.py PASSes on cpu and cuda, stub still fails
closed; ruff/flake8/black/isort/mypy clean. Golden manifest regenerated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kcu8vXve5YMZ5wqCCKRfvJ
Issue #2 splits P1 by function, not by operator: P1-D1 owns the mHC
forwards, P1-D2 the RMSNorm forwards, P1-D3 every custom backward across
both, P1-D4 the provider adapter, P1-D5 trace/tests/benchmarks and P1-D6
the fixed-K GEMM reference -- eight tasks with P1-S0 and P1-R0.

The kit had invented a one-task-per-operator split instead. That was
wrong in the design doc and, worse, in StubProvider's messages, which
told a developer to "claim it on P1-D1 (hc_split_sinkhorn)" and would
have pointed them at the wrong task.

- StubProvider now names the real owning task per (operator, direction);
  every *_bwd points at P1-D3.
- design doc: the eight tasks with owners, the dependency line, and a
  note that operators and tasks are different axes.
- provider docstring and tests updated to match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kcu8vXve5YMZ5wqCCKRfvJ
#2 has eight GitHub sub-issues, and they are split one per operator --
not by function as the older 任务分工 section in the issue body reads.
The previous commit followed that superseded section and mislabelled
everything; this restores the operator split and uses the authoritative
issue numbers, matching how P5 labels its sub-tasks.

- P1-1 (#14) hc_split_sinkhorn, P1-2 (#15) fp32_gemm_rms, P1-3 (#16)
  mhc_post, P1-4 (#17) mhc_pre/h_aggregate, P1-5 (#18) rmsnorm_residual
  -- each fwd + bwd together; P1-6..P1-8 (#19-#21) are the WS2 TP/SP/CP/
  DP/PP tasks.
- #15 explicitly absorbs the fixed-K / batch-invariant GEMM reference and
  its equivalence harness, so fixed_k_gemm points there rather than at a
  task of its own.
- StubProvider names the owning sub-issue per operator.
- D8 reframed against #18's actual wording: it says prefer TE's
  TEFusedResidualRMSNorm and self-write only if TE fails the
  deterministic contract. TE fails it -- it refuses to expose the
  pre-normalization intermediate -- so unfused v1 is the escape hatch
  #18 provides, not a departure from it.
- design doc: the eight sub-issues with stage, issue number and the WS2
  attachment points (placement + check_capability).

54 tests pass; check_p1.py reference PASS, stub fails closed; lint clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kcu8vXve5YMZ5wqCCKRfvJ
Implements shared_expert_mlp_fwd/bwd per the P5-S0 contract: every valid
token runs fc1 -> one-round SwiGLU -> fc2 on BF16 frozen weights, backward
returns dX only (FP32 accumulator), and the shared output stays independent
of the routed path.

Both backends reproduce the FP32 oracle's numeric profile
oracle-fp32-serial-v1 byte-for-byte on the same device: one lane owns one
output element and reduces serially in ascending k, multiply and add rounded
separately (__fmul_rn/__fadd_rn on CUDA, uncontracted IEEE fp32 in Triton),
sigmoid computed as 1/(1+expf(-x)) to match torch.sigmoid on FP32 CUDA
tensors. No cross-lane floating-point reduction exists anywhere, so results
are batch/padding invariant by construction (fwd(x)[t] == fwd(x[t:t+1])
byte-equal). The one-round SwiGLU core runs in shared mode (p_s=None, no
clamp, per S0 decision D6) and is the reuse point for P5-2 (#63).

Providers subclass ReferenceProvider and override only the two shared-expert
methods, so the full acceptance command runs unchanged; unsupported input
(non-CUDA device, missing extension or triton, foreign numeric profile)
raises instead of falling back (fail-closed). Provenance records split_k=1 /
serial-ascending-k / no-FMA per the P5-5 provenance requirement.

Acceptance:
  python scripts/check_p5.py --provider rl_engine.moe.backends.shared_expert:CudaSharedExpertProvider --device cuda
  python scripts/check_p5.py --provider rl_engine.moe.backends.shared_expert:TritonSharedExpertProvider --device cuda
  pytest tests/test_shared_expert_mlp.py
  python benchmarks/benchmark_shared_expert_mlp.py


Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
tl.exp is the fast exp2-based path and does not bit-match torch.sigmoid;
libdevice __nv_expf does (0/4M mismatches on the device probe).


Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
Neither tl.exp (exp2-based) nor libdevice __nv_expf bit-matches the nvcc
expf inside torch.sigmoid (~45% / ~10% of fp32 values differ by 1 ulp); the
tiny fixtures passed only because the BF16 round absorbed the difference,
and the T=256 benchmark cross-check caught the divergence. The Triton path
now takes torch.sigmoid(gate) as a kernel input and fuses the remaining
SwiGLU math; a (256, 1024, 512) cross-backend byte-equality test locks the
regression in.

Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
… _rn ops

The compiler may contract a * b + c into an FMA; at T=256 that rounded
dsilu = sig * (1 + g * (1 - sig)) differently on 2/262144 dgate elements
(1 ulp after the BF16 round). All mul/add/sub in the Triton strict GEMM and
SwiGLU kernels now go through libdevice add_rn/mul_rn/sub_rn, the exact
Triton spelling of the CUDA kernel's __fadd_rn/__fmul_rn/__fsub_rn.

Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
Signed-off-by: Hsiu-I Liao <hsiu-iliao@Hsiu-Is-MacBook-Pro.local>
Signed-off-by: Hsiu-I Liao <hsiu-iliao@Hsiu-Is-MacBook-Pro.local>
Signed-off-by: Hsiu-I Liao <hsiu-iliao@Hsiu-Is-MacBook-Pro.local>
Signed-off-by: Hsiu-I Liao <hsiu-iliao@Hsiu-Is-MacBook-Pro.local>
Signed-off-by: Hsiu-I Liao <hsiu-iliao@Hsiu-Is-MacBook-Pro.local>
…ct path

Adds two explicitly-selected performance providers that keep the P5-5 round
positions (fc1 output and dX stay FP32; y/dh round once to BF16) and swap
only the GEMM reduction order, per review feedback on #387:

- shared-expert-cuda-det (p5-det-gemm-v1): reuses det_gemm_kernel.cu (fixed
  K-order, no split-K; TMA+mma.sync on SM90+, scalar K-tree fallback
  elsewhere) through two new FP32-output wrappers that expose the existing
  gemm_dispatch output_fp32 path.
- shared-expert-triton-det (p5-triton-dot-v1): tl.dot with fixed 64x64x32
  tiles, ascending-k, no autotune and no split-K.

Both are deterministic and batch-invariant but not byte-equal to
oracle-fp32-serial-v1; selecting them by provider name is the explicit
opt-in, and the strict providers remain the oracle-parity gate. New tests
pin repeat-run byte-equality, per-row batch invariance, and closeness to
the oracle; the benchmark now reports all four backends.

Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
det_gemm merges K with a BF16 mid-split tree (its TP-equivalence design), so
its deviation from the FP32-serial oracle is BF16-tree-sized; the tl.dot path
keeps FP32 accumulators and only carries reduction-order noise. Each provider
now declares its own closeness tolerance instead of sharing one number.

Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
Signed-off-by: Bignonia7 <Bignonia7@users.noreply.github.com>
[P5-2] Add deterministic clamp_swiglu_weighted CUDA kernel
[DSv4][P5-5] Shared Expert MLP: strict CUDA + Triton kernels
Implements the LoRA delta on the routed expert path (#62):

  U  = X @ A^T          FP32 accumulate, round to BF16
  dY = (U @ B^T) * a    FP32 accumulate, FP32 output

Backward follows the same graph and returns (dX, dA, dB); there is no dW,
since the base weights are frozen. dY*alpha and dU round to BF16 between the
GEMMs (contract D4) and the association order is frozen as written.

Three providers, each overriding only this operator so the other eight stay
on the oracle and the acceptance command runs end to end:

  LoRADeltaProvider        torch-native benchmark baseline; not invariant
  LoRADeltaCudaProvider    composed from _C.det_gemm_*
  LoRADeltaTritonProvider  composed from TritonDetGemmOp

Neither deterministic path adds a GEMM -- both compose the repo's existing
fixed-K primitives, per the guidance that GEMMs route through the shared
batch-invariant originals.

The two differ in how close they stay to the oracle, and neither declares
ORACLE_PROFILE.

The Triton path holds one FP32 accumulator across the whole K loop and rounds
once at the store, which is what _serial_dot does -- but the oracle adds one
product at a time while this path adds one BLOCK_K=32 tile at a time, and the
two parenthesisations agree only while nothing rounds at all. BF16 carries 8
mantissa bits, so a BF16xBF16 product needs at most 16 and lands in FP32's 24
exactly; while the partial sums stay inside that headroom the accumulation
order cannot change a single bit. As K grows the accumulator climbs into the
remaining bits and starts dropping low ones, so the agreement degrades rather
than breaking at a threshold -- over 12 seeds at M=24 N=2048, byte-equal holds
12/12 up to K=512, then 11/12 at 1024, 9/12 at 2048 and 7/12 at 4096.

The P5 fixture geometry is K=128 for fc1 and K=64 for fc2, comfortably inside
the headroom, which is why check_p5.py passes. The gap at production width is
FP32-epsilon sized, ~1e-6 relative on a handful of elements.

The CUDA path reuses det_gemm's mid-split tree, which rounds to BF16 at every
32-wide leaf and so sits ~5e-3 from the oracle at all K. It buys something the
Triton path lacks: a contiguous half-K shard is one child of the tree, so TP=2
reproduces TP=1 bitwise, which is likely what P5-8 needs.

Batch invariance is a separate property -- it compares a backend against
itself -- and holds for both at every probed shape, production width included.

Batch invariance did not require avoiding tensor cores, which contradicts a
reasonable first guess. tl.dot is deterministic once the tile shape stops
depending on M; what matters is pinned BLOCK sizes (autotune would pick
per-shape configs), no split-K, and allow_tf32=False.

provenance() reports the reduction strategy per P5-6's fail-closed contract.

Signed-off-by: Loring Ni <93922846+ScottNi0809@users.noreply.github.com>
Follows benchmark_det_gemm.py: reports overhead against a fair baseline
(cuBLAS with TF32 disabled) rather than a speedup, since both deterministic
paths disable TF32 by construction.

Forward on RTX A2000 (sm_86), median of 5x100 iterations:

  shape        M    native fp32   Triton   CUDA    Triton ovh   CUDA ovh
  decode_m1    1      0.200        0.248   0.205      1.2x        1.0x
  small_m8     8      0.176        0.233   0.201      1.3x        1.1x
  batch_m64    64     0.193        0.344   0.352      1.8x        1.8x
  batch_m256   256    0.214        0.240   0.718      1.1x        3.4x

The Triton path stays within 1.1-1.8x of cuBLAS while being byte-equal to the
oracle. The CUDA path degrades with M because det_gemm falls back to its naive
scalar kernel below SM90; an SM90 runner would take the TMA path instead.

Profiling explains the absolute numbers: at M=64 the two BF16 GEMMs cost
0.070 ms but the whole native forward costs 0.222 ms, so 68% of the wall clock
is the elementwise .float() / *alpha / BF16-cast traffic rather than the
matmuls. Traffic analysis agrees -- at X=[M,4096], N=2048, r=8 the arithmetic
intensity is about r = 8 FLOP/byte and the split is ~66% reading X, ~33%
writing Y, and only 0.26% for the U round trip.

So fusing away the intermediate U is not where the headroom is; folding the
scale and the cast into the GEMM2 epilogue is, and that is #58 (Step 10),
deliberately out of scope here. It also means small-M rows are launch-bound,
which is why the TF32 column can land below the FP32 one -- the table should
be read as relative overhead, not GEMM throughput.

Signed-off-by: Loring Ni <93922846+ScottNi0809@users.noreply.github.com>
tests/test_p5_lora_delta.py -- 43 cases parametrised over the CUDA and Triton
providers, each validated independently. The torch-native provider is excluded
on purpose: it is the benchmark baseline and fails batch-invariance by design.
Skips cleanly on CPU-only runners.

Covers the five acceptance criteria from the P5-3 brief:

  golden vector    base-only / LoRA-only / base+LoRA, via the oracle
  tamper fixture   flipping a packed base byte must reach the output
  no dW            frozen weights untouched, backward returns exactly 3 grads
  finite/nonzero   the adapters actually receive signal
  fc1 and fc2      both insertion points, whose shapes differ

plus return order and dtypes, chunked-prefill and padding invariance,
run-to-run stability, and provenance naming the backend that really ran.

Two tests bracket the Triton path's agreement with the oracle instead of
claiming it outright: one pins the byte-equality at fixture width, the other
pins the bounded divergence at production width and fails if that divergence
ever disappears -- which would mean the docstring and the profile have gone
stale. The second sweeps seeds rather than pinning one, since whether a given
input diverges depends both on where its values fall relative to a rounding
boundary and on which instruction tl.dot lowers to (MMA on Ampere, WGMMA on
Hopper).

scripts/check_p5_lora_delta_invariance.py -- 100 probes per backend
(5 seeds x 5 sub-batch sizes x 4 shapes), reporting per-shape and per-boundary
counts so a real pass can be told apart from a lucky one:

  backend            invariant      u     y    dX   rerun
  torch-native         47/100     100    87    47      ok
  cuda det_gemm       100/100     100   100   100      ok
  triton det_gemm     100/100     100   100   100      ok

Three findings shaped both files. Mutating four real bugs (swapped return
order, dropped alpha, transposed dX, a GEMM replaced by torch.matmul) showed
the invariance checks were running at the fixture's 128x64 geometry, where
cuBLAS keeps one strategy across all M and a non-deterministic implementation
passes unnoticed; production geometry (4096x2048) is now probed too. GEMM1
cannot carry an invariance check at all -- u is [M, 8], too narrow for cuBLAS
to switch strategy. And dA/dB reduce over every row, so a sub-batch result is
a different quantity rather than a slice; they are covered by the rerun check.

The tamper test carries a control assertion that an untampered rerun is
bitwise identical, so the not-equal check cannot pass on incidental noise.
One flipped byte moves 494/3072 output elements by up to 2.2e-2.

Signed-off-by: Loring Ni <93922846+ScottNi0809@users.noreply.github.com>
Signed-off-by: Bignonia7 <326766516+Bignonia7@users.noreply.github.com>
…5 oracle

P5-1: MXFP8 activation quantization (OCP E4M3 elements, block-32 E8M0
shared scales) plus the straight-through backward, in both CUDA and Triton.

Both backends reproduce rl_engine.moe.mx_format.mx_quantize byte for byte, so
they keep the oracle numeric profile (oracle-fp32-serial-v1) instead of
registering a relaxed one:

  the block amax is an integer max over |x| bit patterns, floor(log2) is read
  off the FP32 exponent field after the FLT_MIN clamp (exact integer
  equivalent of the oracle's frexp path), the scale is assembled from its bit
  pattern so code 0 lands on the subnormal 2**-127, and the division is an
  explicit non-.ftz div.rn.f32 / tl.fdiv(ieee_rounding=True). Nothing on the
  subnormal-sensitive path can be flushed by --use_fast_math; a fast-math build
  is a regression test. E4M3 encoding is clamp-then-RNE (start-kit decision
  D1), which is what cvt.rn.satfinite.e4m3 does in hardware.

Determinism: the amax reduction is row-local over a fixed 32-element window and
every element is then transformed independently, so the bytes do not depend on
grid shape, block size, or how many rows are in flight.

Non-finite input is fail-closed in both backends and in the providers (P5-1 spec s4);
the STE backward accepts any floating dtype (P5-1 spec s2).

Providers live in rl_engine/moe/backends/mxfp8_act_quant.py, following the
backends package P5-5 introduced. scripts/check_p5.py --device cuda passes
byte-equal on every boundary for both {Cuda,Triton}MXFP8ActQuantProvider,
with act_quant1.*/act_quant2.* hashed and act_quant_bwd noted as "ste".

Two outside MX implementations are byte-identical to these kernels and are
asserted as tests when installed: torchao's to_mx (FLOOR) and the Triton repo's
triton_kernels.downcast_to_mxfp (ROUND_DOWN), the quantizer vLLM runs for its
MXFP4 path. The benchmark measures against both, against vLLM's production
per_token_group_quant_fp8 CUDA kernel (a different format: group 128, FP32
scale), and against a plain fp8 cast as the memory-bandwidth lower bound. On
H100 the kernels reach 2.99 TB/s at 16384x7168, at that bound.

The CUDA wrapper prefers the compiled rl_engine._C symbols and otherwise
JIT-builds the single .cu file, so the tests and the benchmark run in a source
tree without a full extension build (RL_KERNEL_P5_DISABLE_JIT=1 to require AOT).

Signed-off-by: Ryan Huang <ryankert01@gmail.com>
…nd past 2**31 elements

The benchmark shapes (4096x7168 and 16384x7168, bf16 and fp32, random exponents
over 2**+-60) were only timed before; they are now asserted byte-equal against
the oracle, at ~0.2 s each.

A 2**31-element bf16 input (262144 x 8192) checks that the Triton kernel's int64
element offsets do not wrap: the oracle cannot run at that size, so rows at the
start, middle and far end of the tensor are compared with their standalone
quantization, which the row-local contract requires to be identical. Skipped
when the device has less than ~7 GiB free.

Signed-off-by: Ryan Huang <ryankert01@gmail.com>
…indings

- Providers resolve their backend once in __init__ and raise
  NotImplementedError when it is unavailable, matching the other P5 providers
  and what scripts/check_p5.py renders as a FAIL row; the per-call import and
  the _Ops bag are gone, and the recorded linkage is the one that resolved.
- The CUDA wrapper resolves rl_engine._C or the JIT module once per process and
  refuses the JIT fallback under RL_KERNEL_REQUIRE_EXT=1 (the repo's existing
  "extension must be present" switch, set by the GPU CI) instead of a new flag.
- Input gates and the fail-closed read-back live in
  rl_engine/kernels/ops/moe_common.py, shared by both backends;
  rl_engine/moe/mx_format.py is back to defining only the format.
- The non-finite check is derived from the block amax bit pattern
  (inf/NaN >= 0x7F800000) instead of a second full reduction, in both kernels;
  bytes are unchanged.
- csrc/ops.cpp binds the _cuda entry points directly, as the sibling P5 ops do;
  the six load_vec specializations collapse to one template with a 16-bit
  converter; rl_engine.moe no longer re-exports the providers (the backends
  package is the registry).
- Tests: the --use_fast_math build uses torch's build cache instead of a fresh
  tmp_path per session (~50 s saved per run); one _wide_range generator.
- Benchmark: dead _speedup helper removed; one byte-equality check.

Signed-off-by: Ryan Huang <ryankert01@gmail.com>
[DSv4][P5-4] MXFP8×MXFP4 Routed Expert grouped GEMM: wgmma CUDA kernel
…rapper

Review on #419: `envs.py` is a build-time helper next to setup.py and is not
part of the installed rl_engine package, so importing it unconditionally broke
a non-editable install. Import it optionally and fall back to reading
RL_KERNEL_REQUIRE_EXT directly (the spelling the CI scripts export).

Signed-off-by: Ryan Huang <ryankert01@gmail.com>
KJLdefeated and others added 30 commits November 11, 2026 16:04
m_grouped_bf16_gemm_nt_contiguous gives the bytes of one bf16_gemm_nt per expert in
both the m-indices layout and the psum layout (each expert starts at the previous
end aligned up to 128), over 5 repeats and uneven / decode / large row
distributions; a row alone in its expert gives the same bytes;
k_grouped_bf16_gemm_tn_contiguous wgrad is within 8e-7 of FP32. H200, 32 experts,
H 4096, rank 64: 0.06-0.77 ms against 0.22-0.92 ms for the per-expert loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
rl_engine/models/deepseek_v4/layerwise_gate.py runs the same sequences as a
continuous-batching rollout (staggered arrivals, chunked prefill under a token
budget, decode next to prefill chunks), as the actor recompute (one packed prefill,
reversed request order) and through forward_train, then compares every layer's
streams per token and reports the first layer that differs. forward_train gains
return_streams. scripts/check_dsv4_layerwise_gate.py runs it on a random checkpoint
or the first --layers layers of a real one.

H200, real dimensions, nonzero LoRA, window / C4 / C128 with hash and scored
routing, 2108 tokens: every layer, the final hidden state and the logprobs are
byte-identical for 4 layers (with training) and 8 layers. Negative control: a
one-element change in layer 2 is reported at layer 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
Per-expert MoE adapters are deferred until verl needs them; when built they use
DeepGEMM's grouped BF16 GEMM (probed byte-equal to per-expert GEMMs) under a new
profile with a BF16-rounded delta. Chunked training across sequence chunks is not
planned; packed multi-sequence batches already work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
… removal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
The Megatron cross-config training provider (adapters, readback, gradient ranks),
tests/p7_doubles and the test_megatron_training_* suite are not on the DSv4 path
and reference APIs that main no longer has (mypy). The DSv4 logprob head replaces
P7; verl integration will be redone on its own. Drop the CI step and the
conftest sys.path hook that only these helpers needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
rl_engine/p2, rl_engine/p6, their tests, CI workflows, smoke script, design
notes and validation reports came from the start-kit phase; the DSv4 attention
and MoE combine replace them and nothing imports them. The start-kit design notes
for P1, P5 and P7 go too. Duplicate validation reports (the same probe on the old
toolchain) are dropped; the torch 2.13 / CUDA 13 reports stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
… use

The P1 (unfused mHC, fp32_gemm_rms, rmsnorm_residual, h_aggregate) and P5
(providers, LoRA delta, clamp SwiGLU, Triton lanes, BF16 shared expert) kernels
stay in the tree but are not on the DSv4 path; their tests, check scripts and
benchmarks go, as does the P6 moe_merge test harness. Tests of modules DSv4 does
use move to tests/ops/moe (mx_format, MXFP8 act quant, MoE contract, oracle), and
the golden-hash fixtures move from tests/fixtures/p1, p5 to mhc, moe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
- DSv4: Any for the MoE block's object-typed fields; overloads for
  silu_and_mul_clamp_quant_cuda's return_h; narrow Optional cache / weight
  fields with asserts; type the layerwise gate's locals; rename a reused
  variable in random_checkpoint.
- Import _C from rl_engine.backends.extension (where main moved it) instead of
  the kernels.ops.base shim mypy cannot see through.
- mxfp8_act_quant parses RL_KERNEL_REQUIRE_EXT like build_tools.envs.env_flag
  instead of importing the repo-root envs shim.
- Legacy P1 / P5 code: TritonDetGemmOp from its current module, Any for the
  extension handle, and targeted type: ignore where the old code cannot be typed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
docs/architecture/dsv4-design.md replaces the plan / todo log (dsv4-cuda-design.md),
the batch-invariant profile and the attention-backward proposal, and folds in the
performance table. It covers the goal and guarantees, environment, build, the
regression / benchmark / probe commands, the module map (code, kernels, operator page,
tests), decisions, pinning rules and the reduction order of every module, numeric
profiles, training, validation including the layerwise gate, what the rollout must
patch, legacy code that DSv4 does not use, open work and numerics lessons.

- Every DSv4 operator page links to it and drops the P1-P7 / T0x labels.
- The MoE page describes the DeepGEMM shared expert DSv4 runs (it still described the
  legacy strict kernels) and lists the legacy backends instead of their ROCm results.
- Legacy operator pages (fp32_gemm_rms, h_aggregate, rmsnorm_residual, moe_merge) say
  they are not used by DSv4.
- p7-t02-m10-sampling-contract.md becomes dsv4-sampling-contract.md.
- CLAUDE.md points to the design doc and drops the account-specific environment notes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
… count

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
Every CTA of the routed-expert forward and backward GEMMs scanned all E
expert offsets to find its token block, and the grid was sized from
ceil(M / BM) + E. At small M most CTAs only scanned and exited: E = 256,
T = 1 launched about 8k CTAs for 6 rows and took 1.9 ms on H200.

A one-block kernel now builds tile -> (expert, first row) and the tile
count once per GEMM (moe_tile_map.cuh); a CTA decodes its block with two
loads. The grid bound is ceil(M / BM) + min(E, M). Scheduling only: tile
bytes do not depend on which CTA computes them, so outputs are unchanged.

H200, routed experts (BF16 in -> combined out), CUDA-graph replay:
  E = 256, T = 1:     1.88 -> 0.131 ms
  E = 256, T = 64:    3.47 -> 2.27 ms
  E = 256, T = 16384: 17.2 -> 12.9 ms
  E = 32,  T = 1..256 unchanged within 10%

DSv4 regression: 1217 passed, 72 skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
check_dsv4_moe_ep.py --lora ran on 2 H200s. Forward is byte equal to EP = 1
for scored and hash layers in every split. dX differed (relative L2 up to
8.3e-5), deterministic run to run.

Cause: lora_backward runs FP32 torch.mm, and cuBLAS picks its kernel from M,
so a row's result depends on the row count of the call (rows of
[M, 1024] @ [1024, 16] at M = 1, 2, 12 differ from M = 97). On one GPU the
routed fc2 adapter already shows it (block dX alone vs in batch, 5e-5); EP
changes each rank's row count, so it shows up against EP = 1. The design
already does not require the LoRA dX path to be deterministic.

The check now requires, with --lora: forward byte equal, dX run-to-run equal
and within 1e-3 relative of EP = 1, adapter gradients within 1e-3 (seen
2.2e-4). Without LoRA, dX must still be byte equal (passes). The design doc
records the result, the cause and the lesson.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
… 8 GPUs

scripts/verl_e2e/ lets an engineer or a coding agent on an 8 x H100 / H200
machine check whether verl + vLLM batch-invariant mode (verl #7688, image
iseekyan/verl:ds4_vllm_align.preview-amd64 by digest, pinned verl and
Megatron-LM commits) gives bitwise train/rollout log-probs for
DeepSeek-V4-Flash, and what it costs:

- setup.sh / launch_container.sh: preflight, pull, pinned clones, model
  and DAPO data, a long-lived container.
- patch_verl_bitwise_metric.py: verl's rollout_probs_diff compares
  exp(log_prob) and reads 0.0 for log-probs 1 ulp apart (checked on the
  real metrics.py); the patch adds the count of response tokens whose FP32
  bits differ.
- run_e2e.sh: T1 (the recipe's 4-GPU quick test), T2 (8 GPUs, EP8, 4
  layers), T3 (more layers, longer responses); summarize_metrics.py gives
  the per-step verdict and timing.
- bench_rollout.sh / probe_server_bi.py: vLLM DP8+EP8 throughput with BI
  on, off and native experts, plus a black-box alone-vs-batch probe.

Checked here: shell syntax, the metric patch on verl's file (1-ulp case
caught), the summarizer on a synthetic log. Nothing has run on 8 GPUs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
T4 trains 200 steps on the first 8 layers of DeepSeek-V4-Flash (32
prompts x 8 samples, one update per step, LR 1e-5) and checks the FP32
log-prob bits every step. A truncated model scores 0 on the math reward,
so GRPO advantages and gradients would vanish and the weights would never
move; custom_reward.py (fraction of digits) is a dense synthetic reward
that keeps real updates flowing to the rollout. The summarizer counts
steps with a nonzero grad norm and shortens long per-step tables. The
README now targets 8 x H100 (memory and OOM fallbacks, T3 as the memory
probe for T4, tmux, no checkpoints at ~700 GB each).

Checked here: shell syntax, the reward function, the summarizer on a
synthetic 200-step log. Nothing has run on 8 GPUs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
The guide now runs setup -> T1 (environment check) -> rollout speed (BI
on / off, full model) -> T4 (50 steps, 8 layers, early check at step 5,
OOM fallbacks inline), about 6-10 h of machine time instead of ~1.5 days.
T2 / T3 stay in run_e2e.sh as optional variants. The baseline recipe
trains all parameters on Hopper, so T4 keeps the truncated model; the
full-model LoRA run belongs to the RL-Kernel stack once it is in verl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
The guide has two training tests now: T1 (the recipe's quick check) and
T2 (8 GPUs, 8 layers, 50 steps). run_e2e.sh drops the old T2 / T3
variants so the names match the README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
rl_engine/integrations/dsv4/vllm_model.py registers RlkDeepseekV4ForCausalLM under vLLM's
DeepseekV4ForCausalLM name (RL_KERNEL_DSV4_ROLLOUT=1 general plugin; no vLLM source
changes). It keeps vLLM's scheduler, paged KV cache, FlashMLA layer, indexer and FP8
projections, and runs RL-Kernel's kernels where the vLLM fork rounds differently:

- mHC pre / post / head (four identical first-layer streams), whole MoE block (Dsv4MoE,
  loaded from the checkpoint; vLLM allocates no experts), plain RoPE on window layers
- q / kv RMSNorm, q-norm / RoPE / KV insert into vLLM's fp8_ds_mla cache, indexer q
  RoPE + FP8 quant, output projection (BI mode on Hopper swaps fp8_einsum for Triton)
- FlashMLA: RL-Kernel's pinned build serves vLLM's interface (same commit, other build
  flags; both register torch.ops._flashmla_C, so ours loads first)
- sparse decode kernel (VLLM_DS4_DECODE_KERNEL=sparse), one stream in attention
- FP32 logits head; sampled-token log-prob from the 64-tile LSE (V1 and V2 runners)
- unmerged LoRA on wq_a / wkv / wq_b / wo_b and in Dsv4MoE
- VLLM_BATCH_INVARIANT starts on the pinned DeepGEMM (vLLM's set_batch_invariant check
  guards MoE kernels this model does not run)

scripts/check_dsv4_vllm_gate.py: vLLM with chunked prefill (decode next to prefill
chunks) against one packed Dsv4Model prefill, every layer, sub-layer and sampled-token
log-prob in bytes. Toy checkpoint (4 layers: window, window, C4, C128): PASS without
LoRA (2102 tokens, 54 log-probs at T = 0.8). With LoRA the C4 layer still differs.

Also: build_lora_adapters (attach_lora's adapters, reusable by the rollout), FP32 in
LoraAdapter.init (vLLM loads under a BF16 default dtype), load_output_projection, and
scripts/probe_dsv4_vllm_attention_ops.py (q-norm / RoPE / KV insert vs vLLM's kernel).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
… with LoRA

vLLM's compress -> RMSNorm -> RoPE -> FP8 quant -> store step (cutedsl for the main C4 /
C128 KV, Triton for the indexer K) rounds a rare compressed entry differently; with LoRA
the toy data hit one in the C4 layer (positions 35 and 303), and every later query
attends to it. The plugin routes that step to compress_main_cuda / compress_indexer_cuda
(same state-cache and block-table addressing); vLLM's save_partial_states stays.

scripts/check_dsv4_vllm_gate.py, toy checkpoint (window, window, C4, C128), chunked
prefill, T = 0.8: PASS without LoRA and with rank-16 adapters on wq_a / wkv / wq_b /
wo_b and the MoE (2102 tokens at every layer and sub-layer, 54 sampled-token log-probs).
The report also lists the (request, position) of every differing attention output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
…tatus

Section 10 now records the rollout plugin: each vLLM fork kernel the gate caught and what
replaces it (the earlier "already equal to vLLM" list was wrong for the q / kv norm, the
q-norm / RoPE / KV insert, the compressor store and the FlashMLA build). Section 12: the
plugin is done on one GPU with LoRA; the actor is a verl engine backend on Dsv4Model
(2026-11-15 decision). Section 13: two lessons from the gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
persist_w8 (default on) expands every expert's folded FP8 weights once, on first
use, and caches the scratch with the prepared weight. Same bytes as the per-call
expansion; costs E x N x K bytes of E4M3 per weight.

Job 181224 (H200): 1237 passed / 52 skipped, layerwise gate PASS (2108 tokens),
EP2 checks pass. E=256 MoE vs vLLM BI: T=4096 1.70x -> 0.88x, T=16384 1.62x ->
1.35x; E=32 T=1024 2.39x -> 1.32x.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
A verl BaseEngine around Dsv4Model, registered in EngineRegistry on import
(model.external_lib) and selected through hydra config groups shipped in
rl_engine (model_engine=rlkernel, hydra.searchpath=pkg://...verl_config).
verl's source is unchanged.

- forward_backward_batch on verl's micro-batching: one packed full-prefill
  forward per micro-batch, jagged next-token log-probs (dsv4-logprob-v1),
  per-sample temperatures grouped per row; entropy as an FP32 metric only.
- LoRA only: FP32 master adapters with verl's build_optimizer and LR
  schedules, FP32 grad accumulation, one flat DP all-reduce, clip, step.
- get_per_tensor_param exports the BF16 adapters with a PEFT-style config;
  disable_adapter gives the base model; checkpoint save / load.
- LoRA rank lives in the engine config: verl's model.lora_rank would turn on
  vLLM's own LoRA in the rollout.

scripts/check_dsv4_verl_engine.py (torchrun, 2 x H200, toy checkpoint):
log-probs equal byte for byte across micro-batch layouts, against each
sequence alone and with per-sample temperatures, before and after training;
adapters identical on both ranks; disable_adapter equals the base model;
export and checkpoint round trip. VERL ENGINE PASS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
…to the actor

The rlkernel engine (actor) and the vLLM rollout plugin now run together
under verl with no change to verl or vLLM:

- vllm_worker.py: RlkWorkerExtension installs the adapters the engine
  exports (BF16 lora_A / lora_B) on the plugin's layers instead of vLLM's
  LoRA manager; RlkWorker keeps level-1 sleep, since the base weights load
  once from the checkpoint and are never re-sent. Selected through
  rollout.engine_kwargs.vllm.worker_cls / worker_extension_cls.
  RL_KERNEL_DSV4_SKIP_ADAPTER_SYNC=1 is a negative control.
- vllm_model.py: vLLM's spawned processes unpickle the DeepSeek-V4 quant
  config, which imports vLLM's DeepSeek-V4 package and its FlashMLA build
  before any plugin runs; register a subclass defined in this module so the
  plugin (and RL-Kernel's FlashMLA build) loads first. Refuse
  load_format=dummy.
- verl_engine.py: accept verl's trainer-side rollout quantizer (offered for
  quantized checkpoints) and leave it unused; only adapters are synced.
- scripts/verl_e2e/run_dsv4_rlkernel_grpo.sh: the launcher.

Toy GRPO on 2 x H200 (toy checkpoint, rank-16 LoRA, 8 x 4 samples x 128
tokens, synthetic reward): training/rollout_logprobs_mismatch_count 0 at
all 4 steps. Negative control: 0 at step 1, 4096 / 4096 from step 2. The
vLLM gate still passes with LoRA (2102 tokens).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
How the rlkernel engine, the vLLM plugin and the worker extension plug into
verl and vLLM, the toy GRPO result and its negative control, and the open
items: the 8-GPU guide against verl #7483, EP > 1 in the engine, memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
Batch, lengths, token budget, rollout memory, LR, logger and experiment name are
environment variables; the defaults remain the 2-GPU toy run. Checked on the toy
checkpoint (1 x H200, file logger): rollout_logprobs_mismatch_count 0 at all 4 steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
scripts/verl_e2e_lora/: setup of both stacks on one verl commit (#7483's head), the
real checkpoint's first LAYERS layers, RL-Kernel's gate on that checkpoint, the
RL-Kernel run, the negative control, #7483's own launcher on the same workload,
and compare.py (bitwise verdict per step, timing breakdown, side by side).
The RL-Kernel env pins the tested 2-GPU env's packages and the local DeepGEMM fix.
Not run here (2 GPUs per job); the BI-off ablation needs a plugin switch first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
CLAUDE.md now carries the owner's decisions, the non-obvious verl integration facts,
status and next steps, and this cluster's environment (limits, envs, job scripts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
Formatting only. The gate's plugin-before-vLLM import order is kept with isort: off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
RL_KERNEL_DSV4_NON_BI=1 with VLLM_BATCH_INVARIANT=0 (the launcher's BI=0): the
rollout plugin keeps vLLM's own attention paths (non-invariant planning, paged FP8
decode, fused q / kv norm, q norm / RoPE / KV insert, indexer query, compressor
store, aux streams) and RL-Kernel's mHC, MoE, output projection, LoRA, RoPE tables
and FP32 head, so the model stays the one the actor trains. Mixing the two env
settings is refused. check_dsv4_vllm_gate.py --non-bi reports the difference.

Toy checkpoint, 1 x H200:
- BI gate with LoRA still byte-identical.
- --non-bi with LoRA: 302 of 2102 tokens differ after layer 0 (decode tokens and
  some prefill chunks), the rest are byte-identical; differences then grow through
  the random-weight layers.
- GRPO BI=0: every response token mismatches (as intended), timing_s/gen 0.79x
  and timing_s/step 0.85x of the BI run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>
run.sh ablation (BI=0) and a report-only --non-bi check in run.sh gate; section 10
describes the switch and the toy numbers; section 12 and CLAUDE.md status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tkttC4AW5Y1FLX1EvSsYj
Signed-off-by: KJLdefeated <linkai0508@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-close DSv4 platform: cuda Specific optimizations or bugs in NVIDIA graphics cards (such as FlashInfer, TMA optimizations) priority: high Severe congestion issues require the highest priority for resolution. status: needs-review The code is complete and awaits review by the core maintainers. WIP Work In Progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.