Repository navigation
Description
The CUDA backend was fixed for a stale-vector-load bug after a cast (issue #4131, pull request #4138). The same code pattern is still present in the HIP and CuTeDSL cast codegen, so the same wrong-result behaviour should be expected there. This report is a code-level finding only; see the evidence limits at the bottom.
The defect
CodeGenSourceBase::SSAGetID caches an entry keyed by the printed text of an expression and returns that entry until the enclosing scope ends. Nothing invalidates an entry when a store writes to the buffer whose load the entry holds. So if a vectorized cast passes a buffer load through SSAGetID, and the same buffer is stored later in the same scope, the next statement that prints the same load text gets the pre-store value.
The CUDA fix replaced the caching call with a fresh temporary at the two affected sites:
std::string src = name_supply_->FreshName("cast_src");
PrintSSAAssign(src, PrintExpr(op->value), from_ty);src/cuda/codegen/codegen_cuda.cc now has no SSAGetID left in VisitExpr_(const CastNode*); the fp8 and fp4 branches route through the same two materialization points, so they are covered as well.
Where the pattern remains
src/rocm/codegen/codegen_hip.cc, VisitExpr_(const CastNode*) (starts at line 1128). The generic vectorized branch is the direct counterpart of the fixed CUDA line:
1331: std::string sret = name_supply_->FreshName("_");
1336: std::string src = SSAGetID(PrintExpr(op->value), from_ty);All vectorized casts that do not hit a special-cased dtype pair go through line 1336, so this is the common path for float16, bfloat16, float8 and integer conversions on ROCm.
The FP4 pair-wise branches use the same call for their source operand: lines 1156, 1181, 1203, 1224, 1247, 1271, 1292, 1312. FP4 hardware is gfx950-only, but the source operand handling is identical.
The vector binary path has the usual SSAGetID calls as well (lines 754, 755 and 816, the counterparts of the CUDA sites that tested clean), so I am not claiming a stale case there.
src/cuda/codegen/codegen_cutedsl.cc, VisitExpr_(const CastNode*) (starts at line 584), generic vectorized branch:
714: std::string src = SSAGetID(PrintExpr_(op->value), from_ty);Lines 631 and 665 are scalar-source fp8 helpers and hold a scalar load rather than a vector one; they are listed for completeness rather than claimed as exploitable.
Reproduction on a machine with an AMD GPU
The kernel from the #4131 regression, lowered for a HIP target, is the direct check. The two statements after the cast must observe the cast's store, not the load it consumed:
with T.Kernel(1, threads=256):
fragment = T.alloc_fragment((32, 16), "float32")
T.copy(A, fragment)
for i, j in T.Parallel(32, 16):
fragment[i, j] = T.cast(T.cast(fragment[i, j], "float16"), "float32")
for i, j in T.Parallel(32, 16):
fragment[i, j] = fragment[i, j] + 1.0
for i, j in T.Parallel(32, 16):
fragment[i, j] = fragment[i, j] + 3.0
T.copy(fragment, B)On CUDA this returns the pre-cast value for the + 1 statement before #4138 and the correct value after it. The expected HIP behaviour is the same failure, because the emitted cast would bind src to a cached load of fragment and the following store to fragment would not invalidate it.
Proposed change
The three generic sites take the same two-line change as CUDA:
- std::string src = SSAGetID(PrintExpr(op->value), from_ty);
+ std::string src = name_supply_->FreshName("cast_src");
+ PrintSSAAssign(src, PrintExpr(op->value), from_ty);codegen_cutedsl.cc:714 uses PrintExpr_ rather than PrintExpr, so the argument there follows its local naming. The FP4 branches would take the same replacement if maintainers want them covered now, though they cannot be exercised without gfx950.
Evidence limits
This environment is one RTX 4090 machine with no AMD GPU and no ROCm toolchain, and the built library registers no HIP backend at all: tvm.get_global_func("target.build.tilelang_hip_without_compile", allow_missing=True) returns nothing, so I could not lower the kernel above for HIP to confirm the emitted source. The CuTeDSL backend is likewise not built here. I have therefore not run any of this, and I am not claiming a reproduced numerical failure on either backend. What I did verify on hardware is the CUDA half: the base commit returns the stale value for that kernel and the patched tree returns the correct one, which is what pull request #4138 contains.
Two smaller notes from the same investigation, offered as negative results so they are not re-checked:
- Vectorized add, subtract, division and exp do not show the stale behaviour on SM89. Their operands are materialized inside a per-statement scope, so the cache entry does not outlive the statement.
Broadcastof a scalar load reachesSSAGetIDatcodegen_cuda.cc:6142and:6159for the fp4 and fp8 splat paths, but those only run for those dtypes and I could not construct a stale case for them.
Disclosure
Prepared with Claude Code. The CUDA runs cited above were executed on the remote GPU machine described in #4131; the HIP and CuTeDSL observations are source reading only.
Activity
wynneyin commentedon Nov 15, 2026
Correction on the CuTeDSL part of this report, after checking the code again and trying to lower for that backend.
The proposed replacement does not work there.
CodeGenC::PrintSSAAssignemits C syntax with a type prefix and a terminating semicolon, while the CuTeDSL codegen emits Python. There is noPrintSSAAssignoverride incodegen_cutedsl.ccor its header, so writing a fresh temporary the way the CUDA backends do would put an invalid statement into the generated source.codegen_cutedsl.cc:714needs a materialization expressed in that backend's own syntax, not a copy of the CUDA two-liner.I also could not confirm the defect itself for CuTeDSL. Lowering is blocked in this environment by the Python side of the backend, before any of its codegen runs:
So please treat the CuTeDSL claim in the report above as unassessed rather than as a finding: the
SSAGetIDcall is there, but whether the cached name can go stale depends on how that backend materializes buffer loads, and I could not produce a lowered kernel to look at.The HIP part stands, and a patch is ready:
SSAGetIDcalls insrc/rocm/codegen/codegen_hip.ccwith the fresh-temporary form, the generic branch at line 1336 plus the eight FP4 branches.g++ -std=c++17 -fsyntax-onlyovercodegen_hip.ccwith the tree's TVM and CUTLASS includes passes, before and after. This build has no ROCm toolchain and no AMD GPU, so the HIP backend is not even compiled into the library here, and no lowering or numerical run was possible. The CUDA half of the same change is the one that was verified on hardware in [CUDA] Prevent stale loads after vectorized casts #4138.Prepared with Claude Code.
wynneyin commentedon Nov 15, 2026
Second correction, and it retracts part of the first one.
The first correction said the two-line CUDA change does not work for CuTeDSL because
PrintSSAAssignemits C syntax and there is no override incodegen_cutedsl.cc. That statement is wrong, and leaving it standing would be worse than the original overreach.CodeGenTileLangCuTeDSLderives from the Python codegen base, not fromCodeGenC:and that base supplies a Python-syntax implementation of the hook:
So
PrintSSAAssign(target, src, type)in this backend emitstarget = src, which is valid Python. I had only looked for an override inside the CuTeDSL files and did not follow the class to its base. That was the error in the first correction.What this changes: the same replacement that #4138 applies to CUDA,
is valid for
codegen_cutedsl.cc:714as written, using that file'sPrintExpr_spelling. The static reason I gave for not touching the line does not hold.What this does not change: I still cannot lower for the CuTeDSL backend in this environment (
nvidia-cutlass-dslversion gate, quoted in the first correction), so whether line 714 can actually go stale remains unassessed. The CuTeDSL half is still not a verified finding, in either direction.Pull request #4207 is unaffected and stays HIP-only.
Prepared with Claude Code. This comment is source reading of the classes named above; nothing was lowered or run for CuTeDSL.