Skip to content

[BUG] Vectorized cast codegen in the HIP and CuTeDSL backends still caches the operand load and can return a stale value after a store #4206

Open

Description

@wynneyin

The CUDA backend was fixed for a stale-vector-load bug after a cast (issue #4131, pull request #4138). The same code pattern is still present in the HIP and CuTeDSL cast codegen, so the same wrong-result behaviour should be expected there. This report is a code-level finding only; see the evidence limits at the bottom.

The defect

CodeGenSourceBase::SSAGetID caches an entry keyed by the printed text of an expression and returns that entry until the enclosing scope ends. Nothing invalidates an entry when a store writes to the buffer whose load the entry holds. So if a vectorized cast passes a buffer load through SSAGetID, and the same buffer is stored later in the same scope, the next statement that prints the same load text gets the pre-store value.

The CUDA fix replaced the caching call with a fresh temporary at the two affected sites:

std::string src = name_supply_->FreshName("cast_src");
PrintSSAAssign(src, PrintExpr(op->value), from_ty);

src/cuda/codegen/codegen_cuda.cc now has no SSAGetID left in VisitExpr_(const CastNode*); the fp8 and fp4 branches route through the same two materialization points, so they are covered as well.

Where the pattern remains

src/rocm/codegen/codegen_hip.cc, VisitExpr_(const CastNode*) (starts at line 1128). The generic vectorized branch is the direct counterpart of the fixed CUDA line:

1331:  std::string sret = name_supply_->FreshName("_");
1336:    std::string src = SSAGetID(PrintExpr(op->value), from_ty);

All vectorized casts that do not hit a special-cased dtype pair go through line 1336, so this is the common path for float16, bfloat16, float8 and integer conversions on ROCm.

The FP4 pair-wise branches use the same call for their source operand: lines 1156, 1181, 1203, 1224, 1247, 1271, 1292, 1312. FP4 hardware is gfx950-only, but the source operand handling is identical.

The vector binary path has the usual SSAGetID calls as well (lines 754, 755 and 816, the counterparts of the CUDA sites that tested clean), so I am not claiming a stale case there.

src/cuda/codegen/codegen_cutedsl.cc, VisitExpr_(const CastNode*) (starts at line 584), generic vectorized branch:

714:  std::string src = SSAGetID(PrintExpr_(op->value), from_ty);

Lines 631 and 665 are scalar-source fp8 helpers and hold a scalar load rather than a vector one; they are listed for completeness rather than claimed as exploitable.

Reproduction on a machine with an AMD GPU

The kernel from the #4131 regression, lowered for a HIP target, is the direct check. The two statements after the cast must observe the cast's store, not the load it consumed:

with T.Kernel(1, threads=256):
    fragment = T.alloc_fragment((32, 16), "float32")
    T.copy(A, fragment)
    for i, j in T.Parallel(32, 16):
        fragment[i, j] = T.cast(T.cast(fragment[i, j], "float16"), "float32")
    for i, j in T.Parallel(32, 16):
        fragment[i, j] = fragment[i, j] + 1.0
    for i, j in T.Parallel(32, 16):
        fragment[i, j] = fragment[i, j] + 3.0
    T.copy(fragment, B)

On CUDA this returns the pre-cast value for the + 1 statement before #4138 and the correct value after it. The expected HIP behaviour is the same failure, because the emitted cast would bind src to a cached load of fragment and the following store to fragment would not invalidate it.

Proposed change

The three generic sites take the same two-line change as CUDA:

-    std::string src = SSAGetID(PrintExpr(op->value), from_ty);
+    std::string src = name_supply_->FreshName("cast_src");
+    PrintSSAAssign(src, PrintExpr(op->value), from_ty);

codegen_cutedsl.cc:714 uses PrintExpr_ rather than PrintExpr, so the argument there follows its local naming. The FP4 branches would take the same replacement if maintainers want them covered now, though they cannot be exercised without gfx950.

Evidence limits

This environment is one RTX 4090 machine with no AMD GPU and no ROCm toolchain, and the built library registers no HIP backend at all: tvm.get_global_func("target.build.tilelang_hip_without_compile", allow_missing=True) returns nothing, so I could not lower the kernel above for HIP to confirm the emitted source. The CuTeDSL backend is likewise not built here. I have therefore not run any of this, and I am not claiming a reproduced numerical failure on either backend. What I did verify on hardware is the CUDA half: the base commit returns the stale value for that kernel and the patched tree returns the correct one, which is what pull request #4138 contains.

Two smaller notes from the same investigation, offered as negative results so they are not re-checked:

  • Vectorized add, subtract, division and exp do not show the stale behaviour on SM89. Their operands are materialized inside a per-statement scope, so the cache entry does not outlive the statement.
  • Broadcast of a scalar load reaches SSAGetID at codegen_cuda.cc:6142 and :6159 for the fp4 and fp8 splat paths, but those only run for those dtypes and I could not construct a stale case for them.

Disclosure

Prepared with Claude Code. The CUDA runs cited above were executed on the remote GPU machine described in #4131; the HIP and CuTeDSL observations are source reading only.

Activity

  1. wynneyin commented on Nov 15, 2026

    @wynneyin
    Author

    Correction on the CuTeDSL part of this report, after checking the code again and trying to lower for that backend.

    The proposed replacement does not work there. CodeGenC::PrintSSAAssign emits C syntax with a type prefix and a terminating semicolon, while the CuTeDSL codegen emits Python. There is no PrintSSAAssign override in codegen_cutedsl.cc or its header, so writing a fresh temporary the way the CUDA backends do would put an invalid statement into the generated source. codegen_cutedsl.cc:714 needs a materialization expressed in that backend's own syntax, not a copy of the CUDA two-liner.

    I also could not confirm the defect itself for CuTeDSL. Lowering is blocked in this environment by the Python side of the backend, before any of its codegen runs:

    AssertionError("CuTeDSL backend is not available. Please install tilelang-cutedsl package.
    CuTeDSL backend requires `nvidia-cutlass-dsl>=4.7.0,!=4.3.4`, but found version `4.4.1`.")
    

    So please treat the CuTeDSL claim in the report above as unassessed rather than as a finding: the SSAGetID call is there, but whether the cached name can go stale depends on how that backend materializes buffer loads, and I could not produce a lowered kernel to look at.

    The HIP part stands, and a patch is ready:

    Prepared with Claude Code.

  2. wynneyin commented on Nov 15, 2026

    @wynneyin
    Author

    Second correction, and it retracts part of the first one.

    The first correction said the two-line CUDA change does not work for CuTeDSL because PrintSSAAssign emits C syntax and there is no override in codegen_cutedsl.cc. That statement is wrong, and leaving it standing would be worse than the original overreach.

    CodeGenTileLangCuTeDSL derives from the Python codegen base, not from CodeGenC:

    src/cuda/codegen/codegen_cutedsl.h:25   class CodeGenTileLangCuTeDSL final : public CodeGenTileLangPY
    

    and that base supplies a Python-syntax implementation of the hook:

    src/backend/common/codegen/codegen_py.h:100        void PrintSSAAssign(const std::string &target, const std::string &src, DataType t) override;
    src/backend/common/codegen/codegen_py.cc:160-164   PrintIndent();
                                                       stream << target << " = " << RemoveOutermostParentheses(src) << "\n";
    

    So PrintSSAAssign(target, src, type) in this backend emits target = src, which is valid Python. I had only looked for an override inside the CuTeDSL files and did not follow the class to its base. That was the error in the first correction.

    What this changes: the same replacement that #4138 applies to CUDA,

    std::string src = name_supply_->FreshName("cast_src");
    PrintSSAAssign(src, PrintExpr_(op->value), from_ty);

    is valid for codegen_cutedsl.cc:714 as written, using that file's PrintExpr_ spelling. The static reason I gave for not touching the line does not hold.

    What this does not change: I still cannot lower for the CuTeDSL backend in this environment (nvidia-cutlass-dsl version gate, quoted in the first correction), so whether line 714 can actually go stale remains unassessed. The CuTeDSL half is still not a verified finding, in either direction.

    Pull request #4207 is unaffected and stays HIP-only.

    Prepared with Claude Code. This comment is source reading of the classes named above; nothing was lowered or run for CuTeDSL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions