Repository navigation
Description
Summary
In a warpgroup-guarded region, a T.Parallel loop that reads one shared buffer (s_s, the column scale) and writes another (c_s, a swizzled staging tile) gets a named barrier, __sync_thread_partial(3, 128), inside every iteration. The loop reads s_s and stores c_s at addresses that never overlap; the two buffers are separate views of the dynamic shared memory pool. The barrier sits between the register cast and the shared store, so every element pair waits on all 128 threads. pass_configs={"tl.disable_thread_storage_sync": True} removes it and keeps the explicit barriers. It appears when the accumulator has a WGMMA layout, where the s_s index changes with the loop variable; with a loop-invariant s_s read it does not.
Environment
| Item | Value |
|---|---|
| TileLang | 0.1.12 |
| PyTorch | 2.13.0+cu132 |
| CUDA | 13.2 |
| Python | 3.12.13 |
| GPU | NVIDIA H200, driver 595.71.05 (compile only) |
Reproduction
python repro.py (writes repro_barrier_wgmma.cu)
"""ThreadSync inserts a named barrier in every iteration of a T.Parallel loop that reads one
shared buffer and writes another, inside a warpgroup-guarded region. Prints the loop."""
import re
import tilelang
import tilelang.language as T
M, N = 64, 128
@tilelang.jit(out_idx=[-1], pass_configs={"tl.disable_warp_specialized": True})
def kernel():
@T.prim_func
def main(A: T.Tensor((M, 64), "float16"), B: T.Tensor((N, 64), "float16"), S: T.Tensor((N,), "float32"), C: T.Tensor((M, N), "bfloat16")):
with T.Kernel(1, threads=256):
s_s = T.alloc_shared((N,), "float32")
a_s = T.alloc_shared((M, 64), "float16")
b_s = T.alloc_shared((N, 64), "float16")
c_s = T.alloc_shared((M, N), "bfloat16")
T.annotate_layout({c_s: tilelang.layout.make_swizzled_layout(c_s)})
tx = T.get_thread_binding()
if tx >= 128:
acc = T.alloc_fragment((M, N), "float32")
T.copy(A, a_s)
T.copy(B, b_s)
T.wgmma_gemm(a_s, b_s, acc, transpose_B=True, policy=T.GemmWarpPolicy.FullRow, clear_accum=True)
T.wait_wgmma(0)
for j in T.Parallel(N):
s_s[j] = S[j]
T.sync_threads(barrier_id=8, arrive_count=128)
for i, j in T.Parallel(M, N):
c_s[i, j] = T.cast(acc[i, j] * s_s[j], "bfloat16")
T.fence_proxy_async()
T.sync_threads(barrier_id=8, arrive_count=128)
T.tma_copy(c_s, C[0:M, 0:N])
T.tma_store_wait(0)
return main
src = kernel().get_kernel_source()
open("repro_barrier_wgmma.cu", "w").write(src)
print("barriers:", re.findall(r"__syncthreads\(\)|__sync_thread_partial\([^)]*\)", src))
i = src.find("c_s_local_cast")
print(src[max(0, i - 600): i + 500])Observed
Barriers in the generated kernel (grep -n sync_thread_partial):
46: tl::__sync_thread_partial(3, 128);
60: tl::__sync_thread_partial(3, 128);
62: tl::__sync_thread_partial(8, 128);
74: tl::__sync_thread_partial(3, 128);
78: tl::__sync_thread_partial(8, 128);
The epilogue loop, 32 iterations, each with a barrier:
for (int i_2 = 0; i_2 < 32; ++i_2) {
*(float2*)(s_s_local_cast_1 + 0) = *(float2*)(((float*)s_s) + (((i_2 >> 1) * 8) + ((((int)threadIdx.x) & 3) * 2)));
uint1 __1;
float2 __2;
float2 v_ = *(float2*)(acc + (i_2 * 2));
float2 v__1 = *(float2*)(s_s_local_cast_1 + 0);
__2.x = (v_.x*v__1.x);
__2.y = (v_.y*v__1.y);
(reinterpret_cast<__nv_bfloat162*>(&__1))[0] = __float22bfloat162_rn(((float2*)(&__2))[0]);
*(uint1*)(c_s_local_cast + 0) = __1;
tl::__sync_thread_partial(3, 128);
*(uint1*)(((bfloat16_t*)c_s) + ((((((((((i_2 >> 4) * 4096) + ((((int)threadIdx.x) >> 5) * 1024)) + ((i_2 & 1) * 512)) + (((((int)threadIdx.x) & 31) >> 2) * 64)) + (((((((int)threadIdx.x) & 31) >> 4) + ((i_2 & 15) >> 3)) & 1) * 32)) + (((((((int)threadIdx.x) & 15) >> 3) + ((i_2 & 7) >> 2)) & 1) * 16)) + (((((((int)threadIdx.x) & 7) >> 2) + ((i_2 & 3) >> 1)) & 1) * 8)) + ((((int)threadIdx.x) & 3) * 2)) - 4096)) = *(uint1*)(c_s_local_cast + 0);
}Expected
No barrier inside the loop: within one thread the loop's shared reads (s_s) and writes (c_s) never touch the same bytes, and the explicit T.sync_threads(barrier_id=8, ...) before and after the loop orders it against the other threads.
Activity
lcy-seso commentedon Nov 15, 2026
Placement of these inserted barriers also depends on where the source file lives. The same kernel file, compiled from three checkouts at different paths, gave three different programs (deterministic for each path). They differed only in which epilogue branches carry
__sync_thread_partial(3, 128)/__sync_thread_partial(4, 128), for example:So the generated CUDA for one source is not reproducible across install locations.