Repository navigation
Conversation
…buffer A pipelined loop can stage a copy's source index through a shared buffer that the loop body updates for the next iteration (`offset = index_shared[0]; T.copy(A[offset : offset + 64], a_shared); ...; index_shared[0] = (k + 1) * 64`). Both pipeline schedulers issue the next iteration's copy before that store, so the copy reads the stale index and the kernel silently returns wrong data (tile-ai#3301, F077). Warp specialization: the producer evaluates the index before its back-pressure wait while the store stays in the consumer, and no barrier orders the two. Once the split is planned, fall back to the sequential loop if a producer-side read can observe a write other than an earlier producer-side statement. Accesses are compared by backing data and tile-op access regions, so the prefix stores the split already moves ahead of the read and writes to a copy's own destination keep warp specialization. Software pipeline: PipelinePlanning falls back to the sequential schedule when a copy, or a binding that feeds it, reads a buffer that a later statement writes. An explicit order/stage schedule that runs such a store in a different stage from its reader, or before it, is rejected with a diagnostic.
|
👋 Hi! Thank you for contributing to the TileLang project. Please remember to run We appreciate you taking this step! Our team will review your contribution, and we look forward to your awesome work! 🚀 |
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Problem
Part of #3301 (F077). A pipelined loop that stores a copy's source index into shared memory for the next iteration silently returns wrong data:
On
0176ba14, every iteration copiesA[0:64], so 192 of 256 elements are wrong withnum_stages=1or2. With 4-element tiles, which are too small for TMA and take the software pipeline, 12 of 16 are wrong. The serial loop is correct.Approach
Both pipeline schedulers issue the next iteration's copy before this iteration's store.
offsetbefore its back-pressure wait, while the store stays in the consumer after its release, and no barrier orders the two. After the split decides which prefix statements move to the producer, the loop now takes the existing sequential fallback if a producer-side read can observe a write that is not an earlier producer-side statement. Accesses are compared by backing data, and tile ops report their operands throughGetAccessRegions(). A store that the split already moves ahead of the read, as intest_producer_bind_kernel, and writes to a copy's own destination keep warp specialization.PipelinePlanningfalls back to the sequential schedule when a copy, or a binding that feeds its operands, reads a buffer that a later statement writes. For explicitorder/stageannotations, the same check rejects a schedule that runs the store in a different stage from its reader, or before it, and accepts one that keeps the store after the copy in the same stage.#2822 made the planner follow store -> binding -> copy chains within one iteration and left the warp-specialized scheduler out of scope; neither path checked the loop back-edge.
Falling back is the smallest correct change: with
num_stages >= 2the producer can run more than one iteration ahead, so moving a barrier cannot order the store before the next copy. The check works at buffer granularity, so a loop whose consumer writes a disjoint region of the copied global tensor also falls back.Tested on RTX 5070 Ti (sm_120), CUDA 13.3:
0176ba14with wrong values, or without a diagnostic for the unsafe explicit schedule, and passes with this change. It covers the TMA path at 1 and 2 stages, aT.fillupdate, a non-TMA copy, and both explicit schedules.test_producer_bind_kernelstill emitstl::tma_load, and a loop that updates the copied tile in place keeps TMA and warp specialization.T.Pipelinedgive the same results as0176ba14. Intest_tilelang_transform_producer_consumer_ws.py, the two wide shared-memory cases fail on both revisions because this GPU cannot allocate 128/144 KB of dynamic shared memory.