From f0c2758c78133c4d9ca4a87356df8c9a4d9c7afd Mon Sep 17 00:00:00 2001
From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Date: Fri, 16 Oct 2026 03:53:11 -0700
Subject: [PATCH 1/8] docs(qwen3.6): add data-format ablations and move SciCode
 diagnosis to ABLATIONS.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
---
 .../tutorials/Qwen3.6-35B-A3B/ABLATIONS.md    | 107 ++++++++++++++++++
 .../tutorials/Qwen3.6-35B-A3B/README.md       |  52 +--------
 examples/megatron_bridge/tutorials/README.md  |   2 +-
 3 files changed, 112 insertions(+), 49 deletions(-)
 create mode 100644 examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md

diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
new file mode 100644
index 00000000000..33789d9b34d
--- /dev/null
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
@@ -0,0 +1,107 @@
+# Ablations: Qwen3.6-35B-A3B W4A4 QAD
+
+Experiments behind the [main tutorial](README.md) that did not become the recommended recipe. All builds use the same W4A4 NVFP4 PTQ student, the same evaluation setup and the same `mean ± sem` pooling as the [main results](README.md#results); deltas are against BF16.
+
+- [Data format: pre-tokenized blend vs chat SFT data](#data-format-pre-tokenized-blend-vs-chat-sft-data)
+- [SciCode non-termination: diagnosis and decode-time fix](#scicode-non-termination-diagnosis-and-decode-time-fix)
+
+## Data format: pre-tokenized blend vs chat SFT data
+
+The main run trains on the **pre-tokenized blend**: conversations rendered with the chat template, packed into 32K windows, with the KD loss on **every** token, prompts included. Two alternatives feed the same Cascade-2 conversations as a chat `.jsonl` through `distill.py --sft --sft_hf_dataset`, with the loss on **assistant tokens only**:
+
+- **Chat SFT:** the Cascade-2 responses as they are.
+- **Self-distilled chat SFT:** each final assistant turn regenerated by the BF16 teacher (thinking on, `T=0.6`, `top_p=0.95`, responses that hit 32K dropped), so the student trains on the teacher's own response style and length.
+
+Everything else matches the main run: 32 nodes, EP=8, ~16M tokens per step, LR 1e-5 → 1e-6 over a 500-iteration schedule, top-4096 KD. Both alternatives were first compared at iteration 50, where the main run's SciCode runaway rate peaks; chat SFT, the stronger of the two, was then continued to 500.
+
+### Results
+
+| Benchmark (runs) | BF16 | Pre-tok it50 | Chat SFT it50 | Self-distilled it50 | Pre-tok it300 | Chat SFT it300 | **Pre-tok it500** | Chat SFT it500 |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| GPQA Diamond (1) | 84.7 | −0.8 | +0.5 | −0.3 | −0.4 | −0.4 | **−0.4** | pending |
+| MMMU-Pro (3 / 8)<sup>1</sup> | 74.6 | −0.8 | −0.5 | −0.9 | −1.0 | −0.7 | **−0.7** | −1.2 |
+| SciCode (8) | 39.9 | +0.2 | +0.2 | −1.7 | −1.2 | −1.0 | **+0.3** | −0.4 |
+| IFBench (8) | 60.0 | −1.3 | −1.9 | −0.0 | −1.9 | −2.6 | **−0.4** | −0.7 |
+| AA-LCR (8) | 69.1 | −0.9 | +1.0 | −1.4 | −0.6 | +1.1 | **+0.3** | +0.1 |
+| tau2-bench Telecom (3) | 94.2 | −4.2 | −1.4 | −2.9 | −1.2 | −2.2 | **−0.9** | −2.8 |
+| **Mean delta** | | −1.30 | **−0.34** | −1.19 | −1.05 | −0.98 | **−0.30** | −0.99<sup>2</sup> |
+| SciCode capped sub-steps | 0.7% | 4.3% | 3.0% | 3.1% | 3.1% | 5.9% | **3.6%** | 5.7% |
+| SciCode median tokens | 1,806 | 2,684 | 2,244 | 2,236 | 3,726 | 3,208 | **3,482** | 2,766 |
+
+><sup>1</sup> 3 runs at iterations 50 and 300, 8 at 500. <sup>2</sup> Over the five finished benchmarks; the pre-tokenized run averages −0.28 on the same five.
+>
+> *Capped* = hit the 131,072-token limit without a stop token, out of 2,704 SciCode sub-steps (338 × 8 runs). See [SciCode non-termination](#scicode-non-termination-diagnosis-and-decode-time-fix).
+
+**Chat SFT's early lead does not last.** At iteration 50 it is ahead on 5 of 6 benchmarks (mean −0.34 vs −1.30) and cuts the SciCode runaway rate from 4.3% to 3.0%. At 300 the two formats tie, and at 500 the pre-tokenized run is better (−0.28 vs −0.99 on the same five benchmarks, mostly tau2-bench and MMMU-Pro). Chat SFT keeps the median SciCode response shorter, but its capped rate *rises* with training, to ~5.8% against the pre-tokenized run's 3.1-3.6%. Masking the loss to assistant tokens is therefore not what drives the runaway.
+
+**Self-distillation does not fix the length drift either.** At iteration 50 its SciCode capped rate and median length match chat SFT's, though its training targets are the teacher's own responses; SciCode scores 1.7 points below BF16. It recovers IFBench fully (the only iteration-50 build to do so) but is no better on average and costs extra generation, so it was not continued.
+
+### Cost
+
+32 nodes × 4 GB200 = 128 GPUs for every run.
+
+| | Pre-tokenized | Chat SFT | Self-distilled chat SFT |
+| --- | --- | --- | --- |
+| Loss on | all tokens, prompts included | assistant tokens | assistant tokens |
+| Data prep | tokenize: 5.8 h on one 96-CPU node | jsonl build, Bridge scan, row pairing (CPU only) | + teacher generation: ~43 GPU-h per 50 iterations of data |
+| Tokens per micro-batch | 32,768 (packed window) | ~9K (two conversations) | ~10K (two conversations) |
+| Micro-batches per rank per step | 4 | 14 | 12 |
+| Steady step time | ~37 s | ~95-105 s | ~90 s |
+| QAD GPU-h, 50 iterations | ~100 | ~290 (Triton autotune cache still filling) | ~194 |
+| QAD GPU-h, 500 iterations | **~735** (5.7 h) | ~2,150 (16.8 h) | — |
+
+Chat SFT costs **~2.9× the GPU-hours** for the same tokens. Each micro-batch holds two unpadded conversations, ~9K tokens on average, instead of a full 32K window, so a step runs 3.5× as many micro-batches, and the fixed per-micro-batch work (teacher and student forward, full recompute, MoE dispatch) dominates. Pairs cannot grow past ~32K tokens: the 248,320-token vocabulary's fp32 logits are ~30 GiB per 32K tokens.
+
+### Running QAD on chat SFT data
+
+These runs needed two `distill.py` options that it does not expose, so they are not reproducible from this repository as is; they were not upstreamed because the format did not win. For anyone repeating the comparison:
+
+- **Pack rows in the batch.** With EP > 1, Megatron-Bridge pads every SFT row to `--seq_length`, which made a step ~8× the pre-tokenized cost here. In-batch packing (micro-batch size ≥ 2, unpadded THD) avoids that. Pair long rows with short ones so each pack stays under ~32K tokens.
+- **Disable the MoE auxiliary loss** (coefficient 0). It runs over padding tokens, and the linear-attention layers amplify the resulting gradient into the pad embedding row; after clipping, everything else gets near-zero updates.
+- **Drop rows that Bridge's assistant-mask builder rejects** (~1.2% of Cascade-2 rows) and rows containing a literal `<image>`, `<video>` or `<audio>` string (HTML in code answers): a single such row switches the whole dataset to the VLM collator, which then fails for a text tokenizer.
+- **Persist Triton autotuning.** The linear-attention kernels re-autotune for every new packed length. Set `TRITON_CACHE_AUTOTUNING=1` with a node-local `TRITON_CACHE_DIR` seeded from shared storage; 128 ranks writing into one Lustre cache directory stalled a run.
+
+## SciCode non-termination: diagnosis and decode-time fix
+
+QAD 500 raises SciCode's mean completion length by +90.9% over BF16 at an unchanged score ([Output length](README.md#output-length)). It is not verbosity. It is a **failure to terminate on a small fraction of sub-steps** — SciCode splits its 80 problems into 338 sub-steps, and `Subtask` scores the fraction of those whose code passes:
+
+- Sub-steps that hit the 131,072-token cap inside `<think>` go from **0.7% (20/2704, BF16) to 3.6% (96/2704, QAD 500)**. Almost all return **zero answer tokens** (93 of those 96) — the model writes a complete solution, says *"I think I've been going in circles"*, and writes it again. In the case we inspected, a 20-word window repeats **352 times** and 97.7% of the trace's 20-word windows are duplicates.
+- Those 3.6% of sub-steps burn **45.7% of all completion tokens**, so they dominate the mean: excluding them it is **+30.4%** rather than +90.9%.
+- The **median** also roughly doubles (+91.6%), so the whole distribution shifted right — this is not *only* a tail effect.
+- Capped rate peaks at **iteration 50** (4.3%) and settles at 3.1% / 3.6% by 300 / 500; it is not gradual drift.
+
+The obvious suspect — that `--logit_kl_top_k 4096` leaves the stop tokens outside the loss — **did not hold up**. Probing the BF16 teacher over one runaway trace: `</think>` does fall outside top-4096 at 35% of positions overall, but *in the looping region* the teacher gives `<|im_end|>` a median rank of **5** and `</think>` ~570, both well inside top-k. The teacher is signalling "stop here" at positions the loss did cover, and the student still does not stop. Neither assistant-only loss nor training on the teacher's own responses removes it ([data format](#data-format-pre-tokenized-blend-vs-chat-sft-data)). More likely, a teacher-forced loss never exercises free-running generation 10K+ tokens deep.
+
+### `presence_penalty: 1.5` removes it at decode time
+
+Qwen's own thinking-mode recommendation for this model removes nearly all of it, with no retraining. Every cell is SciCode **without → with** the penalty, 8 runs per side, capped counts out of 2,704 sub-steps:
+
+| Model | Capped sub-steps | SciCode (Subtask) | Mean tokens | Median tokens |
+| --- | --- | --- | --- | --- |
+| **BF16** (teacher) | 20 → **0** | 39.9 → 39.9 | 5,348 → 3,590 | 1,817 → 1,927 |
+| W4A16 NVFP4 PTQ | 37 → **0** | 38.5 → 39.4 | 6,572 → 3,838 | 1,917 → 1,978 |
+| **W4A4 NVFP4 PTQ** | 32 → **1** | 39.1 → 39.1 | 6,713 → 4,052 | 2,116 → 2,184 |
+| **↳ + QAD 500 iters** | 96 → **2** | 40.2 → 39.6 | 10,209 → 4,641 | 3,482 → 3,302 |
+
+Accuracy is neutral — the deltas span −0.59 to +0.89, run-to-run noise in both directions. The QAD row recovers 94 capped sub-steps, up to +3.5 pp of mechanical headroom, and the score does not move. Whether those sub-steps simply fail their tests anyway or the penalty costs a little elsewhere is not separable at this noise level — either way the win is cost, not accuracy: SciCode wall-clock generation drops 12-44%.
+
+Means are `avg_completion_tokens`, so the left-hand column matches the main README's output-length table; medians come from the per-response `output.jsonl`.
+
+**The penalty removes the tail; it does not make the model concise.** The median is flat or slightly *up* on three of four builds, so a typical response is unchanged; the mean falls 33-55% purely because the runaway generations are gone. QAD's median is still **+71%** over BF16 with the penalty on, against +91.6% without.
+
+**Scope it to this failure mode.** On GPQA Diamond, which never caps, the same setting leaves accuracy unchanged and costs **4.8× throughput**. The penalty has a *rate* cost that scales with concurrency, and SciCode pays it back in tokens it no longer emits:
+
+| Benchmark | `parallelism` | tok/s without → with | rate cost | tokens removed | net |
+| --- | --- | --- | --- | --- | --- |
+| SciCode | 8 | 1,475 → 1,192 | 1.24× | −55% | **faster** |
+| GPQA Diamond | 32 | 5,109 → 1,067 | 4.8× | −7% | **slower** |
+
+Measured on single-Slurm-segment runs on both sides, so neither figure includes requeue overhead. The likely cause is that the penalty is applied per sequence across the 248,320-token vocabulary on every decode step, so its cost grows with the number of concurrent sequences. We measured the correlation with `parallelism`, not the implementation, so treat the attribution as inference. The practical rule is the same either way: **the penalty pays for itself only where it removes enough tokens to cover its rate cost**, and that means workloads that actually cap.
+
+[Lotfi et al., *Quantized Reasoning Models Think They Need to Think Longer, but They Do Not*](https://arxiv.org/abs/2606.00206) reports a related effect — PTQ amplifies "overthinking", where the model reaches a correct answer and then talks itself out of it — and proposes a logit penalty on hesitation markers ("Wait", "But", "Alternatively"). We tested their penalty here: it helps, but less than `presence_penalty` and at a small accuracy cost, and their marker-density signature does not reproduce on our W4A4 PTQ. Their failure opens *new* reasoning branches; ours repeats a finished solution.
+
+### For the next QAD run
+
+- Track the **length-capped rate** as a first-class metric alongside accuracy: a benchmark score can stay flat while 3.6% of responses return nothing.
+- Consider **top-p instead of top-k** for the KD loss, so coverage adapts to the teacher's entropy rather than a fixed rank.
+- If memory allows, use **full-vocab KL**. At 32K on this 248,320-token vocabulary the dense fp32 logits are 30.31 GiB per tensor, which is why top-k was used here.
diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md
index 6638098494e..58cf97dacd6 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md
@@ -9,7 +9,7 @@ End-to-end W4A4 optimization of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qw
 5. **[Evaluation](#5-evaluation)** — benchmarking with NeMo Evaluator across MMMU-Pro, GPQA Diamond, SciCode, and more
 6. **[vLLM Inference Benchmarking](#6-vllm-inference-benchmarking)** — throughput comparison against BF16 on GB200
 
-It closes with [potential further improvements](#potential-further-improvements) — the levers this run did not explore, and what the evidence says each might buy.
+It closes with [potential further improvements](#potential-further-improvements) — the levers this run did not explore, and what the evidence says each might buy. Experiments that did not become the recommended recipe — chat-format SFT data with assistant-only loss, self-distilled data, and the SciCode non-termination diagnosis — are in [ABLATIONS.md](ABLATIONS.md).
 
 > [!NOTE]
 > This tutorial complements the [Nemotron-3-Nano-30B-A3B tutorial](../NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/README.md), which covers pruning + distillation + FP8 PTQ. Here the starting point is an unpruned model and the technique under test is **W4A4** — aggressive enough that PTQ alone leaves a measurable accuracy gap, which is exactly what QAD exists to close.
@@ -310,53 +310,9 @@ Accuracy is only half the serving cost — a model that scores the same while em
 | MMMU-Pro | 9,382 | +3.0% | +6.2% | −3.8% | 8 |
 | GPQA Diamond | 13,561 | +17.1% | +6.6% | −8.5% | 1 |
 
-**4-bit weights lengthen SciCode outputs by ~23-25% on their own** — the published W4A16 checkpoint does it too, so it is not something QAD or W4A4 introduced. **QAD then pushes SciCode to +90.9%**, for an unchanged score (40.2 vs 39.9), while pulling GPQA and MMMU-Pro back toward BF16. The throughput table is measured at fixed output length, so it does not capture this. The SciCode figure is a termination failure with a **decode-time mitigation** — see below.
+**4-bit weights lengthen SciCode outputs by ~23-25% on their own** — the published W4A16 checkpoint does it too, so it is not something QAD or W4A4 introduced. **QAD then pushes SciCode to +90.9%**, for an unchanged score (40.2 vs 39.9), while pulling GPQA and MMMU-Pro back toward BF16. The throughput table is measured at fixed output length, so it does not capture this.
 
-<details>
-<summary><b>What the SciCode number actually is, and how to fix it — worth reading before running QAD on another model</b></summary>
-
-It is not verbosity. It is a **failure to terminate on a small fraction of sub-steps** — SciCode splits its 80 problems into 338 sub-steps, and `Subtask` scores the fraction of those whose code passes:
-
-- Sub-steps that hit the 131,072-token cap inside `<think>` go from **0.7% (20/2704, BF16) to 3.6% (96/2704, QAD 500)**. Almost all return **zero answer tokens** (93 of those 96) — the model writes a complete solution, says *"I think I've been going in circles"*, and writes it again. In the case we inspected, a 20-word window repeats **352 times** and 97.7% of the trace's 20-word windows are duplicates.
-- Those 3.6% of sub-steps burn **45.7% of all completion tokens**, so they dominate the mean: excluding them it is **+30.4%** rather than +90.9%.
-- The **median** also roughly doubles (+91.6%), so the whole distribution shifted right — this is not *only* a tail effect.
-- Capped rate peaks at **iteration 50** (4.3%) and settles at 3.1% / 3.6% by 300 / 500; it is not gradual drift.
-
-The obvious suspect — that `--logit_kl_top_k 4096` leaves the stop tokens outside the loss — **did not hold up**. Probing the BF16 teacher over one runaway trace: `</think>` does fall outside top-4096 at 35% of positions overall, but *in the looping region* the teacher gives `<|im_end|>` a median rank of **5** and `</think>` ~570, both well inside top-k. The teacher is signalling "stop here" at positions the loss did cover, and the student still does not stop. More likely: the blend has few "the answer is written, now stop" positions in this style, and a teacher-forced loss never exercises free-running generation 10K+ tokens deep.
-
-**It is fixable at decode time.** Adding `presence_penalty: 1.5` (Qwen's own thinking-mode recommendation for this model) removes nearly all of it, with no retraining. Every cell is SciCode **without → with** the penalty, 8 runs per side. *Capped* = hit the 131,072-token limit with no stop token; almost all such sub-steps return nothing and score zero. Counts are pooled over all 8 runs, so the denominator is 338 × 8 = 2,704 sub-steps:
-
-| Model | Capped sub-steps | SciCode (Subtask) | Mean tokens | Median tokens |
-| --- | --- | --- | --- | --- |
-| **BF16** (teacher) | 20 → **0** | 39.9 → 39.9 | 5,348 → 3,590 | 1,817 → 1,927 |
-| W4A16 NVFP4 PTQ | 37 → **0** | 38.5 → 39.4 | 6,572 → 3,838 | 1,917 → 1,978 |
-| **W4A4 NVFP4 PTQ** | 32 → **1** | 39.1 → 39.1 | 6,713 → 4,052 | 2,116 → 2,184 |
-| **↳ + QAD 500 iters** | 96 → **2** | 40.2 → 39.6 | 10,209 → 4,641 | 3,482 → 3,302 |
-
-Accuracy is neutral — the deltas span −0.59 to +0.89, run-to-run noise in both directions. Note what that implies: the QAD row recovers 94 capped sub-steps, up to +3.5 pp of mechanical headroom, and the score does not move. Whether those sub-steps simply fail their tests anyway or the penalty costs a little elsewhere is not separable at this noise level — either way the win is cost, not accuracy: SciCode wall-clock generation drops 12-44%, which the token columns explain — see the rate-cost table below.
-
-Means are `avg_completion_tokens`, so the left-hand column matches the table above; medians come from the per-response `output.jsonl`, which is the only place they exist.
-
-**Read the two token columns together: the penalty removes the tail, it does not make the model concise.** The median is flat or slightly *up* on three of four builds, so a typical response is unchanged; the mean falls 33-55% purely because the runaway generations are gone. And the underlying shift survives — QAD's median is still **+71%** over BF16 with the penalty on, against +91.6% without. This suppresses the pathology, not the verbosity QAD introduced.
-
-The shipped `eval_configs/scicode.yaml` carries the line already, commented out, so it reproduces the results table above — uncomment `presence_penalty: 1.5` in its `params_to_add` block to get the right-hand column.
-
-**Scope it to this failure mode, though.** On GPQA Diamond, which never caps, the same setting leaves accuracy unchanged and costs **4.8× throughput**. Use it where runaway generation actually occurs; do not make it a global default.
-
-The two wall-clock results look contradictory and are not — the penalty has a *rate* cost that scales with concurrency, and SciCode pays it back in tokens it no longer emits:
-
-| Benchmark | `parallelism` | tok/s without → with | rate cost | tokens removed | net |
-| --- | --- | --- | --- | --- | --- |
-| SciCode | 8 | 1,475 → 1,192 | 1.24× | −55% | **faster** |
-| GPQA Diamond | 32 | 5,109 → 1,067 | 4.8× | −7% | **slower** |
-
-Measured on single-Slurm-segment runs on both sides, so neither figure includes requeue overhead. The likely cause is that the penalty is applied per sequence across the 248,320-token vocabulary on every decode step, so its cost grows with the number of concurrent sequences — at `parallelism: 8` it is a 24% tax, at 32 it dominates. We measured the correlation with `parallelism`, not the implementation, so treat the attribution as inference. The practical rule is the same either way: **the penalty pays for itself only where it removes enough tokens to cover its rate cost**, and that means workloads that actually cap.
-
-**For the next QAD run**, three things follow: track the **length-capped rate** as a first-class metric alongside accuracy (a benchmark score can stay flat while 3.6% of responses return nothing); consider **top-p instead of top-k** for the KD loss so coverage adapts to the teacher's entropy rather than a fixed rank; and if memory allows, **full-vocab KL** — at 32K on this 248,320-token vocabulary the dense fp32 logits are 30.31 GiB per tensor, which is why top-k was used here, but more GPU memory or a smaller model or shorter sequence may afford it.
-
-[Lotfi et al., *Quantized Reasoning Models Think They Need to Think Longer, but They Do Not*](https://arxiv.org/abs/2606.00206) reports a related effect — PTQ amplifies "overthinking", where the model reaches a correct answer and then talks itself out of it — and proposes a logit penalty on hesitation markers ("Wait", "But", "Alternatively"). We tested their penalty here: it helps, but less than `presence_penalty` and at a small accuracy cost, and their marker-density signature does not reproduce on our W4A4 PTQ. Their failure opens *new* reasoning branches; ours repeats a finished solution.
-
-</details>
+The SciCode figure is not verbosity: **3.6% of sub-steps never terminate** (0.7% for BF16), hit the 131,072-token cap with no answer, and burn 45.7% of all completion tokens. **`presence_penalty: 1.5` fixes it at decode time**, with no retraining: capped sub-steps drop from 96 to 2 of 2,704 with accuracy unchanged, and SciCode generation gets 12-44% faster. The shipped [scicode.yaml](eval_configs/scicode.yaml) carries the line commented out, so it reproduces the results table; uncomment it in `params_to_add` to apply the fix. Do not make it a global default: on GPQA Diamond, which never caps, it costs 4.8× throughput for no accuracy change. The diagnosis, per-build numbers and the rate-cost trade-off are in [ABLATIONS.md](ABLATIONS.md#scicode-non-termination-diagnosis-and-decode-time-fix).
 
 ---
 
@@ -409,6 +365,6 @@ This is a single 500-iteration run at 32K on a text-only blend. Each of those th
 
 **2. Longer sequence length (64K).** The model serves at 262K context, so 32K trains on a fraction of it; AA-LCR, the one long-context benchmark, showed no movement either way. At 32K the run already needs top-k KD plus full recompute, and memory is dominated by the resident student + teacher + fp32 gradient buffer rather than by sequence length — so 64K needs a different memory lever.
 
-**3. A better data blend.** The clearest signal in the study: **MMMU-Pro is multimodal, the blend is text-only, and MMMU-Pro is the one deficit QAD did not close.** Since QAD supports language-model distillation with text-only data, recovering a multimodal benchmark that way asks the method to do something it is not set up for. Agentic coverage is also thin (agent/SWE splits under 9%, and tau2-bench showed the transient 8× `agent_error` spike), and weights here are simply proportional to token count — contrast the [Nemotron-3-Nano blend](../NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/README.md#data-blend), which is deliberately designed against target benchmarks and [ablated](../NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/ABLATIONS.md#effect-of-data-blend-tool_calling).
+**3. A better data blend.** The clearest signal in the study: **MMMU-Pro is multimodal, the blend is text-only, and MMMU-Pro is the one deficit QAD did not close.** Since QAD supports language-model distillation with text-only data, recovering a multimodal benchmark that way asks the method to do something it is not set up for. Agentic coverage is also thin (agent/SWE splits under 9%, and tau2-bench showed the transient 8× `agent_error` spike), and weights here are simply proportional to token count — contrast the [Nemotron-3-Nano blend](../NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/README.md#data-blend), which is deliberately designed against target benchmarks and [ablated](../NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/ABLATIONS.md#effect-of-data-blend-tool_calling). Changing the data *format* alone does not help: chat-templated rows with the loss on assistant tokens only led at iteration 50 but trailed the pre-tokenized blend by 500 iterations, at ~2.9× the GPU-hours ([data-format ablation](ABLATIONS.md#data-format-pre-tokenized-blend-vs-chat-sft-data)). The lever is *what* the blend contains.
 
 If you run only one, run **(1)**: no engineering risk, the trajectory says the gain is still arriving, and it produces the evidence to judge whether (2) and (3) are worth their cost.
diff --git a/examples/megatron_bridge/tutorials/README.md b/examples/megatron_bridge/tutorials/README.md
index 454403083e8..0552cd075f6 100644
--- a/examples/megatron_bridge/tutorials/README.md
+++ b/examples/megatron_bridge/tutorials/README.md
@@ -9,4 +9,4 @@ Each one walks through a complete workflow using the scripts in [examples/megatr
 | --- | --- |
 | [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/README.md) | End-to-end optimization of the Nemotron-3-Nano-30B-A3B-BF16 (MoE + Mamba-Transformer hybrid) model: Minitron structured pruning (31.6B/A3.6B → 22B/A3.0B) → two-phase knowledge distillation (100B tokens, 8K then 32K seq length) → quantization → vLLM deployment. Includes data-blend preparation, evaluation setup, and detailed pruning / data-blend / long-context ablations. |
 | [NemotronH-Omni-W4A4-QAD](NemotronH-Omni-W4A4-QAD/README.md) | **W4A4** quantization of the language model of a NemotronH omni vision-language checkpoint (`NemotronH_Omni_Reasoning_V3`, Mamba-Transformer MoE): mixed NVFP4/FP8 PTQ → KD-only QAD on chat data → Hugging Face export → vLLM serving. |
-| [Qwen3.6-35B-A3B](Qwen3.6-35B-A3B/README.md) | End-to-end **W4A4 NVFP4** optimization of the Qwen3.6-35B-A3B (MoE + hybrid linear/full attention VLM) model: NVFP4 W4A4 post-training quantization → quantization-aware distillation (QAD) to recover the accuracy W4A4 costs → evaluation → vLLM deployment. |
+| [Qwen3.6-35B-A3B](Qwen3.6-35B-A3B/README.md) | End-to-end **W4A4 NVFP4** optimization of the Qwen3.6-35B-A3B (MoE + hybrid linear/full attention VLM) model: NVFP4 W4A4 post-training quantization → quantization-aware distillation (QAD) to recover the accuracy W4A4 costs → evaluation → vLLM deployment. Includes data-format (pre-tokenized vs chat SFT) and SciCode non-termination ablations. |

From 47047e5aa0720a1430316f1fb2f08c4d54cf663e Mon Sep 17 00:00:00 2001
From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Date: Fri, 16 Oct 2026 03:55:40 -0700
Subject: [PATCH 2/8] docs(qwen3.6): fill in the chat-SFT iter-500 GPQA result

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
---
 .../tutorials/Qwen3.6-35B-A3B/ABLATIONS.md                | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
index 33789d9b34d..d34b56e2120 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
@@ -18,21 +18,21 @@ Everything else matches the main run: 32 nodes, EP=8, ~16M tokens per step, LR 1
 
 | Benchmark (runs) | BF16 | Pre-tok it50 | Chat SFT it50 | Self-distilled it50 | Pre-tok it300 | Chat SFT it300 | **Pre-tok it500** | Chat SFT it500 |
 | --- | --- | --- | --- | --- | --- | --- | --- | --- |
-| GPQA Diamond (1) | 84.7 | −0.8 | +0.5 | −0.3 | −0.4 | −0.4 | **−0.4** | pending |
+| GPQA Diamond (1) | 84.7 | −0.8 | +0.5 | −0.3 | −0.4 | −0.4 | **−0.4** | 0.0 |
 | MMMU-Pro (3 / 8)<sup>1</sup> | 74.6 | −0.8 | −0.5 | −0.9 | −1.0 | −0.7 | **−0.7** | −1.2 |
 | SciCode (8) | 39.9 | +0.2 | +0.2 | −1.7 | −1.2 | −1.0 | **+0.3** | −0.4 |
 | IFBench (8) | 60.0 | −1.3 | −1.9 | −0.0 | −1.9 | −2.6 | **−0.4** | −0.7 |
 | AA-LCR (8) | 69.1 | −0.9 | +1.0 | −1.4 | −0.6 | +1.1 | **+0.3** | +0.1 |
 | tau2-bench Telecom (3) | 94.2 | −4.2 | −1.4 | −2.9 | −1.2 | −2.2 | **−0.9** | −2.8 |
-| **Mean delta** | | −1.30 | **−0.34** | −1.19 | −1.05 | −0.98 | **−0.30** | −0.99<sup>2</sup> |
+| **Mean delta** | | −1.30 | **−0.34** | −1.19 | −1.05 | −0.98 | **−0.30** | −0.83 |
 | SciCode capped sub-steps | 0.7% | 4.3% | 3.0% | 3.1% | 3.1% | 5.9% | **3.6%** | 5.7% |
 | SciCode median tokens | 1,806 | 2,684 | 2,244 | 2,236 | 3,726 | 3,208 | **3,482** | 2,766 |
 
-><sup>1</sup> 3 runs at iterations 50 and 300, 8 at 500. <sup>2</sup> Over the five finished benchmarks; the pre-tokenized run averages −0.28 on the same five.
+><sup>1</sup> 3 runs at iterations 50 and 300, 8 at 500.
 >
 > *Capped* = hit the 131,072-token limit without a stop token, out of 2,704 SciCode sub-steps (338 × 8 runs). See [SciCode non-termination](#scicode-non-termination-diagnosis-and-decode-time-fix).
 
-**Chat SFT's early lead does not last.** At iteration 50 it is ahead on 5 of 6 benchmarks (mean −0.34 vs −1.30) and cuts the SciCode runaway rate from 4.3% to 3.0%. At 300 the two formats tie, and at 500 the pre-tokenized run is better (−0.28 vs −0.99 on the same five benchmarks, mostly tau2-bench and MMMU-Pro). Chat SFT keeps the median SciCode response shorter, but its capped rate *rises* with training, to ~5.8% against the pre-tokenized run's 3.1-3.6%. Masking the loss to assistant tokens is therefore not what drives the runaway.
+**Chat SFT's early lead does not last.** At iteration 50 it is ahead on 5 of 6 benchmarks (mean −0.34 vs −1.30) and cuts the SciCode runaway rate from 4.3% to 3.0%. At 300 the two formats tie, and at 500 the pre-tokenized run is better (mean −0.30 vs −0.83, mostly tau2-bench and MMMU-Pro). Chat SFT keeps the median SciCode response shorter, but its capped rate *rises* with training, to ~5.8% against the pre-tokenized run's 3.1-3.6%. Masking the loss to assistant tokens is therefore not what drives the runaway.
 
 **Self-distillation does not fix the length drift either.** At iteration 50 its SciCode capped rate and median length match chat SFT's, though its training targets are the teacher's own responses; SciCode scores 1.7 points below BF16. It recovers IFBench fully (the only iteration-50 build to do so) but is no better on average and costs extra generation, so it was not continued.
 

From 5472cc0cccca8200319eadfa2cf37b8f2c5ec3e1 Mon Sep 17 00:00:00 2001
From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Date: Fri, 16 Oct 2026 04:54:47 -0700
Subject: [PATCH 3/8] docs(qwen3.6): correct the SciCode stop-token probe with
 the eval's real prompts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
---
 .../tutorials/Qwen3.6-35B-A3B/ABLATIONS.md           | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
index d34b56e2120..923efe1eb9f 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
@@ -70,7 +70,13 @@ QAD 500 raises SciCode's mean completion length by +90.9% over BF16 at an unchan
 - The **median** also roughly doubles (+91.6%), so the whole distribution shifted right — this is not *only* a tail effect.
 - Capped rate peaks at **iteration 50** (4.3%) and settles at 3.1% / 3.6% by 300 / 500; it is not gradual drift.
 
-The obvious suspect — that `--logit_kl_top_k 4096` leaves the stop tokens outside the loss — **did not hold up**. Probing the BF16 teacher over one runaway trace: `</think>` does fall outside top-4096 at 35% of positions overall, but *in the looping region* the teacher gives `<|im_end|>` a median rank of **5** and `</think>` ~570, both well inside top-k. The teacher is signalling "stop here" at positions the loss did cover, and the student still does not stop. Neither assistant-only loss nor training on the teacher's own responses removes it ([data format](#data-format-pre-tokenized-blend-vs-chat-sft-data)). More likely, a teacher-forced loss never exercises free-running generation 10K+ tokens deep.
+The obvious suspect — that `--logit_kl_top_k 4096` leaves the stop tokens outside the loss — **did not hold up**. We scored 16 BF16 sub-steps that stop normally and 16 QAD-500 runaway sub-steps, with the eval's exact prompts, under the BF16 teacher and the PTQ, QAD-50 and QAD-500 students, over the full vocabulary:
+
+- **The top-4096 tokens hold essentially all of the teacher's probability**: on average 1e-5 falls outside. `</think>` is often ranked outside the top-4096 mid-reasoning, but only with negligible probability, so the top-k loss sees nearly the whole distribution.
+- **QAD does not suppress stopping.** Fed the teacher's text, every student puts nearly all probability on `</think>` exactly where the teacher stops.
+- **Inside the loops the teacher keeps going too.** Its stop probability there is about e<sup>-22</sup>, and it predicts the next repeated token with probability ~0.98.
+
+So once a loop has started, a teacher-forced loss has nothing to correct: teacher and student agree. The difference is that the student enters loops in its own free-running generation, which KD on fixed text never exercises. Neither assistant-only loss nor training on the teacher's own responses removes it ([data format](#data-format-pre-tokenized-blend-vs-chat-sft-data)). The students were scored with their quantized weights dequantized to BF16 and BF16 activations; the first and last findings depend only on the teacher.
 
 ### `presence_penalty: 1.5` removes it at decode time
 
@@ -103,5 +109,5 @@ Measured on single-Slurm-segment runs on both sides, so neither figure includes
 ### For the next QAD run
 
 - Track the **length-capped rate** as a first-class metric alongside accuracy: a benchmark score can stay flat while 3.6% of responses return nothing.
-- Consider **top-p instead of top-k** for the KD loss, so coverage adapts to the teacher's entropy rather than a fixed rank.
-- If memory allows, use **full-vocab KL**. At 32K on this 248,320-token vocabulary the dense fp32 logits are 30.31 GiB per tensor, which is why top-k was used here.
+- Changing the KD loss's vocabulary coverage (top-p, full-vocab KL) is unlikely to help: the top-4096 tokens already hold nearly all of the teacher's probability.
+- The training-side lever this points to is **on-policy distillation**: KD on the student's own generations, so the loss sees the states where the student starts to loop.

From 499d48a51193db31010f5757adce3f87f6d9815a Mon Sep 17 00:00:00 2001
From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Date: Fri, 16 Oct 2026 13:33:53 -0700
Subject: [PATCH 4/8] docs(qwen3.6): add the ghost-token KD control to
 ABLATIONS.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
---
 examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
index 923efe1eb9f..f5ce1d9f9a0 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
@@ -109,5 +109,5 @@ Measured on single-Slurm-segment runs on both sides, so neither figure includes
 ### For the next QAD run
 
 - Track the **length-capped rate** as a first-class metric alongside accuracy: a benchmark score can stay flat while 3.6% of responses return nothing.
-- Changing the KD loss's vocabulary coverage (top-p, full-vocab KL) is unlikely to help: the top-4096 tokens already hold nearly all of the teacher's probability.
+- Changing the KD loss's vocabulary coverage (top-p, full-vocab KL) is unlikely to help: the top-4096 tokens already hold nearly all of the teacher's probability. We confirmed it with a 50-iteration rerun that adds a "ghost" token carrying the teacher's mass outside the top-4096: SciCode 39.35 ± 0.56 vs 40.13 ± 0.60, length-capped 5.0% vs 4.3%, all within noise.
 - The training-side lever this points to is **on-policy distillation**: KD on the student's own generations, so the loss sees the states where the student starts to loop.

From 8e5147dea334d172ca8786d04425030069cb9cab Mon Sep 17 00:00:00 2001
From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Date: Fri, 16 Oct 2026 13:36:23 -0700
Subject: [PATCH 5/8] docs(qwen3.6): correct the chat-SFT iter-50 benchmark
 count

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
---
 examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
index f5ce1d9f9a0..06f5fa0bc1a 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
@@ -32,7 +32,7 @@ Everything else matches the main run: 32 nodes, EP=8, ~16M tokens per step, LR 1
 >
 > *Capped* = hit the 131,072-token limit without a stop token, out of 2,704 SciCode sub-steps (338 × 8 runs). See [SciCode non-termination](#scicode-non-termination-diagnosis-and-decode-time-fix).
 
-**Chat SFT's early lead does not last.** At iteration 50 it is ahead on 5 of 6 benchmarks (mean −0.34 vs −1.30) and cuts the SciCode runaway rate from 4.3% to 3.0%. At 300 the two formats tie, and at 500 the pre-tokenized run is better (mean −0.30 vs −0.83, mostly tau2-bench and MMMU-Pro). Chat SFT keeps the median SciCode response shorter, but its capped rate *rises* with training, to ~5.8% against the pre-tokenized run's 3.1-3.6%. Masking the loss to assistant tokens is therefore not what drives the runaway.
+**Chat SFT's early lead does not last.** At iteration 50 it leads on 4 of 6 benchmarks, ties SciCode and trails only on IFBench (mean −0.34 vs −1.30); it also cuts the SciCode runaway rate from 4.3% to 3.0%. At 300 the two formats tie, and at 500 the pre-tokenized run is better (mean −0.30 vs −0.83, mostly tau2-bench and MMMU-Pro). Chat SFT keeps the median SciCode response shorter, but its capped rate *rises* with training, to ~5.8% against the pre-tokenized run's 3.1-3.6%. Masking the loss to assistant tokens is therefore not what drives the runaway.
 
 **Self-distillation does not fix the length drift either.** At iteration 50 its SciCode capped rate and median length match chat SFT's, though its training targets are the teacher's own responses; SciCode scores 1.7 points below BF16. It recovers IFBench fully (the only iteration-50 build to do so) but is no better on average and costs extra generation, so it was not continued.
 

From cd6e8330011d1551a72638784a09fa5df7c90e80 Mon Sep 17 00:00:00 2001
From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Date: Fri, 16 Oct 2026 23:47:11 -0700
Subject: [PATCH 6/8] docs(qwen3.6): add the on-policy distillation result to
 ABLATIONS.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
---
 examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
index 06f5fa0bc1a..ed1fb1faa6e 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
@@ -110,4 +110,4 @@ Measured on single-Slurm-segment runs on both sides, so neither figure includes
 
 - Track the **length-capped rate** as a first-class metric alongside accuracy: a benchmark score can stay flat while 3.6% of responses return nothing.
 - Changing the KD loss's vocabulary coverage (top-p, full-vocab KL) is unlikely to help: the top-4096 tokens already hold nearly all of the teacher's probability. We confirmed it with a 50-iteration rerun that adds a "ghost" token carrying the teacher's mass outside the top-4096: SciCode 39.35 ± 0.56 vs 40.13 ± 0.60, length-capped 5.0% vs 4.3%, all within noise.
-- The training-side lever this points to is **on-policy distillation**: KD on the student's own generations, so the loss sees the states where the student starts to loop.
+- **On-policy distillation did not fix it either**, in one offline round. From the iteration-50 weights, 50 more KD iterations on 189K responses generated by the student itself (looping ones kept) give SciCode 39.24 ± 0.47 with 5.2% length-capped, against 38.50 ± 0.74 and 5.5% for the same 50 iterations on the teacher's responses to the same prompts, and 5.0% at the start. The test is weak: on these training prompts the student loops barely more than the teacher (4.1% vs 3.4% capped), so its rollouts hold few loop entries; a retest would need SciCode-like prompts. `presence_penalty` remains the fix.

From 424aa8479a2538e3d758e9d48c17167c591ff13c Mon Sep 17 00:00:00 2001
From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Date: Sat, 17 Oct 2026 00:01:44 -0700
Subject: [PATCH 7/8] docs(qwen3.6): note packed-THD teacher noise in the
 chat-SFT ablation

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
---
 examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
index ed1fb1faa6e..156290fee4e 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
@@ -36,6 +36,8 @@ Everything else matches the main run: 32 nodes, EP=8, ~16M tokens per step, LR 1
 
 **Self-distillation does not fix the length drift either.** At iteration 50 its SciCode capped rate and median length match chat SFT's, though its training targets are the teacher's own responses; SciCode scores 1.7 points below BF16. It recovers IFBench fully (the only iteration-50 build to do so) but is no better on average and costs extra generation, so it was not continued.
 
+**Caveat: the chat-SFT teachers were noisier.** Both chat-SFT variants pack two conversations into one THD micro-batch (`cu_seqlens`), and on that path the Gated DeltaNet kernels are not reproducible: the BF16 teacher's logits for the same packed input differ run to run by 0.001–0.007 nats/token of KL on average, against a KD loss of ~0.03. The same conversations unpacked are bit-exact, and the pre-tokenized run feeds unpacked 32K windows, so only the chat-SFT arms trained on these noisier targets. Treat their gap to the pre-tokenized run as provisional until the kernel is fixed.
+
 ### Cost
 
 32 nodes × 4 GB200 = 128 GPUs for every run.

From aac152a3f4be3b7dfd2908b2798c3d89c8ccb707 Mon Sep 17 00:00:00 2001
From: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Date: Sat, 17 Oct 2026 02:01:54 -0700
Subject: [PATCH 8/8] docs(qwen3.6): address review comments on ablation
 numbers

Fix the BF16 SciCode median (an aborted run was pooled), name SciCode as the
second iteration-500 contributor, call iteration 300 nearly tied, and scope the
presence_penalty accuracy claim to run-to-run noise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
---
 .../megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md    | 4 ++--
 examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
index 156290fee4e..06b67e85651 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/ABLATIONS.md
@@ -26,13 +26,13 @@ Everything else matches the main run: 32 nodes, EP=8, ~16M tokens per step, LR 1
 | tau2-bench Telecom (3) | 94.2 | −4.2 | −1.4 | −2.9 | −1.2 | −2.2 | **−0.9** | −2.8 |
 | **Mean delta** | | −1.30 | **−0.34** | −1.19 | −1.05 | −0.98 | **−0.30** | −0.83 |
 | SciCode capped sub-steps | 0.7% | 4.3% | 3.0% | 3.1% | 3.1% | 5.9% | **3.6%** | 5.7% |
-| SciCode median tokens | 1,806 | 2,684 | 2,244 | 2,236 | 3,726 | 3,208 | **3,482** | 2,766 |
+| SciCode median tokens | 1,817 | 2,684 | 2,244 | 2,236 | 3,726 | 3,208 | **3,482** | 2,766 |
 
 ><sup>1</sup> 3 runs at iterations 50 and 300, 8 at 500.
 >
 > *Capped* = hit the 131,072-token limit without a stop token, out of 2,704 SciCode sub-steps (338 × 8 runs). See [SciCode non-termination](#scicode-non-termination-diagnosis-and-decode-time-fix).
 
-**Chat SFT's early lead does not last.** At iteration 50 it leads on 4 of 6 benchmarks, ties SciCode and trails only on IFBench (mean −0.34 vs −1.30); it also cuts the SciCode runaway rate from 4.3% to 3.0%. At 300 the two formats tie, and at 500 the pre-tokenized run is better (mean −0.30 vs −0.83, mostly tau2-bench and MMMU-Pro). Chat SFT keeps the median SciCode response shorter, but its capped rate *rises* with training, to ~5.8% against the pre-tokenized run's 3.1-3.6%. Masking the loss to assistant tokens is therefore not what drives the runaway.
+**Chat SFT's early lead does not last.** At iteration 50 it leads on 4 of 6 benchmarks, ties SciCode and trails only on IFBench (mean −0.34 vs −1.30); it also cuts the SciCode runaway rate from 4.3% to 3.0%. At 300 the two formats are nearly tied (mean −1.05 vs −0.98), and at 500 the pre-tokenized run is better (mean −0.30 vs −0.83, mostly tau2-bench and SciCode). Chat SFT keeps the median SciCode response shorter, but its capped rate *rises* with training, to ~5.8% against the pre-tokenized run's 3.1-3.6%. Masking the loss to assistant tokens is therefore not what drives the runaway.
 
 **Self-distillation does not fix the length drift either.** At iteration 50 its SciCode capped rate and median length match chat SFT's, though its training targets are the teacher's own responses; SciCode scores 1.7 points below BF16. It recovers IFBench fully (the only iteration-50 build to do so) but is no better on average and costs extra generation, so it was not continued.
 
diff --git a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md
index 58cf97dacd6..fe41ec294b3 100644
--- a/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md
+++ b/examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B/README.md
@@ -312,7 +312,7 @@ Accuracy is only half the serving cost — a model that scores the same while em
 
 **4-bit weights lengthen SciCode outputs by ~23-25% on their own** — the published W4A16 checkpoint does it too, so it is not something QAD or W4A4 introduced. **QAD then pushes SciCode to +90.9%**, for an unchanged score (40.2 vs 39.9), while pulling GPQA and MMMU-Pro back toward BF16. The throughput table is measured at fixed output length, so it does not capture this.
 
-The SciCode figure is not verbosity: **3.6% of sub-steps never terminate** (0.7% for BF16), hit the 131,072-token cap with no answer, and burn 45.7% of all completion tokens. **`presence_penalty: 1.5` fixes it at decode time**, with no retraining: capped sub-steps drop from 96 to 2 of 2,704 with accuracy unchanged, and SciCode generation gets 12-44% faster. The shipped [scicode.yaml](eval_configs/scicode.yaml) carries the line commented out, so it reproduces the results table; uncomment it in `params_to_add` to apply the fix. Do not make it a global default: on GPQA Diamond, which never caps, it costs 4.8× throughput for no accuracy change. The diagnosis, per-build numbers and the rate-cost trade-off are in [ABLATIONS.md](ABLATIONS.md#scicode-non-termination-diagnosis-and-decode-time-fix).
+The SciCode figure is not verbosity: **3.6% of sub-steps never terminate** (0.7% for BF16), hit the 131,072-token cap with no answer, and burn 45.7% of all completion tokens. **`presence_penalty: 1.5` fixes it at decode time**, with no retraining: capped sub-steps drop from 96 to 2 of 2,704, with accuracy flat to within run-to-run noise, and SciCode generation gets 12-44% faster. The shipped [scicode.yaml](eval_configs/scicode.yaml) carries the line commented out, so it reproduces the results table; uncomment it in `params_to_add` to apply the fix. Do not make it a global default: on GPQA Diamond, which never caps, it costs 4.8× throughput for no accuracy change. The diagnosis, per-build numbers and the rate-cost trade-off are in [ABLATIONS.md](ABLATIONS.md#scicode-non-termination-diagnosis-and-decode-time-fix).
 
 ---