Skip to content

[9/11] Trim the torch_onnx example lane - #2853

Draft
shengliangxu wants to merge 2 commits into
shengliangx/ci-8-vllm-dflash-testsfrom
shengliangx/ci-9-torch-onnx-lane
Draft

shengliangxu wants to merge 2 commits into
shengliangx/ci-8-vllm-dflash-testsfrom
shengliangx/ci-9-torch-onnx-lane

Conversation

@shengliangxu

@shengliangxu shengliangxu commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

Type of change: CI (test runtime); the only non-test file is examples/torch_onnx/torch_quant_to_onnx.py

Three cuts to the torch_onnx example lane:

  • No DataLoader workers in the calibration loaders. load_calibration_data builds its DataLoaders over lists of tensors that are already on the GPU, so the four workers load nothing, and since the script forced the spawn start method (CUDA cannot be used in forked workers), every loader pass spawned four processes that re-imported torch, timm and modelopt and received the dataset over CUDA IPC: about 14 s per pass and about 300 s per torch_onnx CI job. Both loaders use num_workers=0, and the module-level mp.set_start_method("spawn", force=True) is dropped, which also stops importing the script from changing the start method of the importing process (two tests import it inside the pytest process). Nothing else in the script uses multiprocessing. The --evaluate path keeps the CPU-only DataLoader workers of examples/onnx_ptq/evaluation.py, now under the default start method, exactly as in the onnx_ptq example that calls the same function.
  • swinv2 int8, mxfp8 and auto are skipped on pull requests. swinv2_tiny runs all five formats on top of swin_tiny, which already covers every format through the same exporter and TensorRT path; swinv2 only adds its cosine attention and post-norm blocks. On PRs (GITHUB_REF under refs/heads/pull-request/) fp8 and nvfp4 still run and exercise those blocks; scheduled and local runs keep every combination, and test ids are unchanged. swinv2_tiny[auto] alone averages 121 s over 304 CI runs; the three skipped cases cost about 110 s of every PR run.
  • build_trt_engine runs trtexec for one iteration. It let trtexec run its default benchmark after the build (200 ms warm-up, then at least 3 s and 10 iterations) and discarded the output. --warmUp=0 --duration=0 --iterations=1 still executes the engine once and removes about 3 s per engine, roughly 40 s per lane run. --trt_build users see no difference besides it returning sooner.

Time improvement (measured in CI)

torch_onnx lane; before = #2846, #2848 and #2850 (875-880 s; #2849 ran on a slower runner: 1,060 s), after = #2853, #2854 and #2855 (452-463 s).

before after change
lane pytest 877 s 458 s -419 s (-48%)
of which: the three swinv2 cases skipped on PRs 143 s 0 -143 s
of which: the cases that still run (no DataLoader workers, trtexec runs once) -277 s
whole job 18.6 min 12.2 min -6.4 min

This lane repeats to within 1% (875-880 s before, 452-463 s after), so it is the cleanest measurement of the series. For example swin_tiny with auto quantization goes from 66 s to 24 s and vit_tiny with auto from 60 s to 17 s.

Method: every PR of the series has its own CI run, so "before" is the run of the PR(s) below that lack the change and "after" the runs that carry it. Times come from the job logs (--durations, or the gap between consecutive result lines). Where runner speed matters, the tests that no PR of the series touches scale each run, and the raw and the scaled medians are both shown. One test varies by 15-30% from run to run on its own, so smaller differences are not evidence.

Usage

No API change. The skip is keyed on GITHUB_REF, so a local run keeps every combination.

Testing

py_compile, ruff 0.15.20 and pytest --collect-only on CPU (24 tests, ids unchanged). With the example command stubbed, GITHUB_REF=refs/heads/pull-request/123 skips exactly swinv2 int8, mxfp8 and auto, and any other ref skips none; the two in-process mapping tests that import the script pass on CPU, and build_trt_engine passes the three flags to a fake trtexec. Not run: the lane itself (it needs a GPU, TensorRT and model downloads) and the flags against a real trtexec, so this PR's torch_onnx job is the check. Pre-commit passes. #2685 (open) also edits this example; the changes touch different lines.

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

  • Is this change backward compatible?: N/A (CI and tests)
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: no new dependency
  • Did you write any new necessary tests?: N/A (existing tests; the PR-only skip keeps test ids)
  • Did you update Changelog?: N/A (CI and tests)
  • Did you get Claude approval on this PR?: N/A (draft)

Additional Information

Slice 9 of 11 of the GPU-CI speed-up work, split out of the reference draft #2730 (not for merge; it holds the whole series with its measurements). Merge in order; each PR stands on its own and passes CI without the later ones.

  1. [1/11] Run the unit tests in parallel with pytest-xdist #2845 [1/11] Run the unit tests in parallel with pytest-xdist
  2. [2/11] Use the sysmon coverage core in CI #2846 [2/11] Use the sysmon coverage core in CI
  3. [3/11] Run release-marked tests on nightly and on-demand GPU runs #2847 [3/11] Run release-marked tests on nightly and on-demand GPU runs
  4. [4/11] Give gpu_megatron's slow cold starts more time #2848 [4/11] Give gpu_megatron's slow cold starts more time
  5. [5/11] Make the ONNX tests cheaper: CPU-only ones to the unit lane, small models #2849 [5/11] Make the ONNX tests cheaper: CPU-only ones to the unit lane, small models
  6. [6/11] Use local tokenizers and calibration text in GPU tests, shrink puzzletron data #2850 [6/11] Use local tokenizers and calibration text in GPU tests, shrink puzzletron data
  7. [7/11] Trim the llm_eval, llm_qat, EAGLE and megatron_bridge example tests #2851 [7/11] Trim the llm_eval, llm_qat, EAGLE and megatron_bridge example tests
  8. [8/11] Trim the DFlash regression test #2852 [8/11] Trim the DFlash regression test
  9. [9/11] Trim the torch_onnx example lane #2853 [9/11] Trim the torch_onnx example lane (this PR)
  10. [10/11] Make the distributed worker pool fail fast when a worker dies #2854 [10/11] Make the distributed worker pool fail fast when a worker dies
  11. [11/11] Share one Megatron worker pool across the gpu_megatron modules #2855 [11/11] Share one Megatron worker pool across the gpu_megatron modules

@copy-pr-bot

copy-pr-bot Bot commented Oct 17, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@shengliangxu
shengliangxu added this pull request to stack #2856 October 17, 2026 01:41
@coderabbitai

coderabbitai Bot commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.82%. Comparing base (307f189) to head (cae95d3).

Additional details and impacted files
@@                         Coverage Diff                         @@
##           shengliangx/ci-8-vllm-dflash-tests    #2853   +/-   ##
===================================================================
  Coverage                               78.82%   78.82%           
===================================================================
  Files                                     658      658           
  Lines                                   73649    73649           
===================================================================
  Hits                                    58051    58051           
  Misses                                  15598    15598           
Flag Coverage Δ
examples-torch_onnx 21.44% <ø> (-0.02%) ⬇️
examples-torch_trt 15.26% <ø> (ø)
unit 60.40% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@shengliangxu

Copy link
Copy Markdown
Collaborator Author

/ok to test 595189c

@shengliangxu shengliangxu changed the title [9/11] Trim the torch_onnx example lane [9/12] Trim the torch_onnx example lane Oct 17, 2026
@shengliangxu
shengliangxu removed this pull request from stack #2856 October 17, 2026 06:58
load_calibration_data builds its DataLoaders over lists of tensors that are
already on the GPU, so the four workers load nothing. They only add cost: the
script forced the "spawn" start method (CUDA cannot be used in forked workers),
so every loader pass spawned four processes that re-imported torch, timm and
modelopt and received the dataset over CUDA IPC. That is about 14 s per pass
and about 300 s per torch_onnx CI job.

Use num_workers=0 in both loaders and drop the module-level
mp.set_start_method("spawn", force=True). The latter also stops importing the
script from changing the start method of the importing process (two tests
import it inside the pytest process). Nothing else in the script uses
multiprocessing. The --evaluate path keeps the CPU-only DataLoader workers of
examples/onnx_ptq/evaluation.py; they now run with the default start method,
exactly as in the onnx_ptq example that calls the same function.

Testing: py_compile and ruff 0.15.20 (check and format) on the script, and read
the file for other users of mp and of the start method (none). Not run:
tests/examples/torch_onnx needs a GPU, TensorRT and model downloads, so GPU CI
validation is still needed.

Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
Two cuts to the torch_onnx example lane.

swinv2_tiny runs all five formats on top of swin_tiny, which already covers
every format through the same exporter and TensorRT path; swinv2 only adds its
cosine attention and post-norm blocks. On PRs (GITHUB_REF under
refs/heads/pull-request/) skip swinv2 int8, mxfp8 and auto and keep fp8 and
nvfp4, which still exercise those blocks. Scheduled and local runs keep every
combination, and test ids are unchanged. swinv2_tiny[auto] alone averages
121 s over 304 CI runs (the 22nd slowest example test); the three together
cost about 110 s of every PR run.

build_trt_engine let trtexec run its default benchmark after the build (200 ms
warm-up, then at least 3 s and 10 iterations), and the output is discarded on
success. Run a single iteration instead (--warmUp=0 --duration=0
--iterations=1): the engine still executes once, but about 3 s of timing per
engine goes away, roughly 40 s per lane run. --trt_build users see no
difference besides it returning sooner.

Testing: ruff 0.15.20 (check and format), py_compile and pytest --collect-only
on CPU (24 tests, ids unchanged). With the example command stubbed,
GITHUB_REF=refs/heads/pull-request/123 skips exactly swinv2 int8, mxfp8 and
auto, and any other ref skips none. The two in-process mapping tests that
import the script pass on CPU, and build_trt_engine passes the three flags to
a fake trtexec. Not run: the lane itself (needs a GPU, TensorRT and model
downloads) and the flags against a real trtexec, so GPU CI validation is still
needed.

Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
@shengliangxu
shengliangxu force-pushed the shengliangx/ci-9-torch-onnx-lane branch from 595189c to cae95d3 Compare October 17, 2026 07:02
@shengliangxu
shengliangxu added this pull request to stack #2859 October 17, 2026 07:02
@shengliangxu shengliangxu changed the title [9/12] Trim the torch_onnx example lane [9/11] Trim the torch_onnx example lane Oct 17, 2026
@shengliangxu

Copy link
Copy Markdown
Collaborator Author

/ok to test cae95d3

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant