Repository navigation
[9/11] Trim the torch_onnx example lane - #2853
Draft
shengliangxu wants to merge 2 commits into
Draft
shengliangxu wants to merge 2 commits into
shengliangxu wants to merge 2 commits into
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
shengliangxu
added this pull request to stack #2856
October 17, 2026 01:41
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
This was referenced Oct 17, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## shengliangx/ci-8-vllm-dflash-tests #2853 +/- ##
===================================================================
Coverage 78.82% 78.82%
===================================================================
Files 658 658
Lines 73649 73649
===================================================================
Hits 58051 58051
Misses 15598 15598
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Collaborator
Author
|
/ok to test 595189c |
shengliangxu
removed this pull request from stack #2856
October 17, 2026 06:58
load_calibration_data builds its DataLoaders over lists of tensors that are
already on the GPU, so the four workers load nothing. They only add cost: the
script forced the "spawn" start method (CUDA cannot be used in forked workers),
so every loader pass spawned four processes that re-imported torch, timm and
modelopt and received the dataset over CUDA IPC. That is about 14 s per pass
and about 300 s per torch_onnx CI job.
Use num_workers=0 in both loaders and drop the module-level
mp.set_start_method("spawn", force=True). The latter also stops importing the
script from changing the start method of the importing process (two tests
import it inside the pytest process). Nothing else in the script uses
multiprocessing. The --evaluate path keeps the CPU-only DataLoader workers of
examples/onnx_ptq/evaluation.py; they now run with the default start method,
exactly as in the onnx_ptq example that calls the same function.
Testing: py_compile and ruff 0.15.20 (check and format) on the script, and read
the file for other users of mp and of the start method (none). Not run:
tests/examples/torch_onnx needs a GPU, TensorRT and model downloads, so GPU CI
validation is still needed.
Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
Two cuts to the torch_onnx example lane. swinv2_tiny runs all five formats on top of swin_tiny, which already covers every format through the same exporter and TensorRT path; swinv2 only adds its cosine attention and post-norm blocks. On PRs (GITHUB_REF under refs/heads/pull-request/) skip swinv2 int8, mxfp8 and auto and keep fp8 and nvfp4, which still exercise those blocks. Scheduled and local runs keep every combination, and test ids are unchanged. swinv2_tiny[auto] alone averages 121 s over 304 CI runs (the 22nd slowest example test); the three together cost about 110 s of every PR run. build_trt_engine let trtexec run its default benchmark after the build (200 ms warm-up, then at least 3 s and 10 iterations), and the output is discarded on success. Run a single iteration instead (--warmUp=0 --duration=0 --iterations=1): the engine still executes once, but about 3 s of timing per engine goes away, roughly 40 s per lane run. --trt_build users see no difference besides it returning sooner. Testing: ruff 0.15.20 (check and format), py_compile and pytest --collect-only on CPU (24 tests, ids unchanged). With the example command stubbed, GITHUB_REF=refs/heads/pull-request/123 skips exactly swinv2 int8, mxfp8 and auto, and any other ref skips none. The two in-process mapping tests that import the script pass on CPU, and build_trt_engine passes the three flags to a fake trtexec. Not run: the lane itself (needs a GPU, TensorRT and model downloads) and the flags against a real trtexec, so GPU CI validation is still needed. Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
shengliangxu
force-pushed
the
shengliangx/ci-9-torch-onnx-lane
branch
from
October 17, 2026 07:02
595189c to
cae95d3
Compare
shengliangxu
added this pull request to stack #2859
October 17, 2026 07:02
Collaborator
Author
|
/ok to test cae95d3 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Type of change: CI (test runtime); the only non-test file is
examples/torch_onnx/torch_quant_to_onnx.pyThree cuts to the torch_onnx example lane:
load_calibration_databuilds its DataLoaders over lists of tensors that are already on the GPU, so the four workers load nothing, and since the script forced thespawnstart method (CUDA cannot be used in forked workers), every loader pass spawned four processes that re-imported torch, timm and modelopt and received the dataset over CUDA IPC: about 14 s per pass and about 300 s per torch_onnx CI job. Both loaders usenum_workers=0, and the module-levelmp.set_start_method("spawn", force=True)is dropped, which also stops importing the script from changing the start method of the importing process (two tests import it inside the pytest process). Nothing else in the script uses multiprocessing. The--evaluatepath keeps the CPU-only DataLoader workers ofexamples/onnx_ptq/evaluation.py, now under the default start method, exactly as in theonnx_ptqexample that calls the same function.swinv2_tinyruns all five formats on top ofswin_tiny, which already covers every format through the same exporter and TensorRT path; swinv2 only adds its cosine attention and post-norm blocks. On PRs (GITHUB_REFunderrefs/heads/pull-request/) fp8 and nvfp4 still run and exercise those blocks; scheduled and local runs keep every combination, and test ids are unchanged.swinv2_tiny[auto]alone averages 121 s over 304 CI runs; the three skipped cases cost about 110 s of every PR run.build_trt_engineruns trtexec for one iteration. It let trtexec run its default benchmark after the build (200 ms warm-up, then at least 3 s and 10 iterations) and discarded the output.--warmUp=0 --duration=0 --iterations=1still executes the engine once and removes about 3 s per engine, roughly 40 s per lane run.--trt_buildusers see no difference besides it returning sooner.Time improvement (measured in CI)
torch_onnx lane; before = #2846, #2848 and #2850 (875-880 s; #2849 ran on a slower runner: 1,060 s), after = #2853, #2854 and #2855 (452-463 s).
This lane repeats to within 1% (875-880 s before, 452-463 s after), so it is the cleanest measurement of the series. For example
swin_tinywith auto quantization goes from 66 s to 24 s andvit_tinywith auto from 60 s to 17 s.Method: every PR of the series has its own CI run, so "before" is the run of the PR(s) below that lack the change and "after" the runs that carry it. Times come from the job logs (
--durations, or the gap between consecutive result lines). Where runner speed matters, the tests that no PR of the series touches scale each run, and the raw and the scaled medians are both shown. One test varies by 15-30% from run to run on its own, so smaller differences are not evidence.Usage
No API change. The skip is keyed on
GITHUB_REF, so a local run keeps every combination.Testing
py_compile, ruff 0.15.20 andpytest --collect-onlyon CPU (24 tests, ids unchanged). With the example command stubbed,GITHUB_REF=refs/heads/pull-request/123skips exactly swinv2 int8, mxfp8 and auto, and any other ref skips none; the two in-process mapping tests that import the script pass on CPU, andbuild_trt_enginepasses the three flags to a fake trtexec. Not run: the lane itself (it needs a GPU, TensorRT and model downloads) and the flags against a real trtexec, so this PR's torch_onnx job is the check. Pre-commit passes. #2685 (open) also edits this example; the changes touch different lines.Before your PR is "Ready for review"
Make sure you read and follow Contributor guidelines and your commits are signed (
git commit -s -S).CONTRIBUTING.md: no new dependencyAdditional Information
Slice 9 of 11 of the GPU-CI speed-up work, split out of the reference draft #2730 (not for merge; it holds the whole series with its measurements). Merge in order; each PR stands on its own and passes CI without the later ones.