GitHub - vipshop/cache-dit: A PyTorch-native inference engine with cache, parallelism, quantization and cpu offload for 100+ DiTs.

⚡️🎉A PyTorch-native Inference Engine with Cache,
Parallelism, Quantization and CPU Offload for DiTs

🤗Why Cache-DiT❓❓Cache-DiT is built on top of the 🤗Diffusers library and now supports nearly ALL DiTs from Diffusers. It provides hybrid cache acceleration (DBCache, TaylorSeer, SCM, etc.) and comprehensive parallelism optimizations, including Context Parallelism, Tensor Parallelism, hybrid 2D or 3D parallelism, and dedicated extra parallelism support for Text Encoder, VAE, and ControlNet.

Cache-DiT is compatible with compilation, offload, and quantization, fully integrates with SGLang Diffusion, vLLM-Omni, TensorRT-LLM, xDiT, and runs natively on NVIDIA GPUs, Huawei Ascend NPUs and AMD GPUs. Cache-DiT is fast, easy to use, and flexible for various DiTs (online docs at 📘cache-dit.io).

⚡️9x speedup by Cache-DiT with Cache, Context Parallelism and Compilation

🚀Quick Start: Cache, Parallelism and Quantization

pip3 install -U cache-dit # PyPI, stable release.
pip3 install git+https://github.com/vipshop/cache-dit.git # latest.

>>> import cache_dit
>>> from diffusers import DiffusionPipeline
>>> pipe = DiffusionPipeline.from_pretrained(...).to("cuda")
>>> cache_dit.enable_cache(pipe) # Cache Acceleration with One-line code.
>>> from cache_dit import DBCacheConfig, ParallelismConfig
>>> cache_dit.enable_cache( # Or, Hybrid Cache Acceleration + Parallelism.
...   pipe, cache_config=DBCacheConfig(), # w/ default
...   parallelism_config=ParallelismConfig(ulysses_size=2))
>>> from cache_dit import DBCacheConfig, ParallelismConfig, QuantizeConfig
>>> cache_dit.enable_cache( # Or, Hybrid Cache + Parallelism + Quantization.
...   pipe, cache_config=DBCacheConfig(), # w/ default
...   parallelism_config=ParallelismConfig(ulysses_size=2),
...   quantize_config=QuantizeConfig(quant_type=...))
>>> output = pipe(...) # Then, just call the pipe as normal.

🚀Quick Start: SVDQuant (W4A4) PTQ/DQ workflow

# Required: CUDA >=13.0, PyTorch >=2.11 (a CUDA 13 build), GLIBC 2.38.
pip3 install -U cache-dit --extra-index-url https://vipshop.github.io/cache-dit/whl/cu13/ 
CACHE_DIT_BUILD_SVDQUANT=1 uv pip install -e ".[quantization]" # latest.

>>> from cache_dit import QuantizeConfig
>>> pipe = DiffusionPipeline.from_pretrained(...).to("cuda")
>>> # ✅ PTQ: "...{dtype}_r{rank}", required `calibrate_fn`.
>>> def calibrate_fn(**_: object) -> None:
...  for prompt in calibration_prompts: _ = pipe(...)
>>> pipe.transformer = cache_dit.quantize(
...  pipe.transformer, quant_config=QuantizeConfig(
...  quant_type="svdq_nvfp4_r32", calibrate_fn=calibrate_fn,
...  serialize_to="path"))
>>> pipe.transformer = cache_dit.load(pipe.transformer, "path")

>>> # ✅ DQ: "...{dtype}_r{rank}_dq", NO calibration need.
>>> pipe.transformer = cache_dit.quantize(
...   pipe.transformer, quant_config=QuantizeConfig(
...   quant_type="svdq_{int4|nvfp4}_r{32|64|128|256|...}_dq",
...   svdq_kwargs={"smooth_strategy": "few_shot"})) 
>>> output = pipe(...) # Then, just call the pipe as normal.

🚀Quick Start: Bucket-style Layerwise CPU Offload

>>> import cache_dit
>>> cache_dit.layerwise_offload(
...   pipe, # nn.Module: pipe, transformer, text_encoder, etc.
...   onload_device="cuda",
...   offload_device="cpu",
...   async_transfer=True,
...   transfer_buckets=4,
...   persistent_buckets=64,
...   persistent_bins=8,
...   prefetch_limit=True,
...   max_copy_streams=4,
...   max_inflight_prefetch_bytes="8gib")
>>> output = pipe(...) # Then, just call the pipe as normal.

🚪Supported DiT Models

Cache-DiT supports 60+ DiT pipeline families (150+ Variants) from 🤗Diffusers, covering the vast majority of DiT-based pipelines. For full support matrix and detailed usage, please refer to our documentation at 📘cache-dit.io.

Modality	Pipeline Series	Model Sources	Variants	C/P/Q/OF
Video	SANA-WM (I2V) 🔥	`🤗diffusers@0.42.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	FLUX.3 Klein 🔥	`🤗diffusers@0.42.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	Wan-Animate-2 🔥	`🤗diffusers@0.42.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	Bria-Fibo 2.0 🔥	`🤗diffusers@0.42.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Video	Kandinsky 6.0 (T2VA) 🔥	`🤗diffusers@0.42.0`, `🤗HF`	1	✔️/✔️/✔️/✖️
Audio	Stable Audio 3 (T2A) 🔥	`🤗diffusers@0.42.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Audio	LongCat-AudioDiT (TTS) 🔥	`🤗diffusers@0.42.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	Qwen-Image-2.1 🔥	`🤗diffusers@0.41.0`, `🤗HF`	3	✔️/✔️/✔️/✔️
Video	LTX2.5 (DFR/T2V/I2V) 🔥	`🤗diffusers@0.41.0`, `🤗HF`	6	✔️/✔️/✔️/✔️
Video	Echo (JoyAI-Echo) 🔥	`🤗diffusers@0.41.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	MiniMax-H3 (T2VA) 🔥	`🤗diffusers@0.40.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	MiniMax-H3 (FL2VA) 🔥	`🤗diffusers@0.40.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	MiniMax-H3 (Ref2VA) 🔥	`🤗diffusers@0.40.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Joy-Image	`🤗diffusers@0.39.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Krea-2	`🤗diffusers@0.39.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Ideogram 4 🔥	`🤗diffusers@0.39.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Anima	`🤗diffusers@0.39.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	Any-Flow	`🤗diffusers@0.39.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	Any-Flow-FAR	`🤗diffusers@0.39.0`, `🤗HF`	1	✖️/✖️/✔️/✔️
Video	Cosmos3 (Omni) 🔥	`🤗diffusers@0.39.0`, `🤗HF`	2	✔️/✖️/✔️/✔️
Video	Motif-Video (T2V/I2V) 🔥	`🤗diffusers@0.39.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	ERNIE-Image 🔥	`🤗diffusers@0.38.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	NucleusMoE-Image 🔥	`🤗diffusers@0.38.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	FLUX.2 Klein 🔥	`🤗diffusers@0.37.0`, `🤗HF`	3	✔️/✔️/✔️/✔️
Image	GLM-Image 🔥	`🤗diffusers@0.37.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	LongCat-Image 🔥	`🤗diffusers@0.37.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Video	LTX2	`🤗diffusers@0.37.0`, `🤗HF`	6	✔️/✔️/✔️/✔️
Video	Helios 🔥	`🤗diffusers@0.37.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	FLUX.2 🔥	`🤗diffusers@0.36.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Z-Image 🔥	`🤗diffusers@0.36.0`, `🤗HF`	6	✔️/✔️/✔️/✔️
Image	Bria-Fibo	`🤗diffusers@0.36.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	Ovis-Image	`🤗diffusers@0.36.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Hunyuan-Image 🔥	`🤗diffusers@0.36.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	Bria	`🤗diffusers@0.36.0`, `🤗HF`	1	✔️/✖️/✔️/✔️
Image	PRX / PRXPixel 🔥	`🤗diffusers@0.36.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Video	HunyuanVideo 1.5 🔥	`🤗diffusers@0.36.0`, `🤗HF`	2	✔️/✖️/✔️/✔️
Video	ChronoEdit	`🤗diffusers@0.36.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	Kandinsky5 (T2V/I2V)	`🤗diffusers@0.36.0`, `🤗HF`	4	✔️/✔️/✔️/✔️
Other	LucyEdit	`🤗diffusers@0.36.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Qwen-Image 🔥	`🤗diffusers@0.35.0`, `🤗HF`	9+	✔️/✔️/✔️/✔️
Video	Wan 2.2 (T2V/I2V) 🔥	`🤗diffusers@0.35.0`, `🤗HF`	3	✔️/✔️/✔️/✔️
Video	SkyReelsV2	`🤗diffusers@0.35.0`, `🤗HF`	5	✔️/✔️/✔️/✔️
Image	HiDream	`🤗diffusers@0.34.0`, `🤗HF`	1	✔️/✖️/✔️/✔️
Image	Chroma	`🤗diffusers@0.34.0`, `🤗HF`	3	✔️/✔️/✔️/✔️
Image	VisualCloze	`🤗diffusers@0.34.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Video	Cosmos	`🤗diffusers@0.34.0`, `🤗HF`	7+	✔️/✖️/✔️/✔️
Video	Wan-VACE 🔥	`🤗diffusers@0.34.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	CogView4	`🤗diffusers@0.33.0`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	OmniGen	`🤗diffusers@0.33.0`, `🤗HF`	1	✔️/✖️/✔️/✔️
Image	Lumina 2	`🤗diffusers@0.33.0`, `🤗HF`	2	✔️/✖️/✔️/✔️
Video	Wan 2.1 (T2V/I2V) 🔥	`🤗diffusers@0.33.0`, `🤗HF`	3	✔️/✔️/✔️/✔️
Video	EasyAnimate	`🤗diffusers@0.33.0`, `🤗HF`	3	✔️/✖️/✔️/✔️
Video	ConsisID	`🤗diffusers@0.33.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Sana (image)	`🤗diffusers@0.32.0`, `🤗HF`	4	✔️/✖️/✔️/✔️
Video	HunyuanVideo	`🤗diffusers@0.32.0`, `🤗HF`	4	✔️/✔️/✔️/✔️
Video	Mochi	`🤗diffusers@0.32.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	Allegro	`🤗diffusers@0.32.0`, `🤗HF`	1	✔️/✖️/✔️/✔️
Video	LTX	`🤗diffusers@0.32.0`, `🤗HF`	5	✔️/✔️/✔️/✔️
Image	CogView3Plus	`🤗diffusers@0.31.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	FLUX	`🤗diffusers@0.30.0`, `🤗HF`	10+	✔️/✔️/✔️/✔️
Image	HunyuanDiT-PAG	`🤗diffusers@0.30.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	AuraFlow	`🤗diffusers@0.30.0`, `🤗HF`	1	✔️/✖️/✔️/✔️
Image	Lumina	`🤗diffusers@0.30.0`, `🤗HF`	2	✔️/✖️/✔️/✔️
Video	CogVideoX	`🤗diffusers@0.30.0`, `🤗HF`	4	✔️/✔️/✔️/✔️
Audio	StableAudio	`🤗diffusers@0.30.0`, `🤗HF`	1	✔️/✖️/✔️/✔️
Image	SD3	`🤗diffusers@0.29.0`, `🤗HF`	3	✔️/✖️/✔️/✔️
Image	HunyuanDiT	`🤗diffusers@0.28.1`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	PixArt-Sigma	`🤗diffusers@0.28.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	Amused	`🤗diffusers@0.25.0`, `🤗HF`	1	✔️/✖️/✔️/✔️
Image	PixArt-Alpha	`🤗diffusers@0.22.0`, `🤗HF`	1	✔️/✔️/✔️/✔️
Other	Shap-E	`🤗diffusers@0.18.0`, `🤗HF`	1	✔️/✖️/✔️/✔️
Image	DiT	`🤗diffusers@0.12.0`, `🤗HF`	1	✔️/✔️/✔️/✔️

C: Hybrid Cache (DBCache + Calibrator: TaylorSeer/DMD/FoCa/SCM); P: Parallelism (Ulysses/Ring/USP
TP/TE-P/VAE-P/2D-P/3D-P); Q: Quantization; OF: Bucket-style Layerwise Offload; @: Minimum version.

🚀Built-in Pipelines Beyond 🤗Diffusers

Modality	Pipeline Series	Model Sources	Variants	C/P/Q/OF
Video	SANA-Video 2.0 (T2V/TI2V) 🔥	`🎯cache-dit@main`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	LingBot-Video Dense 1.3B 🔥	`🎯cache-dit@main`, `🤗HF`	1	✔️/✔️/✔️/✔️
Image	MOSS-Image 🔥	`🎯cache-dit@main`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	Ming-Image 0.1 🔥	`🎯cache-dit@main`, `🤗HF`	2	✔️/✔️/✔️/✔️
Image	HiDream-O1-Image 🔥	`🎯cache-dit@main`, `🤗HF`	1	✔️/✔️/✔️/✔️
Video	Fast-H3 (`fastvideo@main`) 🔥	`🎯cache-dit@main`, `🤗HF`	2	✔️/✔️/✔️/✔️
Video	CausVid-Wan 🔥	`🎯cache-dit@main`, `🤗HF`	1	✖️/✔️/✔️/✔️
Video	Wan2.2-S2V (S2V) 🔥	`🎯cache-dit@main`, `🤗HF`	1	✔️/✔️/✔️/✔️

Cache-DiT also provides built-in pipelines for models that are not currently included in 🤗Diffusers

🤖Agentic Workflows

Cache-DiT provides a model-integration SKILL to help users integrate new DiT pipelines into Cache-DiT, including Cache, CP, TP, TE-P, VAE-P and carefully designed test cases. Users can use it with Coding Agents, e.g, GitHub Copilot, Claude Code, Open Code.

gh skill install vipshop/cache-dit cache-dit-model-integration --allow-hidden-dirs

Model integration: The SKILL guides the Coding Agent through: search and download the model weights (e.g. from 🤗Hugging Face or ModelScope), match the model against the corresponding 🤗Diffusers pipeline and transformer code, and then implement support for it following the SKILL's instructions.

/cache-dit-model-integration "Please download and support MiniMax-H3 in Cache-DiT."

Performance: It can also run the performance matrix for models that Cache-DiT already supports. For examples:

/cache-dit-model-integration "Run the perf matrix for MiniMax-H3 with 1/2/4/8 GPUs."

🌐Community Integration

Special thanks to vipshop's Computer Vision AI Team for supporting testing and deployment of this project. We learned and reused codes from: Diffusers, SGLang, vLLM-Omni, Nunchaku, xDiT and TaylorSeer.

©️Citations

@misc{cache-dit@2025,
  title={Cache-DiT: A PyTorch-native Inference Engine with Cache, Parallelism, Quantization and CPU Offload for DiTs.},
  url={https://github.com/vipshop/cache-dit.git},
  note={Open-source software available at https://github.com/vipshop/cache-dit.git},
  author={DefTruth, vipshop.com, etc.},
  year={2025}
}

Name		Name	Last commit message	Last commit date
.github		.github
assets		assets
bench		bench
csrc		csrc
docs		docs
examples		examples
src/cache_dit		src/cache_dit
tests		tests
tools		tools
.clang-format		.clang-format
.editorconfig		.editorconfig
.gitignore		.gitignore
.gitmodules		.gitmodules
.pre-commit-config.yaml		.pre-commit-config.yaml
.readthedocs.yml		.readthedocs.yml
.style.yapf		.style.yapf
LICENSE		LICENSE
MANIFEST.in		MANIFEST.in
README.md		README.md
collect_env.py		collect_env.py
mkdocs.yml		mkdocs.yml
pyproject.toml		pyproject.toml
setup.cfg		setup.cfg
setup.py		setup.py

Navigation Menu

Repository navigation

About

Repository files navigation

⚡️🎉A PyTorch-native Inference Engine with Cache,
Parallelism, Quantization and CPU Offload for DiTs

🚀Quick Start: Cache, Parallelism and Quantization

🚀Quick Start: SVDQuant (W4A4) PTQ/DQ workflow

🚀Quick Start: Bucket-style Layerwise CPU Offload

🚪Supported DiT Models

🚀Built-in Pipelines Beyond 🤗Diffusers

🤖Agentic Workflows

🌐Community Integration

©️Acknowledgements

©️Citations

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

⚡️🎉A PyTorch-native Inference Engine with Cache, Parallelism, Quantization and CPU Offload for DiTs

🚀Quick Start: Cache, Parallelism and Quantization

🚀Quick Start: SVDQuant (W4A4) PTQ/DQ workflow

🚀Quick Start: Bucket-style Layerwise CPU Offload

🚪Supported DiT Models

🚀Built-in Pipelines Beyond 🤗Diffusers

🤖Agentic Workflows

🌐Community Integration

©️Acknowledgements

©️Citations

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

⚡️🎉A PyTorch-native Inference Engine with Cache,
Parallelism, Quantization and CPU Offload for DiTs