SGLang is a high-performance serving framework for large language models and multimodal models.
Python 38.9k 10.5k
SGLang's TorchTPU Plugin
Python 82 50
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
Python 1.3k 394
High Performance Serving Runtime For Interactive Super Intelligence System
Python 1.8k 743
A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.
Python 5.6k 987
JAX backend for SGL
Python 379 163
SGLang kernel library for NPU
SGLang Kernel Wheel Index
FlashMLA: Efficient Multi-head Latent Attention Kernels