Usation Labs

Popular repositories Loading

1. modelsight modelsight Public

   Benchmark models on modal via API.

   Python 1

2. aiperf aiperf Public

   Forked from ai-dynamo/aiperf

   AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.

   Python

3. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python

4. dynamo dynamo Public

   Forked from ai-dynamo/dynamo

   A Datacenter Scale Distributed Inference Serving Framework

   Rust

5. router router Public

   Forked from vllm-project/router

   A high-performance and light-weight router for vLLM large scale deployment

   Rust