Python
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
1036Curated repositories
46,902.8kTotal GitHub stars
Open sourceBuilt by the community
inference repositories
Hand-picked tools to improve your workflow.
Python · Trending
SGLang is a high-performance serving framework for large language models and multimodal models.
Go · Trending
An open, programmable decision layer for models and compute.
Python · Trending
SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.
Go · Trending
llm-d Router: The intelligent entry point for inference requests