AI & Machine Learning

vLLM

High-throughput LLM inference.

Website Docs GitHub

vLLM server configs, tensor parallelism and quantization settings for fast LLM serving.