🧠 MLOps
Serving LLMs (vLLM)
vLLM: high-throughput LLM serving, OpenAI API, quantization.
skill_view("serving-llms-vllm")
📖 Guide
Load with skill_view("serving-llms-vllm"). Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
🔧 How To
Skill sections: vLLM - High-Performance LLM Serving, When to use, Quick start, Query with OpenAI SDK, Common workflows, Workflow 1: Production API deployment.
⚠️ Pitfalls
Focuses on Serving LLMs (vLLM). Read the skill before acting - it encodes the exact commands.
Related Skills
More skills in MLOps