Separating GGUF Weights, Tokenizer, and Config in vLLM
Distinguish quantized weights, tokenizer, and Hugging Face configuration when starting GGUF models with vLLM.
#vLLM
Articles tagged “vLLM”
Migrating vLLM Structured Outputs from Legacy Guided Fields
Move from guided_json to structured_outputs or response_format while keeping JSON Schema and semantic validation boundaries explicit.
Separating vLLM Token Limits and Reasoning Settings
Tune local inference by separating max-model-len, request limits, batching, GPU memory, and reasoning parsers.