Separating GGUF Weights, Tokenizer, and Config in vLLM
Distinguish quantized weights, tokenizer, and Hugging Face configuration when starting GGUF models with vLLM.
#quantization
Articles tagged “quantization”
SignalcraftSignalcraftArticles tagged “quantization”
Distinguish quantized weights, tokenizer, and Hugging Face configuration when starting GGUF models with vLLM.