Install
Easy, fast, and cheap LLM serving for everyone vLLM is a fast and easy-to-use library for LLM inference and serving. Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evolved into a community-driven project with contributions from both academia and industry. vLLM is fast with: Efficient management
- 37articles · 30d
- 4+ day agolatest article
- Aug 15, 2026earliest in window
- 0%with images
- 177avg words
- Science & Technology 37
- Software Dev. 37
- Computers & Electronics 36
- Jobs & Education 1
Please confirm you are human
This browser or connection looks automated. Press and continuously hold the control for 3 seconds to enable Google-hosted web results and, when separately allowed, AI-assisted answers.
A successful check enables 100 search requests. Interactive access does not authorize scraping, systematic collection, or reuse of search output.
News
dspark
4+ day, 5+ hour ago (47+ words) vLLM CPU DeepSeek-V4 DSpark (speculative decoding) — deferred, not implemented. DSparkDeepseekV4ForCausalLM is only instantiated when the speculative- decoding registry resolves it by name for a DSpark speculative config; a plain (non-spec) DeepSeek-V4 CPU model never constructs this class. This stub exists solely…...
DeepSeek R1 with 128xH100??? Vime
6+ day, 15+ hour ago (580+ words) This is an example of doing DeepSeek R1 RL training using 128xH100 GPUs. We will use bf16 for training, and an fp8 format with 128x128 blockwise quantization for inference. The maximum response length is 32k, and dynamic sampling will be used to filter data during training....
GLM-4.7-Flash with 8??H100??? Vime
6+ day, 16+ hour ago (364+ words) The environment setup, data, and checkpoint conversion are the same as for the Qwen3-4B model. You can refer to Example: Qwen3-4B Model, replacing mentions of Qwen3-4B with GLM-4.7-Flash. To convert the Hugging Face checkpoint to torch_dist format: Execute the training script: Here, we…...
DeepSeek-V4-Flash-Vision-Exp (Experimental)
1+ week, 5+ hour ago (920+ words) DeepSeek-V4-Flash-Vision-Exp is a multimodal mixture-of-experts model in the DeepSeek-V4 family. It combines the DeepSeek-V4 language model with a vision encoder and aligner, and accepts text, single-image, and multi-image requests through the OpenAI-compatible chat API. Support on vLLM Ascend is…...
low_latency_gemm
2+ week, 3+ hour ago (20+ words) vLLM Qwen4Exp low-latency GEMM hook for AMD ROCm. Keep the standard vLLM linear methods on AMD ROCm....
model
2+ week, 3+ hour ago (106+ words) Return all MambaSpecs for this model (GDN layers + PLE layer). The PLE layer uses a separate short_conv MambaSpec whose page_size_bytes may exceed the GDN spec; callers should take the maximum. Qwen3-VL vision tower backed by the Qwen4Exp language model. Expose Qwen4Exp routed experts through…...
qwen4_exp
2+ week, 3+ hour ago (78+ words) This implementation uses only PyTorch operators. Tensor-parallel collectives are supplied by its caller. Mix: RMSNorm -> low-rank gate -> gated mean. Broadcast the block output back to every stream. Average the HC streams into a single block input. Return all MambaSpecs for this…...
model_state
2+ week, 3+ hour ago (18+ words) vLLM Model-runner state for Qwen4Exp PLE inputs. Add rollback-safe PLE n-gram context to the model inputs....
attention
2+ week, 23+ hour ago (434+ words) MLA attention and lightning indexer for HY V4 (NVIDIA). The per-head learnable sink is supported through.flashmla_sparse, which subclasses the platform's sparse MLA backend to forward attn_sink. The sink is applied by binding the sink-capable backend from.flashmla_sparse; if no backend on this platform…...
hy_v4_reasoning_parser
2+ week, 23+ hour ago (286+ words) Reasoning extraction for HYV4, on plain data. Count reasoning tokens (0 in no_think). The start token is injected at the END of the prompt, so it is normally absent from token_ids. Only skip a leading start token when it actually appears first (legacy / no…...