Install
Easy, fast, and cheap LLM serving for everyone vLLM is a fast and easy-to-use library for LLM inference and serving. Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evolved into a community-driven project with contributions from both academia and industry. vLLM is fast with: Efficient management
- 37articles · 30d
- 4+ day agolatest article
- Aug 15, 2026earliest in window
- 0%with images
- 177avg words
- Science & Technology 37
- Software Dev. 37
- Computers & Electronics 36
- Jobs & Education 1
Please confirm you are human
This browser or connection looks automated. Press and continuously hold the control for 3 seconds to enable Google-hosted web results and, when separately allowed, AI-assisted answers.
A successful check enables 100 search requests. Interactive access does not authorize scraping, systematic collection, or reuse of search output.
News
protocol
2+ week, 1+ hour ago (53+ words) vLLM Prompt tokens contributed by each input modality, keyed by modality name multimodal_tokens = None class-attribute instance-attribute ¶ Prompt tokens contributed by each input modality, keyed by modality name (e.g. image, audio, video ). A breakdown of the multimodal placeholder tokens already counted…...
low_latency_gemm
2+ week, 2+ hour ago (20+ words) vLLM Qwen4Exp low-latency GEMM hook for AMD ROCm. Keep the standard vLLM linear methods on AMD ROCm....
model
2+ week, 2+ hour ago (106+ words) Return all MambaSpecs for this model (GDN layers + PLE layer). The PLE layer uses a separate short_conv MambaSpec whose page_size_bytes may exceed the GDN spec; callers should take the maximum. Qwen3-VL vision tower backed by the Qwen4Exp language model. Expose Qwen4Exp routed experts through…...
model_state
2+ week, 2+ hour ago (18+ words) vLLM Model-runner state for Qwen4Exp PLE inputs. Add rollback-safe PLE n-gram context to the model inputs....
hy_v4_tool_parser
2+ week, 22+ hour ago (620+ words) Pure tool-call parsing logic for HYV4. Returns plain dicts; no vLLM types. Holds only parser state: the structural token strings/ids, the compiled regexes, and the streaming incremental state. Stateless argument parsing is delegated to the module-level utilities above. Streaming has…...
sse_keep_alive
3+ week, 8+ hour ago (48+ words) vLLM SSE keep-alive comments for idle streaming responses. Emit an SSE keep-alive comment when generator is idle. A non-positive or non-finite interval returns generator unchanged, so the default path has no overhead. Otherwise a keep-alive comment is yielded whenever no…...
glm52_low_latency_gemm
1+ mon, 1+ week ago (26+ words) vLLM GLM-5.2 decode GEMM selection for unquantized BF16 on SM103. Plan for a weight the walk below cannot reach (a plain nn.Linear )....
server_utils
3+ week, 4+ day ago (47+ words) vLLM Get the uvicorn log config based on the provided arguments. Priority: 1. If log_config_file is specified, use it 2. If disable_access_log_for_endpoints is specified, create a config with the access log filter 3. Otherwise, return None (use uvicorn defaults)...
dots_tool_parser
4+ week, 1+ day ago (55+ words) vLLM Parse Dots tool calls in their XML wrapper format. The canonical body contains one or more invoke elements:: A JSON object with name and arguments is also accepted as a fallback. Multiple wrapper blocks and multiple invokes per block…...