vLLM 0.31 locks down per-request media knobs.
vLLM v0.31.0 ships with safer multimodal defaults
The headline breaking change: requests that carry mm_processor_kwargs or media_io_kwargs are refused unless the server starts with --trust-request-mm-kwargs. The context is a Sep 23 vLLM advisory: on deployments serving Qwen2-VL or Qwen3-VL, an unauthenticated caller could raise request-level video frame and fps knobs and exhaust API-server memory — the project says 74 extra bytes of JSON took peak RSS from 2,271 MiB to 13,629 MiB (patched in 0.30.0). The same release also tags prefix-cache keys by source so a LoRA name and a cache_salt can no longer collide, and folds the LoRA path into the block hash.
The rest is a big serving drop: DeepSeek-V4.1-Flash performance work on NVIDIA Blackwell-class GPUs, a vllm preload weight-cache daemon for fast engine restarts, experimental CRIU engine snapshots, speculative decoding on Model Runner V2 (draft models, async DFlash), and new scheduling caps. Speed claims are project-reported and mostly datacenter CUDA paths — not an Apple Silicon story.
Monday action: before you bump the pin, grep your clients for per-request multimodal kwargs and only re-enable them for trusted callers. Other breaks to check: tokenizer_mode="slow" is gone, online quantization="fp8" becomes the fp8_per_tensor shorthand, a Mamba prefix-cache flag was renamed, and --enforce-eager now also skips JIT kernel warmup.
Sources: vLLM v0.31.0 release · vLLM advisory GHSA-x6mc-67gf-chw4 (Sep 23)
| Model / package | Org | Notes |
|---|---|---|
| vLLM v0.31.0 Oct 5 |
vLLM | GA mm kwargs gated · preload daemon · MRV2 spec decode |
| Ollama v0.35.1 Oct 2 · GA |
Ollama | GA Clef / Clef Flash · /v1/systemone · images |
| Ollama v0.40.0-rc3 Oct 4 |
Ollama | Watch MLX default on Apple Silicon · decision models on MLX · pre-release |
| llama.cpp tips b11361–b11417 Oct 2–5 · Metal b11362 / b11404 |
ggml | Tip /v1/systemone · Clef / Nimble · Metal spec-decode · GA still v0.5.0 |
| Kolibri 1 Oct 3 |
Aleph Alpha | Open 78B / ~3.5B active MoE · Apache-2.0 · FP8 · mlx-lm PR open |
| AstaBrief 8B Oct 2 |
Ai2 | Open cited reports · Apache-2.0 · SFT data released |
| Ray 2.59.0 Oct 2 |
Anyscale / Ray | GA Data / Serve LLM GA · local token auth default |
| MLX / mlx-lm / mlx-vlm unchanged |
Apple ML / Blaizzy | Watch 0.32.3 / 0.32.0 / 0.7.4 · SGLang still 0.5.21 |
| Item | Beat | Status |
|---|---|---|
| vLLM per-request mm kwargs v0.31.0 · Oct 5 |
Rejected unless --trust-request-mm-kwargs · related Sep 23 DoS advisory |
Active |
| Ray local token auth 2.59.0 · Oct 2 |
On by default for local clusters · opt out via RAY_AUTH_MODE | Active |
| AgentCore Gateway private CA Oct 2 |
Private TLS trust for MCP / OpenAPI / proxy targets | Active |
| Sonnet 4.5 retirement Claude API · Nov 30 |
Migrate to Sonnet 5.5 · partner clouds set own schedules | Watch |
| GPT-Rosalind billing starts Oct 5 |
Trusted-access life-sciences model · $5 / $25 per 1M | Watch |
| NIST NCCoE Agentic webinar Oct 28 · comments Nov 9 |
DevSecOps / agentic AI scoping | Watch |
Critical · upgrade breaks · tip vs pin
vLLM 0.31.0 rejects per-request multimodal kwargs by default and drops tokenizer_mode="slow". Ray 2.59 local clusters now require a token. llama.cpp tips ≠ GA v0.5.0. Claude Sonnet 4.5 still retires on the Claude API Nov 30.
Watch · Ollama RC · Kolibri on MLX · Spark ship date
Ollama 0.40.0-rc3 is still pre-release. Kolibri’s mlx-lm support is an open PR; community quants are unofficial. DGX Spark 64GB ships Oct 23; the 1.7× cluster figure is vendor. OpenAI’s Oct 2 GPT-6 model guide restates Astra / Sol / Luna pricing — useful, not new SKUs.
Opportunity · end-state evals · private MCP
Run a few ThinkingBox tasks 20× against your agent model before trusting a leaderboard. If your MCP servers sit behind a private CA, AgentCore Gateway can now reach them without an ALB. AWS Well-Architected Agent (preview, Oct 1) reviews Terraform / CDK / CloudFormation.
Retest speculative decoding on your Studio
Build or download a llama.cpp tip at or after the Oct 5 Metal change, then run the same model with and without your draft-model / DFlash setup on a fixed prompt set. Compare decode tok/s against serial and against GA v0.5.0. Keep the GA binary around for rollback.
Put a decision model in front of your agent
On Ollama 0.35.1, pull clef-flash and POST a real ticket or screenshot state to /v1/systemone with one noul and one choice question. Route on the probabilities instead of asking a chat model to “classify.” Check ollama show lists it as decision only.
vLLM 0.31 upgrade checklist
Before bumping: search client code for mm_processor_kwargs / media_io_kwargs; replace quantization="fp8" with fp8_per_tensor; drop tokenizer_mode="slow"; rename the Mamba prefix-cache flag. Only add --trust-request-mm-kwargs for endpoints with trusted callers.
Issue 31 · Monday, October 5, 2026 · Not investment advice · Sources cited inline · Window: after Issue 30 (Fri Oct 2 morning ET) through Mon Oct 5 morning ET · Issue 30 distillation / SGLang 0.5.21 / Clef MLX quants / Olmo-core 3 / AutoSynthData not re-led · Issue 29 UltraFast / Sonnet 4.5 deprecation not re-led · Issue 28 BMA not re-led · Issue 27 Sonnet 5.5 / Sol / DevDay not re-led · Haiku 5.5 not invented · Sol Ultrafast not GA · Listen omitted (episode not published)
Friday after Issue 30 through Monday morning. ASTGL / Fraser takes — sourced, not recap theater.
Request knobs are attack surface — vLLM finally treats them that way
Letting callers tune frame counts, fps, and processor settings per request was a convenience. The Sep 23 advisory showed what that costs: a few bytes of JSON and your API server is swapping. Defaulting those knobs to “off unless trusted” in v0.31.0 is the right call, even though it will break somebody’s client this week. If you run a vision model behind any OpenAI-compatible endpoint — vLLM on a box or a local server on your LAN — check what a caller is allowed to set, and pin the expensive limits server-side.
Sources: vLLM v0.31.0 · GHSA-x6mc
The database disagreed — and that is the eval you should be running
Microsoft’s ThinkingBox (on Hugging Face Saturday) grades agents on the records they leave behind across 507 stateful business workflows, each run 20 times. The finding that matters: one success is not reliability. Claude Opus 5.5 edges Opus 5 on single-attempt score, yet both pass exactly 241 tasks on all 20 runs; Kimi-K3 solves 93.89% of tasks at least once but only 13.41% every time; roughly four in five failures are tool handling, not reasoning. Numbers are author-reported and costs are estimates at list rates, but the method is the point — check terminal state, not the transcript.
Source: Microsoft / Hugging Face — ThinkingBox
Decision models are now a standard local endpoint
In one weekend,
/v1/systemonestopped being one runtime’s experiment. Ollama 0.35.1 went GA Friday afternoon with Cloudflare’s Clef and Clef Flash (images alongside text state). Friday night llama.cpp’s server grew its own/v1/systemone, added Nimble support the same night, and text-only Clef support Saturday morning. Sunday’s Ollama 0.40 RC runs Nimble, tev1, and both Clefs on MLX. The classifier tier for agents is local now; just remember Cloudflare’s Decision Index wins and latency table are vendor-reported.Sources: Ollama v0.35.1 · llama.cpp b11361 · b11371 · Cloudflare — Clef
Studio owners: retest speculative decoding
The most Apple-specific line of the weekend sits in a llama.cpp tip note: without the tensor API, Metal ran the few-token verify step with mat-vec kernels, so DFlash2 decoding on an M3 Ultra was slower than plain serial decoding. Monday’s tip adds few-row matrix kernels for that step on Apple GPUs without the tensor API, and Friday night’s tip added a tensor-API flash-attention kernel for F16 KV. If you wrote off draft models on your Studio, this is the week to rerun the numbers. Tip builds, not GA — keep v0.5.0 pinned for rollback.
Sources: llama.cpp b11404 · b11362
Kolibri is the shape unified memory loves — wait for merged support
Aleph Alpha shipped Kolibri on the Day of German Reunification: a German/English reasoning MoE with 78B total and about 3.5B active parameters, Apache-2.0, 1M-token context validated (≤262k recommended). Big total, tiny active is exactly what a high-memory Mac runs well. But the official weights are FP8 sized for datacenter GPUs, mlx-lm support is still an open PR, and the weekend’s MLX and GGUF uploads are community conversions. Quality claims are vendor-reported. Try it if you like; do not build on it until support merges.
Sources: Aleph Alpha blog · Kolibri-1 model card · mlx-lm PR #1945
DGX Spark 64GB lowers the CUDA desk-box entry
NVIDIA’s Friday post puts a 64GB DGX Spark on OEM shelves Oct 23 from $4,999, same GB10 chip and software stack as the 128GB unit, with two boxes clustering over ConnectX-7. For CUDA-native teams that is a real local option. For Mac shops it is a reminder that “pool two boxes” is not the same as one big unified memory pool — NVIDIA’s own 1.7× two-unit figure is vendor-reported, so measure before you buy a pair.
Source: NVIDIA blog — DGX Spark 64GB