OSS tools Agent Skills → Agentic AI Foundation · SageMaker training session APIs · vLLM / SGLang quiet

Agent Skills donated to the Agentic AI Foundation

Anthropic news Oct 22. A skill is “a folder containing a SKILL.md file that teaches an agent a specific task or domain, along with any scripts or reference material it needs.” Standard published December 2025; “More than 70 agent platforms, including ChatGPT, Cursor, GitHub Copilot, and VS Code, have integrated Agent Skills”; “1 million skills have been published and nearly 280 million installed” (Anthropic’s figures). Donation covers the name and logo, GitHub org (agentskills/agentskills) and agentskills.io. “Engineers from Anthropic, Google, OpenAI, and other companies will maintain the standard as individuals, with David Soria Parra and Mahesh Murag as Lead Maintainers.” Proposed roadmap: “versioned releases, a formal proposal process, and security guidance for people who write and install skills.” AAIF already hosts MCP.

SageMaker AI training session APIs

What’s New Oct 22 12:30 PM ET; ML blog 2:00 PM ET. Serverless model customization adds training session APIs: “teams get to write the training loop, whether that’s reproducing SFT or DPO on their own terms or implementing a custom loss, a new RL reward signal, a data curriculum.” Operations (forward_backward(), optim_step(), sample()) are called from your own Python “against a live session, with checkpointing and resume built in. Billing is per token processed, so local computation like reward scoring isn’t billed as GPU time.” Regions per the announcement: US East (N. Virginia), US West (Oregon), Asia Pacific (Mumbai). Same day: serverless customization for Muse Glimmer, Qwen 3.8 and Ministral 3 (4:00 AM ET).

vLLM / SGLang — quiet; MiniMax recipes published

vLLM v0.32.0 (Oct 18) still latest; advisories newest the Oct 20 batch. SGLang v0.5.22 (Oct 19 UTC) still latest; no advisories. Both projects now have MiniMax M3.1 Flash serving pages linked from the model card (vLLM: TP4 on Blackwell or MI355X, TP8 on H100/H200). Datacenter recipes; no Apple Silicon path.

TorchTitan — new models, MoE performance, TitanRL

PyTorch blog Oct 21 19:45Z (≈3:45 PM ET). TitanRL: async rollouts; same model implementation from pretraining through post-training and inside vLLM; batch-invariant mode gives bitwise-identical trainer/generator log-probs in supported on-policy configs (“the gap between them is exactly zero”). Demonstrations: SWE-Rebench easy with Qwen3.5-35B-A3B base at 64K context on 40 GB300; Tmax 15k train / Terminal-Bench 2.1 eval at 128K on 20 GB300. Composable config system (typed Python recipes, model-config transforms, component overrides); Attention Gym kernels (Gated DeltaNet, Kimi Delta Attention) and Helion kernels; fused comm-compute kernels for SM100+; MXFP8 DeepSeek V3 671B+MTP on MLPerf training (internal GB300). Project-reported. Also Oct 21: FlyDSL on AMD GPUs; AWS ML blog “Train Qwen3 on AWS Trainium with torchtitan.” vLLM v0.32.0 (Oct 18) latest; advisories newest Oct 20. SGLang v0.5.22.

Ray 2.58.1 / 2.57.1

Both published Oct 21 00:57Z (≈8:57 PM ET Oct 20). Ray Data: fix out-of-scope block callbacks registered on blocks passed through from input to output (#66369; cherry-picks #66404 / #66403) — spurious callback errors on passthrough datasets. Images: google-cloud-cli installed in base-extra images (#65757). vLLM: v0.32.0 (Oct 18) latest final, v0.32.1rc0 tag; security advisories newest Oct 20 07:37Z (the nine in Issue 42). SGLang v0.5.22 (Oct 19 UTC); no advisories.

vLLM — nine advisories published Oct 20, all patched in 0.32.0

Published Oct 20 07:33–07:37Z (≈3:33–3:37 AM ET); every one lists patched: 0.32.0. High: GHSA-h3rc-j82p-2c3h (CVSS 8.1) — ImageMediaIO passed attacker bytes to Pillow format autodetection; an EPS payload labeled image/png reached the host Ghostscript; with Ghostscript affected by CVE-2024-29510 that is a command-execution path (loader-level reproduced; HTTP path source-traced, not run end to end); affected 0.30.0. GHSA-6757-hggm-hp2w (7.5) — with --mm-processor-cache-type shm, a forged SHM address/monotonic_id pair reaches worker-side pickle.loads() (authenticated caller; listed affected ≤0.25.1). Medium (6.5): GHSA-542m-9234-8gvf — unbounded chat-template Jinja rendering (<100-byte request) wedges the single renderer thread for all tenants (≥0.19.0); GHSA-x567-49g5-p4cf — /v1/responses bypassed trust_request_chat_template for chat_template_kwargs (0.16.0–0.23.0); GHSA-q97c-p368-75c4 — pre-tokenized token id equal to vocab size passes validation and triggers a CUDA device-side assert (≤0.30.0); GHSA-frvg-qw88-hww9 / CVE-2026-94624 — P2P secondary-tier peers never connected exhaust the ZeroMQ socket quota (0.26.0); GHSA-gq5q-89c7-vhvx — DeepSeek-OCR family: an image whose normalized pixels sum to exactly 0 trips an assert and kills the engine (0.30.0); GHSA-8397-8qf6-f2gm — Phi-4 multimodal audio encoder retains full attention matrices; a max-length (600 s) clip OOMs the engine (0.31.0). Low (3.4): GHSA-vvpg-rrww-wpmh — default allowed_origins=["*"] lets any web page read responses from an unauthenticated server the browser can reach. Closes Issue 41 Watch: q97c and frvg (404 yesterday) are now public. Still a mismatch: v0.32.0 notes cite GHSA-4hhp-h66f-j5j7 for chat-template exhaustion; 4hhp is the Aug 13 MiMoV2Omni SSRF (patched ≥0.26.0). The chat-template advisory is GHSA-542m.

Transformers v5.20.2 (patch)

Published Oct 19 17:45Z (≈1:45 PM ET). Fixes inputs losing their original type inside prepare_inputs_for_generation in generate (broke multimodal models; #49731); zero expert gradients in bfloat16 with expert parallelism (#49688); declared input modalities for VLMs that reported text only (#49677); avoids a host-to-device copy in eager_mask (#49684). Note from Issue 41 still applies: vLLM 0.32.0 pins transformers <5.20.0.

Pointers (not reviewed in depth): TorchNeuron; AWS vLLM KV-transfer posts

PyTorch blog Oct 19: “Introducing TorchNeuron: Native PyTorch backend for AWS Trainium” (also HF blog Oct 19). AWS ML blog Oct 19: “Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM” and “Improve TTFT for multi-turn agentic workloads with bi-directional KV transfer in vLLM.” Listed for completeness; titles only.

vLLM v0.32.0

Published Oct 18 06:50Z (≈2:50 AM ET Sun); 814 commits / 363 contributors. Highlights: POST /v1/systemone structured decisions (choice / noul / score) served by default; --tool-call-parser hf / --reasoning-parser hf read the checkpoint’s response_template; NVFP4 KV cache on SM8x/SM12x/SM90; GLM-5.x FP8 KV default on SM100; MRV2 GPU n-gram speculator, Elastic EP; vllm download-kernels, vllm preload --preload-autotune; least-loaded API worker routing (43–85% more req/s in the PR’s Qwen3-0.6B microbenchmark, project-reported). New models: EmbeddingGemma 2, MiniCPM-V 4.7, Nemotron 3.5 ASR, clef-omni, LiquidAI d1-3B. Security: fixes listed for GHSA-4hhp-h66f-j5j7 (“chat template resource exhaustion”, #50300), GHSA-q97c-p368-75c4 (tokenizer max_token_id, #59491), GHSA-frvg-qw88-hww9 (P2P ZMQ socket exhaustion, #51504); CORS default * → none (#61009; set --allowed-origins); untrusted images raster-only (#60022); signed shared-memory MM cache handles (#59357); JSON schema depth >64 rejected (#60036). Flag: on GitHub, q97c and frvg return 404 (unpublished), and the public GHSA-4hhp-h66f-j5j7 is a different issue (MiMoV2Omni SSRF / local file read, published Aug 13, patched ≥0.26.0) — track by PR. Breaking: Python 3.10 dropped; transformers>=5.16.1,<5.20.0; cascade attention, torch.compile SP/AsyncTP passes and mamba_cache_mode="all" removed; max_tokens no longer reserved from context; Responses reasoning stream events renamed. Release notes do not mention Beam or Muse Spark Open. Newest published advisory batch on the repo remains Oct 9.

JAX v0.12.0

GitHub release published Oct 19 11:18Z (≈7:18 AM ET) (tag created Oct 16). New: jax.nn.get_scaled_dot_general_config('mxfp4') (E2M1 + E8M0 per 32; native on AMD gfx950, dequantize fallback elsewhere); proxy backend for IFRT Proxy / Pathways on Cloud (JAX_PLATFORMS=proxy, JAX_BACKEND_TARGET); jax.custom_vjp.defremat, generic_batching; jax.sharding.no_collectives; jax.profiler.ProfileSession; DLPack 1.0; jax.experimental.thunky. Breaking: jax.custom_remat removed; x86-64 requires Haswell+ (AVX2 + FMA); atleast_*d return tuples; random.multinomial shape semantics; vmap + custom_vjp shared-output case now errors; NumPy ≥2.2; jnp.cross 2D removed; jnp.array copy/order/ndmin keyword-only. Deprecated lax.psend/precv.

Transformers v5.20.1

Published Oct 16 13:02Z (≈9:02 AM ET). Adds Reflection Beam (BeamForCausalLM; HF says transformers>=5.20.1). Fixes: HunYuan-VL batched image assignment (#49557), DeepGEMM Mega MoE under EP (#49428), from_pretrained on torch 2.7 — require torch ≥2.8 for distributed checkpointing (#49652), DtensorShardOperation on torch 2.6/2.7 (#49657), auto-mapping bug (#49664). Conflict to watch: vLLM v0.32.0 pins transformers<5.20.0.

TRL v1.16.0

Released Oct 15 18:00Z. AWS Trainium support with guide; pad_to_multiple_of defaults to "auto" (128 on Neuron; Qwen3-0.6B SFT 20 steps 130 s vs 260 s). Fused LM head / distillation kernels without Triton; activation offloading on any accelerator. Tree attention for AsyncGRPO (#7328): micro-batch rows packed as prefix trees, one FA3 varlen call per segment, no mask tensor. Qwen3-1.7B, 32K tokens/rollout, fwd+bwd: packing ratio 3.37 shared prompt 8179 → 2755 ms (2.97×), nested fork 3.56 8172 → 2635 ms (3.10×); peak 35.2 → 13.5–13.6 GiB; ~1% cost when nothing is shared HF-reported. Muse Spark Open: LoRA SFT on 8 nodes × 8 H100 with DistributedConfig(fsdp_size=64, ep_size=8), 14.2 s/step, 131k tokens/step, 40.2 GB/GPU; needs transformers ≥5.20.0 (adapter save also needs transformers#49500 + peft 0.21.2).

Transformers v5.20.0

Released Oct 15 18:05Z. New: Muse Spark Open (MuseSparkForConditionalGeneration; BF16 on 4×8 H200 TP=32, NVFP4 on one B200 node per HF), Ling 3.0 Flash native as BailingHybridForCausalLM (no trust_remote_code; 5 KDA layers per MLA layer), Aleph Alpha Kolibri 1 (MoE, 6 of 384 experts + shared expert), MiniCPM-V 4.7 (canvas M-RoPE), Baidu UnlimitedOcr (3B, reference sliding-window attention, constant KV during decode).

ExecuTorch — Cortex-M backend

PyTorch blog Oct 15, 2026 (Meta + Arm). Runs exported, quantized PyTorch models directly on Arm Cortex-M CPUs with CMSIS-NN; complements the Ethos-U NPU path (Cortex-M passes run after Ethos-U partitioning). Demo models on Alif Ensemble E8 run in ≤21.5 ms and <90 KiB RAM on one Cortex-M55 core; core runtime <20 KiB flash with selective operator build Meta/Arm-reported. Latest ExecuTorch release tag still v1.5.1 (Sep 23).

vLLM — 11 advisories on Oct 9 (5 High, 6 Medium); all fixed by 0.31.0

Published 2026-10-09 06:39Z–08:27Z ≈ 2:39–4:27 AM ET. Patched-version column from each advisory header. Highs: GHSA-rhcx-5729-88vg (High, CVSS 7.7, AV:N/AC:L/PR:L/S:C/A:H) — LogprobsTensors.empty_cpu() allocates with torch.empty(); with a KVConnectorBase_V1 that reports externally computed prefix tokens (in-tree: LMCacheConnectorV1 on any prefix hit, NixlConnector on the decode side), positions [0..num_computed_tokens-1] stay uninitialized and prompt_logprobs (or logprobs + echo=true) decode raises OverflowError, killing EngineCore and all co-tenant requests; affected ≥0.8.5, patched ≥0.31.0. GHSA-8c65-hq7q-r7jm (High 7.5) — allowed_token_ids request with two requests in flight during speculative decoding crashes the worker; ≥0.11.1 <0.18.0, patched 0.18.0. GHSA-vfp2-c8pq-v6h6 (High) — OpenAIServing._base_request_id copies X-Request-Id verbatim; P2pNcclConnector.parse_request_id extracts host:port via regex ___decode_addr_(.*):(\d+); prefill worker ZMQ-connects out, sends the NCCL unique_id, then ncclSends the request’s KV cache to the attacker; requires kv_connector: P2pNcclConnector, kv_role: kv_producer; ≥0.9.2 <0.24.0, patched 0.24.0. GHSA-mm38-5g96-7j6x (High) — nemotron_vl.py and nano_nemotron_vl.py set Image.MAX_IMAGE_PIXELS = None at import, disabling Pillow’s decompression-bomb guard process-wide; unauthenticated image DoS; header ≤0.8.5, patched ≥0.25.0 (PR #47010; adds VLLM_MAX_IMAGE_PIXELS). GHSA-4m6g-h9mv-wqjc (High) — media_io_kwargs.video.num_frames=-1 bypasses the inline video/jpeg frame cap; ≥0.18.0 <0.31.0, patched 0.31.0 (#58830 rejects per-request media_io_kwargs unless --trust-request-mm-kwargs). Mediums: GHSA-r45p-7qwx-qwjw (6.5; audio decoding bypasses decompression-bomb duration limit; ≥0.19.0 <0.24.0), GHSA-w2v2-5662-8v5q, GHSA-6pgw-f3pv-4h8q, GHSA-rwfw-xvg3-3937 (5.9 each; Rust gRPC: out-of-vocab prompt IDs, output_candidates.token_ids vs logprob_token_ids, CandidateTokens.top_n vs max_logprobs terminate EngineCore; ≥0.22.0 <0.24.0), GHSA-34qw-q2h2-jp3m (5.9; disaggregated multimodal /inference/v1/generate via kwargs_data/mm_placeholders; ≥0.20.0 <0.30.0), GHSA-w9r6-29q4-pvpr (6.5; unvalidated stop_token_ids → crash and out-of-bounds GPU write; ≥0.20.1 <0.29.0). No CVE IDs listed yet. Several Highs carry no CVSS score in the API. Takeaway: only rhcx and 4m6g need 0.31.0; the rest were fixed in 0.18–0.30 and are disclosed now.

TRL v1.15.0 — fused LM head by default; Liger path deprecated in four trainers

Published 2026-10-09T05:55Z ≈ 1:55 AM ET. SFT, DPO, KTO, GRPO, RLOO and distillation score tokens through a Triton kernel that projects hidden states through the LM head in tiles and reduces straight to per-token log-probs and entropy, so the [batch, seq, vocab] logits tensor is never built (#7386, #7387, #7389, #7390, #7466, #7465). On by default. HF benchmark (1× B300 capped at 79 GiB, gemma-3-1b-pt random bf16 weights, batch 1, gradient checkpointing, sdpa): max length DPO 10,240 → 59,392 (5.80×), KTO 9,216 → 63,488 (6.89×), GRPO scoring 28,672 → 114,688, RLOO 23,552 → 100,352, SFT nll 20,480 → 107,520; SFT default chunked_nll unchanged. At 8,192 tokens peak memory −52% to −82%, steps 2.3–10.9% faster (HF-reported; not measured: H100 itself, flash-attention, real generation, multi-GPU). Needs Triton on a GPU (Linux with CUDA, ROCm or XPU). 💥 Breaking: full-logits scoring path, use_liger_kernel chunked path and _forward_redirection removed from GRPO/RLOO/DPO/KTO; a PEFT adapter on lm_head raises (use modules_to_save=["lm_head"]); use_liger_kernel=True deprecated (removal v2.0.0; use model_init_kwargs={"use_kernels": True}); DPO/KTO compute_metrics full logits come from an extra forward (deprecated). Also: selective activation checkpointing in SFT (gradient_checkpointing_kwargs={"selective": True}; PEFT + ZeRO-3 rejected) (#7194); assistant-only loss on vision datasets.

vLLM GHSA-h3rc-6mm3-gc2m — request-controlled code_revision → RCE

Published 2026-10-06T07:06Z ≈ 3:06 AM ET Oct 6. Severity High, CVSS 8.1 (AV:N/AC:H/PR:N/UI:N). Affected ≥0.7.3, <0.31.0; patched ≥0.31.0 (#58830). A request can set mm_processor_kwargs.code_revision; when the served model’s processor uses dynamic auto_map.AutoProcessor code and the operator enabled --trust-remote-code, the value reaches AutoProcessor.from_pretrained and selects which Python revision Transformers imports, i.e. arbitrary code in the API/renderer process. The model/data revision stays pinned; only the processor code changes. Preconditions: dynamic processor code, trust-remote-code on, the server can fetch or has cached the chosen revision, the attacker can create or control a revision (e.g. an HF community PR ref), and a multimodal request reaches the processor path. mm_processor_kwargs is accepted on TokenizeChatRequest and ChatCompletionRequest; /tokenize is outside the --api-key middleware (which guards /v1, /v2, /inference, /cohere). Default trust_remote_code=False is not affected. Correction: Issues 32 and 33 listed four Oct 6 advisories (p92p, 4xqp, 823j, 6cxc) and missed this one. Context: 0.31.0 also refuses request mm_processor_kwargs unless --trust-request-mm-kwargs is set (Issue 31).

TRL v1.14.2 — two silently wrong training cases, three crashes

Published 2026-10-06T20:30Z ≈ 4:30 PM ET. #7505: GRPO, RLOO and Distillation on the default generation path (use_vllm=False). Models that end a turn with an id only their generation config declares (Gemma 3/4, Phi-3.5, …) did not stop at the end of the turn, and everything generated after it was trained on, up to max_completion_length. “The only visible sign was completions/clipped_ratio close to 1.” With vLLM, generation stopped correctly but metrics were wrong, and mask_truncated_completions=True dropped every finished completion. Models whose tokenizer eos is their only end-of-turn id are unaffected. Also: finite DFT loss on batches with no trainable tokens (#7471); precompute_ref_log_probs under FSDP with no reference model (#7507); completion start where prompt and prompt+completion tokenizations diverge (#7463); guarded requests/urllib3 imports in the vLLM client (#7545).

Transformers v5.19.0 — EmbeddingGemma2 and breaking changes

Published 2026-10-06T16:39Z ≈ 12:39 PM ET. New model: EmbeddingGemma2. 🚨 Breaking: every MoE model whose router computes logits now returns them when output_router_logits=True, following the Qwen3-MoE pattern (#48920). OWLv2 embed_image_query now picks the query box by objectness (#49200). The paged| prefix for sdpa and flash attention is deprecated, and regular attention now supports continuous batching (#49112, #49101); paged|eager remains. The CB cache update for the index and block-table paths is fused (#49088). Parallelism: expert-parallel token dispatch is the default for Qwen3 MoE and Mellum, so EP size no longer needs to equal TP size (#48865), and the Trainer works with EP (#48873). Cache: quantized cache fixed and generate no longer mutates cache_config (#48700); per-layer cache configuration (#48178). Also a WatermarkDetector repeated-ngram counting fix (#49315).

vLLM GHSA-6cxc-2vcg-w5qc — stale allowed_token_ids mask across requests (Low)

Published 2026-10-06T20:45Z ≈ 4:45 PM ET. Severity Low (CVSS 3.1 3.7; AV:N/AC:H/PR:N/UI:N/I:L). V1 InputBatch.condense() copies a moved request’s allowed_token_ids mask row into the freed index but never clears the source row. A later unrestricted request that reuses the slot inherits another client’s allowlist and still returns HTTP 200. Exploitation needs concurrent requests and a condense slot-reuse race, and impact is vocabulary steering, not arbitrary text. Unauthenticated /invocations sits outside --api-key guards. Header: patched in ≥ 0.31.0. The body says the fix (#43931) was not in a tag “as of v0.30.0”, which is consistent. A related swap_states aliasing fix is #48419. This is the fourth vLLM advisory published Oct 6.

vLLM v0.31.1rc0 — release candidate

Tagged 2026-10-06T18:52Z ≈ 2:52 PM ET; headline commit “[Metrics] Expose cached prompt tokens by cache tier (#56318)”. RC only; 0.31.0 remains the fix version for the Oct 6 advisories.

Apple ML Research — RISED: rubrics for multi-environment agent RL

Posted Oct 6. An LLM judge tags each rollout with a rubric vocabulary shared across environments. The rubric profiles then guide online prompt-group selection and policy supervision (self-distillation) when scalar rewards give no within-group contrast, as with all-fail or all-success groups. Author findings.

vLLM GHSA-4xqp-c3mv-qff7 — one request hangs the engine, /health stays 200

Published Oct 6 (~3 AM ET). Severity Moderate; affected < 0.31.0; patched ≥ 0.31.0. A body on /inference/v1/generate with sampling_params.max_tokens null and min_tokens 2147483648 skips the min_tokens <= max_tokens check in the API process. The EngineCore re-validates it on decode, one line above the try that guards request handling, so the input thread dies. That thread was the only reader of the input socket. check_health looks only for a recorded error, so /health keeps answering 200 while every request hangs. Reach: mounted by default on vllm serve in v0.29.0, unauthenticated unless --api-key; on main only with VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 or --tokens-only. “One 92-byte request.” OpenAI routes resolve max_tokens first and return 400.

vLLM GHSA-p92p-rxj5-7p2x — forged multimodal uuid desyncs P0/P1 caches, kills EngineCore

Published Oct 6. Severity Moderate; < 0.31.0; patched ≥ 0.31.0. The user-supplied uuid on image_url / video_url / input_audio parts is used verbatim as mm_hash. The API-process (P0) and EngineCore (P1) LRU caches evict independently, and the P1 hit path does no consistency check. Sequence: seed a small image with uuid U, then send ~70 requests with unique uuids and oversized prompts. Each returns 400, but only after its image is inserted into P0, so U is evicted from P0 alone. Then send a large image with uuid U. The engine substitutes the stale tensor, a placeholder-count ValueError goes uncaught, and the engine dies. /health never recovers until a full restart. Verified on vllm/vllm-openai:v0.29.0 with Qwen2.5-VL-3B-Instruct at default flags.

vLLM GHSA-823j-m4cj-hmjf — MOSS-Audio /tokenize memory exhaustion (High)

Published Oct 6. Severity High; affected > 0.24.0, < 0.31.0; patched ≥ 0.31.0. With a MOSS-Audio model loaded, /tokenize accepts request-controlled mm_processor_kwargs. Each distinct config builds a MossAudioProcessor (with a WhisperFeatureExtractor mel-filter array) stored in an unbounded dictionary. Oversized mel_config values amplify a single request. “/tokenize is intentionally outside vLLM’s API-key-protected route prefixes, so --api-key alone does not restrict access.” Mitigation: upgrade to 0.31.0, and block /tokenize at the proxy if unused. All three advisories were reported by jperezdealgaba.

Apple ML Research — “How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?”

Listed in the Apple ML Research feed on Oct 1 (EPFL / Apple authors). Under equal wall-clock time and the same frontier backbone, open-source state-of-the-art MLE harnesses gave no advantage over a single session of a minimal-harness coding agent with read / write / bash tools. The authors conclude the backbone model, not the scaffolding, is the primary driver. Author findings on the benchmarks studied. Takeaway: baseline a plain agent before adding multi-agent orchestration.

vLLM v0.31.0 GA

Published 2026-10-05T06:44Z ≈ 2:44 AM ET Mon Oct 5 (Latest; rc2–rc5 Sep 30–Oct 2). 717 commits / 307 contributors (96 new). Security: per-request mm_processor_kwargs and media_io_kwargs rejected unless --trust-request-mm-kwargs (#58830); prefix-cache extra keys tagged by source so LoRA name and cache_salt cannot collide (#51899); LoRA path in block hash (#59335); stale multimodal receiver-cache entries can no longer replace fresh payloads (#57833). Context: advisory GHSA-x6mc-67gf-chw4 (Sep 23; affected ≥0.24.0, patched ≥0.30.0) — unauthenticated memory exhaustion via request-level media_io_kwargs.video.max_frames/fps on Qwen2-VL/Qwen3-VL; “74 extra bytes of JSON took the server’s peak RSS from 2 271 MiB to 13 629 MiB.” Breaking: tokenizer_mode="slow" removed (#58545); --enable-mamba-fine-grained-prefix-cache → --enable-mamba-shared-prefix-checkpoint (#57382); online quantization="fp8" → fp8_per_tensor (#53585); Quark silent online quant and AllSpark INT8 W8A16 removed; --enforce-eager also disables JIT warmup (#58197); XPU graphs default on. Perf/features (project-reported, mostly SM100/SM103 CUDA): DeepSeek-V4.1-Flash FlashMLA mega attention with NVFP4 compressed KV default; vllm preload weight-cache daemon (DP, MTP drafts, /health, readiness); experimental CRIU vllm snapshot create/restore (TP1); MRV2 draft-model spec decode, LiLiCorr drafter, async DFlash; MoonEP all2all; --max-num-active-seqs; adaptive --long-prefill-token-threshold; HiSparse fixes. Wheels: CUDA 13.0 default on PyPI; ROCm / XPU indexes; Docker vllm/vllm-openai:v0.31.0.

Ray 2.59.0

Release created 2026-10-02T07:04Z (≈3:04 AM ET Fri Oct 2; feed updated 17:58Z). Ray Data LLM and Ray Serve LLM APIs graduate to GA/Stable (#65194) with vLLM 0.27.0 (#65351). Token authentication on by default for local clusters: ray.init() without an address enables auth and generates ~/.ray/auth_token if missing; ray start --head enables auth when a token exists, otherwise warns; remote/multi-node unchanged; opt out RAY_AUTH_MODE=disabled (#64755). Dashboard redacts runtime_env values in browser-facing endpoints (#65226); read_hudi unpickling guard against RCE (#65780). Ray Data external disk-backed shuffle_v2 (renamed from hash_shuffle_v2); ORC reads; multi-path read_lance; read_videos fps/resize.

Microsoft ThinkingBox / ThinkingBox-Bench on Hugging Face

Joint Microsoft + HF blog 2026-10-03T22:56Z ≈ 6:56 PM ET Sat Oct 3. Sandbox runs agents against isolated MCP tool sessions and grades terminal backend state + side effects with deterministic judges (477 of 507 tasks state-only; 30 add rubric questions). 507 workflows across retail, auto insurance, travel, neobank, consulting; each run 20× from a clean backend; reports pass@1, pass@20, observed 20/20. Author-reported highlights: Claude Opus 5.5 67.16% pass@1 overall (Opus 5 66.50%) yet both pass 241 tasks 20/20; Kimi-K3 strongest open-weights (57.37% pass@1; 93.89% pass@20; 13.41% 20/20); GPT-6 Astra retains 78% of pass@1 at 20/20. Failure signatures: tool usage 79.9%, wrong state updates 10.3%, incomplete resolutions 7.0%, no state-changing action 2.9%. Costs estimated at undiscounted OpenRouter list rates — author-reported, not invoices. Run via OpenEnv (envs/thinkingbox_env), microsoft/thinkingbox-data at thinkingbox-bench-v1.0, tb CLI, Typesense 30.1 in Docker; tested on Linux/WSL with Python 3.11+.

SGLang v0.5.21

GitHub release Fri Oct 2, 2026 (~1:09 AM UTC). Highlights: new model cookbooks including DeepSeek-V4.1 Flash, GigaChat 3.5, MiMo-V2.6, DiffusionGemma, FLUX 3 Action, others; PD instances can switch prefill/decode on the fly without restart; prefix cache on a Rust core by default; new Decisions API (/v1/decisions) and Score API (/v1/score). Project claims: DeepSeek-V4.1 ~22% faster first token on long prompts; Kimi K3 ~20.6% higher prefill throughput in PD — project-reported. Install: uv pip install --prerelease=allow sglang==0.5.21; Docker tags lmsysorg/sglang:v0.5.21 (+ ROCm/XPU variants).

Olmo-core 3 — open MoE training infra

Ai2 / Hugging Face blog Thu Oct 1, 2026. Redesigned open MoE training system (DDP-based experts resident on GPU; expert / pipeline parallelism; distributed optimizer; rowwise expert parallelism; GPU-resident routing; grouped GEMM; MXFP8). Benchmark narrative: expert pool 8→128 selecting 4 experts/token (~3.2B active) with <5% throughput drop; preliminary 47B MoE ~52k tok/s/GPU vs ~19.4k prior on 8× B300 (~2.7×); MXFP8 ~21% higher throughput vs BF16 on 4× B300; 1.2T-parameter config across 512 GPUs (858 TFLOP/s/GPU peak observed with random routing). All benches Ai2-reported. GitHub + technical report linked from post.

AutoSynthData — synthetic training for enterprise agents

ServiceNow CoreAI / Hugging Face blog Fri Oct 2, 2026. Pipeline turns target-model failures + stronger-teacher successes into capability cards, then generates validated agentic tasks (system spec + user prompt + verifier) with positive/negative verification and bounded repair; target + multiply phases. Illustrated on EnterpriseOps Gym. Hybrid: Gemma-4-26B-A4B-it target, Qwen3.8-27B teacher, ~2,000 samples / ~18h — author reports mean Pass@1 +7.2 pp (35% relative) and verifier success 63.01%→68.55%. ITSM: DeepSeek-V4.1-Flash teacher, Pass@1 18.77%→27.18%. All deltas author-reported on named pairs — not a general agent GA claim.

Transformers v5.18.0 · PEFT v0.21.2

Transformers v5.18.0 published 2026-09-30T16:46:27Z ≈ 12:46 PM ET Sep 30. New model adds: Nemotron 3 Diarization (streaming, up to 8 speakers); NemotronH Omni (text/image/video/audio); HyperCLOVAX Vision V2; GTE encoder family. Release notes include [Executorch] Add MLX recipe (#48910); compressed-tensors FP8 kernels on MPS (torch ≥2.15) (#48985); PEFT × DTensor TP (#48485); Add MPS maintainer (#49052). Several breaking changes listed (ROCm gpt-oss FA3 routing, DETR image processing, indexer layer_type remap, vLLM Transformers-backend video token counting, DINO modernization, Kernels version bump). PEFT v0.21.2 published 2026-10-01T10:29:50Z ≈ 6:29 AM ET — fixes encoder-decoder models breaking under Transformers ≥5.18.0 (PR #3845). Do not rehash PEFT 0.21.0 methods (Issue 28).

TypeSafe Jev — System One decision models (prior · Issue 27)

TypeSafe AI blog (canonical launch primary). Jev is a “System One” model: typed Choice/Score/Noul outputs with probabilities; parallel (non-autoregressive) sampling; RLCD training. Pricing $0.042/MTok input, output free; claimed ~70–500ms E2E. Early access. Operator note: typed outputs reduce format bugs; calibration caveats (confidence ≠ accuracy on style-adversarial / reference-free pairs) matter for triage use. Pairs with Ollama v0.35.0 local /v1/systemone.

DeepEP-over-EFA MoE RL on Amazon EKS (prior · Issue 26)

AWS ML blog Fri Sep 25, 2026 ~12:29 PM ET (pubDate 16:29:50 +0000) — after Issue 25 morning ship. Architecture: EKS node groups for rollout vs policy training; EFA inter-node; S3 checkpoints; Spot for interruptible rollout. Stack pins include DeepEP 2.0.0, NCCL 2.31.2, SGLang 0.5.17, PyTorch 2.12.1 (CUDA 13.0), EFA installer 1.49, Miles 0.1.0. Vendor bench: 48× P5en (16 train / 32 infer), “super-sparse” MoE — DeepEP-over-EFA vs prior Slime stack → up to +40% aggregate RL rollout throughput — vendor-eval (mixes DeepEP/EFA and newer full stack vs older baseline; not DeepEP-only A/B). Companion SkyRL HyperPod how-to same afternoon is secondary.

TRL v1.14.0 (prior · Issue 25)

Published 2026-09-25T06:40:07Z ≈ 2:40 AM ET Sep 25. Removes trl.losses fused Liger loss reimplementations; DPO/KTO/GRPO stream selected-token log-probs via _ChunkedLogProbFunction then use each trainer’s own loss. Ships in-tree fused Triton logprob+entropy kernel at trl.kernels (default on CUDA/ROCm/XPU). Breaking: six experimental trainers removed (GRPOWithReplayBufferTrainer, BCOTrainer, PRMTrainer, XPOTrainer, NashMDTrainer, GSPO-token). Supported vLLM window moves to >=0.20.0,<=0.30.0 (adds 0.30.0). Do not re-lead Issue 24 vLLM Sep 23 GHSAs.

vLLM four Moderate advisories — Sep 23

Published Wed Sep 23 ~6:16–6:25 AM ET (10:16–10:25 UTC). Hub index: security/advisories. Availability-only DoS / resource exhaustion; remediation upgrade to ≥ 0.30.0 (ranges vary). Issue 23 covered only the Sep 22 NIXL/Mooncake batch — these Sep 23 GHSAs are the Issue 24 delta.

GHSA-85xf-c7hm-whqw (~6:16 AM ET) — Moderate; affected < 0.30.0 (confirmed path ≤0.25.1); patched ≥ 0.30.0. Three structured-output request paths raise uncaught, engine-fatal exceptions → _send_engine_dead(); one bad request DoSes all tenants on that EngineCore. Sites: latched auto/xgrammar compile; ngram_gpu -1 padding into guidance validate_tokens; Rust frontend empty structured_outputs.json/grammar. Credit: Patch the Planet (Trail of Bits + OpenAI); proposed public PR #51450.

GHSA-58v5-2m8f-94pr (~6:20 AM ET) — Moderate; affected ≥ 0.23.0rc2, < 0.30.0; patched ≥ 0.30.0. Request-level media_io_kwargs.video with video_backend: "glmga" + large fps/max_frames builds huge pre-decode index lists → CPU/memory exhaustion on shared media executor. Reachable without API key when none configured.

GHSA-5fj9-pfhr-6j48 (~6:23 AM ET) — Moderate; affected < v0.30.0 (Rust frontend / vllm-rs); patched ≥ 0.30.0. Prometheus middleware uses raw HTTP method tokens as labels → unbounded cardinality / OOM; /metrics outside API-key guard prefixes. Permanent series until process restart.

GHSA-x6mc-67gf-chw4 (~6:25 AM ET) — Moderate; affected ≥ 0.24.0 (through 0.29.1rc0 / cited main); patched ≥ 0.30.0. Qwen2-VL / Qwen3-VL samplers honor request max_frames/fps (ignore num_frames clamp from related GHSA-vxqj / PR #51969). Advisory cites unauthenticated POST /metrics RSS jump. No CVE IDs assigned on these four pages at research time — stick to GHSA ids.

NVIDIA NeMo Switchyard v0.3.0 (prior · Issue 23)

Published 2026-09-22T18:06:59Z ≈ 2:06 PM ET Sep 22. Advisor Gate / Composite routing / switchyard-server. Do not re-lead.