Local this window: Ollama v0.40.9 final (Oct 22, 10:12 PM ET) adds Phonon text-to-speech on Apple Silicon, d3 decision models, and Flint on Linux/Windows. llama.cpp b11819–b11836 (head Oct 23, 4:23 AM ET): DFlash2 probabilistic drafts, Nemotron-3.5-Embed fix, MiniCPM-OCR. No local MiniMax M3.1 Flash path in either yet.
rc2 (Oct 22 2:38 PM ET): “flint: fix replies with thinking disabled” (#19216). rc3 (6:51 PM ET): “Support GA Gradium Phonon checkpoints” (#19227) — older pre-GA Phonon checkpoints are rejected; 0.40.9 is the minimum. Final v0.40.9 02:12Z Oct 23 (≈10:12 PM ET Oct 22). Notes: speech “with Phonon on Apple Silicon”; ollama run “plays text or a chat model’s reply aloud as the audio is generated”; apps stream through the OpenAI-compatible /v1/audio/speech endpoint; ollama run phonon "Hello from Ollama.". Tags: phonon:latest = phonon:en = phonon:31m-en (digest 6667dd188d2e, 104 MB, Apache 2.0 upstream); de/fr/es/pt/multilingual = the 40M checkpoints (~123 MB; upstream CC BY-NC 4.0); 8K context. d3 “brings vLLM Semantic Router’s decision models to Ollama … classify, answer yes or no, or score options with probabilities” in 2B / 4B / 9B / 27B; example ollama run d3 "A customer says their card was charged twice." --ask "Is this a billing issue?". Flint “now runs on Linux and Windows as well as macOS.” Streaming OpenAI Responses and Anthropic Messages requests “now report generation failures instead of ending silently after partial output.” ollama launch passes cloud models’ context lengths and capabilities to coding tools. Anthropic-compatible web search uses the model’s supported thinking setting in follow-ups. minimax-m3.1-flash remains :cloud only (976K context listed).
PR #30928 opened Oct 21 14:48Z, merged 21:26Z (≈5:26 PM ET); release b11803 22:30Z (≈6:30 PM ET). Adds HF→GGUF conversion, tokenizer handling, sliding/full attention, routed + shared experts. Uses adaptive expert-activation scaling from #30861 (b11785) with a 1,024 threshold; PR: “Auto-scaling leads to slowdown of 10-30%, this will have to be addressed in follow-up PRs.” Chat template needs hexadecimal string escapes — b11798 (#30929, Oct 21 16:36Z). Validation: test-llama-archs on CPU/CUDA/Metal; finite logits BF16/NVFP4 on H200/B300, quantized on RTX PRO 6000, Q4_1 on M4 Pro. AI usage disclosed (“Codex substantially assisted”). GGUFs: RecursiveSI/flint-int4-gguf (flint-int4-q4_1.gguf ≈26.0 GB), RecursiveSI/flint-nvfp4-gguf; ggml-org/Flint-GGUF created Oct 22 12:16Z: Q4_K_M ≈23.4 GB, Q8_0 ≈42.4 GB, BF16 ≈79.6 GB. No Apple Silicon tok/s in the PR.
v0.40.8-rc0 Oct 21 14:45Z (“mlx: add Flint model support” #19181: preserve source tool JSON so Flint keeps numeric types and schema order; assemble ModelOpt expert banks on safetensors import; reject direct Flint NVFP4 creation before conversion; optional learned sinks + device causal masks in shared MLX attention). v0.40.8 final 21:46Z (≈5:46 PM ET): “Note: Flint is supported on MacOS. Other GPU support will be enabled in a subsequent release.” Notes repeat Recursive’s 54.9% Terminal-Bench 2.1 / 74.2% SWE-bench Verified (vendor-reported). Library: flint:latest = flint:40b = flint:40b-a2b (23 GB, 256K context, tools + thinking); flint:40b-a2b-gguf-nvfp4 (MLX tag). v0.40.9 01:33Z Oct 22 (≈9:33 PM ET Oct 21): llama.cpp update; rc0 switched to released WoA CUDA toolkit.
RecursiveSI/flint created Oct 21 14:20Z (≈10:20 AM ET), Apache 2.0, BF16 in 15 safetensors shards (“about 80 GB”), plus SGLang serving kit (one B300 or H200 per replica); “Transformers modeling code is not included yet.” Card: 39.8B total, ~2.5B active incl. output head (~2.2B body); 49 layers (2 dense + 47 MoE); 35 sliding-window (513-token window) + 14 full-attention layers; 32 Q / 4 KV heads, head dim 128; QK RMSNorm + learned attention sinks; RoPE on sliding layers only; 256 routed experts, 6 per token + shared expert; documented context 262,144; vocab 150,144; knowledge cutoff June 2026. Evals (vendor-reported): Terminal-Bench 2.1 54.9% (Terminus 2; retrieval-obtained solutions counted as failures), SWE-bench Pro v2 89.7%, SWE-bench Verified 74.2% (mini-swe-agent), pass@1 avg ≥5 attempts, sampled at temp 1.0 / top-p 0.95 / top-k 20. Blog: “more than 125 tokens per second on an M4 Pro MacBook Pro and more than 150 tokens per second on a 64GB NVIDIA RTX Spark at short contexts”; context “up to 512k… recommend staying within 256k”; Eureka Machine searched architecture (from a 35B-A3B baseline), quantized lm_head to 4-bit with offline self-distillation (NVFP4 “within 0.7 points” of 16-bit), and wrote a Metal megakernel fusing the decode pass on M4/M5. Contradiction: NVIDIA’s Oct 21 post said “over 220 tokens per second on a 64GB RTX Spark.” Blog lists llama.cpp, Ollama, Unsloth, LM Studio, vLLM, SGLang support (we verified llama.cpp, Ollama, mlx-vlm only).
Repo created Oct 21 15:35Z, Apache 2.0; v0.1.0 release 15:58Z (≈11:58 AM ET). Requirements: “Apple Silicon Mac with an M4 Pro, M5 Pro, or M5 Max,” ≥48 GB unified memory, ~30 GB disk, Node.js 22 for the pi harness. Install via curl … install.sh | bash (installs to ~/.local/bin); rec launch pi (preferred) / rec launch opencode. Claims 1.4–2.6× llama.cpp decode throughput across 1K–252K contexts on an M4 Pro (48 GB), 4-bit, prefill excluded (vendor-reported). “Optimized for single-session usage.” M1–M3, M4 Max, and Ultra chips are not listed — on an M3 Ultra Studio use Ollama or llama.cpp. We did not run the installer.
HF blog Oct 21 14:58Z (≈10:58 AM ET). ibm-granite/granite-speech-5.0-2b (Apache 2.0): ASR in EN/FR/DE/ES/PT/JA; translation to/from English plus EN→IT and EN→Mandarin; trained on 390,000 hours of public audio plus synthetic data; language-ID prompting; keyword biasing up to 2,000 terms; punctuation/capitalization on request (cap F1 89.7–99.5). “Support in Transformers, vLLM, llama.cpp, and mlx-audio for Apple Silicon.” IBM chart: #2 open-weight model on Open ASR Leaderboard English average WER 5.06% (IBM-presented, “as of 21 October 2026”). granite-speech-5.0-470m-turboctc: adds JA/ES/DE/FR/IT/PT, native punctuation, reverb robustness; 19,870 RTFx (from ~12,600; ~1.6×). 4.1 2B Plus: WDER −7%, cpWER −7%, timestamp shift halved, biased-keyword errors −45% (IBM-reported). Earlier English-only Turbo CTC covered Aug 26.
PR #30636 by ngxson (opened Oct 16, follow-up to #30629) merged Oct 20 22:04Z (≈6:04 PM ET); release b11780 Oct 21 04:11Z (≈12:11 AM ET). WebSocket /v1/realtime “Transcribes streamed audio, for models with a streaming audio encoder. Compatible with OAI /v1/realtime schema.” Input: PCM16 mono at the model sample rate (rate returned in session.created as audio.input.format.rate); client events session.update (accepted for compatibility), input_audio_buffer.append (base64 PCM16), input_audio_buffer.commit (flushes held-back words; "final": true ends input), input_audio_buffer.clear (accepted, cannot discard sent audio). Server events: conversation.item.input_audio_transcription.delta / .completed, input_audio_buffer.committed, error. Router mode selects the model from the model query parameter; PR includes a websocket proxy. Test GGUF: ggml-org/Voxtral-Mini-4B-Realtime-2602-GGUF (Apache-2.0 on HF; created Oct 16). PR discloses AI usage (“yes”). Transcription only — no speech-to-speech. No Apple Silicon latency figures published.
PR #26603 by ngxson (opened Aug 4, follow-up to #26254) merged Oct 20 23:19Z (≈7:19 PM ET); release b11781 Oct 21 04:36Z (≈12:36 AM ET); adds an mtmd audio-out stream API. Returns raw audio (audio/wav default). Options: input (alias prompt), lang, speaker_ref_b64 or multipart speaker_ref file (file wins), top_k/top_p for the acoustic code predictor, repeat_penalty (default 1.05), seed, n_predict (frames, default 512), response_formatwav|pcm (float32 LE mono; rate in Content-Type), stream (WAV header with unknown size). Router mode reports TTS capability. tools/tts README documents Qwen3-TTS (ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF; speaker reference “highly recommended”) and Kyutai Pocket TTS (reference required). Not OpenAI-compatible yet: “Support OAI-compat TTS endpoint” and --tts-speaker-dir listed as next PRs. PR discloses “most code written by AI, based on my plan.”
Published Oct 21 04:01Z (≈12:01 AM ET; rc1 Oct 20 5:15 PM ET deduplicated MLX CUDA runtime payloads). Adds pplx-decider (27B decision model from Perplexity, fine-tuned from Qwen3.8-27B). Responses API: continuing a conversation no longer fails when switching between thinking and non-thinking models or when history contains another provider’s compaction items; streaming fix where client tool calls (e.g., Codex with web search) emitted before open reasoning items. Remote-model requests cancelled on ollama.com on client disconnect. ollama show + API gain renderer, parser, template_source. llama-server infers draft type for speculative decoding. ollama launch claude clamps CLAUDE_CODE_AUTO_COMPACT_WINDOW. Updated llama.cpp and MLX.
NVIDIA blog Oct 21 10:50Z (≈6:50 AM ET). DGX Spark: clusters up to eight units in manual configuration (“1TB of unified memory”); NVIDIA Sync supports eight Sparks; Sync Model Launcher “later this month” (downloads model + container, connects to OpenCode, loads DGX Spark Skills). DGX Station: 748GB coherent memory, up to 20 PF; Windows version in December; NVIDIA Cloud Functions early access. Recursive Flint: 40B MoE, 2.5B active, Apache 2.0, “over 220 tokens per second on a 64GB RTX Spark,” claimed best-in-class on SWE-Bench Pro v2 / Terminal-Bench 2.1 / SWE-bench Verified; kernels from Recursive’s “Eureka Machine” +55% generation speed — all vendor-reported. HF org RecursiveSI showed “None public yet” at ~8:40 AM ET. FP4-Local checkpoints: nvidia/Qwen3.8-Flash-Next-FP4-Local (single DGX Spark 128GB or RTX PRO 6000), nvidia/GLM-5.3-Flash-FP4-Local (two systems); HF license tag “other”. llama.cpp MTP speculative decoding for Qwen 3.8 Flash Next “up to 1.4x” on RTX PRO 6000; vLLM spec decoding + FlashInfer prefill for Qwen3.8 27B “up to 1.3x” on DGX Spark (NVIDIA-reported). Same post calls MiniMax M3.1 Flash “a new open model” — no weights on HF at check.
PR #28277 opened Sep 2, merged Oct 19 22:23Z (≈6:23 PM ET); release b11744 22:54Z (≈6:54 PM ET). New flags: --connect (env LLAMA_ARG_CONNECT, default disabled; “(experimental) open a peer-to-peer tunnel for server using llama-connect”) and --connect-code CODE (env LLAMA_ARG_CONNECT_CODE; 40 characters = 8-char room code + 32-char pass code from a 54-character alphabet; generated fresh each run from std::random_device unless pinned). Per the PR: WebRTC peer-to-peer, end-to-end encrypted (as WebRTC requires), WebTorrent trackers used only for signaling, no central server. Needs the separate Rust llama-connect binary from ggml-org/llama-connect (a CMake script downloads the prebuilt release asset; no macOS x64 build yet). Server dials llama-connect on the bound port (works with --port 0), maps wildcard hosts to loopback, refuses --connect with only unix sockets, and no longer advertises a code in /props without the tunnel. Related web UI work: #24577. PR discloses “90% of the code is AI-generated.” Our advice (not the project’s): also set --api-key; the README lists it alongside.
No new release since v0.40.7-rc0 (Oct 18 00:28Z ≈ 8:28 PM ET Oct 17). Library “newest”: minimax-m3.1-flash — cloud only (ollama run minimax-m3.1-flash:cloud), 428B parameters, 976K context, vision/tools/thinking; $0.15 in / $0.015 cached / $0.60 out per 1M tokens, free until Oct 22 23:59 UTC (7:59 PM ET); “updated 20 hours ago” at our check. See OSS models page for the weights question.
Release b11671 at Oct 16 00:18Z (≈8:18 PM ET Oct 15). Latent-space MoE (residual 8192, experts 4096 with down/up projection), top-8 sigmoid routing with correction bias; attention interleaves three 2048-window sliding layers with RoPE and one global layer without RoPE, sigmoid output gate, weightless QK-norm. Vision reuses the Muse Glimmer tower; audio is a causal conv pyramid (no spectrogram), one embedding per 80 ms at 24 kHz. Quantizer keeps latent projections and attention gate at Q8_0 (~1% of weights) because low-bit types caused repeating-loop output. Use builds ≥ b11671. Head at check time b11696 (Oct 16 11:26Z).
Stable release created Oct 15 16:11Z (≈12:11 PM ET) (rc0 tag Oct 14). Adds nemotron-3.5-super (120B hybrid Mamba-2/Transformer MoE, 12B active, C-RADIO vision, tool calling, thinking on/off, up to 1M context) and nemotron-3.5-bolt (74B / 9B active, 24 GB NVIDIA target, included drafter). Decision models on cloud: clef:cloud, clef-flash:cloud. Fix: continuing an OpenAI-compatible Responses conversation with custom tool calls. Clearer cloud-access errors. Bumps llama.cpp (Nemotron 3.5) and MLX. Library: muse-spark-open is listed with a cloud tag only, $1.25 input / $0.15 cached / $4.25 output per 1M tokens; run with ollama run muse-spark-open:cloud.
Merged Oct 15, 2:39 AM ET; release b11645 (7:56 AM ET). BF16 variant of the tensor-API FLASH_ATTN_EXT kernel, used with -ctk bf16 -ctv bf16 under the same thresholds as F16; scores/output still accumulated in float. Kernels move to kernels/fa_tensor.metal (built into ggml-tensor.metallib). Contributor M5 Pro (20-core GPU): FLASH_ATTN_EXT BF16 K/V 1.7–2.5× at kv 4096/32768; Qwen3.8-27B Q8_0 pp512@d32768 184.2 → 265.6 t/s (F16 KV 268.5); KLD@32K BF16 4.05e-4 → 5e-6 — contributor-reported.
b11642 (#30498 merged Oct 15, 12:30 AM ET): rerank and embedding slots are cleared on release instead of filling the RAM prompt cache (--cache-ram default 8 GiB); infill and decision tasks can restore from cache; decision models disable the prompt cache by default (re-enable with explicit --cache-ram). PR test: 40 rerank calls on Qwen3-Reranker-0.6B grow host memory 14 MiB instead of 4.8 GiB. b11641 (#30506, merged Oct 14, 11:49 PM ET): LiliCorr speculative decoding (NVIDIA paper, arXiv 2608.20530) re-ranks each DFlash slot’s top-k candidates with a small transformer; new LiliCorrDraftModel conversion; still served with --spec-type draft-dflash. Contributor reports acceptance length similar to the vLLM implementation on an RTX 5090 (SGLang and vLLM PRs are the references). Head b11647 (8:21 AM ET).
Ollama v0.40.5-rc0 published Oct 14, 3:45 PM ET: MLX Metal custom kernels ported to CUDA (#18657), Responses API accepts custom tool calls in history (#18911), MLX version bump. Stable remains v0.40.4. NVIDIA’s Oct 13 blog says Nemotron 3.5 Bolt 74B-A9B 24GB (2-bit QAT) is available Thu Oct 15 via llama.cpp, vLLM, Ollama, Unsloth, LM Studio; not listed on huggingface.co/nvidia or the Ollama library as of 8:40 AM ET. Watch only.
Stable release after Issue 37’s rc1 watch. Decision models via CLI: ollama run <model> --ask "…". Kev decision models on MLX runner (Apple Silicon). Multimodal embeddings through llama-server; small-context multimodal embedding load fix. ollama launch with usage credits without paid plan; Codex launch includes local models in picker. MLX: clip long Laya states; speculative decoding / recurrent state after rollback; Gemma 4 prefill on NVIDIA with MLX runner. llama.cpp updated with native decision-model support.
Published Oct 13, 2026. RTX Spark laptops available Friday Oct 16. PAIR leaves beta with llama.cpp support, Hugging Face downloads, RTX Spark support. Agent apps (Copilot, Hermes, OpenClaw, Perplexity, Pi, WorkBuddy) with OpenShell + MXC sandboxing. Nemotron 3.5 Bolt 74B-A9B 24GB due Oct 15 (llama.cpp / vLLM / Ollama / Unsloth / LM Studio claimed). Broadcast 2.3 MCP. llama.cpp decision-model support + Llama Windows app noted. Performance multipliers in post are NVIDIA/partner-reported.
Merged in b11608 (Oct 14). server_decision_escape_special_tokens() replaces std::regex_replace for d1-omni / kev prompts escaping <|name|>; libstdc++ recursion on long names overflowed the HTTP thread stack (SIGSEGV). Also b11604: KleidiAI SME2 Q4_K and Q6_K matmul (#29219). Head b11612. Metal large-tensor fix already Issue 37 (b11575).
PR merged 2026-10-12T19:57:27Z; tagged in build b11575. Fixes Metal tensor kernel_mul_mm errors for large tensors. Head builds continued through b11583 (Oct 13). GA tag still v0.6.0.
Published 2026-10-11T01:39:04Z. Changes: ollama launch claude no longer warns about Claude connectors (#18878); ollama launch codex no longer warns about service tiers (#18879); “Paused background model upgrade for embedding and other request performance” (#18908). Context from 0.40.2: background upgrades kept original model backups on disk (double space until cleanup).
PR #29600 merged 2026-10-11T12:39:17Z; first tagged in build b11556. Runtime support for Prism Bonsai 2 27B in stock llama.cpp. Use standard Q2_0 group-64 GGUF (*-Q2_0_g64.gguf); heavier Prism-specific quants may still need Prism’s fork (per Prism docs / community notes — verify against Prism’s own format docs before asserting). Head builds continued through b11567 (Oct 12) with unrelated Vulkan/SYCL/OpenCL work. GA tag still v0.6.0.
#30159 “server: use port 9931 by default” merged 2026-10-09T08:40Z ≈ 4:40 AM ET, released as b11521 (09:57Z ≈ 5:57 AM ET). Refs #26116 / #26508. “For docker, we still use 8080 inside the container to avoid breaking any tools using our base image.” Anything that assumed localhost:8080 (scripts, proxies, IDE plugins, launchd/systemd units) should pass --port. #29583 “ui : add the models manager” merged 08:34Z: one table of every model the server can serve (expandable quants; loaded/downloading/favorites/local/hidden sections; filters), load/unload, delete from disk, hide, pause/resume downloads, per-model config drawer; reasoning effort becomes its own control. Follow-ups open in #30228. #30209 “metal: add the 128/96 flash attention kernels” (b11518, 09:29Z): Metal claimed support for K=128/V=96 but had no kernel and crashed on new tests; now supports_op accepts exactly K==V plus 96/64, 128/96, 192/128, 320/256, 576/512; tested on an M5 Max, 4973/4973 FLASH_ATTN_EXT passes; “No model uses 128/96 yet.” Also overnight: CUDA radix top-k (b11513), DFlash output-head sharing fix (b11512), convert support for compressed-tensor mixed-precision NVFP4 checkpoints (master 11:54Z, untagged at check time). /releases/latest still v0.6.0.
Published 2026-10-09T03:37Z ≈ 11:37 PM ET Oct 8 (rc0 15:54Z Oct 8). Release notes: “Models downloaded with earlier versions of Ollama are upgraded in the background the first time you run them, for better performance and compatibility when running on llama.cpp. To make downgrading safe, Ollama keeps the original copy as a backup… A future release will remove these backups automatically.” The notes give a jq pipeline over /api/tags and /api/show that removes manifests where runner == "ggml" for models that also have a llamacpp manifest; downgrading below 0.40 afterwards means re-pulling. Changes: ollama list hides duplicate entries and downgrade guards (#18874); ollama launch claude sets CLAUDE_CODE_MAX_CONTEXT_TOKENS to the model’s context length and respects CLAUDE_CODE_AUTO_COMPACT_WINDOW (#18855); oxi added to community integrations (#18739). Main after the tag: avoid inherited service-tier warnings in Codex launch (#18879). No performance claims.
Windows Developer Blog Oct 7 (18:00Z ≈ 2 PM ET). Policy-driven execution layer for untrusted or generated workloads: contain model output, plugins, tools, the harness, or the whole agent. Policy (isolation backend; command/args/env; writable, read-only and blocked paths; inbound/outbound network incl. loopback; desktop/UI access) stays outside the agent’s control. Unified JSON schema and multi-language SDK. Backends: process container on Windows 11, macOS and Linux (AppContainer / Seatbelt / Bubblewrap); session container (Windows only: separate account, desktop, clipboard, input); WSLc; microVM (Windows 11 and Linux, experimental). Windows 365 support GA. Entra agent identity and Agent 365 controls for local agents “soon”. Supporting now: GitHub Copilot, OpenClaw, OpenAI Codex, Replit, LM Studio, Unsloth (the Experience Blog also lists NVIDIA OpenShell); coming: Claude Code, Box, Egnyte, Heidi Health, Hermes Agent, Manus, Perplexity, Raycast, Simular. “Each containment backend has distinct security properties.”
NVIDIA blog Oct 7 (18:45Z ≈ 2:45 PM ET). RTX Spark: Blackwell RTX GPU with up to 6,144 cores plus an up-to-20-core Grace CPU at 600 GB/s; up to 128GB unified memory; “one petaflop” of FP4. Laptop preorders opened Oct 7 and ship Oct 16; compact desktops in November. OEMs: Acer, ASUS, Dell, HP, Lenovo, Microsoft (Surface Laptop Ultra), MSI, Gigabyte. NVIDIA says Qwen 3.8 Flash Next (125B) “matches the intelligence of many cloud models” (NVIDIA claim). DGX Station for Windows (preview, no date): GB300, 748GB coherent memory, up to 20 PF FP4, “up to a trillion-parameter scale” (NVIDIA claim). Windows blog: MAI Code 1.1 Flash (137B total / 6.8B active) at 3-bit with a 256K local context; an upcoming 70B+ Nemotron at 2-bit in “just over 20GB”; DeepSeek V4 Flash (284B) on RTX Spark; Copilot hybrid local features on Copilot+ PCs “in the coming months”. RTX Spark itself was introduced in June; the delta is ship dates and the Windows software.
Released 2026-10-08T02:09Z ≈ 10:09 PM ET Oct 7 (rc0 6:22 PM ET). Changes: proxy cloud usage and balance APIs (#18829); fix Clef head reads past 2 GiB on Windows (#18777); remove the account step from CLI onboarding (#18826); avoid manifest symlinks on Windows (#18852); docs link fixes; “mlx: drop carried metal residency patch now that it is upstream” (#18854). No performance claims. Mac users: update and re-check memory pressure on long contexts.
Oct 8 tips: b11495 “model : support classifier_activation for rerankers (#29692)” (maps classifier GELU to gelu_erf, accepts tanh; ModernBERT falls back to gelu_erf), 5:44 AM ET; b11490 Hexagon alloc_buffer_n (#30126); head b11500. /releases/latest is still v0.6.0.
Release v0.40.0, not pre-release, 17 assets. The tag was created 2026-10-06T17:46Z ≈ 1:46 PM ET Tue Oct 6, and the releases feed shows the entry finalized at 18:47Z ≈ 2:47 PM ET. The GitHub API reports published_at 2026-09-25T03:31Z, which looks like the draft-creation date; rc5 was still pre-release at 9:16 PM ET Oct 5 and rc6 shipped at 11:08 AM ET Oct 6. Notes: “on Apple Silicon devices, model architectures supported by the MLX runtime will automatically run on MLX”, naming qwen3.8, gemma4, qwen3.6, qwen3.5. Decision models on MLX: Nimble, tev1, clef, clef-flash. “MLX now has support for an embedding model: embeddinggemma-2.” rc6 (#18820) implements the EmbeddingGemma2Model architecture on the MLX runner (24-layer bidirectional text encoder with PLE, shared gemma4 vision/audio towers, mean-pool + L2) and lets /api/embed accept per-item media through input dicts. Full changelog v0.35.1…v0.40.0. The notes give no performance figures and do not describe behavior for architectures MLX does not support. Measure tok/s yourself.
Library page updated Oct 6. Tags: 270m 378 MB (text), 440m 714 MB (text+image), 570m 990 MB (text), 740m / latest 1.3 GB (text+image). ollama pull embeddinggemma-2; embed via POST /api/embed. Contradiction: the library lists a “256K context window” for every tag, while Google’s model card says 8,192 tokens. Use the card’s figure until Ollama or Google reconciles them. Per-tag rows list text or text+image inputs; the page’s capability chips also show audio.
Notable tips Oct 6–7: b11446 “metal : fix excess threadgroup memory in quantized flash attention (#29340)” (1:31 PM ET Oct 6); b11447 “models: support pplx-decider (#30044)”; b11450 “RPC: add -sm tensor (#26610)” (includes an Apple RDMA flush fix; RPC major version bumped); b11454 K2 Horizon dense and MoVA support (#29535); b11455 PLaMo-3 tokenizer pre-segmentation (#30045). Head around b11468 Wed morning. /releases/latest is still v0.6.0. EmbeddingGemma 2 GGUFs from ggml-org run with llama serve -hf ggml-org/embeddinggemma-2-GGUF; see OSS models.
Release published 2026-10-05T16:56Z ≈ 12:56 PM ET Mon Oct 5, marked Latest; nightly build b11429; version bump commit d812350 (#29997). Highlights: llama_batch_ext extended batch API with llama_process() for mixed token/embedding batches and per-token state embeddings for MTP and deepstack models (#24669); GLM-5.3-Flash (GLM5-Next), a 320B KDA/DSA hybrid text+vision model with mHC and MoE (#27773); Clef decision model with text and vision (#29831, #29969); Qwen4Exp high-quality support with MTP speculative decoding — “~1.5x decode speedup on DGX Spark” (#29761, #29751; project-reported); /v1/systemone for laya, julia-1, lev, openjev (+vision), kev (#29818), extended to nimble (#29844); Metal tensor-API flash attention for F16 KV (#29570); Metal few-row MMA mat-mul for speculative and batched decoding — “up to ~3x faster mat-mul on Apple GPUs” (#29869; project-reported); llama_prefetch_rows() MADVISE prefetch of PLE tensors in Qwen4Exp and Gemma4 (#29599). Also Ling 3.0 VL (#29151), LFM2.5-Encoder-230M/350M (#29862), reranker classifier_pooling (#29627). API changes: new llama_get_causal_attn() (#28876); session formats bumped to LLAMA_SESSION_VERSION 11 and LLAMA_STATE_SEQ_VERSION 4; mtmd_get_memory_usage() now returns an mtmd_memory_usage struct (#29773). Server:GET /v1/models and /models report input/output modalities in a new architecture object (#29987); /v1/embeddings accepts typed vision/audio/video (#29556) and returns HTTP 400 for invalid requests (#29060); partial media truncation rejected (#24076); laya abort fixed by limiting n_batch to n_ubatch (#29903). Speculative decoding: probabilistic sampling for simple draft and MTP (#27694); n-gram drafts at temp > 0 fixed (#29924); stop accepting drafts at EOG (#29638). UI: model download pipeline (#27959), HF Hub data layer (#27947), memory-fit estimation (#27957). ggml v0.26.0: sparse flash attention on SYCL, Vulkan and Metal; CUDA W4A4 (NVFP4/MXFP4) mul_mat path; Windows ARM64 MSVC builds; stricter GGUF size validation. Tips continued after GA through b11439 (~3:59 AM ET Tue Oct 6; includes a k-pool scatter data-race fix at b11435). Those are pre-release, not GA.
Repo created 2026-10-04T13:06Z ≈ 9:06 AM ET Sun Oct 4 under google/; the card states “This is not an officially supported Google product.” Fine-tuned from Gemma 4 E4B with a LoRA (rank 256) merged into the weights. Input is the ASR transcript with speaker tokens, and output is a corrected transcript. Prompt format <|turn>user\n{hyp} --> <turn|>\n<|turn>model\n; card llama.cpp example uses --temp 0.0 -n 128. Files: GGUF Q4_K_M ~5.30 GB and Q4_0 ~5.15 GB for llama.cpp / Ollama. Card-reported WDER vs the original DiarizationLM-8b: Fisher 5.32 → 2.99 (p<0.0001); improvements also on Callhome, ICSI, AMI (author-reported).
rc4 published 2026-10-06T00:41Z ≈ 8:41 PM ET Mon Oct 5 (MLX version bump). rc5 published 01:16Z ≈ 9:16 PM ET, titled “v0.40.0” but flagged Pre-release. Notes unchanged from rc3: MLX is the default engine on Apple Silicon for qwen3.8, gemma4, qwen3.6, qwen3.5; decision models Nimble, tev1, clef, clef-flash run on MLX. Recent main-branch commits: “mlx: mitigate high latency after GPU idle (#18807)”, “Reduce model lookup and MLX decision request overhead (#18806)”, “llama.cpp: version update (#18761)”, “mlx: fix patch for recent mlx update (#18812)”. GA remains v0.35.1. Do not present 0.40 as GA.
Tip published 2026-10-05T06:06:22Z ≈ 2:06 AM ET Mon Oct 5 (pre-release). metal : few-row MMA mat-mul (#29869). Note: speculative decoding verifies a few draft tokens per step; without the tensor API, Metal ran these mat-muls with mat-vec kernels whose time grows with every src1 row, “so DFlash2 decoding on an M3 Ultra was slower than serial decoding.” Adds mat-mul kernels for 2..16 src1 rows on 8×8 simdgroup matrices (each weight dequantized once for all rows; simdgroups split K). Q4_0, Q8_0, Q5_K get dedicated kernels; F32, F16, Q4_1, Q5_0, Q5_1, Q4_K, Q6_K use a generic path. Used only on MTLGPUFamilyApple7+without the tensor API, from the row count at which they beat mat-vec on an M3 Ultra (F32: 6; F16, Q4_K, Q5_0, Q5_1: 3; others: 2). MUL_MAT + ADD residual fusion in the MMA store; batched-copy fusion was removed in review. Tests added in test-backend-ops. macOS arm64 assets. Tip head now b11417 (~7:43 AM ET, CUDA NVFP4 mmq). GA still v0.5.0.
Tip published 2026-10-03T00:26:08Z ≈ 8:26 PM ET Fri Oct 2. metal : add tensor API flash attention kernel for F16 KV (#29570); continuation commits add tensor FA kernels for DK=DV=512, DK=576/DV=512 and DK=192/DV=128, and support attention sinks, ALiBi and logit softcap in the tensor FA kernel. Applies to Apple GPUs that expose the Metal tensor API (b11404 covers the others). No benchmarks in the note — measure locally.
b11361 published 2026-10-02T23:53:18Z ≈ 7:53 PM ET Fri Oct 2: llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) (#29818) — conversion script, server code, shared prompt-prefix support, vision support, openjev tiny test model, docs; notes clarify date_facts is not supported. b11364 (≈9:03 PM ET Oct 2): model: support nimble decision model (#29844). b11371 (2026-10-03T08:34:10Z ≈ 4:34 AM ET Sat Oct 3): model: add support for clef decision model (text-only) (#29831) — Clef vision path not included. Tip builds; GA still v0.5.0.
Release marked Latest (non-pre-release). Assets uploaded 2026-10-02T17:10Z ≈ 1:10 PM ET Fri Oct 2; release updated 18:28Z ≈ 2:28 PM ET (API published_at reads 2026-09-29T20:14Z from the draft — use the Oct 2 asset time). Issue 30 (morning Oct 2) correctly said no GA yet. Notes: supports Clef (27B) and Clef Flash (9B) through /v1/systemone; multimodal — requests can include images alongside the text state, shared by all questions and scored jointly. Web search up to ten searches per response (from three); Modelfile CAPABILITY declarations preserved through GGUF/safetensors create, inheritance and export; ollama show and the model list report only decision for decision models so clients stop offering them for chat/tools/thinking; llama.cpp + MLX engine updated. Changelog v0.35.0...v0.35.1.
rc1 2026-10-04T14:45Z ≈ 10:45 AM ET Sun Oct 4 — mlx: match publisher tokenizer semantics (#18779) (pretokenizer order, split behavior, Unicode boundaries, added-token normalization, ranked BPE merges). rc2 ≈ 11:48 AM ET (merge). rc3 updated 2026-10-05T00:17Z ≈ 8:17 PM ET Sun Oct 4; release title reads “v0.40.0” but tag is v0.40.0-rc3 and it is marked Pre-release. Notes: models supported by the MLX runner run on MLX by default on Apple Silicon (qwen3.8, gemma4, qwen3.6, qwen3.5); decision models on MLX: Nimble, tev1, clef, clef-flash. Prefer v0.35.1 for pins until non-RC 0.40.
NVIDIA blog Oct 2, 2026 (13:00Z ≈ 9:00 AM ET, after Issue 30 shipped). 64GB unified-memory DGX Spark available exclusively from Acer, ASUS, Dell, Gigabyte, HP and MSI on Fri Oct 23, starting at $4,999. Same GB10 Grace Blackwell Superchip, DGX OS and software stack as the 128GB model; “supports up to 100-billion-parameter models.” Two units connect via ConnectX-7 (QSFP) to pool 128GB / up to 200B params; NVIDIA’s Qwen 3.8 27B test: up to 1.7× vs one system (vendor). NVIDIA Sync Cluster Assistant configures the link; Sync Model Launcher (Qwen3.8 27B + OpenCode) “coming at the end of the month.” Supported runtimes listed: llama.cpp, Ollama, vLLM, LM Studio.
Tip releases Fri Oct 2, 2026 (ET). After Issue 29 tip head b11320: b11334 — metal : release temporary private transfer buffers (#29777); macOS arm64 assets. Tip head b11344 (~4:54 AM ET) — CUDA: fix 2 broken Volta FA cases (#29803). Same cascade: Hexagon shared strided DMA (b11338); WebGPU bf16 MUL_MAT (b11333); Qwen4Exp MTP (b11330). Tip / pre-release — GA still v0.5.0. Do not rehash b11316 mxfp4 as today’s lead.
GitHub pre-release Tue Sep 29, 2026. Issue 29 still listed v0.35.1-rc0; tip advanced to rc2. Notes: allow ten web searches per response; MLX version bump; llama.cpp bump to b11232; create supports explicit model capabilities. Still prerelease — no GA 0.35.1 tag. Do not re-lead v0.35.0 Decision Models or v0.40.0-rc0 as new.
Tip head b11320 published 2026-10-01T11:20:29Z ≈ 7:20 AM ET Oct 1. Apple-relevant b11316 (2026-10-01T08:58:32Z ≈ 4:58 AM ET): metal : use bf16 math for mxfp4 mul-mat (#29770); macOS arm64 assets published. Same-day cluster also includes b11313 (model : re-enable -sm tensor for qwen4exp #28569) and narrower tips (LLM-jp-4.1 Harmony, PLaMo vocab, WebGPU SSM_SCAN, CUDA iq4_nl, Hexagon ALLREDUCE). Tip / pre-release — GA still v0.5.0. Do not rehash Issue 28 b11279 GLM-5.3-Flash as today’s lead. Superseded tip note: Ollama now v0.35.1-rc2 (still no GA 0.35.1).
Published 2026-09-28 (~21:23 UTC ≈ 5:23 PM ET; prerelease). Adds decision models through /v1/systemone, based on TypeSafe’s Jev API. Models return choices, probabilities, and scores (not free-form text) for triage, routing, and classification. Launch models: Nimble (Bespoke Labs), Tev1 (Together AI); ollama pull nimble. Question types: choice, noul, score. Also: settings open without waiting for model discovery; macOS update-menu fix; stalled MLX model downloads no longer hang indefinitely; deprecated typical_p now warns instead of failing. Pre-release — prefer GA v0.34.4 for production pins. Do not re-lead v0.40.0-rc0 as the decision-models story.
Hugging Face blog 2026-09-22. Load GGUF via AutoModelForCausalLM.from_pretrained(..., gguf_file=...) with packed Metal kernels from the kernels library (ggml-quantization, ggml-norm, ggml-attn, ggml-gated-delta-net). Initial focus: interactive, unpadded generation on Apple Silicon; architecture coverage starts at Qwen3.5 dense/MoE. transformers serve OpenAI-compatible endpoint. Blog states llama.cpp remains the recommended dedicated local engine; this path is for Python/PyTorch workflows, eval, and conversion validation. Requires transformers main (until next release) + compatible kernels + recent PyTorch. Throughput charts vs llama.cpp are HF-measured on M2 Max 32 GB (Transformers includes prefill; llama-bench column decode-only — not identical conditions).
Published ~2026-09-28 22:26 UTC ≈ 6:26 PM ET. Server supports typed content (vision/audio/video) input on /v1/embeddings (Qwen3-VL-Embedding OpenAI-style content arrays; KV prefix reuse disabled for embedding/rerank). Tip signal only — do not re-lead Issue 26 b11228 Metal PAD. GA still v0.5.0.
Published 2026-09-28T11:06:50Z ≈ 7:06 AM ET Sep 28 (pre-release tip). metal: support left and circular padding in GGML_OP_PAD (#29561). Aligns Metal with CPU, CUDA, and Vulkan: shifts source coordinates by left paddings, wraps with the same wrap_around when circular, reads through nb00 (also fixes right padding of a permuted source); test coverage added. Drops slower/failing f32_4 pad kernel path; circular pad uses Metal function constant (FC_PAD). macOS Apple Silicon arm64 binary among assets. GA still v0.5.0.
Published 2026-09-28T09:29:57Z ≈ 5:29 AM ET Sep 28. context : do not re-reserve the scheduler when toggling causal_attn (#28751). For vision inputs, Gemma models flip causal_attn twice around each non-causal image chunk; prior path forced two expensive sched_reserve() passes per image. Re-reserve unnecessary because the flag only changes KQ mask values, not tensor shapes/buffer sizes. Notes include llama-server benches on gemma-4-26B-A4B (CUDA GPUs — not Apple Silicon-measured). Tip cluster also includes b11184 Metal FWHT widths >512 (~1:57 PM ET Sep 25).
Published 2026-09-25T03:31:52Z ≈ 11:31 PM ET Sep 24 (prerelease=yes). Headline: model architectures supported by the MLX runner run on MLX by default on Apple Silicon. Example in notes: ollama pull qwen3.8 / ollama run qwen3.8. Changelog span v0.34.4...v0.40.0-rc0. Darwin assets: ollama-darwin.tgz, Ollama-darwin.zip, Ollama.dmg. Notes: additional models tested/enabled during RC — do not claim universal architecture coverage. Release list title may say “v0.40.0” but tag is v0.40.0-rc0. Prefer v0.34.4 for production pins until non-RC 0.40.
Newest tip b11178 published 2026-09-25T11:16:00Z ≈ 7:16 AM ET Sep 25 — MUSA/PH1 (MTT S5000) operator/build fixes (#29193), not Metal. Apple Silicon operators: b11173 (~8:06 PM ET Sep 24) Metal graph capture / empty graphs (#29390); b11172 (~7:35 PM ET Sep 24) Metal sparse FlashAttention optimize (#29377). GA still v0.5.0 (Issue 24). Tip advanced past Issue 24’s b11158.
Published 2026-09-23T20:50:06Z ≈ 4:50 PM ET Sep 23 (prerelease=false). Numbered GA after Issue 23 tip-only coverage. Highlights: CUDA conv2d implicit GEMM (#29135); Metal MoE and SSM_CONV fusion (#28948); server bind to multiple addresses (#28690); input_image on function-call outputs (#22575); --host comma-separated TCP + UNIX sockets; HRM-Text / DFM Mimir 1B (#27625); MiMo-V2.6 convert (#29257); DFlash HunyuanOCR (#28890); ggml 0.25.0; router eviction/lifecycle fixes; cpp-httplib 0.57.1; RPC protocol major v7. Assets note nightly build b11146. Cite convert/support only for MiMo/HunyuanOCR — do not re-lead those models. KleidiAI macOS asset still DISABLED on nightlies. Prefer v0.5.0 for pin-able installs vs tip nightlies.
Tag published 2026-09-23T02:24:43Z ≈ 10:24 PM ET Sep 22 (prerelease=false); darwin/install assets updated_at ~2026-09-24T04:18Z ≈ 12:18 AM ET Sep 24. Issue 23 shipped as v0.34.4-rc0; this is GA. Release body (unchanged from RC): structured outputs on thinking models in a single pass; fixed intermittent “model not found” with a large local library; fixed macOS app unresponsive when checking ChatGPT/Codex; Qwen 3.8 prompt processing faster on Apple Silicon; Gemma 4 on Apple Silicon picks best image resolution per image; updated llama.cpp, MLX, and XGrammar. Prefer GA URL over -rc0. Do not rehash v0.34.0–0.34.3 features.
v1.5.1 published 2026-09-23T16:58:34Z ≈ 12:58 PM ET Sep 23. Patch: CMSIS-NN to v8.0.0 + aligned Arm runner deps; public get_thread_count(); expanded Arm support for symbolic shapes and dynamic convolution/pooling; folded redundant Q/DQ pairs in shared EXIR to_edge path. v1.5.0 published 2026-09-16T16:08:14Z ≈ 12:08 PM ET Sep 16 (missed as quiet in Issue 23): Python wheels expanded into a C++ SDK with linkable CUDA, Core ML, MLX, OpenVINO, Qualcomm delegates + TorchAO kernels; LLM serving multi-method export / batched scheduling / bounded cancellation / off-graph KV-cache; Core ML/MLX linkable libs + SwiftPM; ETDump; MLX off-graph KV flat/ring/cell; fused attention; Metal addmm / split_copy. Attribute feature bullets to 1.5.0, patch to 1.5.1. Not a substitute for mlx-vlm GA.
Tip b11158 published 2026-09-24T11:40:xxZ ≈ 7:40 AM ET Sep 24 (pre-release). Headline: vulkan: tune KHR cooperative matrix support for Adreno GPUs (#29328). b11156 (~4:08 AM ET Sep 24): model Ling 3.0 VL folded into BailingMoeV3 (#29151). After Issue 23 tip b11126: also notable b11136 OpenAI video_url / data: video URIs; b11132 Gemma4 DSpark draft. Prefer v0.5.0 unless tip features are required. Do not rehash b11126 as tip.
Published 2026-09-23T02:24:43Z ≈ 10:24 PM ET Sep 22. Pre-release at Issue 23 time; superseded by v0.34.4 GA above. Release body matched GA notes (MLX Qwen 3.8 PP #18550; Gemma 4 dynamic image resolution #18603).
Tip b11126 published 2026-09-23T12:01:xxZ ≈ 8:01 AM ET Sep 23 (pre-release nightly). Headline PR: vulkan: add IQ4_XS MMQ/MMV matmul kernels (#28415). Superseded as tip by b11158; do not re-lead.
