MLX MiniMax M3.1 Flash on MLX (lead): mlx-lm #2003 + mlx-community Text 3/4-bit + oQ builds · MLX quiet
Apple Silicon this window: mlx-lm #2003 (load the original MiniMax M3.1 checkpoint) merged early Oct 23 ET; mlx-community posted MiniMax-M3.1-Flash-Text-3bit / -Text-4bit and oQ3e / oQ3.5e / oQ4e-mtp builds overnight. MLX 0.32.4 / mlx-lm 0.32.2 still the latest tagged releases.
Opened by jundot (implementation credited to pcuenca), approved and merged by zcbenz on Oct 23 (early morning ET). Loads MiniMax’s native-precision files directly: routed experts NVFP4 with a per-expert global scale, attention / shared expert / dense MLP MXFP8. “The router stays float32 on convert, like the checkpoint. With a bf16 router about 1% of tokens per layer pick a different top-4 set.” “No vision yet.” PR verification: mlx_lm.generate --model MiniMaxAI/MiniMax-M3.1-Flash … -m 1024 → prompt 199 tokens at 104.7 tok/s, generation 858 tokens at 23.8 tok/s, peak memory 252.3 GB. Hardware not stated; one run; project-reported, not ours. Not in a tagged release: PyPI mlx-lm is still 0.32.2 (Oct 15 ET). To use it on the original checkpoint, install from main.
Both created Oct 23 ~01:43Z (≈9:43 PM ET Oct 22; note this is before the 03:37Z timestamp on MiniMax’s single “Init” commit — HF shows last-modified time, so the weights may have gone public earlier than the commit date suggests). Both cards: “This is a text-only conversion. The vision tower is not included.” “Tested with mlx-lm 0.32.2.” Text-4bit: “keep the original quantization” (NVFP4 experts with per-expert global scale; MXFP8 attention/shared/dense; router float32); ≈248.4 GB; peak ≈249 GB in a short test — does not realistically fit 256 GB with macOS and KV cache. Text-3bit: “dequantized and then quantized to 3-bit affine with group size 64 (3.5 bits per weight). The router is kept at 8 bits.” ≈186.5 GB; peak ≈187 GB in a short generation test — the 256 GB Studio fit. Dequant/requant means quality may differ from the official weights (MiniMax: “third-party quantizations may differ”). Not benchmarked here.
Quantized with oQ (oMLX v0.7.3.dev1). oQ3e-mtp: routed experts NVFP4 → affine 3-bit; MXFP8 kept for shared experts, sparse-attention indexer and about a third of attention projections; vision encoder (0.677B, 1.35 GB) and DSpark draft (2.011B, 2.30 GB) included; total 429.063B params, 196.15 GB; 3.631 bpw (3.657 incl. draft). oQ4e-mtp ≈248.8 GB. Built for the oMLX runtime (github.com/jundot/omlx), not necessarily stock mlx-lm. License wording contradiction: the oQ3e card says commercial use needs “obtaining prior written authorization from MiniMax”; the actual LICENSE requires written authorization only above US$20M yearly revenue, otherwise a one-time notice.
MLX v0.32.4 (Oct 14) and mlx-lm v0.32.2 (Oct 15 ET) still latest on GitHub and PyPI; mlx-swift / mlx-swift-lm newest Sep 30; mlx-vlm v0.7.10 (Oct 21; Issue 44) still latest; mlx-audio newest Oct 20. No GitHub Security advisories for MLX since Nov 2025, none ever for mlx-lm. Flint support in mlx-lm (#1998) not re-checked this morning (GitHub API rate limit).
Published Oct 21 17:46Z (≈1:46 PM ET). #2565 native Flint BF16 support (Blaizzy); #2542 Muse Spark Open with audio/video/images (lucasnewman); #2538 preserve NVFP4 scales + batch offloaded MoE execution; #2566 fix fused NVFP4 rowwise scales in expert-offload repacking. Note: mlx-vlm is a VLM package; Flint is text-only — this is a runtime path, not a vision feature.
mlx-community/flint-mixed-3-4bit created Oct 22 08:29Z (≈4:29 AM ET). mlx-community/DeepSeek-V4.1-Flash-1017-oQ2e/oQ2.5e/oQ3e/oQ4e-ngram-mtp created Oct 22 ~03:53–03:56Z (≈midnight ET). Community uploads (also many third-party Flint MLX/GGUF repos); check each card for conversion details. Not benchmarked here.
MLX v0.32.4 (Oct 14) and mlx-lm v0.32.2 (Oct 15 ET) still latest on GitHub and PyPI. mlx-lm: #1998 “Recursive Flint model support” open; merged Oct 22 ~09:46–09:48Z #1986 Mistral3 medium and #1955 keep NVFP4 global scales in convert --dtype / --dequantize; open #1945 Kolibri 1, #1997 GLM sparse prefill, #1988 Dory, #1982 Nemotron 3.5 Bolt. MLX: merged #4712 Metal qmv_mma/qmv_nax for 6–16 rows with large N/K (Oct 22 08:22Z); open #4734 sliding-window attention (relevant to Flint’s 35 sliding layers), #4722 int8 affine qmm path.
Hugging Face blog “MPS in PyTorch 2.15,” published Oct 20, 2026 by HF’s PyTorch team (Irakli Salia, Khushi Agrawal). Status: release candidate — post says “give the release candidate a try”; RC install uv pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/test/cpu; PyPI latest final at our check (≈8:20 AM ET) is 2.14.1 (Sep 30). Requirements: MPS needs macOS 15+; PyTorch 2.15 needs Python 3.11+; the MPS availability check now says why it is unavailable. FP8:torch.float8_e4m3fn tensors on MPS (1 byte/value) and F.scaled_mm with tensorwise and rowwise scaling. Decode:F.linear now uses the MPS GEMV kernels (since 2.14 only for torch.mm) with eligible bias fused — 1.23× geometric-mean over 96 BF16 layer shapes; native F.grouped_mm Metal kernels read group offsets on the GPU (Metal Performance Primitives where available) plus a small-group decode kernel (~1.3× MoE tok/s on its own); MoE decode 5–6.5× vs 2.14 on an M3 Ultra with torch.compile: Qwen3.6-35B-A3B 9.4 → 61.3 tok/s, LFM2.5-8B-A1B 23.6 → 123.6 tok/s; vocabulary softmax ~8× (57 µs → 7 µs in HF’s example). Linear algebra: complex64 LU / LU solve / triangular solve on MPS; single-kernel batched FP32 inverse up to 4×4; blocked QR; Gated DeltaNet fast path for 16/32/64 FP32 triangular systems with ≥8 RHS; SVD robustness fixes. Conv/padding/training: specialized conv1d forward kernels + im2col path; constant padding moved to Metal — Wan2.2-TI2V-5B tiled VAE decode (8,764 padding calls) peak driver memory ≈270 GiB → 8 GiB; reflection/replication padding on Metal with backward fixes; Qwen-Image-2.1 VAE encoder corruption on 2.14 (reported in ComfyUI) fixed — round-trip PSNR vs CPU 10.06 → 29.95 dB. Public MPS streams; SDPA math path batch/head broadcasting fix. 2.16 preview (nightlies/in review): command buffer committed every 16 kernels — compiled BF16 decode at 2,048 ctx: LFM2.5-8B-A1B 108.8 → 126.0 (+16%), Qwen3.6-35B-A3B 52.9 → 62.8 (+19%), Qwen3.8-27B 12.1 → 12.6 (+4%); SDPA fully on Metal (M5 Pro GQA decode at 4K: 465 µs → 65 µs). Flag: the 2.16 table’s “before” for Qwen3.6-35B-A3B (52.9 tok/s) is below the 2.15 headline (61.3) — different test settings, not reconciled in the post; treat absolute numbers as setup-specific. All figures HF-reported.
Published Oct 15, 2026. 816B total / 42B active MoE; 62 layers, hidden 8192, 128 Q / 16 KV heads (GQA, head dim 64), 256 experts/layer with top-8 routing (experts ≈97% of params; latent 8192→4096 expert projection per llama.cpp #30616); untied input/output embeddings ≈1.66B each; vocab 202,048; 1.8B perception encoder; convolutional audio encoder at one token per 80 ms (12.5 tok/s of audio). Context 1,048,576 tokens shared across modalities; KV ≈250 KB/token BF16 (≈266 GB at 1M, per HF). Reasoning cannot be turned off (low/medium/high/xhigh = 16/64/256/512; default xhigh). Recommended sampling temp 1.0, top_p 1.0, top_k off. Checkpoints: meta-models/Muse-Spark-Open-816B-A42B (BF16, safetensors total ≈1.64 TB) and -NVFP4; HF gate is manual approval. Model card says ≥700 GB GPU memory for NVFP4, 2 TB for BF16, and “optimized to run on two Mac Studios with M5 Ultra using MLX.” Hosted: Baseten, Fireworks, Groq, Modal, Nebius, OpenRouter, Together; weights also via SageMaker JumpStart (AWS What’s New Oct 15), Google Cloud Model Garden, Microsoft Foundry. Shares the base model with Muse Spark 1.3 (PyTorch blog). License: Muse Model License — MIT-style grant (use, modify, distill, sell) subject to (1) Usage Policy, (2) prominent “Built with Muse Spark Open” display for products using the Software or Derivatives, (3) a separate written agreement before offering it as Model-as-a-Service if group gross revenue exceeds US$20M in any 12 months (internal use and non-MaaS product features exempt). Benchmarks are Meta-reported (xhigh): e.g. MCP-Atlas 89.2, JobBench 61.8, Terminal-Bench 2.1 76.4 (trails four listed comparators at 80.1–85.0), GDPVal-AA v2 1401 (trails four comparators at 1424–1653). Full evaluation report “available soon.”
Published Oct 15, 2026. Benchmarks via mlx_lm.benchmark at 8,192 and 65,536 context, 128 generated tokens, batch 1. Models: Muse Spark Open (NVFP4), Mistral Medium 3.5 128B (FP8, dense), Qwen 3.8 27B (BF16, dense). Single box (512 GB): M5 Ultra TTFT up to 4.6× faster than M3 Ultra (GPU Neural Accelerators); decode ≈1.6× (1.2 TB/s vs 819 GB/s); peak power ≈200 W (M3 Ultra) / ≈300 W (M5 Ultra); Qwen 3.8 and Mistral Medium take up to 68 GB / 158 GB at 65K. Muse Spark Open NVFP4 ≈493 GB weights → does not fit one machine with intermediates. Multi-node: macOS 26.2 RDMA-over-Thunderbolt 5 driver + MLX ≥0.30.1 jaccl backend; two TB cables per link. Pipeline over 2 nodes leaves decode ≈flat and gives up to ≈1.8× prefill at 65K (chunked prefill overlap, --prefill-step-size 8192). Muse decode at 8K (Figure 1): 15.74 tok/s (2× pipeline), 24.99 (2× tensor), 36.73 (4× tensor). 4× M5 at 65K: two cables cut Muse TTFT from ≈50.5 s to 40.6 s. Two-machine tensor parallel lags pipeline on Muse prefill at 65K (expert-output normalization adds comms). Flag: post says “the latest version of MLX, released today”; GitHub/PyPI show MLX v0.32.4 on Oct 14 ET and mlx-lm 0.32.1/0.32.2 on Oct 15 ET. All numbers Apple-reported.
HF blog (Oct 15): mlx-community/Muse-Spark-Open-816B-A42B-oQ3e on an M3 Ultra 512 GB ≈344 GB, ≈225 tok/s prefill and ≈21 tok/s decode (1,024-token prompt, 128 generated). oQ3e/oQ4e published for oMLX; mlx-community also has 3bit/4bit (Oct 16). llama.cpp imatrix 2-bit (ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q2_K) ≈304 GB, 18 tok/s on M3 Ultra 512 GB; Q2_K, Q4_K_S, Q8 tested on Apple Silicon. HF: “even the 3-bit quantization requires more than 256 GB of RAM.” HF-reported. Contrast with Meta/Apple “two Mac Studios” framing: that is for NVFP4; lower-bit community quants fit one 512 GB box.
v0.32.1 Oct 15 21:22Z (≈5:22 PM ET): fix Muse template (#1977). v0.32.2 Oct 16 02:49Z (≈10:49 PM ET Oct 15): MiMo-V2-Flash attention value scaling (#1971, open in Issue 39’s check), SMM mask fix (#1979), Muse Spark HF names (#1981). The GitHub release object for v0.32.0 was published Oct 15 and lists meta-models/Muse-Spark-Open-816B-A42B-NVFP4 (#1976), nvfp4 MoE global scale (#1844), Hadamard-rotated 2-bit Qwen3.5 (#1941), unconverted-model support (#1962), tool-call argument parsing fixes (#1904, #1910); PyPI shows 0.32.0 uploaded Oct 1, so pin 0.32.2 for Muse.
Released Oct 13, 2026. EmbeddingGemma 2 multimodal embedding support (#2446). Clef decision model (#2459). LiquidAI d1-3B and d1-omni-600M decision models (#2467). Kolibri1 (#2423). Xing4.0 (#2465). Qwen3.5 packed gated-delta prefill via mlx-lm (#2442). RF-DETR / tracker RoPE / MoE offload fixes. MLX framework tag still 0.32.3; main continues with Metal/NAX commits Oct 13–14.
After Issue 36’s #4672/#4673: #4660 “Fix missing add_temporary in pad_gpu/fill_gpu” committed 2026-10-13; #4679 arctan2/radians/degrees scalar inputs 2026-10-12. Latest release tag remains v0.32.3 (Sep 29). No new GA tag in this window.
#3161 “Add 1-bit affine quantization support (Metal)” merged 2026-10-09T01:18Z ≈ 9:18 PM ET Oct 8. Extends affine widths from {2,3,4,5,6,8} to {1,2,3,4,5,6,8} using w_hat = scale * w_q + bias; affine 1-bit (bit 0 → w_min, bit 1 → w_max) and symmetric {−d,+d} both run through the affine path (no dedicated scale-only mode yet). CPU quantize/dequantize/qmm and all Metal quantized kernels. The PR says it “assumes the model has already been quantized externally (e.g. during training)” — packing and inference, not a 1-bit quantizer for arbitrary models. #4643 “[Metal] Fix SDPA max threadgroup for M1/M2 generation” (07:13Z ≈ 3:13 AM ET Oct 9): recent SDPA optimizations could exceed thread limits on older GPUs (RuntimeError: Maximum threads per threadgroup); repro used mlx-community/Qwen3.8-27B-4bit with a 1,024-token prompt cache. Also Oct 9: thin-M GEMM dispatch (#4654), FloorDivide primitive (#4642), .npy shape validation on load (#4657), compile scalar-merge fix (#4658), Metal negative-stride index fixes in scatter and SDPA masks, bfloat16 pickle strides (#4649). Latest release v0.32.3 (Sep 29); mlx-swift 0.32.3 / mlx-swift-lm 3.32.3 (Sep 30).
#2467 “feat: add LiquidAI d1-3B and d1-omni-600M decision models” merged 2026-10-08T20:30Z ≈ 4:30 PM ET; companion “let decision models declare the media they read” (13:36Z). Also: inkling uses Metal kernels only when Metal is available (#2462), RF-DETR segmentation fast upsample (#2468). Unreleased; latest v0.7.6 (Oct 5). d1 license is LFM 1.0 (see Issue 34).
Google Developers Blog, 2026-10-08. ML Drift is the GPU compute engine inside LiteRT, now standalone and open under Apache 2.0 at github.com/google-ai-edge/ml-drift. Backends: OpenGL ES, OpenCL, Metal, WebGPU; desktop previews on Windows and Linux by compiling the WebGPU backend natively with Dawn, alongside native Metal on macOS. Architecture: tensor virtualization (logical tensors decoupled from textures/buffers; one shader template set across backends), custom-op framework with low-level shading access and an agentic SKILL.md for coding agents to author/register/verify shaders, 5D tensors (3D conv, YOLO 11n, MobileViT v2, Swin v2), stage-aware LLM kernels (convolution-aligned KV-cache layout and in-kernel activation quantization during decode). Production users named: Chrome built-in AI (Prompt/Summarize/Writer APIs on Gemini Nano), YouTube Shorts (up to 40% lower frame latency), Photos, Meet, AI Edge Gallery, Adobe Lightroom/Photoshop mobile (up to 30% faster), Snap (30% latency). “Up to 12% lower memory overhead than other frameworks” on Gemma; all figures Google/partner-reported. Partners: Arm (Mali/Immortalis), Intel (Xe3 / XMX via WebGPU), Qualcomm (Adreno OpenCL). The legacy TFLite GPU delegate gets no new features; ML Drift is in standalone LiteRT packages now and “coming soon” to LiteRT in Google Play Services.
Released 2026-10-07T21:26Z ≈ 5:26 PM ET. Single-kernel Flash-Decode and FlashAttention-2 prefill GPU kernels on Metal with GQA (odml.sdpa_transposed). Google-reported, “Tested on MacBook M4 pro”: Qwen3-0.6B decode ~38% faster; 32K prefill 2.98× (Qwen3-0.6B) and 1.40× (Qwen3-4B); >4 GB peak RAM saved at 32K. Fused composite kernels odml.qkv_norm_rope and odml.swiglu (Metal and OpenCL; opt-in fusion passes): −69% Qwen3 decode dispatches, ~12% decode throughput (Google-reported). Dynamic Metal residency via MTLResidencySet on macOS and iOS for weights and external KV cache. Type-safe Swift API over the C API. Also: Intel NPU weight sharing (Gemma4 12B AOT 11.6 → 6.3 GB), zero-copy mmap program cache, separate litert-gpu Maven package, Android minSdk 24, MediaTek NPU on Android S+, vendor NPU Maven packages (the runtime-libraries zip is dropped), ATS coverage on macOS Metal/WebGPU.
“Add EmbeddingGemma 2 multimodal embedding support (#2446)” merged 2026-10-06T18:35Z ≈ 2:35 PM ET; “adding kolibri1 support (#2423)” merged 02:22Z Oct 6 ≈ 10:22 PM ET Oct 5. Also on main: fused SDPA in the RF-DETR backbone (#2443) and the packed gated-delta kernel for multi-token qwen3_5 calls (#2442). Latest release is still v0.7.6, so install from main or wait for a tag. Google’s launch post links mlx-vlm as the MLX path. mlx-community posted EmbeddingGemma 2 nvfp4 / mxfp4 / mxfp8 conversions around 11:54 AM ET Oct 6. Those are community quants, not Google files.
Issue 32 said PR #1945 (Kolibri 1 for mlx-lm) was still open. It now shows Closed, with last activity Oct 5, while Kolibri support landed on mlx-vlm main instead (see above; unreleased). mlx-lm main Oct 6: a Qwen3 Coder tool-argument parsing fix and a max_tokens: 0 server fix. MLX core main, Oct 6–7: Metal reductions 32-bit dispatch grid overflow (#4437), CPU compile-cache cross-process race (#4638), float16 overflow in clip_grad_norm (#4626), GPU boolean masking on 0-dim arrays (#4635). None are in a release; MLX remains 0.32.3.
v0.7.5 published 2026-10-05T18:31Z ≈ 2:31 PM ET: Decider and shared typed decision API (#2396); Laya on the decision API (#2397); decision CLI (#2402); conversation compaction in the Responses API (#2408); Qwen3-Omni multimodal batching (#2349); fixes for prequantized Qwen4 PLE loading, PhiMoE routing, top-p computed in float32, Qwen3-VL video timestamps, and MoE router stop-gradient (needs MLX ≥ 0.32.1). v0.7.6 published 19:14Z ≈ 3:14 PM ET: stream compaction progress and recover oversized requests (#2418).
MLX main commits on Oct 6: “[metal] Add env var based residency refresh interval (#4633)” and “Tune M1 quantized matmul dispatch (#4629)”. Neither is in a release (latest 0.32.3). mlx-lm last commit Oct 2. Kolibri 1 support PR #1945 still open (updated Oct 4). An mlx-vlm PR #2424 “Add Kolibri 1 (Aleph Alpha) text model” references it. Do not present either as shipped.
Opened Oct 4, 2026 by a community contributor: “add support for the (German language) Kolibri 1 model by Aleph Alpha.” Not merged at publish time. A related mlx-vlm PR (Blaizzy/mlx-vlm#2424) was closed. Community MLX conversions appeared on the Hub Oct 3–4 (e.g. 3/4/6/8-bit, MXFP4) — these are unofficial, dequantize the official FP8 weights and re-quantize; their size/perplexity/tok-s figures are uploader-reported. No mlx-community upload yet. Official weights: Aleph-Alpha/Kolibri-1 (FP8) and Kolibri-1-BF16.
Pre-release, updated ≈8:17 PM ET Sun Oct 4. On Apple Silicon, architectures supported by the MLX runner run on MLX by default; decision models Nimble, tev1, clef, clef-flash now available on MLX. rc1 aligned MLX tokenization with publisher tokenizers. See Local page for full notes.
Hub uploads Fri Oct 2, 2026. Cloudflare Clef and Clef Flash converted to MLX (4-bit and 8-bit). Decision / System One models: state (text, JSON, images, or video) + typed questions → probabilities in one forward pass. Not a chat model — mlx_vlm.generate / mlx_lm.generate / LM Studio load the backbone but produce meaningless text. Use bundled clef_mlx.py (model.systemone(...)). Backbone via mlx_vlm.convert -q; joint schema head copied bf16. License Apache-2.0 following Cloudflare/clef. Card tables (parity vs PyTorch, Decision Index sample 57.02 vs published 61.21, M5 Max latency) are uploader-reported. Related: clef-flash-4bit, clef-8bit, clef-flash-8bit. Do not rehash K2-Horizon.
Published ~2026-09-29 00:37 UTC ≈ 8:37 PM ET Sep 28. First mlx core release since v0.32.2 (2026-08-25). Apple-facing: Metal kernels for gated delta nets (#4020); M5 Ultra tunings for non-quantized matmuls (#4447); Metal SDPA D512 + D96/V64 support and memory improvements for D256/D512; Metal global scale for qmm; adaptive I/O concurrency on load; GIL release when copying Metal DLPack inputs (mx↔PyTorch MPS); various Metal/CUDA correctness fixes.
HF Hub API sample ~8:35 AM ET Sep 24. Notable after Issue 23 morning: mlx-community/LensVLM-9B-OptiQ-4bit (created ~5:30 AM ET Sep 24); mlx-community/Qwen3.8-Flash-Next-4bit-mtp (~4:48 PM ET Sep 23); also SAM2.1-hiera-small-fp16 and Nemotron-3-Diarization(-8bit) mid-day Sep 23. Community quants — verify license + mlx-lm / mlx-vlm arch support before try-this. Already in Issue 23 table (do not re-lead): Qwen3.6-35B-A3B-DFlash2-4bit.
