Supported conversion matrix¶
Which GGUF architectures convert, which are inspection-only, and what end-to-end evidence exists for each.
Split out of the top-level README on 2026-10-20; content is unchanged apart from heading levels.
gguf2mlx distinguishes between:
- recognized for inspection via GGUF metadata, and
- conversion enabled through an explicit tensor adapter.
Language-model conversion is enabled only for explicit adapters. Text conditioning adds a separate, opt-in contract for the four Qwen-VL identifiers above. Full-VLM detection and conversion provide LLaVA, LLaVA-NeXT (anyres), Qwen2-VL, Qwen2.5-VL, Qwen3-VL, and a dense MiniCPM-V 4.7 1B adapter.
Conversion-enabled architectures¶
| Family | GGUF architecture IDs | Conversion | End-to-end | MoE | Mixed precision | .mlx bundle |
VLM |
|---|---|---|---|---|---|---|---|
| Llama | llama, mistral |
✅ | ✅ Llama-3.2-1B Q4_0/Q4_K_M/Q8_0 real GGUF in CI | ❌ | — | ✅ generic package/verify | ❌ |
| Mixtral | mixtral, or llama with llama.expert_count > 0 |
✅ both on-disk layouts (requant only; preserve refuses it) |
✅ fixture-pinned name and shape conformance against mlx_lm.models.mixtral; no pinned real GGUF (smallest public quant is ~18 GB) |
✅ | — | ✅ generic package/verify | ❌ |
| Qwen | qwen2, dense qwen3 |
✅ | ✅ Qwen2.5-0.5B Q4_0/Q4_K_M/Q8_0 and Qwen3-0.6B Q8_0 in CI | ❌ | — | ✅ generic package/verify | ❌ |
| Qwen3.5 family | qwen35; aliases qwen3_5, qwen3.5, qwen36, qwen3_6, qwen3.6, qwen38, qwen3_8, qwen3.8 |
✅ dense layouts + preserve | ✅ Qwen3.5-0.8B Q4_K_M preserve: local run passes convert → weight gate → preserve gate → mlx_lm.load → finite logits (nightly job pending); aliases fixture-tested |
❌ | — | ✅ generic package/verify | ❌ |
| Qwen MoE | qwen2moe, qwen3moe |
✅ | ✅ Qwen1.5 and Qwen3-30B-A3B real-GGUF uniform 4-bit smoke tests | ✅ | ⚠️ experimental; JANG runtime unverified | ✅ generic package/verify | ❌ |
| DeepSeek | deepseek2, deepseek3 |
✅ | ✅ V2 Lite real Q2_K smoke test; V3 runtime unverified | ✅ | ⚠️ experimental; JANG runtime unverified | ✅ generic package/verify | ❌ |
| GLM | glm4moe |
✅ | ❌ | ✅ | ⚠️ experimental; JANG runtime unverified | ✅ generic package/verify | ❌ |
| Gemma | gemma, gemma2, gemma3, text-only gemma4 |
✅ | ✅ Gemma-2-2B Q4_K_M/Q8_0 real GGUF in CI (gemma, gemma3, gemma4 stay fixture-only) |
Gemma4 MoE layout supported | — | ✅ generic package/verify | ❌ |
| Phi | phi3 (Phi-3 and Phi-4; Phi-4 alias) |
✅ Phi-3 4K/LongRoPE and Phi-4 fixtures | ✅ Phi-3-mini Q4_K_M/Q8_0 real GGUF in CI plus synthetic GGUF | ❌ | — | ✅ generic package/verify | ❌ |
| StableLM | stablelm |
✅ | ❌ | ❌ | — | ✅ generic package/verify | ❌ |
| GLM | glm-dsa |
⚠️ experimental | ❌ | ✅ | ⚠️ experimental; JANG runtime unverified | ✅ generic package/verify | ❌ |
| LLaVA | llava, or llama + CLIP/mmproj |
✅ direct/hybrid fixture-validated | Tiny image-path parity; pretrained quality unverified | ❌ | Language-only 4-bit | ✅ generic package/verify | ✅ Llama + CLIP only |
| LLaVA-NeXT (anyres) | llava_next + CLIP-L/336 mmproj |
✅ direct (no HF vision) | ✅ llava-v1.6-vicuna-7b Q4_K_M + F16 mmproj: 378/378 mmproj tensors consumed, 380/380 emitted names match the HF index by name and shape, 375/375 tower parameters load strictly into mlx-vlm and the tower runs. The text half must come from the same HF checkpoint: llama.cpp's LLaVA exports drop the <image> row and are refused by name |
❌ | Not measured | ✅ generic package/verify | ✅ CLIP + anyres projector + model.image_newline |
| Qwen2-VL | qwen2vl + Qwen2-VL/mmproj |
✅ direct/hybrid fixture-validated | Named 2B direct checkpoint: 12/12 image diagnostic; see report | ❌ | Uniform 4-bit: 11/12 exact; source-preserved Q4_K/float: 12/12 | ✅ generic package/verify | ✅ ViT + patch merger |
| Qwen2.5-VL | qwen2vl + clip.projector_type=qwen2.5vl_merger |
✅ direct (no HF vision) | ✅ Qwen2.5-VL-3B-Instruct Q4_K_M + F16 mmproj: 824/824 tensors match the HF index by name and shape; fused QKV / stacked patch embedding / merger within 3e-8 of the checkpoint | ❌ | Not measured | ✅ generic package/verify | ✅ RMSNorm ViT + gated SiLU MLP + RMSNorm merger |
| Qwen3-VL | qwen3vl + clip.projector_type=qwen3vl_merger |
✅ direct (no HF vision) | ✅ Qwen3-VL-2B-Instruct Q4_K_M + F16 mmproj: 625/625 tensors match the HF index by name and shape; deepstack mergers mapped by rank | ❌ | Not measured | ✅ generic package/verify | ✅ fused QKV + deepstack mergers + learned pos_embed |
| MiniCPM-V 4.7 1B | dense qwen35 + clip.projector_type=minicpmv4_7 |
✅ portable + native image path | Native image-path CI; pretrained caption quality unverified | ❌ (35B-A3B excluded) | ❌ | ✅ generic package/verify | ✅ SigLIP + window/final mergers |
| Qwen-VL text encoder | qwen3vl, qwen3_vl, qwen2vl, qwen2_vl |
✅ --type text-encoder, BF16 default |
Portable conversion only; native hidden-state/image quality unverified | ❌ | Explicit direct 4-bit only | ✅ generic package/verify | Text only; vision skipped |
“Conversion” means that a code path exists. “End-to-end” means a converted model
has been loaded and exercised; ⚠️ rows still require the opt-in model fixtures.
The mixed-precision contract validator checks agreement between
conversion_report.json and jang_config.json; this is not a substitute for
loading the result in a JANG-compatible runtime. .mlx bundle packaging and
integrity verification are architecture-independent operations on emitted model
directories and do not assert runtime compatibility.
The Qwen2.5-VL and Qwen3-VL vision claims have two independent checks behind
them, because neither is sufficient alone. tests/test_vlm_qwen25.py and
tests/test_vlm_qwen3.py drive synthetic fixtures that prove the converter
assembles what each adapter declares; tests/test_vlm_real_mmproj.py reads the
downloaded mmproj files and proves each declaration matches what llama.cpp
actually shipped — every source named, every shape right, and the assembled
target shape reproduced from real sources. Neither test can substitute for the
other: a fixture can only contain the tensors the plan asks for, so a plan that
invents a tensor passes it. The tensor count is pinned in that test so a
regenerated fixture changes the count instead of passing quietly.
The ✅ end-to-end cells are produced by the Load matrix workflow, which
downloads each pinned GGUF by SHA-256, converts it, runs the weight-fidelity
gate, loads the result with mlx_lm and requires finite logits. All twelve
pinned entries are green in CI as of 2026-10-28, including the Qwen3.5 preserve
entry added in this cycle; the earlier published run
(39505897872,
2026-10-21) covered eleven. The entry list lives in tests/load_matrix.json, and
each job uploads the conversion report it produced.
Pinned versus declared-only¶
CONVERTIBLE_ARCHES lists 31 architecture IDs — 23 canonical plus 8 alias
spellings; the pinned matrix covers six of those canonical architectures
(llama, qwen2, qwen3, qwen35, gemma2, phi3). The
difference is not implied — it is written down and enforced.
tests/test_load_matrix.py fails when an architecture is convertible but is
neither pinned nor listed in DECLARED_COVERAGE_GAPS with a reason, so a new
adapter cannot quietly widen the support claim. Alias spellings (qwen3_5,
qwen3.5, qwen2vl, …) share one canonical ID and one case. Separately,
tests/test_runtime_registry.py checks every emitted model_type against the
minimum pinned mlx-lm/mlx-vlm registries.
| Architecture IDs without a pinned case | Reason |
|---|---|
deepseek2, deepseek3, glm4moe, glm-dsa, qwen2moe, qwen3moe |
MoE/MLA sources; the smallest public GGUFs exceed the per-runner download budget, and router protection needs its own case. weekly-moe.yml can cover qwen3moe and deepseek2 instead, but only on a runner that already holds those files: it is inactive until GGUF2MLX_MOE_MODELS is set, and reports skipped rather than passing. |
gemma, gemma3, gemma4 |
share gemma2's adapter path, which is pinned; no small pinned GGUF for the others yet |
mistral, phi4 |
share llama's tensor map and phi3's sliding-window adapter, both pinned |
mixtral |
8x7B MoE; the smallest public quant is ~18 GB, over the per-runner budget. tests/test_mixtral.py pins both on-disk layouts and checks every emitted name against mlx_lm.models.mixtral instead |
qwen36, qwen38 (and their alias spellings) |
newer hybrid variants share the dense Qwen3.5 adapter and are fixture-tested; only qwen35 has a pinned real-model case |
stablelm |
adapter ships; no pinned public GGUF yet |
qwen2_vl, qwen3_vl |
full VLM entries need the matching HF repo beside the GGUF; vision.yml owns that path |
Pin an entry by adding it to tests/load_matrix.json and deleting its line from
DECLARED_COVERAGE_GAPS — the test fails on a stale line, so the two cannot
drift apart.
MoE naming matrix¶
Every accepted MoE layout names its router and its experts differently, and the
per-family suites cannot see each other. tests/test_moe_matrix.py holds one row
per layout and checks three things: the router lands on the module mlx-lm
instantiates, the stacked experts land in that family's switch module, and the
shared-expert module is spelled the way that family spells it.
| Layout | Router | Routed experts | Shared expert |
|---|---|---|---|
qwen2moe |
mlp.gate |
mlp.switch_mlp.* |
mlp.shared_expert + mlp.shared_expert_gate |
qwen3moe |
mlp.gate |
mlp.switch_mlp.* |
none |
deepseek2, deepseek3, glm4moe, glm-dsa |
mlp.gate (+ e_score_correction_bias) |
mlp.switch_mlp.* |
mlp.shared_experts |
mixtral |
block_sparse_moe.gate |
block_sparse_moe.switch_mlp.* |
none |
gemma4 |
router.proj (+ router.scale) |
experts.switch_glu.* |
none |
The table is checked against mlx_lm.models.<family> parameter trees, not only
against itself, so a plausible-looking row that names a module the runtime does
not have fails. glm-dsa is the one layout outside the runtime half — its
glm_moe_dsa arguments need an indexer configuration this matrix has no other
use for — and a separate assertion fails if that gap changes in either direction,
so it cannot be forgotten once a tiny configuration exists.
Qwen3 MoE real-model validation¶
On 2026-10-18, Qwen3-30B-A3B-Q4_K_M.gguf from
unsloth/Qwen3-30B-A3B-GGUF
was converted and exercised on Apple Silicon with 64 GiB unified memory.
The source is 18,556,686,912 bytes, with SHA-256
9f1a24700a339b09c06009b729b5c809e0b64c213b8af5b711b3dbdfd0c5ba48.
The passing pytest rerun used gguf 0.19.0, mlx 0.32.2, mlx-lm 0.31.3,
and numpy 2.4.6.
| Check | Observed result |
|---|---|
| Mixed-precision conversion and artifact contract | Passed; 48 protected router tensors and 144 compressed expert tensors |
| Uniform direct 4-bit conversion and MLX-LM loading | Passed |
| Uniform-model logits | All finite; shape [1, 8, 151936] |
| First-layer router logits | Shape [1, 8, 128]; normalized entropy 0.895546, within the required (0.05, 0.95) interval |
| Default thinking-mode generation | 32 tokens, 28 unique tokens; begins normal <think> reasoning before reaching the token limit |
Historical test failure, now fixed: the original output.isprintable()
assertion rejected the normal newline after Qwen3's <think> marker. The test
now accepts line breaks and tabs while rejecting empty/whitespace-only output
and other control characters; eight regression cases cover these boundaries.
The actual real-MoE pytest now passes without disabling thinking or changing
its entropy/token-diversity thresholds. A 32-token reasoning fragment does not
establish answer quality: this remains a smoke test, not a semantic benchmark.
Mixed-model inference and JANG runtime compatibility remain unverified.
The source and both conversions require approximately 52 GiB of disk.
Qwen3.5-family variants share the non-MoE Qwen3.5 tensor contract; Qwen3.x MoE
and Gemma4 multimodal/audio components are not included in these text-model
adapters. qwen35 is the only variant with a real-checkpoint run (Qwen3.5-0.8B,
Q4_K_M, preserve); qwen36 and qwen38 remain fixture-tested only.
DeepSeek-V2 Lite real-model validation¶
The pinned DeepSeek-Coder-V2-Lite-Instruct-Q2_K.gguf checkpoint also passes
mixed-artifact validation and uniform direct 4-bit conversion, MLX-LM loading,
finite-logit checks, router entropy, and generation. Its first routed layer
has logits of shape [1, 7, 64] and normalized entropy 0.883369; generation
produced 32 tokens, 29 unique, beginning with an explanation of Rayleigh
scattering.
Enabling this previously skipped test exposed missing DeepSeek-V2 config:
conversion now preserves the source's dense/expert FFN sizes, leading dense
layers, shared/routed experts, query/KV LoRA layout, MLA head dimensions, and
YaRN scaling (including GGUF's log-multiplier encoding). Invalid required
metadata fails explicitly. The entropy probe measures raw router logits even
when the runtime gate returns top-k indices and scores.
This validates the named V2 Lite checkpoint, not DeepSeek-V3 or mixed-model
inference in a JANG runtime. The checkpoint's deepseek-llm tokenizer profile
still emits the existing legacy-BPE fallback warning; exact native tokenizer
parity for DeepSeek was not established by this smoke test.
Inspection only (not converted)¶
These may still be recognized by metadata or --skip-weights, but they are
rejected during conversion until a dedicated, tested adapter exists:
arctic, baichuan, bert, bitnet, bloom, chameleon, chatglm,
codeshell, command-r, command-r-plus, dbrx, exaone, falcon,
gpt2, gptneox, granite, grok-1, jais, minicpm,
minicpm3, mpt, nemotron, olmo, olmo2, openelm, orion, phi,
phi2, plamo, refact, smolm, starcoder, t5, and xverse.
That means no more silent "Llama fallback" producing invalid outputs for unrelated architectures.
MiniCPM-V support is limited to the dense 4.7 1B checkpoint, supplied as a
Qwen3.5 text GGUF plus its matching minicpmv4_7 vision/projector GGUF and
original HF assets. The 35B-A3B MoE and older 4.6 layouts are rejected.
Plain MiniCPM3 is a separate text-only architecture and is not handled by the
MiniCPM-V adapter.